Skip to content

Integrate threading modules from python-isal #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Dec 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ Changelog

version 0.4.0-dev
-----------------
+ Add a ``gzip_ng_threaded`` module that contains the ``gzip_ng_threaded.open``
function. This allows using multithreaded compression as well as escaping the
GIL.
+ The internal ``gzip_ng._GzipReader`` has been rewritten in C. As a result the
overhead of decompressing files has significantly been reduced.
+ The ``gzip_ng._GzipReader`` in C is now used in ``gzip_ng.decompress``. The
Expand Down
8 changes: 7 additions & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ by providing Python bindings for the zlib-ng library.
This package provides Python bindings for the `zlib-ng
<https://github.com/zlib-ng/zlib-ng>`_ library.

``python-zlib-ng`` provides the bindings by offering two modules:
``python-zlib-ng`` provides the bindings by offering three modules:

+ ``zlib_ng``: A drop-in replacement for the zlib module that uses zlib-ng to
accelerate its performance.
Expand All @@ -51,6 +51,11 @@ This package provides Python bindings for the `zlib-ng
instead of ``zlib`` to perform its compression and checksum tasks, which
improves performance.

+ ``gzip_ng_threaded`` offers an ``open`` function which returns buffered read
or write streams that can be used to read and write large files while
escaping the GIL using one or multiple threads. This functionality only
works for streaming, seeking is not supported.

``zlib_ng`` and ``gzip_ng`` are almost fully compatible with ``zlib`` and
``gzip`` from the Python standard library. There are some minor differences
see: differences-with-zlib-and-gzip-modules_.
Expand All @@ -68,6 +73,7 @@ The python-zlib-ng modules can be imported as follows

from zlib_ng import zlib_ng
from zlib_ng import gzip_ng
from zlib_ng import gzip_ng_threaded

``zlib_ng`` and ``gzip_ng`` are meant to be used as drop in replacements so
their api and functions are the same as the stdlib's modules.
Expand Down
7 changes: 7 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,13 @@ API-documentation: zlib_ng.gzip_ng
:members:
:special-members: __init__

===========================================
API-documentation: zlib_ng.gzip_ng_threaded
===========================================

.. automodule:: zlib_ng.gzip_ng_threaded
:members: open

===============================
python -m zlib_ng.gzip_ng usage
===============================
Expand Down
Loading