New Features
- zstd library upgraded from 1.3.6 to 1.3.8.
ZstdCompressor.stream_reader()
,ZstdCompressor.stream_writer()
,ZstdDecompressor.stream_reader()
, andZstdDecompressor.stream_writer()
should now completely implement theio.RawIOBase
interface, allowing them to be used as fully-featured streams. One can combine an instance with say anio.TextIOWrapper
,codecs.StreamReader
, orcodecs.StreamWriter
to transparently convert between binary and textual data or apply encodings to data. A stream reader or writer should interoperate with any Python code that accepts a stream-like object. Please note that some implementation details may change slightly in the future in order to ensure better compatibility with the stream interfaces.ZstdDecompressor.stream_reader()
can now read inputs that have multiple zstd frames. Look for future releases to improve support for reading sources with multiple frames.
Backwards Compatibility Notes
ZstdDecompressor.read()
now allows reading sizes of-1
or0
and defaults to-1
, per the documented behavior of
io.RawIOBase.read()
. Previously, we required an argument that was
a positive value.- The
readline()
,readlines()
,__iter__
, and__next__
methods
ofZstdDecompressionReader()
now raiseio.UnsupportedOperation
instead ofNotImplementedError
. ZstdDecompressor.stream_reader()
now accepts aread_across_frames
argument. The default value will likely be changed in a future release
and consumers are advised to pass the argument to avoid unwanted change
of behavior in the future.setup.py
now always disables the CFFI backend if the installed
CFFI package does not meet the minimum version requirements. Before, it was
possible for the CFFI backend to be generated and a run-time error to
occur.- In the CFFI backend,
CompressionReader
andDecompressionReader
were renamed toZstdCompressionReader
andZstdDecompressionReader
,
respectively so naming is identical to the C extension. This should have
no meaningful end-user impact, as instances aren't meant to be
constructed directly. ZstdDecompressor.stream_writer()
now accepts awrite_return_read
argument to control whetherwrite()
returns the number of bytes
read from the source / written to the decompressor. It defaults to off,
which preserves the existing behavior of returning the number of bytes
emitted from the decompressor. The default will change in a future release
so behavior aligns with the specified behavior ofio.RawIOBase
.ZstdDecompressionWriter.__exit__
now callsself.close()
. This
will result in that stream plus the underlying stream being closed as
well. If this behavior is not desirable, do not use instances as
context managers.ZstdCompressor.stream_writer()
now accepts awrite_return_read
argument to control whetherwrite()
returns the number of bytes read
from the source / written to the compressor. It defaults to off, which
preserves the existing behavior of returning the number of bytes emitted
from the compressor. The default will change in a future release so
behavior aligns with the specified behavior ofio.RawIOBase
.ZstdCompressionWriter.__exit__
now callsself.close()
. This will
result in that stream plus any underlying stream being closed as well. If
this behavior is not desirable, do not use instances as context managers.ZstdDecompressionWriter
no longer requires being used as a context
manager (#57).ZstdCompressionWriter
no longer requires being used as a context
manager (#57).- The
overlap_size_log
attribute onCompressionParameters
instances
has been deprecated and will be removed in a future release. The
overlap_log
attribute should be used instead. - The
overlap_size_log
argument toCompressionParameters
has been
deprecated and will be removed in a future release. Theoverlap_log
argument should be used instead. - The
ldm_hash_every_log
attribute onCompressionParameters
instances
has been deprecated and will be removed in a future release. The
ldm_hash_rate_log
attribute should be used instead. - The
ldm_hash_every_log
argument toCompressionParameters
has been
deprecated and will be removed in a future release. Theldm_hash_rate_log
argument should be used instead. - The
compression_strategy
argument toCompressionParameters
has been
deprecated and will be removed in a future release. Thestrategy
argument should be used instead. - The
SEARCHLENGTH_MIN
andSEARCHLENGTH_MAX
constants are deprecated
and will be removed in a future release. UseMINMATCH_MIN
and
MINMATCH_MAX
instead. - The
zstd_cffi
module has been renamed tozstandard.cffi
. As had
been documented in theREADME
file since the0.9.0
release, the
module should not be imported directly at its new location. Instead,
import zstandard
to cause an appropriate backend module to be loaded
automatically.
Bug Fixes
- CFFI backend could encounter a failure when sending an empty chunk into
ZstdDecompressionObj.decompress()
. The issue has been fixed. - CFFI backend could encounter an error when calling
ZstdDecompressionReader.read()
if there was data remaining in an
internal buffer. The issue has been fixed. (#71)
Changes
ZstDecompressionObj.decompress()
now properly handles empty inputs in
the CFFI backend.ZstdCompressionReader
now implementsread1()
andreadinto1()
.
These are part of theio.BufferedIOBase
interface.ZstdCompressionReader
has gained areadinto(b)
method for reading
compressed output into an existing buffer.ZstdCompressionReader.read()
now defaults tosize=-1
and accepts
read sizes of-1
and0
. The new behavior aligns with the documented
behavior ofio.RawIOBase
.ZstdCompressionReader
now implementsreadall()
. Previously, this
method raisedNotImplementedError
.ZstdDecompressionReader
now implementsread1()
andreadinto1()
.
These are part of theio.BufferedIOBase
interface.ZstdDecompressionReader.read()
now defaults tosize=-1
and accepts
read sizes of-1
and0
. The new behavior aligns with the documented
behavior ofio.RawIOBase
.ZstdDecompressionReader()
now implementsreadall()
. Previously, this
method raisedNotImplementedError
.- The
readline()
,readlines()
,__iter__
, and__next__
methods
ofZstdDecompressionReader()
now raiseio.UnsupportedOperation
instead ofNotImplementedError
. This reflects a decision to never
implement text-based I/O on (de)compressors and keep the low-level API
operating in the binary domain. (#13) README.rst
now documented how to achieve linewise iteration using
anio.TextIOWrapper
with aZstdDecompressionReader
.ZstdDecompressionReader
has gained areadinto(b)
method for
reading decompressed output into an existing buffer. This allows chaining
to anio.TextIOWrapper
on Python 3 without using anio.BufferedReader
.ZstdDecompressor.stream_reader()
now accepts aread_across_frames
argument to control behavior when the input data has multiple zstd
frames. WhenFalse
(the default for backwards compatibility), a
read()
will stop when the end of a zstd frame is encountered. When
True
,read()
can potentially return data spanning multiple zstd
frames. The default will likely be changed toTrue
in a future
release.setup.py
now performs CFFI version sniffing and disables the CFFI
backend if CFFI is too old. Previously, we only usedinstall_requires
to enforce the CFFI version and not all build modes would properly enforce
the minimum CFFI version. (#69)- CFFI's
ZstdDecompressionReader.read()
now properly handles data
remaining in any internal buffer. Before, repeatedread()
could
result in random errors. (#71) - Upgraded various Python packages in CI environment.
- Upgrade to hypothesis 4.5.11.
- In the CFFI backend,
CompressionReader
andDecompressionReader
were renamed toZstdCompressionReader
andZstdDecompressionReader
,
respectively. ZstdDecompressor.stream_writer()
now accepts awrite_return_read
argument to control whetherwrite()
returns the number of bytes read
from the source. It defaults toFalse
to preserve backwards
compatibility.ZstdDecompressor.stream_writer()
now implements theio.RawIOBase
interface and behaves as a proper stream object.ZstdCompressor.stream_writer()
now accepts awrite_return_read
argument to control whetherwrite()
returns the number of bytes read
from the source. It defaults toFalse
to preserve backwards
compatibility.ZstdCompressionWriter
now implements theio.RawIOBase
interface and
behaves as a proper stream object.close()
will now close the stream
and the underlying stream (if possible).__exit__
will now call
close()
. Methods likewritable()
andfileno()
are implemented.ZstdDecompressionWriter
no longer must be used as a context manager.ZstdCompressionWriter
no longer must be used as a context manager.
When not using as a context manager, it is important to call
flush(FRAME_FRAME)
or the compression stream won't be properly
terminated and decoders may complain about malformed input.ZstdCompressionWriter.flush()
(what is returned from
ZstdCompressor.stream_writer()
) now accepts an argument controlling the
flush behavior. Its value can be one of the new constants
FLUSH_BLOCK
orFLUSH_FRAME
.ZstdDecompressionObj
instances now have aflush([length=None])
method.
This provides parity with standard library equivalent types. (#65)CompressionParameters
no longer redundantly store individual compression
parameters on each instance. Instead, compression parameters are stored inside
the underlyingZSTD_CCtx_params
instance. Attributes for obtaining
parameters are now properties rather than instance variables.- Exposed the
STRATEGY_BTULTRA2
constant. CompressionParameters
instances now expose anoverlap_log
attribute.
This behaves identically to theoverlap_size_log
attribute.CompressionParameters()
now accepts anoverlap_log
argument that
behaves identically to theoverlap_size_log
argument. An error will be
raised if both arguments are specified.CompressionParameters
instances now expose anldm_hash_rate_log
attribute. This behaves identically to theldm_hash_every_log
attribute.CompressionParameters()
now accepts aldm_hash_rate_log
argument that
behaves identically to theldm_hash_every_log
argument. An error will be
raised if both arguments are specified.CompressionParameters()
now accepts astrategy
argument that behaves
identically to thecompression_strategy
argument. An error will be raised
if both arguments are specified.- The
MINMATCH_MIN
andMINMATCH_MAX
constants were added. They are
semantically equivalent to the oldSEARCHLENGTH_MIN
and
SEARCHLENGTH_MAX
constants. - Bundled zstandard library upgraded from 1.3.7 to 1.3.8.
setup.py
denotes support for Python 3.7 (Python 3.7 was supported and
tested in the 0.10 release).zstd_cffi
module has been renamed tozstandard.cffi
.ZstdCompressor.stream_writer()
now reuses a buffer in order to avoid
allocating a new buffer for every operation. This should result in faster
performance in cases wherewrite()
orflush()
are being called
frequently. (#62)- Bundled zstandard library upgraded from 1.3.6 to 1.3.7.