Backwards Compatibility Notes
- Support for Python 2.7 has been dropped. Python 3.5 is now the
minimum required Python version. (#109) train_dictionary()
now uses thefastcover
training mechanism
(as opposed tocover
). Some parameter values that worked with the old
mechanism may not work with the new one. e.g.d
must be6
or8
if it is defined.train_dictionary()
now always calls
ZDICT_optimizeTrainFromBuffer_fastCover()
instead of different APIs
depending on which arguments were passed.- The names of various Python modules have been changed. The C extension
is now built aszstandard.backend_c
instead ofzstd
. The
CFFI extension module is now built aszstandard._cffi
instead of
_zstd_cffi
. The CFFI backend is nowzstandard.backend_cffi
instead
ofzstandard.cffi
. ZstdDecompressionReader.seekable()
now returnsFalse
instead of
True
because not all seek operations are supported and some Python
code in the wild keys off this value to determine ifseek()
can be
called for all scenarios.ZstdDecompressionReader.seek()
now raisesOSError
instead of
ValueError
when the seek cannot be fulfilled. (#107)ZstdDecompressionReader.readline()
and
ZstdDecompressionReader.readlines()
now accept an integer argument.
This makes them conform with the IO interface. The methods still raise
io.UnsupportedOperation
.ZstdCompressionReader.__enter__
andZstdDecompressionReader.__enter__
now raiseValueError
if the instance was already closed.- The deprecated
overlap_size_log
attribute onZstdCompressionParameters
instances has been removed. Theoverlap_log
attribute should be used
instead. - The deprecated
overlap_size_log
argument toZstdCompressionParameters
has been removed. Theoverlap_log
argument should be used instead. - The deprecated
ldm_hash_every_log
attribute on
ZstdCompressionParameters
instances has been removed. The
ldm_hash_rate_log
attribute should be used instead. - The deprecated
ldm_hash_every_log
argument to
ZstdCompressionParameters
has been removed. Theldm_hash_rate_log
argument should be used instead. - The deprecated
CompressionParameters
type alias to
ZstdCompressionParamaters
has been removed. Use
ZstdCompressionParameters
. - The deprecated aliases
ZstdCompressor.read_from()
and
ZstdDecompressor.read_from()
have been removed. Use the corresponding
read_to_iter()
methods instead. - The deprecated aliases
ZstdCompressor.write_to()
and
ZstdDecompressor.write_to()
have been removed. Use the corresponding
stream_writer()
methods instead. ZstdCompressor.copy_stream()
,ZstdCompressorIterator.__next__()
,
andZstdDecompressor.copy_stream()
now raise the original exception
on error calling the source stream'sread()
instead of raising
ZstdError
. This only affects the C backend.ZstdDecompressionObj.flush()
now returnsbytes
instead of
None
. This makes it behave more similarly toflush()
methods
for similar types in the Python standard library. (#78)ZstdCompressionWriter.__exit__()
now always callsclose()
.
Previously,close()
would not be called if the context manager
raised an exception. The old behavior was inconsistent with other
stream types in this package and with the behavior of Python's
standard library IO types. (#86)- Distribution metadata no longer lists
cffi
as aninstall_requires
except when running on PyPy. Instead,cffi
is listed as an
extras_require
. ZstdCompressor.stream_reader()
andZstdDecompressor.stream_reader()
now default to closing the source stream when the instance is itself
closed. To change this behavior, passclosefd=False
. (#76)- The
CFFI
backend now defines
ZstdCompressor.multi_compress_to_buffer()
and
ZstdDecompressor.multi_decompress_to_buffer()
. However, they
raiseNotImplementedError
, as they are not yet implemented. - The
CFFI
backend now exposes the typesZstdCompressionChunker
,
ZstdCompressionObj
,ZstdCompressionReader
,
ZstdCompressionWriter
,ZstdDecompressionObj
,
ZstdDecompressionReader
, andZstdDecompressionWriter
as
symbols on thezstandard
module. - The
CFFI
backend now exposes the typesBufferSegment
,
BufferSegments
,BufferWithSegments
, and
BufferWithSegmentsCollection
. However, they are not implemented. ZstdCompressionWriter.flush()
now callsflush()
on the inner stream
if such a method exists. However, whenclose()
itself calls
self.flush()
,flush()
is not called on the inner stream.ZstdDecompressionWriter.close()
no longer callsflush()
on
the inner stream. However,ZstdDecompressionWriter.flush()
still
callsflush()
on the inner stream.ZstdCompressor.stream_writer()
andZstdDecompressor.stream_writer()
now have theirwrite_return_read
argument default toTrue
.
This brings the behavior ofwrite()
in compliance with the
io.RawIOBase
interface by default. The argument may be removed
in a future release.ZstdCompressionParameters
no longer exposes acompression_strategy
property. Its constructor no longer accepts acompression_strategy
argument. Use thestrategy
property/argument instead.
Bug Fixes
- Fix a memory leak in
stream_reader
decompressor when reader is closed
before reading everything. (Patch by Pierre Fersing.) - The C backend now properly checks for errors after calling IO methods
on inner streams in various methods.ZstdCompressionWriter.write()
now catches exceptions when calling the inner stream'swrite()
.
ZstdCompressionWriter.flush()
on inner stream'swrite()
.
ZstdCompressor.copy_stream()
on dest stream'swrite()
.
ZstdDecompressionWriter.write()
on inner stream'swrite()
.
ZstdDecompressor.copy_stream()
on dest stream'swrite()
. (#102)
Changes
- Bundled zstandard library upgraded from 1.4.5 to 1.4.8.
- The bundled zstandard library is now using the single C source file
distribution. The 2 main header files are still present, as these are
needed by CFFI to generate the CFFI bindings. PyBuffer
instances are no longer checked to be C contiguous and
have a single dimension. The former was redundant with what
PyArg_ParseTuple()
already did and the latter is not necessary
in practice because very few extension modules create buffers with
more than 1 dimension. (#124)- Added Python typing stub file for the
zstandard
module. (#120) - The
make_cffi.py
script should now respect theCC
environment
variable for locating the compiler. (#103) - CI now properly uses the
cffi
backend when running all tests. train_dictionary()
has been rewritten to use thefastcover
APIs
and to consistently callZDICT_optimizeTrainFromBuffer_fastCover()
instead of different C APIs depending on what arguments were passed.
The function also now accepts argumentsf
,split_point
, and
accel
, which are parameters unique tofastcover
.- CI now tests and builds wheels for Python 3.9.
zstd.c
file renamed toc-ext/backend_c.c
.- All built/installed Python modules are now in the
zstandard
package. Previously, there were modules in other packages. (#115) - C source code is now automatically formatted with
clang-format
. ZstdCompressor.stream_writer()
,ZstdCompressor.stream_reader()
,
ZstdDecompressor.stream_writer()
, and
ZstdDecompressor.stream_reader()
now accept aclosefd
argument to control whether the underlying stream should be closed
when theZstdCompressionWriter
,ZstdCompressReader
,
ZstdDecompressionWriter
, orZstdDecompressionReader
is closed.
(#76)- There is now a
zstandard.open()
function for returning a file
object with zstd (de)compression. (#64) - The
zstandard
module now exposes abackend_features
attribute containing a set of strings denoting optional features
present in that backend. This can be used to sniff feature support
by performing a string lookup instead of sniffing for API presence
or behavior. - Python docstrings have been moved from the C backend to the CFFI
backend. Sphinx docs have been updated to generate API documentation
via the CFFI backend. Documentation for Python APIs is now fully
defined via Python docstrings instead of spread across Sphinx ReST
files and source code. ZstdCompressionParameters
now exposes astrategy
property.- There are now
compress()
anddecompress()
convenience functions
on thezstandard
module. These are simply wrappers around the
corresponding APIs onZstdCompressor
andZstdDecompressor
.