github indygreg/python-zstandard 0.15.0

latest releases: 0.23.0, 0.21.0, 0.22.0...
3 years ago

Backwards Compatibility Notes

  • Support for Python 2.7 has been dropped. Python 3.5 is now the
    minimum required Python version. (#109)
  • train_dictionary() now uses the fastcover training mechanism
    (as opposed to cover). Some parameter values that worked with the old
    mechanism may not work with the new one. e.g. d must be 6 or 8
    if it is defined.
  • train_dictionary() now always calls
    ZDICT_optimizeTrainFromBuffer_fastCover() instead of different APIs
    depending on which arguments were passed.
  • The names of various Python modules have been changed. The C extension
    is now built as zstandard.backend_c instead of zstd. The
    CFFI extension module is now built as zstandard._cffi instead of
    _zstd_cffi. The CFFI backend is now zstandard.backend_cffi instead
    of zstandard.cffi.
  • ZstdDecompressionReader.seekable() now returns False instead of
    True because not all seek operations are supported and some Python
    code in the wild keys off this value to determine if seek() can be
    called for all scenarios.
  • ZstdDecompressionReader.seek() now raises OSError instead of
    ValueError when the seek cannot be fulfilled. (#107)
  • ZstdDecompressionReader.readline() and
    ZstdDecompressionReader.readlines() now accept an integer argument.
    This makes them conform with the IO interface. The methods still raise
    io.UnsupportedOperation.
  • ZstdCompressionReader.__enter__ and ZstdDecompressionReader.__enter__
    now raise ValueError if the instance was already closed.
  • The deprecated overlap_size_log attribute on ZstdCompressionParameters
    instances has been removed. The overlap_log attribute should be used
    instead.
  • The deprecated overlap_size_log argument to ZstdCompressionParameters
    has been removed. The overlap_log argument should be used instead.
  • The deprecated ldm_hash_every_log attribute on
    ZstdCompressionParameters instances has been removed. The
    ldm_hash_rate_log attribute should be used instead.
  • The deprecated ldm_hash_every_log argument to
    ZstdCompressionParameters has been removed. The ldm_hash_rate_log
    argument should be used instead.
  • The deprecated CompressionParameters type alias to
    ZstdCompressionParamaters has been removed. Use
    ZstdCompressionParameters.
  • The deprecated aliases ZstdCompressor.read_from() and
    ZstdDecompressor.read_from() have been removed. Use the corresponding
    read_to_iter() methods instead.
  • The deprecated aliases ZstdCompressor.write_to() and
    ZstdDecompressor.write_to() have been removed. Use the corresponding
    stream_writer() methods instead.
  • ZstdCompressor.copy_stream(), ZstdCompressorIterator.__next__(),
    and ZstdDecompressor.copy_stream() now raise the original exception
    on error calling the source stream's read() instead of raising
    ZstdError. This only affects the C backend.
  • ZstdDecompressionObj.flush() now returns bytes instead of
    None. This makes it behave more similarly to flush() methods
    for similar types in the Python standard library. (#78)
  • ZstdCompressionWriter.__exit__() now always calls close().
    Previously, close() would not be called if the context manager
    raised an exception. The old behavior was inconsistent with other
    stream types in this package and with the behavior of Python's
    standard library IO types. (#86)
  • Distribution metadata no longer lists cffi as an install_requires
    except when running on PyPy. Instead, cffi is listed as an
    extras_require.
  • ZstdCompressor.stream_reader() and ZstdDecompressor.stream_reader()
    now default to closing the source stream when the instance is itself
    closed. To change this behavior, pass closefd=False. (#76)
  • The CFFI backend now defines
    ZstdCompressor.multi_compress_to_buffer() and
    ZstdDecompressor.multi_decompress_to_buffer(). However, they
    raise NotImplementedError, as they are not yet implemented.
  • The CFFI backend now exposes the types ZstdCompressionChunker,
    ZstdCompressionObj, ZstdCompressionReader,
    ZstdCompressionWriter, ZstdDecompressionObj,
    ZstdDecompressionReader, and ZstdDecompressionWriter as
    symbols on the zstandard module.
  • The CFFI backend now exposes the types BufferSegment,
    BufferSegments, BufferWithSegments, and
    BufferWithSegmentsCollection. However, they are not implemented.
  • ZstdCompressionWriter.flush() now calls flush() on the inner stream
    if such a method exists. However, when close() itself calls
    self.flush(), flush() is not called on the inner stream.
  • ZstdDecompressionWriter.close() no longer calls flush() on
    the inner stream. However, ZstdDecompressionWriter.flush() still
    calls flush() on the inner stream.
  • ZstdCompressor.stream_writer() and ZstdDecompressor.stream_writer()
    now have their write_return_read argument default to True.
    This brings the behavior of write() in compliance with the
    io.RawIOBase interface by default. The argument may be removed
    in a future release.
  • ZstdCompressionParameters no longer exposes a compression_strategy
    property. Its constructor no longer accepts a compression_strategy
    argument. Use the strategy property/argument instead.

Bug Fixes

  • Fix a memory leak in stream_reader decompressor when reader is closed
    before reading everything. (Patch by Pierre Fersing.)
  • The C backend now properly checks for errors after calling IO methods
    on inner streams in various methods. ZstdCompressionWriter.write()
    now catches exceptions when calling the inner stream's write().
    ZstdCompressionWriter.flush() on inner stream's write().
    ZstdCompressor.copy_stream() on dest stream's write().
    ZstdDecompressionWriter.write() on inner stream's write().
    ZstdDecompressor.copy_stream() on dest stream's write(). (#102)

Changes

  • Bundled zstandard library upgraded from 1.4.5 to 1.4.8.
  • The bundled zstandard library is now using the single C source file
    distribution. The 2 main header files are still present, as these are
    needed by CFFI to generate the CFFI bindings.
  • PyBuffer instances are no longer checked to be C contiguous and
    have a single dimension. The former was redundant with what
    PyArg_ParseTuple() already did and the latter is not necessary
    in practice because very few extension modules create buffers with
    more than 1 dimension. (#124)
  • Added Python typing stub file for the zstandard module. (#120)
  • The make_cffi.py script should now respect the CC environment
    variable for locating the compiler. (#103)
  • CI now properly uses the cffi backend when running all tests.
  • train_dictionary() has been rewritten to use the fastcover APIs
    and to consistently call ZDICT_optimizeTrainFromBuffer_fastCover()
    instead of different C APIs depending on what arguments were passed.
    The function also now accepts arguments f, split_point, and
    accel, which are parameters unique to fastcover.
  • CI now tests and builds wheels for Python 3.9.
  • zstd.c file renamed to c-ext/backend_c.c.
  • All built/installed Python modules are now in the zstandard
    package. Previously, there were modules in other packages. (#115)
  • C source code is now automatically formatted with clang-format.
  • ZstdCompressor.stream_writer(), ZstdCompressor.stream_reader(),
    ZstdDecompressor.stream_writer(), and
    ZstdDecompressor.stream_reader() now accept a closefd
    argument to control whether the underlying stream should be closed
    when the ZstdCompressionWriter, ZstdCompressReader,
    ZstdDecompressionWriter, or ZstdDecompressionReader is closed.
    (#76)
  • There is now a zstandard.open() function for returning a file
    object with zstd (de)compression. (#64)
  • The zstandard module now exposes a backend_features
    attribute containing a set of strings denoting optional features
    present in that backend. This can be used to sniff feature support
    by performing a string lookup instead of sniffing for API presence
    or behavior.
  • Python docstrings have been moved from the C backend to the CFFI
    backend. Sphinx docs have been updated to generate API documentation
    via the CFFI backend. Documentation for Python APIs is now fully
    defined via Python docstrings instead of spread across Sphinx ReST
    files and source code.
  • ZstdCompressionParameters now exposes a strategy property.
  • There are now compress() and decompress() convenience functions
    on the zstandard module. These are simply wrappers around the
    corresponding APIs on ZstdCompressor and ZstdDecompressor.

Don't miss a new python-zstandard release

NewReleases is sending notifications on new releases.