github apache/arrow-rs 57.0.0
arrow 57.0.0
on GitHub

one day ago

Changelog

57.0.0 (2025-10-19)

Full Changelog

Breaking changes:

Implemented enhancements:

  • Added Avro support (new arrow-avro crate) #4886
  • parquet-rewrite: supports compression level and write batch size #8639
  • Error not panic when int96 stastistics aren't size 12 #8614 [parquet]
  • [Variant] Make VariantArray iterable #8612
  • [Variant] impl PartialEq for VariantArray #8610
  • [Variant] Remove potential panics when probing VariantArray #8609
  • [Variant] Remove ceremony of going from list of Variant to VariantArray #8606
  • Eliminate redundant validation in RecordBatch::project #8591 [arrow]
  • [PARQUET][BENCH] Arrow writer bench with compression and/or page v2 #8559 [parquet]
  • [Variant] casting functions are confusingly named #8531 [parquet]
  • Support writing GeospatialStatistics in Parquet writer #8523 [parquet]
  • [thrift-remodel] Optimize convert_row_groups #8517 [parquet]
  • [Variant] Add variant to arrow primitive support for boolean/timestamp/time #8515
  • Test thrift-remodel branch with DataFusion #8513 [parquet]
  • Make UnionArray::is_dense Method Public #8503 [arrow]
  • Add append_n method to FixedSizeBinaryDictionaryBuilder #8497 [arrow]
  • [Parquet] Reduce size of ParquetMetadata when encryption feature is enabled #8469 [parquet]
  • [Parquet] Remove useless mut requirements in geting bloom filter function #8461 [parquet]
  • Change serde dependency to serde_core where applicable #8451 [arrow]
  • [Parquet] Split ParquetMetadataReader into IO/decoder state machine and thrift parsing #8439 [parquet]
  • Remove compiler warning for redundant config enablement #8412 [arrow]
  • Add geospatial statistics creation support for GEOMETRY/GEOGRAPHY Parquet logical types #8411 [arrow]
  • arrow_json lacks with_timestamp_format functions like arrow_csv had offered #8398 [arrow]
  • Unify API for writing column chunks / row groups in parallel #8389 [parquet]
  • Reuse zstd context in arrow IPC writer #8386 [arrow] [arrow-flight]
  • [Variant] Support reading/writing Parquet Variant LogicalType #8370 [parquet]
  • [Variant] Implement a shred_variant function #8361
  • [Parquet] Expose ReadPlan and ReadPlanBuilder #8347 [parquet]
  • [Variant] [Shredding] Support typed_access for List #8337 [parquet]
  • [Variant] [Shredding] Support typed_access for Struct #8336 [parquet]
  • [Variant] [Shredding] Support typed_access for Time64(Microsecond) #8334 [parquet]
  • [Variant] [Shredding] Support typed_access for Decimal128 #8332 [parquet]
  • [Variant] [Shredding] Support typed_access for Timestamp(Microsecond, _) and Timestamp(Nanosecond, _) #8331 [parquet]
  • [Variant] [Shredding] Support typed_access for Date32 #8330 [parquet]
  • [Variant] Support strict casting for all data types #8303
  • [Variant] Support typed access for string types in variant_get #8285
  • [Variant]: Implement DataType::FixedSizeList support for cast_to_variant kernel #8281

Fixed bugs:

  • Fix arrow-avro Writer Documentation related to AvroBinaryFormat #8631 [arrow]
  • Decimal -> Decimal cast wrongly fails for large scale reduction #8579 [arrow]
  • [Parquet] Avoid fetching multiple pages when max_predicate_cache_sizeis 0 #8542 [parquet]
  • DataType parsing no longer works correctly for old formatted timestamps #8539 [parquet] [arrow]
  • [Parquet] ArrowWriter flush does not work #8534 [parquet]
  • arrow::compute::interleave fails with struct arrays with no fields #8533 [arrow]
  • [Parquet] Over memory consumation for writer page v1 compressed #8526 [parquet]
  • Incorrect Behavior of Collecting a filtered iterator to a BooleanArray #8505 [arrow]
  • [Parquet] ProjectionMask::columns name handling is bug prone #8443 [parquet]
  • [Variant] Shredded typed_value columns must have valid variant types #8435 [parquet]
  • cargo test -p parquet fails with default ulimit #8406 [parquet]
  • Column with List(Struct) causes failed to decode level data for struct array #8404 [parquet]
  • Binaryview Utf8 Cast Issue #8403 [arrow]
  • Decimal precision validation displays value without accounting for scale #8382 [arrow]
  • [Variant] VariantArray::data_type returns StructType, causing Array::as_struct to panic #8319 [parquet]
  • [Variant] writing a VariantArray to parquet panics #8296 [parquet]

Documentation updates:

Performance improvements:

  • [parquet] Improve encoding mask API (wrap bare i32 in a struct w/ docs) #8588 [parquet]
  • bench: create zip kernel benchmarks #8654 [arrow] (rluvaton)
  • Skip redundant validation checks in RecordBatch#project #8583 [arrow] (pepijnve)
  • [thrift-remodel] Remove conversion functions for row group and column metadata #8574 [parquet] (etseidl)
  • [PARQUET] Improve memory efficency for compressed writer parquet 1.0 #8527 [parquet] (lilianm)
  • perf: improve GenericByteBuilder::append_array to use SIMD for extending the offsets #8388 [arrow] (rluvaton)

Closed issues:

  • Utf-8, LargeUtf8, Utf8View #8601
  • [Variant] Improve the get type logic for DataType in variant to arrow row builder #8538
  • Add a README.md for arrow-avro #8504 [arrow]
  • Fix UnionArray references to "positive" values #8418 [arrow]
  • [Variant] metadata field should be marked is non-nullable #8410 [parquet]
  • [Avro] Example read_with_utf8view.rs fails to run with error "Error: ParseError("Unexpected EOF while reading Avro header")" #8380 [arrow]
  • [Geospatial]: Add CI checks for parquet-geospatial crate #8377
  • [Geospatial] Create new parquet-geometry crate #8374

Merged pull requests:

* This Changelog was automatically generated by github_changelog_generator

Check out latest releases or
releases around apache/arrow-rs 57.0.0

Don't miss a new arrow-rs release

NewReleases is sending notifications on new releases.

Get notifications