Changelog

59.1.0 (2026-07-03)

Full Changelog

Implemented enhancements:

• Fast path for nested DictionaryArray casting #10247 [arrow]
• parquet/arrow: reading multiple nested columns fails with "Not all children array length are the same!" when a list continues across DataPageV2 page boundary #10243 [parquet]
• Add product aggregate kernel to arrow-rs #10150 [arrow]
• Stricter DataType parsing #10146 [arrow]
• Support validating CSV headers against Schema #10143 [arrow]
• arrow-ipc: Supports compression level configuration for arrow-ipc writer #10132 [arrow]
• [Variant] VariantArray field API naming #10093
• Add StructArray::field_ APIs symmetric to StructArray::column_ ones #10092 [arrow]
• arrow-buffer: implement Saturating, CheckedShl, Not num-traits for i256 #10087 [arrow]
• feat: native concat for MapArray #10047 [arrow]
• [Variant] Add variant_to_arrow Dictionary/REE type support #10013

Fixed bugs:

• arrow-row on fixed size binary/list with size 0 and no nulls return wrong length #10270
• casting list to 0-size fixedsizelist can cause incorrect output length #10227 [arrow]
• Buffer count mismatched with metadata when encoding records with dictionary of dictionaries #10213 [arrow]
• Buffer::into_mutable is not consistent regarding sliced data and can lead to panics #10117 [arrow]
• parquet_derive: cannot read or write columns whose name is a Rust keyword (raw identifiers like r#type become column "r#type") #10112
• parquet: fix OffsetBuffer panic on corrupt input #10107 [parquet]
• Parquet geospatial conversion uses metadata key "algorithm" instead of "edges" in geoarrow metadata #9929 [parquet]

Documentation updates:

• doc: More comments to concat_batches #10178 (2010YOUY01)
• Minor: improve PageStore docs with a temp-file spilling example #10074 [parquet] (alamb)

Performance improvements:

• perf: interleave_list for List<Primitive> could be optimized? #10022 [arrow]
• perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #9990 [arrow]
• Replace conversion of binary->string in arrow-row from arraydata to direct construction #10261 [arrow] (Jefffrey)
• [arrow-flight encode path]re-use flatbufferbuilder #10220 [arrow] (Rich-T-kid)
• [10125] arrow-flight decode path optimizations (add skip_validation to arrow-flight) #10206 [arrow] [arrow-flight] (Rich-T-kid)
• Improve performance of concat_elements ByteViewArray concatenation #10161 [arrow] (pepijnve)
• [arrow-flight] Optimize flight, remove some allocations, add dictionary focused benchmarks #10126 [arrow] [arrow-flight] (Rich-T-kid)
• optimize(concat): concat map implementation #10048 [arrow] (mapleFU)
• Reduce copies in Arrow IPC writer #10044 [arrow] (Rich-T-kid)
• perf(interleave): Optimize list interleave_list when child is primitive #10025 [arrow] (mapleFU)

Closed issues:

• Soundness: Unsound alignment contract in public FromBytes trait and BitReader::get_batch #10164 [parquet]
• ParquetPushDecoder: expose the next row-group index that try_next_reader will yield #10148 [parquet]
• arrow-ipc: Extend writer benchmarks to include dictionaries #10119 [arrow]
• bench(parquet): benchmark for nested list write #10083 [parquet]
• Support i256 implement From<i128> #10080 [arrow]

Merged pull requests:

• fix(arrow-row): allow to convert non empty fixed size binary/list array with size length 0 and no nulls #10271 [arrow] (rluvaton)
• chore: Fix audit CI run by ignore quick-xml audit advisories #10267 (alamb)
• fix main: parquet test compilation failure #10266 [parquet] (Jefffrey)
• minor: drive-by refactors for dicts in substring & filter #10264 [arrow] (Jefffrey)
• Add validated row decode benchmark #10259 [arrow] (alamb)
• arrow-cast: Add optimized path for unnesting a dict #10248 [arrow] (brancz)
• feat: support uuid from fixed type of length 16 #10241 [arrow] (ariel-miculas)
• chore(deps): bump actions/cache from 6.0.0 to 6.1.0 #10240 (dependabot[bot])
• fix: Rename parquet feature flag 'flate2-rust_backened' to 'flate2-rust_backend' #10239 [parquet] (dannycjones)
• chore: Make clippy::question_mark happy #10231 (Tpt)
• fix(ipc): reject dictionary-encoded dictionary values #10230 [arrow] (goutamadwant)
• Replace ArrayData with direct Array construction in arrow-row #10229 [arrow] (Jefffrey)
• fix: casting list to fixedsizelist didn't respect input length #10228 [arrow] (Jefffrey)
• chore: Fix clippy::byte_char_slices (use byte strings instead of explicit arrays) #10225 [parquet] [arrow] (Tpt)
• nit: arrow-pyarrow: Use string interning #10224 (Tpt)
• Support concatenation of mixed FixedSizeBinary via concat_elements_dyn #10222 [arrow] (pepijnve)
• rename Compression struct #10221 [arrow] [arrow-flight] (Rich-T-kid)
• chore(deps): bump the all-other-cargo-deps group across 1 directory with 16 updates #10218 (dependabot[bot])
• chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0 #10210 (dependabot[bot])
• [10125] Introduce mult-batch decode benchmarks #10207 [arrow] [arrow-flight] (Rich-T-kid)
• chore(deps): bump actions/cache from 5.0.5 to 6.0.0 #10203 (dependabot[bot])
• introduce decode benchmarks #10202 [arrow] [arrow-flight] (Rich-T-kid)
• Fix merge_kernels benchmark panic due to not wrapping with Scalar #10199 [arrow] (Jefffrey)
• Benchmarks and performance improvement for parquet boolean reader #10196 [parquet] (jhorstmann)
• add stale PR workflow #10194 (Jefffrey)
• chore: group minor/patch dependabot updates #10193 (Jefffrey)
• chore(deps): bump http from 1.4.0 to 1.4.2 #10191 (dependabot[bot])
• chore(deps): bump syn from 2.0.117 to 2.0.118 #10190 (dependabot[bot])
• chore(deps): bump chrono from 0.4.44 to 0.4.45 #10188 (dependabot[bot])
• chore(deps): bump uuid from 1.23.1 to 1.23.3 #10186 (dependabot[bot])
• chore: run cargo update to bump quinn #10181 (Jefffrey)
• test: cover signed integers and bool in BitReader::get_batch test #10180 [parquet] (alamb)
• [arrow-select] perf: Replace ArrayData with direct Array construction in take kernels #10176 [arrow] (liamzwbao)
• Return PyValueError for nullable PyArrow struct imports #10174 (fallintoplace)
• Fix Variant time microsecond JSON formatting #10173 (fallintoplace)
• Split traits for plain and bitpacked decoding and fix soundness issue in BitReader::get_batch #10172 [parquet] (jhorstmann)
• fix: switch generic usages of i128 to IntervalMonthDayNano for MonthDayNano type #10171 [arrow] (Jefffrey)
• chore: specify --locked when cargo installing cargo-audit #10170 (Jefffrey)
• chore: Fix clippy::useless_borrows_in_formatting #10163 [parquet] [arrow] (Tpt)
• fix(arrow-cast): respect cast safety for overflowing temporal casts #10162 [arrow] (SAY-5)
• chore(deps): bump actions/checkout from 6 to 7 #10159 (dependabot[bot])
• feat(parquet): add ParquetPushDecoder::peek_next_row_group() #10158 [parquet] (zhuqi-lucas)
• feat(pyarrow) FromPyArrow on Vec<T>: allow any iterable for input #10155 (Tpt)
• nit: pyarrow: simplify class validation error creation #10154 (Tpt)
• [Variant] add doc reference to VariantArrayBuilder #10152 (sdf-jkl)
• feat: Adds product aggregate compute kernel #10151 [arrow] (devanbenz)
• Stricter datatype parsing for decimals, fixedsizelists and time32/64 #10147 [arrow] (Jefffrey)
• feat(arrow_csv): add header validation option #10144 [arrow] (XiNiHa)
• [Parquet] route dictionary page through the PageStore #10142 [parquet] (liamzwbao)
• chore: update pyo3 dependency to 0.29 #10134 (timsaucer)
• feat(ipc): Supports compression level configuration #10133 [arrow] (wForget)
• fix: write error for dbg output of out of range timestamps #10130 [arrow] (Jefffrey)
• [Variant] VariantArray field API naming #10124 (sdf-jkl)
• feat(arrow_array): add helper function to create MapArray from Vec<Option<Vec<(Key, Option<Value>)>>> for tests #10123 [arrow] [arrow-flight] (rluvaton)
• perf(arrow-ipc): Add writer benchmarks for dictionaries #10122 [arrow] (JakeDern)
• feat: support MapArray in lengths kernel #10121 [arrow] (rluvaton)
• feat: add OffsetBuffer::subtract to allow to shift offsets by value #10120 [arrow] (rluvaton)
• fix: Buffer::into_mutable return error instead of panic for converting owned sliced when not start at 0 and fix returned Mutable length #10118 [arrow] (rluvaton)
• chore: update Bytes visibility to correctly reflect the actual visibility #10115 [arrow] (rluvaton)
• fix(parquet_derive): support raw identifiers as column names #10113 (cbmixx)
• removed clippy ignore statment #10111 [arrow] (Rich-T-kid)
• Add StructArray::field_ APIs symmetric to StructArray::column_ ones #10110 [arrow] (sdf-jkl)
• fix(parquet): return error instead of panicking in pad_nulls on corrupt input #10108 [parquet] (thepenguinco)
• Minor: Add interleave tests for List<Decimal128> and List<Timestamp(tz)> #10099 [arrow] (alamb)
• Add arrow-flight test coverage for IPC compression #10097 [arrow] [arrow-flight] (alamb)
• chore(deps): bump pyspark from 3.3.2 to 3.4.4 in /parquet/pytest #10091 [parquet] (dependabot[bot])
• refactor(parquet): bundle array reader recursion args into ReaderArgs #10089 [parquet] (HippoBaro)
• arrow-buffer: implement Saturating, Checked num-traits for i256 #10088 [arrow] (theirix)
• bench(parquet): add nested list writer benchmarks #10084 [parquet] (mapleFU)
• Implement From<i128> for i256 #10081 [arrow] (AdamGS)
• test(parquet): drop confusing main reference in page-roundtrip test comment #10072 [parquet] (adriangb)
• ci: Split miri tests into 4 parallel shards #10067 (AdamGS)
• Add tests and fix corner cases for Parquet/GeoArrow extension type conversion #10065 [parquet] (paleolimbot)
• Support writing REE arrays directly to Parquet #10064 [parquet] (Rich-T-kid)
• test(arrow-select): additional tests for inline-view filter fast path (tests for #9755) #10054 [arrow] (alamb)
• test(arrow-select): add take_bytes coverage for sliced values and nullable offset overflow #10053 [arrow] (alamb)
• Consolidate filter_null_mask into FilterPredicate::filter_nulls #10049 [arrow] (alamb)
• [Variant] Add VariantBuilder values check #10016 (sdf-jkl)
• [Variant] Preserve UUID extension type metadata for Parquet writer #10015 (sdf-jkl)
• feat(parquet-variant): add Dictionary and REE variant_to_arrow support #10014 (mneetika)
• perf(arrow-ord): Avoid full index materialization for small-limit lexsorts #9991 [arrow] (pchintar)

* This Changelog was automatically generated by github_changelog_generator