What's Changed
- fix(compute): detect temporal rounding overflow by @fallintoplace in #1127
- chore: Bump github.com/klauspost/compress from 1.19.2 to 1.20.0 by @dependabot[bot] in #1294
- chore: Bump modernc.org/sqlite from 1.57.0 to 1.58.0 by @dependabot[bot] in #1295
- fix(parquet/variant): prevent metadata offset cursor overflow by @matanper in #1290
- fix(arrow/extensions): avoid child values for missing variant lists by @matanper in #1293
- fix(parquet/utils): make unpack32 NEON profiler-safe by @zeroshade in #1300
- ci: run parquet/file alone on macOS instead of serializing all packages by @zeroshade in #1301
- docs(compute): state which function kinds the registry implements by @singhpratech in #1303
- docs(arrow/memory): document allocator alignment and which allocator to use when C retains buffers by @singhpratech in #1302
- chore: Bump github.com/andybalholm/brotli from 1.2.3 to 1.2.4 by @dependabot[bot] in #1318
- chore: Bump github.com/twmb/avro from 1.8.0 to 1.9.0 by @dependabot[bot] in #1316
- perf(internal/hashing): avoid boxing byte slices in InsertOrGet by @fallintoplace in #1310
- perf(parquet): use typed memo insertion for fixed-width dictionaries by @fallintoplace in #1312
- fix(arrow): support zero-length fixed-size lists by @jakthom in #1308
- fix(arrow/decimal): widen Decimal64 rescale to 64 bits by @winklemad in #1307
- perf(parquet): decode nullable plain values into valid runs by @fallintoplace in #1314
- ci: use clang ASan runtime for -asan tests by @zeroshade in #1299
- perf(arrow/array): append dictionary indices directly by @fallintoplace in #1313
- chore: require Go 1.26 by @zeroshade in #1326
- ci: shard the benchmark workflow across parallel jobs by @zeroshade in #958
- chore: Bump golang.org/x/sys from 0.47.0 to 0.48.0 by @dependabot[bot] in #1319
- perf(parquet): reuse DELTA decoder scratch by @fallintoplace in #1322
- perf(parquet): encode dictionary indices in RLE batches by @fallintoplace in #1328
- chore: Bump golang.org/x/sync from 0.22.0 to 0.23.0 by @dependabot[bot] in #1317
- test(flightsql): isolate fixture setup timeout by @zeroshade in #1332
- docs(compute): attach the NewDatumWithoutOwning doc comment and say what Release does to it by @singhpratech in #1320
- chore: Bump modernc.org/sqlite from 1.58.0 to 1.59.0 by @dependabot[bot] in #1333
- chore: Bump google.golang.org/grpc from 1.83.2 to 1.84.0 by @dependabot[bot] in #1334
- perf(compute): vectorize mixed 64-bit filters by @fallintoplace in #1311
- chore: Bump github.com/pierrec/lz4/v4 from 4.1.29 to 4.1.30 by @dependabot[bot] in #1335
- perf(arrow/array): copy view buffers directly by @fallintoplace in #1324
- feat(arrow/compute): scalar aggregate framework with the count and sum kernels by @singhpratech in #1336
- perf(parquet): reuse DELTA_BYTE_ARRAY discard storage by @fallintoplace in #1321
- chore: Bump github.com/klauspost/compress from 1.20.0 to 1.20.1 by @dependabot[bot] in #1339
- chore: Bump github.com/andybalholm/brotli from 1.2.4 to 1.2.5 by @dependabot[bot] in #1340
- feat(arrow/ipc): add Reader.ContinueFrom for streams split across blobs by @zeroshade in #1346
- chore: Bump github.com/andybalholm/brotli from 1.2.5 to 1.2.6 by @dependabot[bot] in #1358
- chore: Bump modernc.org/sqlite from 1.59.0 to 1.60.1 by @dependabot[bot] in #1356
- fix(array/record): bldr stop ignore unexpected EOF by @Willem-J-an in #1341
- docs(arrow/compute): say what MergeAll returns for no states and where Ordered takes effect by @singhpratech in #1338
- chore: Bump github.com/pierrec/lz4/v4 from 4.1.30 to 4.1.32 by @dependabot[bot] in #1354
- chore: Bump github.com/apache/thrift from 0.24.0 to 0.25.0 by @dependabot[bot] in #1355
- fix(parquet/variant): return "" for an empty long-form string by @CaptainAni187 in #1353
- perf(compute): use typed memo insertion for binary hashes by @fallintoplace in #1315
- perf(arrow/array): concatenate list offsets directly by @fallintoplace in #1323
- perf(arrow/array): compare approximate list values by valid runs by @fallintoplace in #1330
- perf(arrow/array): compare fixed-size binary values in bulk by @fallintoplace in #1347
- perf(compute): vectorize mixed 16-bit filters by @fallintoplace in #1349
- perf(arrow/compute): avoid getter wrappers in chunked binary take by @fallintoplace in #1359
- perf: zstd implement Pooling + DecodeAll with no new reader. by @samueleresca in #1331
- fix: expose internal offsets in json/csv readers by @Willem-J-an in #1342
- ci: fix MinGW CGO job under Git Bash's UCRT64 MSYSTEM by @zeroshade in #1369
- fix(compute/exec): do not carry a cached null count across ArraySpan.SetSlice by @singhpratech in #1306
- perf(compute): filter dictionary indices directly by @fallintoplace in #1352
- fix(arrow/ipc): open mapped files with dictionary fields by @hpaliwal8 in #1365
- fix(parquet/metadata): count non-null values from the validity bitmap in UpdateSpaced by @CaptainAni187 in #1367
- perf(parquet): avoid scanning plain byte-array offsets by @fallintoplace in #1362
- perf(arrow): reduce metadata sorting overhead by @fallintoplace in #1361
- perf(arrow/compute): read flat take values directly by @fallintoplace in #1360
- fix(arrow/compute): allow casting nullable struct fields to non-nullable when no nulls by @sankalpsthakur in #1363
- chore: Bump actions/download-artifact from 8.0.1 to 8.0.2 by @dependabot[bot] in #1375
- chore: Bump actions/upload-artifact from 7.0.1 to 7.0.2 by @dependabot[bot] in #1374
- chore: Bump golang.org/x/sys from 0.48.0 to 0.49.0 by @dependabot[bot] in #1373
- chore: Bump golang.org/x/sync from 0.23.0 to 0.24.0 by @dependabot[bot] in #1372
- chore: Bump github.com/pierrec/lz4/v4 from 4.1.32 to 4.1.33 by @dependabot[bot] in #1370
- chore: Bump github.com/pterm/pterm from 0.12.83 to 0.12.84 by @dependabot[bot] in #1371
- perf(arrow/array): compare optional validity bitmaps in bulk by @fallintoplace in #1327
- perf(compute): reuse amd64 SIMD comparisons for numeric-to-bool casts by @fallintoplace in #1351
- perf(compute): add AVX2 take-index compaction by @fallintoplace in #1348
- perf(compute): vectorize mixed 8-bit filters by @fallintoplace in #1350
- test(arrow/flight/flightsql): add client tests for Substrait prepared statements by @sankalpsthakur in #1383
- feat(arrow/ipc): add WithZstdLevel option for IPC writer by @Asthenia0412 in #1376
- chore: Bump github.com/klauspost/compress from 1.20.1 to 1.20.3 by @dependabot[bot] in #1386
- fix(arrow/compute): respect GOMAXPROCS for default NumParallel by @samueleresca in #1384
- chore: Bump github.com/klauspost/cpuid/v2 from 2.4.0 to 2.4.1 by @dependabot[bot] in #1387
- fix(parquet/metadata): stop bloom filter asm clobbering the frame pointer by @zeroshade in #1379
- fix(arrow): handle dense union slices and empty validation by @skyon0522-ai in #1385
- perf(parquet): speed up delta byte array prefix matching by @fallintoplace in #1388
- perf(parquet/metadata): pool bloom filter hash scratch buffers by @zeroshade in #1378
- fix(arrow/bitutil): preserve overlapping bitmap transfers by @fallintoplace in #1389
- perf(parquet): copy DELTA_LENGTH Arrow payloads by valid runs by @fallintoplace in #1392
- fix(parquet/internal/encoding): reset delta bit pack encoder state after flush by @zeroshade in #1400
- ci: pin RC Verify Linux job to ubuntu-24.04 by @zeroshade in #1401
- fix(parquet/internal/encoding): do not leak delta bit width buffers from decoders by @zeroshade in #1394
- perf(arrow/csv): stream full-file CSV reads into builders by @fallintoplace in #1393
- fix(parquet/file): read encrypted page headers written by parquet-mr by @sankalpsthakur in #1390
- perf(arrow/compute): unify changing dictionaries incrementally by @fallintoplace in #1395
- perf(parquet): fast-path constant delta bitpacked blocks by @fallintoplace in #1396
- perf(bitutils): use four-word fast path for unaligned bitmaps by @fallintoplace in #1398
- perf(compute): radix-sort null-free integer indices by @fallintoplace in #1391
- perf(bitutil): count offset bits by byte mask by @fallintoplace in #1397
- fix(parquet/pqarrow): honor offset and null slots when writing fixed-size lists by @zeroshade in #1382
- perf(arrow/array): reuse dictionary mappings and handle empty chunks in Concatenate by @fallintoplace in #1403
- perf(parquet): expand nullable boolean bitmaps by byte by @fallintoplace in #1405
- fix(arrow/compute): do not return a result from a cancelled call by @zeroshade in #1410
- fix(parquet): return an error instead of panicking on a hidden encrypted column by @ump45nose in #1406
- ci: key workflow concurrency groups by pull request number by @zeroshade in #1411
- test(arrow/array): make run-end-encoded concat tests valid on 32-bit platforms by @zeroshade in #1408
- fix(arrow/decimal): handle native word widths in big integer conversions by @zeroshade in #1409
- fix(arrow/array): validate bufferless run-end encoded arrays by @zeroshade in #1421
- perf(parquet): decode narrow packed levels directly by @fallintoplace in #1404
- feat(arrow/flight/flightsql/driver): accept grpc+tcp, grpc+tls and grpc+unix schemes by @nirajan1111 in #1420
- ci: run parquet/file once on macOS by @zeroshade in #1422
- perf(bitutil): accelerate misaligned bitmap equality by @fallintoplace in #1414
- perf(parquet): batch DELTA_LENGTH byte-array decoding and discard by @fallintoplace in #1415
- chore: Bump github.com/andybalholm/brotli from 1.2.6 to 1.2.7 by @dependabot[bot] in #1424
- chore: Bump modernc.org/sqlite from 1.60.1 to 1.61.0 by @dependabot[bot] in #1425
- chore: Bump docker/login-action from 4.6.0 to 4.7.0 by @dependabot[bot] in #1426
- ci: run benchmark metadata steps on Python 3.12 by @zeroshade in #1427
- perf(bitutil): pack boolean bitmap writes by byte by @fallintoplace in #1402
- fix(parquet): preserve CTR algorithm in plaintext footers by @autoCAT6 in #1412
- perf(parquet): write packed words directly into memory buffers by @fallintoplace in #1418
- perf(parquet): decode RLE booleans directly into bool slices by @fallintoplace in #1417
- chore: Bump actions/setup-python from 7.0.0 to 7.1.0 by @dependabot[bot] in #1423
- perf(arrow/array): write bulk binary offsets and payloads directly by @fallintoplace in #1413
- perf(parquet): pack narrow RLE literal levels eight at a time by @fallintoplace in #1419
- perf(bitutil): bulk-transfer unaligned bitmap ranges by @fallintoplace in #1416
- test: make JSON tests independent of the JSON backend's error details by @zeroshade in #1428
- chore: Bump github.com/goccy/go-json from 0.10.6 to 0.11.2 by @dependabot[bot] in #1357
- fix(arrow/compute): release sort scratch buffers by @zeroshade in #1431
- feat(arrow/compute): add IEEE 754 total-order float sorting option by @zeroshade in #1430
- fix(arrow/flight/flightsql): send schema for empty ingest by @arnoldwakim in #1433
- fix(arrow/compute): correct amd64 kernel assembly ABI by @zeroshade in #1432
- perf(arrow/ipc): reuse zstd encoders across writers by @vdpoora in #1429
- fix(parquet/pqarrow): release partial readers when building a field reader fails by @elsteveogrande in #1435
- test(parquet): cover plaintext-footer CTR interoperability by @autoCAT6 in #1438
- perf(bitutil): process unaligned bitmap operations by word by @fallintoplace in #1437
- fix(arrow): compare LargeListType like ListType in TypeEqual by @paveon in #1440
- chore: Bump github.com/goccy/go-json from 0.11.2 to 0.12.0 by @dependabot[bot] in #1441
- chore: Bump google.golang.org/grpc from 1.84.0 to 1.85.0 by @dependabot[bot] in #1442
- test(parquet): pyarrow encryption interop tests by @EdwardShen125 in #1439
- fix(arrow/compute): honor the input offset in is_not_null by @hcrosse in #1456
- fix(arrow/compute): honor the input offset in not by @hcrosse in #1458
- perf(arrow/array): reuse unchanged dictionaries across builder arrays by @hcrosse in #1444
- perf(parquet/pqarrow): build struct child readers serially when Parallel is false by @elsteveogrande in #1434
- fix(arrow/compute): register is_null, is_not_null and is_nan as unary functions by @hcrosse in #1457
- fix(arrow/compute): treat null dictionary values as null in is_null and is_not_null by @hcrosse in #1461
- feat(arrow/compute): add starts_with kernel by @hcrosse in #1448
- fix(arrow/compute/exprs): accept string and binary dictionary columns in field references by @hcrosse in #1449
- fix(arrow/compute): handle null, union and run-end encoded arrays in is_null and is_not_null by @hcrosse in #1465
- fix(arrow/compute/exprs): avoid leaking field ref on type mismatch by @hcrosse in #1466
- feat(arrow/compute): support string_view and binary_view in starts_with by @hcrosse in #1467
- fix(arrow/compute): fix NewDatumWithoutOwning ownership semantics by @Asthenia0412 in #1377
- ci: move macOS jobs from retired macos-14 runners to macos-15 by @zeroshade in #1472
- perf(parquet/file): use dense decoding for batches without nulls by @hcrosse in #1447
- fix(arrow/scalar): give null dictionary scalars an empty dictionary by @hcrosse in #1464
- fix(arrow/compute/exprs): retain nested field ref results by @hcrosse in #1470
- feat(arrow/compute): support IgnoreCase in starts_with by @hcrosse in #1468
- perf(parquet/internal/encoding): preallocate dictionaries bounded by page data by @zeroshade in #1473
- fix(arrow/compute/exprs): fix offsets, nulls and ownership of nested field references by @zeroshade in #1474
- fix(arrow/compute): honor NullOptions.NanIsNull in is_null by @hcrosse in #1488
- docs: explain test data submodules and env vars for running tests by @AynSiddiqui in #1477
- fix(arrow/scalar): build empty arrays from dictionary scalars of any value type by @hcrosse in #1486
- fix(arrow/array): validate extension arrays by their storage type by @hcrosse in #1487
- fix(arrow/compute): release null scalars and partial exec batches in exprs by @hcrosse in #1485
- chore: Bump golang.org/x/sync from 0.24.0 to 0.25.0 by @dependabot[bot] in #1491
- chore: Bump github.com/klauspost/cpuid/v2 from 2.4.1 to 2.4.2 by @dependabot[bot] in #1495
- chore: Bump docker/login-action from 4.7.0 to 4.8.0 by @dependabot[bot] in #1496
- chore: Bump github.com/andybalholm/brotli from 1.2.7 to 1.2.8 by @dependabot[bot] in #1492
- chore: Bump modernc.org/sqlite from 1.61.0 to 1.61.1 by @dependabot[bot] in #1494
- chore: Bump github.com/tidwall/sjson from 1.2.5 to 1.2.7 by @dependabot[bot] in #1493
- fix(arrow/csv): preserve braces in list values by @peterolkhov in #1499
- fix(arrow/compute): use int16 kernels for casts to int16 by @hcrosse in #1503
- feat(arrow/scalar): add Decimal32 and Decimal64 scalars by @hcrosse in #1501
- feat(arrow/scalar): add StringView and BinaryView scalars by @hcrosse in #1490
- fix(parquet): release writer buffers when a column chunk flush fails by @twuebi in #1506
- fix(arrow/decimal): correct rescaling results and report cast errors by @hcrosse in #1513
- fix(arrow/decimal): reject infinite decimal strings and bound extreme exponents by @hcrosse in #1514
- feat(parquet/file): Write geospatial parquet row group statistics for GEOMETRY column chunks by @C-Loftus in #1504
- fix(arrow/csv): preserve lists holding a single empty string by @zeroshade in #1505
- fix(arrow/decimal): fail clearly for Decimal128 and Decimal256 scales past MaxScale by @hcrosse in #1515
- perf(arrow/decimal): return MaxPrecision as a constant per width by @hcrosse in #1520
- fix(arrow/scalar): use AppendNull when appending null scalars by @hcrosse in #1528
- fix(parquet/metadata): count empty strings in byte array statistics from Arrow by @hcrosse in #1527
- fix(parquet/compress): read to the end of a Brotli stream that fills dst by @hcrosse in #1530
- fix(arrow/csv): keep list items equal to the null value distinct from nulls by @zeroshade in #1508
- fix(arrow/cdata): export extension arrays as their storage by @hcrosse in #1526
- feat(parquet/encryption): KMS support for reading PKMT1 key material by @EdwardShen125 in #1478
- fix(parquet/file): write
null_countstatistics for GEOMETRY and GEOGRAPHY column chunks by @C-Loftus in #1533 - fix(parquet/internal/encoding): clear buffered RLE booleans after each flush by @hcrosse in #1539
- fix(arrow/array): keep the storage dictionary in NewExtensionArrayWithStorage by @hcrosse in #1540
- fix(arrow/scalar): validate missing binary values and null REE scalars by @hcrosse in #1541
- fix(parquet/internal/encoding): clear buffered RLE booleans on Reset by @hcrosse in #1545
- fix(arrow/cdata): export an empty view sizes buffer as the zero region by @hcrosse in #1544
- fix(parquet/file): count V2 page num_nulls from the definition levels by @hcrosse in #1543
- fix(parquet/internal/encoding): clear encoder state on Reset by @hcrosse in #1546
- fix(parquet/file): return an error for dictionary encoded boolean columns by @hcrosse in #1547
- feat(parquet): add ALP encoding for FLOAT and DOUBLE columns by @prtkgaur in #1289
- chore: bump version number by @zeroshade in #1551
New Contributors
- @matanper made their first contribution in #1290
- @singhpratech made their first contribution in #1303
- @jakthom made their first contribution in #1308
- @CaptainAni187 made their first contribution in #1353
- @samueleresca made their first contribution in #1331
- @hpaliwal8 made their first contribution in #1365
- @sankalpsthakur made their first contribution in #1363
- @Asthenia0412 made their first contribution in #1376
- @skyon0522-ai made their first contribution in #1385
- @nirajan1111 made their first contribution in #1420
- @autoCAT6 made their first contribution in #1412
- @vdpoora made their first contribution in #1429
- @elsteveogrande made their first contribution in #1435
- @EdwardShen125 made their first contribution in #1439
- @AynSiddiqui made their first contribution in #1477
- @peterolkhov made their first contribution in #1499
- @prtkgaur made their first contribution in #1289
Full Changelog: v18.8.0...v18.9.0