What's Changed
- MINOR: Fix error message in SimpleGroup.add() for Binary by @takezoe in #3385
- [minor] remove comment from pom.xml by @dossett in #3382
- Minor:
ci-before_install.shshould show wget errors and retry by @dossett in #3381 - Minor: Upgrade spotless plugin by @dossett in #3380
- [minor] Move on from java8 for three core modules by @dossett in #3379
- MINOR:
japicmpchecks are failing after Java 11 release by @dossett in #3388 - MINOR: Bump version to 1.18.0-SNAPSHOT by @wgtmac in #3389
- GH-3372: Enhance ColumnSizeCommand to sort by size by @wgtmac in #3371
- GH-3399: Hint IDE the source code generated by thrift by @HuaHuaY in #3400
- GH-3387: bump aircompressor to mitigate CVE-2025-67721 by @olegbonar in #3403
- GH-3383: Modernize mockito by @dossett in #3384
- GH-3404: fix CI TestByteBitPacking512VectorLE.unpackValuesUsingVector randomly oom by @HuaHuaY in #3405
- GH-3401: Remove redundant MAP_KEY_VALUE annotation at middle level of logical type map by @HuaHuaY in #3402
- GH-3406: fix Float16 statistics handling for NaN and zero values by @HuaHuaY in #3407
- GH-3215: Sort the list of org.apache.parquet.format.Encoding created … by @mbrannstrom in #3413
- Bump dependabot limit by @Fokko in #3416
- Bump org.mockito:mockito-core from 5.21.0 to 5.22.0 by @dependabot[bot] in #3436
- Bump org.apache:apache from 33 to 37 by @dependabot[bot] in #3431
- Bump protobuf.version from 4.33.2 to 4.34.0 by @dependabot[bot] in #3441
- Bump com.github.luben:zstd-jni from 1.5.7-3 to 1.5.7-7 by @dependabot[bot] in #3442
- Bump org.apache.maven.plugins:maven-shade-plugin from 3.6.0 to 3.6.2 by @dependabot[bot] in #3430
- Bump org.xerial.snappy:snappy-java from 1.1.10.7 to 1.1.10.8 by @dependabot[bot] in #3426
- Bump org.codehaus.mojo:exec-maven-plugin from 3.5.0 to 3.6.3 by @dependabot[bot] in #3440
- Bump org.apache.parquet:parquet-pig from 1.15.0 to 1.15.2 by @dependabot[bot] in #3420
- Bump commons-logging:commons-logging from 1.3.5 to 1.3.6 by @dependabot[bot] in #3418
- Bump com.google.re2j:re2j from 1.7 to 1.8 by @dependabot[bot] in #3433
- Bump org.apache.commons:commons-lang3 from 3.17.0 to 3.20.0 by @dependabot[bot] in #3429
- GH-3411 Expose row group index via Parquet reader by @uros7251brick in #3412
- Bump org.mockito:mockito-core from 5.22.0 to 5.23.0 by @dependabot[bot] in #3445
- Bump net.openhft:zero-allocation-hashing from 0.27ea0 to 0.27ea1 by @dependabot[bot] in #3419
- Bump org.tukaani:xz from 1.10 to 1.12 by @dependabot[bot] in #3417
- Bump it.unimi.dsi:fastutil from 8.5.15 to 8.5.18 by @dependabot[bot] in #3422
- Bump org.apache.commons:commons-text from 1.13.1 to 1.15.0 by @dependabot[bot] in #3438
- GH-3449: Fix BinaryStats int overflow by @Jiayi-Wang-db in #3448
- Bump protobuf.version from 4.34.0 to 4.34.1 by @dependabot[bot] in #3453
- Bump org.apache.arrow:arrow-vector from 17.0.0 to 19.0.0 by @dependabot[bot] in #3454
- GH-3410: Bump jackson dependencies from 2.19.2 to 2.21.2 by @manuzhang in #3460
- Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.23.1 to 0.25.4 by @dependabot[bot] in #3437
- Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.4 to 0.25.5 by @dependabot[bot] in #3473
- Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.5 to 0.25.6 by @dependabot[bot] in #3491
- Bump com.google.guava:guava from 33.5.0-jre to 33.6.0-jre by @dependabot[bot] in #3492
- GH-3011: Deny further writes after InternalParquetRecordWriter is aborted by @LuciferYang in #3450
- GH-3461: Avoid unnecessary debug logging overhead in record write hot path by @arouel in #3462
- [GH-3408] Add support for stale PR workflow by @ArnavBalyan in #3409
- Remove Parquet-Pig by @dossett in #3377
- Bump com.google.api.grpc:proto-google-common-protos from 2.59.2 to 2.68.0 by @dependabot[bot] in #3474
- Bump org.apache.maven.plugins:maven-dependency-plugin from 3.8.1 to 3.10.0 by @dependabot[bot] in #3425
- PARQUET-3532: Remove hardcoded versions from
pom.xmlby @Fokko in #3531 - Bump org.codehaus.mojo:build-helper-maven-plugin from 3.6.0 to 3.6.1 by @dependabot[bot] in #3434
- GH-3457: Validate column index for null_pages/null_counts contradiction by @clee704 in #3458
- GH-3112: Fix reading of proto Uint32Value by @0x26res in #3113
- Bump jackson.version from 2.21.2 to 2.21.3 by @dependabot[bot] in #3538
- Bump commons-io:commons-io from 2.21.0 to 2.22.0 by @dependabot[bot] in #3528
- Bump com.github.luben:zstd-jni from 1.5.7-7 to 1.5.7-8 by @dependabot[bot] in #3541
- GH-3464: Improve
DeltaByteArrayWriter.writeBytesto avoid unnecessary allocation and scalar prefix comparison by @arouel in #3465 - GH-3478: Fix LZ4_RAW heap decompression by @yadavay-amzn in #3486
- GH-3497: Fix thread-unsafe singleton in
DefaultValuesWriterFactoryby @arouel in #3498 - GH-3261: Fix integer overflow in
CapacityByteArrayOutputStreamby @yadavay-amzn in #3525 - GH-3545: Use parquet-java in SCM block by @RussellSpitzer in #3546
- MINOR: Fix Spotless formatting in TestCompressionCodec by @RussellSpitzer in #3551
- Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.6 to 0.25.7 by @dependabot[bot] in #3554
- Bump com.google.api.grpc:proto-google-common-protos from 2.68.0 to 2.71.0 by @dependabot[bot] in #3553
- GH-3451. Add a JMH benchmark for variants by @steveloughran in #3452
- GH-3414: Add parseJson to VariantBuilder for JSON-to-Variant conversion by @gaurav7261 in #3415
- Bump actions/stale from 9 to 10 by @dependabot[bot] in #3527
- GH-3482: parquet-hadoop tests to work behind a web proxy by @alexeyroytman in #3483
- GH-3471: Fix
ByteBufferhandling inVariantUtilandVariantBuilderby @rayokota in #3472 - GH-3484: Eliminate per-page heap allocation for CRC32 checksums when using direct
ByteBufferAllocatorby @arouel in #3485 - Optimizing Variant read path with lazy caching by @nssalian in #3481
- Bump com.github.luben:zstd-jni from 1.5.7-8 to 1.5.7-9 by @dependabot[bot] in #3581
- Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.25.7 to 0.26.0 by @dependabot[bot] in #3583
- GH-3573: Fix VariantUtil string decoding to use explicit UTF-8 charset by @mikhail-melnik in #3576
- Bump protobuf.version from 4.34.1 to 4.35.0 by @dependabot[bot] in #3582
- GH-3587: Fix
parquet.thrift.string.size.limitvalidation by @divjotarora in #3588 - GH-3572: Bump thrift to 0.23 by @steveloughran in #3589
- Bump org.apache:apache from 37 to 38 by @dependabot[bot] in #3564
- Bump com.github.siom79.japicmp:japicmp-maven-plugin from 0.26.0 to 0.26.1 by @dependabot[bot] in #3592
- Bump com.fasterxml.jackson.core:jackson-databind from 2.21.3 to 2.21.4 by @dependabot[bot] in #3593
- Bump jackson.version from 2.21.3 to 2.21.4 by @dependabot[bot] in #3594
- GH-3398: Fix potential ClassLoader leak caused by ThreadLocal lambda in Binary.java by @LuciferYang in #3447
- Bump parquet-format to 2.13.0 by @wgtmac in #3611
- Bump protobuf.version from 4.35.0 to 4.35.1 by @dependabot[bot] in #3612
- Bump com.github.luben:zstd-jni from 1.5.7-9 to 1.5.7-11 by @dependabot[bot] in #3613
- Bump com.google.api.grpc:proto-google-common-protos from 2.71.0 to 2.72.0 by @dependabot[bot] in #3602
- GH-3574: parquet-hadoop: Statistics.toParquetStatistics: always set null_count by @mdibaiee in #3575
- Bump com.fasterxml.jackson.core:jackson-databind from 2.21.4 to 2.22.0 by @dependabot[bot] in #3606
- MINOR: Remove Java 8 direct buffer cleaner path by @dossett in #3591
- Simplify CI: collapse codec matrix and add zstd coverage by @iemejia in #3580
- Bump jackson.version from 2.21.4 to 2.22.0 by @dependabot[bot] in #3605
- Bump com.fasterxml.jackson.core:jackson-annotations from 2.21 to 2.22 by @dependabot[bot] in #3604
- GH-3489: Fix Binary.copy() to materialize direct ByteBuffer-backed values to heap by @arouel in #3490
- GH-3632: Fix record-level notIn evaluation by @wgtmac in #3631
- Bump commons-logging:commons-logging from 1.3.6 to 1.4.0 by @dependabot[bot] in #3623
- GH-3627: Add missing onAppend() to VariantBuilder#appendUUIDBytes by @nastra in #3624
- Bump com.squareup.okhttp3:okhttp from 4.12.0 to 5.4.0 by @nastra in #3630
- GH-3598: Expose
getRowRanges(int)by @peter-toth in #3599 - GH-3585: Fix DataPageHeaderV2.num_nulls=-1 when column statistics are disabled by @keuin in #3586
- PARQUET-2249: Add IEEE-754 total order and nan count for floating types by @wgtmac in #3393
- GH-3618: Upgrade JUnit4 to JUnit 5.14.4 by @nastra in #3619
- GH-3542 Fix
LocalInputFileByteBufferread methods to support direct and non-zero-position buffers by @arouel in #3543 - Bump org.cyclonedx:cyclonedx-maven-plugin from 2.9.1 to 2.9.2 by @dependabot[bot] in #3635
- GH-3628: prevent NPE & unclosed releaser by @nastra in #3629
- GH-3625: Compute dataSize correctly when dealing with duplicate keys by @nastra in #3626
- GH-3615: Introduce AssertJ for testing by @nastra in #3616
- Restrict trigger push branch for GitHub Workflow by @apupier in #3641
- GH-3639: Don't drop row groups for IN(..., null) when num_nulls is not set by @nastra in #3640
- GH-3615: Use assertThat checks in parquet-variant tests by @nastra in #3617
- GH-3615: Replace @test(expected=...) with assertThatThrownBy() checks by @nastra in #3636
- Bump org.apache:apache from 38 to 39 by @dependabot[bot] in #3646
- GH-3475: Fix parquet-vector compatiblity with Java > 17 by @iemejia in #3476
- GH-3621: Replace the previous field in the queue instead of appending to the queue by @njlaw in #3622
- MINOR: Fix PrimitiveStringifier class Javadoc to link UnsupportedOperationException by @anxkhn in #3651
- GH-3530: Optimize PLAIN encoding and decoding with direct ByteBuffer I/O by @iemejia in #3565
- Bump com.google.api.grpc:proto-google-common-protos from 2.72.0 to 2.73.0 by @dependabot[bot] in #3657
- Bump jackson.version from 2.22.0 to 2.22.1 by @dependabot[bot] in #3656
- Bump com.fasterxml.jackson.core:jackson-databind from 2.22.0 to 2.22.1 by @dependabot[bot] in #3658
- GH-3596: Add
RowRanges.Builderfor incremental construction from selected row indices by @peter-toth in #3597 - PARQUET-3459: Per column compression by @mengna-lin in #3526
- Add SECURITY.md by @emkornfield in #3643
- GH-3659: Throw IAE when dealing with odd-length/invalid hex string in Parquet CLI by @nastra in #3642
- GH-3637: Fix corrupt output when reusing a dictionary writer across row groups after fallback by @nastra in #3638
- GH-3644: Handle NoClassDefFoundError in DynMethods by @nastra in #3645
- MINOR: Fix withLogicalTypeAnnotation Javadoc @return description by @anxkhn in #3648
- MINOR: Fix VariantUtil.getByte() Javadoc copied from getShort() by @anxkhn in #3649
- GH-3652: Use assertThat checks in parquet-plugins/parquet-protobuf tests by @nastra in #3653
- Fix CI caused by a silent merge conflict by @nastra in #3665
- [INFRA] Set up default rulesets for default and release branches by @asf-gitbox-commits in #3563
- GH-3652: Use assertThat checks in parquet-common tests by @nastra in #3654
- GH-3652: Use assertThat checks in parquet-avro tests by @nastra in #3660
- GH-3652: Use assertThat checks in parquet-cli tests by @nastra in #3661
- GH-3615: Use assertThat checks in parquet-column tests by @nastra in #3664
- GH-3652: Use assertThat checks in parquet-format-structures/parquet-encoding/parquet-thrift tests by @nastra in #3650
- GH-3671: Keep files readable when they use future logical types by @kevinjqliu in #3669
- GH-3652: Use assertThat checks in parquet-hadoop tests by @nastra in #3666
- GH-3618: Migrate multiple modules to JUnit5 by @nastra in #3673
- Fix CI due to silent merge conflict with AssertJ migration by @nastra in #3677
- GH-3618: Migrate parquet-hadoop to JUnit5 by @nastra in #3676
- GH-3618: Ban JUnit4 imports by @nastra in #3681
- MINOR: Update copyright year in NOTICE file by @kevinjqliu in #3685
- PARQUET: Restore LICENSE and NOTICE files in packaged JARs by @kevinjqliu in #3686
- PARQUET: Verify LICENSE and NOTICE in published JARs in CI by @kevinjqliu in #3689
- Revert "GH-3574: Statistics.toParquetStatistics always set null_count (#3575)" by @Fokko in #3688
- PARQUET-2249: Write IEEE 754 total order by default for floating-point columns by @Jiayi-Wang-db in #3699
New Contributors
- @takezoe made their first contribution in #3385
- @HuaHuaY made their first contribution in #3400
- @mbrannstrom made their first contribution in #3413
- @uros7251brick made their first contribution in #3412
- @manuzhang made their first contribution in #3460
- @arouel made their first contribution in #3462
- @clee704 made their first contribution in #3458
- @0x26res made their first contribution in #3113
- @yadavay-amzn made their first contribution in #3486
- @RussellSpitzer made their first contribution in #3546
- @gaurav7261 made their first contribution in #3415
- @alexeyroytman made their first contribution in #3483
- @rayokota made their first contribution in #3472
- @nssalian made their first contribution in #3481
- @mikhail-melnik made their first contribution in #3576
- @divjotarora made their first contribution in #3588
- @mdibaiee made their first contribution in #3575
- @peter-toth made their first contribution in #3599
- @keuin made their first contribution in #3586
- @apupier made their first contribution in #3641
- @njlaw made their first contribution in #3622
- @anxkhn made their first contribution in #3651
- @mengna-lin made their first contribution in #3526
- @asf-gitbox-commits made their first contribution in #3563
- @kevinjqliu made their first contribution in #3669
Full Changelog: apache-parquet-1.17.1...apache-parquet-1.18.0