apache/iceberg apache-iceberg-1.8.0

on GitHub

What's Changed

• Doc: Update rewrite data files spark procedure by @dramaticlly in #11396
• Docs: warn parallelism > 1 doesn't work for migration procedures by @manuzhang in #11417
• Core: log retry sleep time by @sullis in #11413
• Core: Use RoaringPositionBitmap in position index by @aokolnychyi in #11441
• Core: Add validation for table commit properties by @dramaticlly in #11437
• Core: Add cardinality to PositionDeleteIndex by @aokolnychyi in #11442
• Puffin: Add delete-vector-v1 blob type by @rdblue in #11238
• Spec v3: Add deletion vectors to the table spec by @rdblue in #11240
• API, Core: Add data file reference to DeleteFile by @aokolnychyi in #11443
• Build: Bump software.amazon.awssdk:bom from 2.29.1 to 2.29.6 by @dependabot in #11454
• Build: Bump com.gradleup.shadow:shadow-gradle-plugin from 8.3.3 to 8.3.5 by @dependabot in #11452
• Build: Bump com.google.cloud:libraries-bom from 26.49.0 to 26.50.0 by @dependabot in #11451
• Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.4 to 5.4.1 by @dependabot in #11450
• Build: Bump com.azure:azure-sdk-bom from 1.2.28 to 1.2.29 by @dependabot in #11453
• Flink: Maintenance - TableManager + ExpireSnapshots by @pvary in #11144
• Build: Bump mkdocs-material from 9.5.42 to 9.5.43 by @dependabot in #11455
• Build: Bump net.snowflake:snowflake-jdbc from 3.19.1 to 3.20.0 by @dependabot in #11447
• Build: Bump kafka from 3.8.0 to 3.8.1 by @dependabot in #11449
• Build: Bump jackson-bom from 2.18.0 to 2.18.1 by @dependabot in #11448
• Core: Fix generated position delete file spec by @aokolnychyi in #11458
• API, Core: Add content offset and size to DeleteFile by @aokolnychyi in #11446
• Build: Revert Parquet to 1.13.1 by @RussellSpitzer in #11462
• Spark 3.5: Preserve data file reference during manifest rewrites by @aokolnychyi in #11457
• Core: Make PositionDeleteIndex serializable by @aokolnychyi in #11463
• Spark 3.5: Preserve content offset and size during manifest rewrites by @aokolnychyi in #11469
• Spark 3.5: Fix flaky test due to temp directory not empty during delete by @manuzhang in #11470
• Core, Data, Flink, Spark: Improve tableDir initialization for tests by @nastra in #11460
• Core: Support DVs in DeleteFileIndex by @aokolnychyi in #11467
• Core: Adapt commit, scan, and snapshot stats for DVs by @aokolnychyi in #11464
• Spark: Merge new position deletes with old deletes during writing by @amogh-jahagirdar in #11273
• Fix ADLSLocation file parsing by @mrcnc in #11395
• open-api: Build runtime jar for test fixture by @ajantha-bhat in #11279
• Core, Puffin: Add DV file writer by @aokolnychyi in #11476
• Flink: Fix config key typo in error message of SplitComparators by @liuml07 in #11482
• API: Removes Explicit Parameterization of Schema Tests by @RussellSpitzer in #11444
• Docs: Fix verifying release candidate with Spark and Flink by @manuzhang in #11461
• Flink: Port #11144 to v1.19 by @pvary in #11473
• Docs: Fix format of verifying release candidate with Flink by @manuzhang in #11487
• Core: Support DVs in DeleteLoader by @aokolnychyi in #11481
• Infra: Update DOAP.RDF for Apache Iceberg 1.7.0 by @RussellSpitzer in #11492
• Docs: Site Update for 1.7.0 Release by @RussellSpitzer in #11494
• Infra: Add 1.7.0 to issue template by @RussellSpitzer in #11491
• Build: Let revapi compare against 1.7.0 by @RussellSpitzer in #11490
• Docs: Adds Release notes for 1.6.1 by @RussellSpitzer in #11500
• Docs: Fixes Release Formatting for 1.7.0 Release Notes by @RussellSpitzer in #11499
• Spec: Explicitly specify operation as a required field of Snapshot's summary field by @sungwy in #11355
• Exclude reading _pos column if it's not in the scan list by @huaxingao in #11390
• Build: Bump mkdocs-redirects from 1.2.1 to 1.2.2 by @dependabot in #11511
• Build: Bump mkdocs-material from 9.5.43 to 9.5.44 by @dependabot in #11510
• Build: Bump software.amazon.awssdk:bom from 2.29.6 to 2.29.9 by @dependabot in #11509
• Docs: Update multi-engine support after 1.7.0 release by @manuzhang in #11503
• Spark: Fix typo in spark ddl document by @hantangwangd in #11517
• Core: Support commits with DVs by @aokolnychyi in #11495
• Kafka Connect: fix Hadoop dependency exclusion by @bryanck in #11516
• Upgrade to Gradle 8.11.0 by @jbonofre in #11521
• Spark 3.5: Iceberg parser should passthrough unsupported procedure to delegate by @pan3793 in #11480
• use dist/release KEYS by @kevinjqliu in #11526
• Remove iceberg-pig by @jbonofre in #11380
• Core, Flink, Spark: Verify maintenance actions with DVs by @nastra in #11485
• Spark: Use assumeThat(format).isEqualTo(FileFormat.PARQUET) instead of assumeThat(format).isEqualTo(parquet) by @huaxingao in #11540
• Spark 3.4: Support Spark Column Stats by @saitharun15 in #11532
• Docs: Fix rendering lists by @manuzhang in #11546
• Build: Bump kafka from 3.8.1 to 3.9.0 by @dependabot in #11508
• Support WASB scheme in ADLSFileIO by @mrcnc in #11504
• Docs: 4 Spaces are Required for Sublists by @RussellSpitzer in #11549
• API, Core, Spark: Ignore schema merge updates from long -> int by @rocco408 in #11419
• Docs: Fix level of Deletion Vectors by @manuzhang in #11547
• API, Core: Replace deprecated ContentFile#path usage with location by @amogh-jahagirdar in #11550
• API: Add Variant data type by @aihuaxu in #11324
• Spark 3.5: Adapt DeleteFileIndexBenchmark for DVs by @aokolnychyi in #11529
• Spark 3.5: Adapt PlanningBenchmark for DVs by @aokolnychyi in #11531
• Spark 3.5: Add DVReaderBenchmark by @aokolnychyi in #11537
• Build: Bump datamodel-code-generator from 0.26.2 to 0.26.3 by @dependabot in #11572
• Build: Bump software.amazon.awssdk:bom from 2.29.9 to 2.29.15 by @dependabot in #11568
• Build: Bump io.netty:netty-buffer from 4.1.114.Final to 4.1.115.Final by @dependabot in #11569
• Data, Flink, Spark: Test deletes with format-version=3 by @nastra in #11538
• Build: Bump nessie from 0.99.0 to 0.100.0 by @dependabot in #11567
• Build: Bump orc from 1.9.4 to 1.9.5 by @dependabot in #11571
• API, Arrow, Core, Data, Spark: Replace usage of deprecated ContentFile#path API with location API by @amogh-jahagirdar in #11563
• Core: Serialize null when there is no current snapshot by @Fokko in #11560
• Spark 3.4: Iceberg parser should passthrough unsupported procedure to delegate by @pan3793 in #11579
• Spark 3.3: Iceberg parser should passthrough unsupported procedure to delegate by @pan3793 in #11580
• Core: Inherited classes from SnapshotProducer has TableOperations red… by @gaborkaszab in #11578
• Revert "Core: Use encoding/decoding methods for namespaces and deprecate Splitter/Joiner" by @nastra in #11574
• Core: delete temp metadata file when version already exists by @leesf in #11350
• Build: Bump Apache Parquet 1.14.4 by @Fokko in #11502
• core: Filter on live entries when reading the manifest by @Fokko in #9996
• Core: Fix CCE when retrieving TableOps by @nastra in #11585
• API, Core: Remove unnecessary casts to Iterable by @nastra in #11601
• Spark 3.5: Fix NotSerializableException when migrating Spark tables by @manuzhang in #11157
• Spark 3.3: Deprecate support by @aokolnychyi in #11596
• Bugfix for incorrect Deletion of Snapshot Metadata Due to OutOfMemoryError by @ZhendongBai in #11576
• Docs: add iceberg-go to doc site by @zeroshade in #11607
• Procedure to compute table stats by @karuppayya in #10986
• Parquet: Use native getRowIndexOffset support instead of calculating it by @wypoon in #11520
• Spark: Fix changelog table bug for start time older than current snapshot by @Acehaidrey in #11564
• Spark 3.5: Fix flaky TestRemoveOrphanFilesAction3 by @manuzhang in #11616
• Upgrade to Gradle 8.11.1 by @jbonofre in #11619
• Core: Optimize MergingSnapshotProducer to use referenced manifests to determine if manifest needs to be rewritten by @amogh-jahagirdar in #11131
• Revert "Core: Update TableMetadataParser to ensure all streams closed (#11220)" by @hussein-awala in #11621
• Add REST Catalog tests to Spark 3.5 integration test by @haizhou-zhao in #11093
• Spark 3.5: Correct the two-stage parsing strategy of antlr parser by @pan3793 in #11628
• Spark 3.4: Correct the two-stage parsing strategy of antlr parser by @pan3793 in #7734
• Docs: Add new blog post to Iceberg Blogs by @ismailsimsek in #11627
• Docs: Mention look-free requires HIVE-28121 for MySQL/MariaDB-based HMS by @pan3793 in #11631
• Spark 3.4: IcebergSource extends SessionConfigSupport by @pan3793 in #7732
• Spark 3.5: IcebergSource extends SessionConfigSupport by @pan3793 in #11624
• Spark 3.3: IcebergSource extends SessionConfigSupport by @pan3793 in #11625
• Build: Bump testcontainers from 1.20.3 to 1.20.4 by @dependabot in #11640
• Build: Bump mkdocs-material from 9.5.44 to 9.5.45 by @dependabot in #11641
• Spark 3.3: Correct the two-stage parsing strategy of antlr parser by @pan3793 in #11630
• Build: Bump software.amazon.awssdk:bom from 2.29.15 to 2.29.20 by @dependabot in #11639
• Core,Open-API: Don't expose the last-column-id by @Fokko in #11514
• Build: Bump com.google.errorprone:error_prone_annotations from 2.35.1 to 2.36.0 by @dependabot in #11638
• Flink: Add table.exec.iceberg.use-v2-sink option by @arkadius in #11244
• Docs: Use DataFrameWriterV2 in Spark's example by @pan3793 in #11647
• Docs: add WHEN NOT MATCHED BY SOURCE to Spark MERGE INTO doc by @hussein-awala in #11636
• Build: Bump nessie from 0.100.0 to 0.100.2 by @dependabot in #11637
• Build: Delete branch automatically on PR merge by @manuzhang in #11635
• Flink: Test both "new" Flink Avro planned reader and "deprecated" Avro reader by @jbonofre in #11430
• Adding ComputeTableStats Procedure to Spark 3.4 by @jeesou in #11652
• Flink: Backport #11244 to Flink 1.19 (Add table.exec.iceberg.use-v2-sink option) by @arkadius in #11665
• Doc: Fix some Javadoc URLs. by @zhongyujiang in #11666
• Docs: Add blog post showing Nussknacker with Iceberg integration by @arkadius in #11667
• Kafka Connect: Add config to prefix the control consumer group by @hugofriant in #11599
• Backport Avro planned reader (and corresponding tests) on Flink v1.18 and v1.19 by @jbonofre in #11668
• Docs: Add RisingWave by @hengm3467 in #11642
• REST: Docker file for REST Fixture by @ajantha-bhat in #11283
• Core: Propagate custom metrics reporter when table is created/replaced through Transaction by @nastra in #11671
• Spark: remove ROW_POSITION from project schema by @huaxingao in #11610
• Default to overwrite when operation is missing by @Fokko in #11421
• Core,API: Set 503: added_snapshot_id as required by @Fokko in #11626
• Create publish-docker.yml by @sungwy in #11632
• Replace wrong order of assertion by @ebyhr in #11677
• REST: Refactor docker image files by @ajantha-bhat in #11676
• Build: Bump software.amazon.awssdk:bom from 2.29.20 to 2.29.23 by @dependabot in #11683
• Build: Bump mkdocs-material from 9.5.45 to 9.5.46 by @dependabot in #11680
• REST: Use HEAD request to check table existence by @ebyhr in #10999
• Build: Bump jackson-bom from 2.18.1 to 2.18.2 by @dependabot in #11681
• Build: Bump org.xerial:sqlite-jdbc from 3.47.0.0 to 3.47.1.0 by @dependabot in #11682
• Spark-3.5: make where sql case sensitive setting alterable in rewrite data files procedure by @ludlows in #11439
• Spark: Remove extra columns for ColumnBatch by @huaxingao in #11551
• Spark: Add view support to SparkSessionCatalog by @nastra in #11388
• REST: Fix warning message for deprecated OAuth2 server URI by @ebyhr in #11694
• Build: Bump Parquet to 1.15.0 by @Fokko in #11656
• Spark 3.3,3.4: Make where clause case sensitive in rewrite data files by @ludlows in #11696
• Core: Generalize Util::blockLocations by @okumin in #11053
• Document procedure for stats collection by @karuppayya in #11606
• Spark 3.5: Align RewritePositionDeleteFilesSparkAction filter case sensitivity with Spark case sensitivity by @huaxingao in #11700
• Spark: Write DVs for V3 MoR tables by @amogh-jahagirdar in #11561
• Infra: Add 1.7.1 to issue template by @bryanck in #11711
• Update ASF doap.rdf to release 1.7.1 by @bryanck in #11712
• Spark 3.3/3.4: Align RewritePositionDeleteFilesSparkAction filter case sensitivity with Spark case sensitivity by @huaxingao in #11710
• Build: Bump software.amazon.awssdk:bom from 2.29.23 to 2.29.29 by @dependabot in #11723
• Build: Bump com.google.cloud:libraries-bom from 26.50.0 to 26.51.0 by @dependabot in #11724
• Build: Bump mkdocs-material from 9.5.46 to 9.5.47 by @dependabot in #11726
• Add C++ to the list of languages in doap.rdf by @Fokko in #11714
• Build: Bump com.azure:azure-sdk-bom from 1.2.29 to 1.2.30 by @dependabot in #11725
• Added curl to the iceberg rest fixture Docker image #11688 by @dominikhei in #11705
• Build: Bump nessie from 0.100.2 to 0.101.0 by @dependabot in #11722
• Docs: Add release notes for 1.7.1 by @bryanck in #11717
• Docs: Add guidelines for contributors to become committers by @rdblue in #11670
• Core, Flink, Spark: Drop deprecated APIs scheduled for removal in 1.8.0 by @findepi in #11721
• Flink: Fix range distribution npe when value is null by @Guosmilesmile in #11662
• AWS: Enable Adaptive Retries for AWS KMS client by @hsiang-c in #11420
• Core, Flink, Spark, KafkaConnect: Remove remaining usage of deprecated path API by @amogh-jahagirdar in #11744
• docker: Build for arm64 architecture by @Fokko in #11753
• Docs: fix typo in spec by @xxchan in #11759
• Spark3.4,3.5: Fix the BUG of iceberg views when resolved "group by ordinals" by @Ppei-Wang in #11729
• Spark: Remove deprecated SparkAppenderFactory by @ajantha-bhat in #11727
• Hadoop: Log where the missing metadata file is located by @manuzhang in #11643
• Core: Use HEAD request to check if view exists by @nastra in #11760
• Core: Use HEAD request to check if namespace exists by @nastra in #11761
• Hive: Optimize tableExists API in hive catalog by @dramaticlly in #11597
• GCS: Suppress JavaUtilDate in OAuth2RefreshCredentialsHandler by @ebyhr in #11773
• Flink: Add RowConverter for Iceberg Source by @abharath9 in #11301
• Spark 3.5: Fix comment and assertion mismatch in PartitionedWritesTestBase/TestRewritePositionDeleteFilesAction by @wzx140 in #11748
• Build: Bump nessie from 0.101.0 to 0.101.2 by @dependabot in #11791
• Core: Add missing REST endpoint definitions by @ajreid21 in #11756
• Build: Bump software.amazon.awssdk:bom from 2.29.29 to 2.29.34 by @dependabot in #11793
• Spark: Read DVs when reading from .position_deletes table by @nastra in #11657
• Core: Add TableUtil to provide access to a table's format version by @nastra in #11620
• Build: Bump mkdocs-material from 9.5.47 to 9.5.48 by @dependabot in #11790
• Spark 3.4: Add REST catalog to Spark integration tests by @nastra in #11698
• Parquet: Implement defaults for generic data by @rdblue in #11785
• Avro: Support default values for generic data by @rdblue in #11786
• REST: Use apache/iceberg-rest-fixture docker image by @ajantha-bhat in #11673
• Docs: Default value of table level distribution-mode should be not set by @manuzhang in #11663
• Docs: Fix Spark catalog table-override description by @manuzhang in #11684
• API: Add missing deprecations by @Fokko in #11734
• Core, Spark3.5: Fix tests failure due to timeout by @manuzhang in #11654
• Auth Manager API part 1: HTTPRequest, HTTPHeader by @adutra in #11769
• Flink: make StatisticsOrRecord to be correctly serialized and deser… by @huyuanfeng2018 in #11557
• Spark3.4,3.5: In describe extended view command: fix wrong view catal… by @Ppei-Wang in #11751
• Use try-with-resources in TestParallelIterable by @sopel39 in #11810
• Spark 3.5: Support default values in Parquet reader by @rdblue in #11803
• Data: Fix Parquet and Avro defaults date/time representation by @rdblue in #11811
• Revert "Support WASB scheme in ADLSFileIO" by @mrcnc in #11812
• Core, Spark, Flink, Hive: Remove unused failsafe dependency from core and add Failsafe to runtime LICENSE(s) by @amogh-jahagirdar in #11816
• Docs: Change to Flink directory for instructions by @liuml07 in #11031
• Spark 3.5: Support default values in vectorized reads by @rdblue in #11815
• Remove number from assert description by @TQJADE in #11827
• Auth Manager API part 2: AuthManager by @adutra in #11809
• Core: Add Variant implementation to read serialized objects by @rdblue in #11415
• Spark: Test reading default values in Spark by @rdblue in #11832
• Build: Bump datamodel-code-generator from 0.26.3 to 0.26.4 by @dependabot in #11856
• Build: Bump mkdocs-awesome-pages-plugin from 2.9.3 to 2.10.0 by @dependabot in #11855
• Build: Bump mkdocs-material from 9.5.48 to 9.5.49 by @dependabot in #11854
• Build: Bump io.netty:netty-buffer from 4.1.115.Final to 4.1.116.Final by @dependabot in #11853
• Build: Bump software.amazon.awssdk:bom from 2.29.34 to 2.29.39 by @dependabot in #11851
• Build: Bump guava from 33.3.1-jre to 33.4.0-jre by @dependabot in #11850
• Build: Bump junit from 5.11.3 to 5.11.4 by @dependabot in #11849
• Build: Bump org.assertj:assertj-core from 3.26.3 to 3.27.0 by @dependabot in #11847
• Build: Bump nessie from 0.101.2 to 0.101.3 by @dependabot in #11852
• Build: Bump junit-platform from 1.11.3 to 1.11.4 by @dependabot in #11848
• Doc: Add status page for different implementations. by @liurenjie1024 in #11772
• Build: Upgrade to Gradle 8.12 by @jbonofre in #11861
• Build: Fix ignoring .asf.yaml in PR by @manuzhang in #11860
• Gradle 8.12 updated the gradlew with better APP_HOME definition by @jbonofre in #11869
• Core, Spark: Avoid deprecated methods in Guava Files by @ebyhr in #11865
• Core: Don't reset snapshotLog in TableMetadata.removeRef method by @ebyhr in #11779
• Core: Replace deprecated Schema.toString with SchemaFormatter by @ebyhr in #11867
• Build: Fix ignoring license-check.yml in PR by @manuzhang in #11873
• API: Replace deprecated asList with asInstanceOf in TestExceptionUtil by @ebyhr in #11875
• Flink: Avoid RANGE mode broken chain when write parallelism changes by @huyuanfeng2018 in #11702
• Update README.md to add link to iceberg-cpp by @gabeiglio in #11882
• Build: Bump software.amazon.awssdk:bom from 2.29.39 to 2.29.43 by @dependabot in #11886
• Build: Bump mkdocs-awesome-pages-plugin from 2.10.0 to 2.10.1 by @dependabot in #11885
• Core: Fix typo in HadoopTableOperations by @okumin in #11880
• Revert "Hive: close the fileIO client when closing the hive catalog" by @Fokko in #11858
• Docs: Add history to Hive's metadata tables by @okumin in #11902
• Doc: Fix format of Hive by @ebyhr in #11892
• Flink: Backport #11662 Fix range distribution npe when value is null to Flink 1.18 and 1.19 by @Guosmilesmile in #11745
• Spark: Change Delete granularity to file for Spark 3.5 by @amogh-jahagirdar in #11478
• Bump Spark 3.5.4 by @pan3793 in #11731
• Kafka-connect-runtime: remove code duplications in integration tests by @wombatu-kun in #11883
• Flink: Backport #11557 to Flink 1.19 and 1.18 by @huyuanfeng2018 in #11834
• Use ExternalTypeInfo in Rowconverter code instead of deprecated TableSchema.getFieldTypes by @abharath9 in #11838
• Update maintenance.md by @hgeraldino in #11916
• Build: Bump software.amazon.awssdk:bom from 2.29.43 to 2.29.45 by @dependabot in #11910
• Build: Bump org.xerial:sqlite-jdbc from 3.47.1.0 to 3.47.2.0 by @dependabot in #11907
• Hive: Optimize viewExists API in hive catalog by @dramaticlly in #11813
• Core: Add support for view-default property in catalog by @ebyhr in #11064
• Build: Bump io.delta:delta-standalone_2.12 from 3.2.1 to 3.3.0 by @dependabot in #11909
• Build: Bump org.assertj:assertj-core from 3.27.0 to 3.27.2 by @dependabot in #11908
• Doc: Add missing fields to metadata tables in Spark page by @ebyhr in #11897
• API: Support removeUnusedSpecs in ExpireSnapshots by @advancedxy in #10755
• Spark 3.5: Implement RewriteTablePath by @szehon-ho in #11555
• Infra: Add manuzhang to collaborators by @manuzhang in #11927
• Put a CFP Banner on the Homepage by @RussellSpitzer in #11942
• Use compatible column name to set Parquet bloom filter by @huaxingao in #11799
• Core: Allow adding files to multiple partition specs in FastAppend by @anuragmantri in #11771
• Avro: Add internal writer by @ajantha-bhat in #11919
• Core: Fix possible deadlock in ParallelIterable by @sopel39 in #11781
• Hive: Remove Hive runtime by @manuzhang in #11801
• Build: Bump software.amazon.awssdk:bom from 2.29.45 to 2.29.50 by @dependabot in #11949
• Core, Rest: Read the max connection for rest client from properties by @munendrasn in #11522
• Docs: Use the correct YAML text block indicator to prevent formatting issues by @neodon in #11552
• API: Fix parameter of ScanTask by @engraving-knife in #11941
• Build: Bump io.delta:delta-spark_2.12 from 3.2.1 to 3.3.0 by @dependabot in #11911
• Build: Bump net.snowflake:snowflake-jdbc from 3.20.0 to 3.21.0 by @dependabot in #11792
• Build: Bump com.google.cloud:libraries-bom from 26.51.0 to 26.52.0 by @dependabot in #11846
• Core: Add tests for catalogs supporting empty namespaces by @nastra in #9890
• Spec: Add cross-region bucket access property to config by @munendrasn in #11260
• Core: Fix test failure caused by check empty namespace on REST session catalog by @hantangwangd in #11960
• [docs] Update docs link about Apache Doris and update vendors list by @morningman in #11956
• docs: update README.md fix brand name macOS by @jbampton in #11964
• Core: Fix loading a table in CachingCatalog with metadata table name by @gaborkaszab in #11738
• Open-API: Bump to OpenAPI 3.1 by @Fokko in #11955
• Build: Bump openapi-generator plugin from 6.6.0 to 7.10.0 by @nastra in #11970
• Run java-ci on changes in open-api/** by @Fokko in #11972
• AWS, Core, GCP: Support relative credential endpoint / pass OAuth2 token to credential provider by @nastra in #11954
• Doc: Add DELETE ORPHAN-FILES example to Hive page by @ebyhr in #11896
• Spark 3.4: Add view support to SparkSessionCatalog by @nastra in #11797
• Spark 3.4: Backport rewriting historical file-scoped deletes (#11273) to 3.4 by @amogh-jahagirdar in #11975
• Support sanitizing a Literal<?> by @dmgkeke in #11943
• Auth Manager API part 3: OAuth2 Manager by @adutra in #11844
• Spark 3.5: Fix flaky tests withSnapshotIsolation by @manuzhang in #11974
• Spark: Fix empty scan issue when start timestamp retrieves root snapshot and end timestamp is missing by @lliangyu-lin in #11967
• Spark 3.4: Backport support for default values by @rdblue in #11987
• Doc: Add missing content value to manifests table by @ebyhr in #11989
• Spark: Fix Puffin suffix for DV files by @amogh-jahagirdar in #11986
• Spec: Add added-rows field to Snapshot by @RussellSpitzer in #11976
• API: add hashcode cache in StructType by @wzx140 in #11764
• Build: Bump mkdocs-material from 9.5.49 to 9.5.50 by @dependabot in #12005
• Build: Bump datamodel-code-generator from 0.26.4 to 0.26.5 by @dependabot in #12004
• Revert "API: add hashcode cache in StructType (#11764)" by @Fokko in #12007
• Build: Bump io.netty:netty-buffer from 4.1.116.Final to 4.1.117.Final by @dependabot in #11999
• Build: Bump org.apache.datasketches:datasketches-java from 6.1.1 to 6.2.0 by @dependabot in #12000
• Build: Bump org.xerial:sqlite-jdbc from 3.47.2.0 to 3.48.0.0 by @dependabot in #12001
• Build: Bump org.assertj:assertj-core from 3.27.2 to 3.27.3 by @dependabot in #12002
• Build: Bump com.google.cloud:libraries-bom from 26.52.0 to 26.53.0 by @dependabot in #12003
• Spark: Don't skip tests in TestSelect for SparkSessionCatalog by @manuzhang in #11824
• Core: List namespaces/tables when testing identifier with a dot by @nastra in #11991
• Support WASB scheme in ADLSFileIO by @mrcnc in #11830
• Docs: Update Footer Copyright Year by @omkenge in #12011
• Spec: Clarify next-row-id by @Fokko in #12018
• Flink: Upgrade Flink version 1.19.0 => 1.19.1 to mitigate NoSuchMethodError on commit by @mxm in #12021
• Update notice files copyright end year to 2025 by @amogh-jahagirdar in #12013
• Spark 3.4: Backport writing DVs to Spark 3.4 by @amogh-jahagirdar in #12019
• Spark 3.3: Backport support for default values by @rdblue in #11988
• Doc: Fix expired link in vendor page by @ebyhr in #12045
• add RisingWave to the Vendors page by @hengm3467 in #12043
• Build: Nighly build for Iceberg REST fixtures by @Fokko in #12008
• ORC: Fail when initial default support is required by @rdblue in #12026
• REST: Add missing default HEAD endpoints and V1_COMMIT_TRANSACTION by @ebyhr in #11980
• Spark 3.5: Refactor delete logic in batch reading by @huaxingao in #11933
• Delta, AWS, REST: Remove redundant charset lookup by @ebyhr in #12057
• Remove slf4j-api reference in LICENSE as it's now excluded from the distributed jar files by @jbonofre in #12052
• Adding Crunchy Data to Iceberg Vendors list by @elizabeth-christensen in #12020
• Flink 1.20: Support default values in Parquet reader by @jbonofre in #11839
• Flink: Backport default values support in Parquet reader on Flink v1.18 and v1.19 by @jbonofre in #12072
• Spark 3.5: Fix Javadoc in ColumnarBatchUtil by @huaxingao in #12058
• Core, Spark: Rewrite data files with high delete ratio by @nastra in #11825
• Core, Spark: Include content offset/size in PositionDeletesTable by @nastra in #11808
• Core: Retain current view version during expiration by @nastra in #12067
• Spark 3.5: Procedure to rewrite table path by @dramaticlly in #11931
• Spark: Disable rewriting position deletes for V3 tables by @nastra in #12048
• API: Add UnknownType by @Fokko in #12012
• Spark: Fix typo in NoSuchTableException message by @baolsen in #12091
• Spark 3.4: Refactor delete logic in batch reading by @huaxingao in #12061
• Spec, OpenAPI: Adds EnableRowLineage Metadata Update by @RussellSpitzer in #12050
• Spec: Document Snapshot Summary Optional Fields for Standardization by @HonahX in #11660
• Parquet: Add readers and writers for the internal object model by @ajantha-bhat in #11904
• Spark 3.5: Fix broadcasting specs in RewriteTablePath by @manuzhang in #11982
• Build: Upgrade to Gradle 8.12.1 by @jbonofre in #12093
• Spark 3.4: Backport Spark actions and procedures for RewriteTablePath by @dramaticlly in #12111
• Core: Add metadataFileLocation in TableUtil by @dramaticlly in #12082
• Build: Bump actions/stale from 9.0.0 to 9.1.0 by @dependabot in #12110
• OpenAPI: Deprecate snapshot-id of SetStatisticsUpdate by @c-thiel in #12010
• Spark 3.5: Make ColumnVectorWithFilter generic and refactor batch load by @aokolnychyi in #12056
• Spec: Adds in missing ChangeLog metadata columns - Reassigns Row Line… by @RussellSpitzer in #12100
• OpenAPI: Changes for freshness-aware table loading by @gaborkaszab in #11946
• Core: Check referencedDataFile existence for DV by @ebyhr in #12088
• Build: Bump software.amazon.awssdk:bom from 2.29.50 to 2.30.6 by @dependabot in #12109
• Build: Bump org.openapitools:openapi-generator-gradle-plugin from 7.10.0 to 7.11.0 by @dependabot in #12108
• Add IBM to Iceberg vendors list by @alileclerc in #12101
• Build: Bump nessie from 0.101.3 to 0.102.2 by @dependabot in #12107
• Core: Update variant class visibility by @rdblue in #12105
• Parquet: Clean up Parquet generic and internal readers by @rdblue in #12102
• Build: Bump scala-collection-compat from 2.12.0 to 2.13.0 by @manuzhang in #12121
• Minor: typo in url by @Fokko in #12126
• [docs] add spacing to "how to release"'s voting section by @kevinjqliu in #12123
• Spec: Fix minor typo in _last_updated_sequence_number docs by @smaheshwar-pltr in #12128
• Spec: add variant type by @aihuaxu in #10831
• Make view metadata path configurable by write.metadata.path by @tomtongue in #12017
• Core: Support removing keys from EnvironmentContext by @rshkv in #12103
• OpenAPI: add initial/write defaults to schema by @danielcweeks in #12094
• Fix NOTICE and LICENSE in the aws-bundle jar by @jbonofre in #12142
• Iceberg/Comet integration POC by @huaxingao in #9841
• Fix NOTICE and LICENSE in the azure-bundle jar by @jbonofre in #12143
• Spark 3.5: Iceberg / DataFusion Comet integration by @huaxingao in #12147
• Spark 3.5: Remove use of File.Separator in RewriteTablePath by @manuzhang in #12066
• Core, API, Spec: Metadata Row Lineage by @RussellSpitzer in #11948
• Spec: Fix current-version-id in View Spec example by @ldsantos0911 in #12146
• Data: open file using stats in scan by @bryanck in #12151
• Build: Bump nessie from 0.102.2 to 0.102.4 by @dependabot in #12153
• Build: Bump mkdocs-material from 9.5.50 to 9.6.1 by @dependabot in #12157
• Build: Bump software.amazon.awssdk:bom from 2.30.6 to 2.30.11 by @dependabot in #12156
• Build: Bump net.snowflake:snowflake-jdbc from 3.21.0 to 3.22.0 by @dependabot in #12155
• Build: Bump me.champeau.jmh:jmh-gradle-plugin from 0.7.2 to 0.7.3 by @dependabot in #12152
• Build: Bump com.azure:azure-sdk-bom from 1.2.30 to 1.2.31 by @dependabot in #12154
• Spark: make delete file ratio configurable by @MichaelDeSteven in #12148
• Core: Remove TableMetadata::Builder::resetMainBranch by @smaheshwar-pltr in #12149
• Spark: Update benchmark instructions by @Fokko in #12171
• Spark: Test metadata tables with format-version=3 by @nastra in #12135
• Docs: Fix latest and nightly link on javadoc (according to site README.md) by @jbonofre in #12023
• Variants: Implement toString by @rdblue in #12138
• Core: Refactor variants to enable moving interfaces to API module by @rdblue in #12167
• Flink: Add null check to writers to prevent resurrecting null values by @mxm in #12049
• Bump the versions of site/mkdocs.yml by @Fokko in #12180
• Bump Nessie to include updated License/Notice by @amogh-jahagirdar in #12186
• API: Null check for auto-unboxed field id in builder by @danielcweeks in #12165
• Remove jitpack.yml by @Fokko in #12170
• Test Hive: Fix TestHiveMetastore by @2MD in #12140
• Fix NOTICE and LICENSE in the spark-runtime jar by @jbonofre in #12160
• Fix NOTICE and LICENSE in the gcp-bundle jar by @jbonofre in #12144
• Auth Manager API part 4: RESTClient, HTTPClient by @adutra in #11992
• Fix NOTICE and LICENSE in the flink-runtime jar by @jbonofre in #12145
• Update LICENSE/NOTICE in flink-runtime jar files by @jbonofre in #12188
• Update LICENSE/NOTICE for spark-runtime 3.3 and 3.4 by @jbonofre in #12189
• Docs: Minor improvements to Spark Procedures docs by @smaheshwar-pltr in #12190
• Core, Spark: Exclude non live content file in RewriteTablePathUtil by @dramaticlly in #12006
• Spark 3.4: Remove use of File.Separator in RewriteTablePath by @manuzhang in #12173
• Auth Manager API part 5: SigV4 Auth Manager by @adutra in #11995
• Core: Fix RewriteTablePath Incremental Replication by @barronfuentes in #12172
• Spark: support statistics files in RewriteTablePath by @dramaticlly in #11929
• Build: Bump com.gradleup.shadow:shadow-gradle-plugin from 8.3.5 to 8.3.6 by @dependabot in #12210
• Build: Bump mkdocs-material from 9.6.1 to 9.6.3 by @dependabot in #12205
• Build: Bump datamodel-code-generator from 0.26.5 to 0.27.2 by @dependabot in #12204
• Fix LICENSE and NOTICE for the kafka-connect-runtime distributions by @jbonofre in #12195

New Contributors

• @saitharun15 made their first contribution in #11532
• @rocco408 made their first contribution in #11419
• @ZhendongBai made their first contribution in #11576
• @zeroshade made their first contribution in #11607
• @Acehaidrey made their first contribution in #11564
• @hugofriant made their first contribution in #11599
• @hengm3467 made their first contribution in #11642
• @okumin made their first contribution in #11053
• @dominikhei made their first contribution in #11705
• @Guosmilesmile made their first contribution in #11662
• @Ppei-Wang made their first contribution in #11729
• @abharath9 made their first contribution in #11301
• @wzx140 made their first contribution in #11748
• @ajreid21 made their first contribution in #11756
• @sopel39 made their first contribution in #11810
• @TQJADE made their first contribution in #11827
• @liurenjie1024 made their first contribution in #11772
• @gabeiglio made their first contribution in #11882
• @wombatu-kun made their first contribution in #11883
• @hgeraldino made their first contribution in #11916
• @neodon made their first contribution in #11552
• @engraving-knife made their first contribution in #11941
• @morningman made their first contribution in #11956
• @jbampton made their first contribution in #11964
• @dmgkeke made their first contribution in #11943
• @lliangyu-lin made their first contribution in #11967
• @omkenge made their first contribution in #12011
• @mxm made their first contribution in #12021
• @elizabeth-christensen made their first contribution in #12020
• @baolsen made their first contribution in #12091
• @alileclerc made their first contribution in #12101
• @smaheshwar-pltr made their first contribution in #12128
• @rshkv made their first contribution in #12103
• @ldsantos0911 made their first contribution in #12146
• @MichaelDeSteven made their first contribution in #12148
• @2MD made their first contribution in #12140
• @barronfuentes made their first contribution in #12172

Full Changelog: apache-iceberg-1.7.1...apache-iceberg-1.8.0