apache/iceberg apache-iceberg-1.7.0 on GitHub

What's Changed

[DOCS] Fix link on Concepts page by @gaborkaszab in #10718
Core: Support appending files with different specs by @fqaiser94 in #9860
Remove unnecessary class-level synchronized in ManifestFiles by @findepi in #10544
Spec: Clarify which columns can be used for equality delete files. by @emkornfield in #8981
Build: Bump nessie from 0.92.1 to 0.93.1 by @dependabot in #10727
Build: Bump org.testcontainers:testcontainers from 1.19.8 to 1.20.0 by @dependabot in #10730
Build: Bump mkdocs-material from 9.5.28 to 9.5.29 by @dependabot in #10734
Build: Bump org.roaringbitmap:RoaringBitmap from 1.2.0 to 1.2.1 by @dependabot in #10733
Build: Bump software.amazon.awssdk:bom from 2.26.20 to 2.26.21 by @dependabot in #10729
Build: Bump io.netty:netty-buffer from 4.1.111.Final to 4.1.112.Final by @dependabot in #10726
Build: Bump orc from 1.9.3 to 1.9.4 by @dependabot in #10728
Build: Bump com.google.errorprone:error_prone_annotations from 2.28.0 to 2.29.2 by @dependabot in #10731
Flink: Migrate remaining classes to JUnit5 by @tomtongue in #10684
Api, Build: Fix typo in comments in Table and gradlew by @hantangwangd in #10744
Flink: parameterize Flink table source tests to test both old and FLIP-27 source implementations by @stevenzwu in #10741
Core: Limit memory used by ParallelIterable by @findepi in #10691
Flink: handle rescale properly and refactor statistics by @stevenzwu in #10457
Flink: Backport other remaining classes by @tomtongue in #10749
Spec: Clarify time travel implementation in Iceberg by @emkornfield in #8982
Build: Let revapi compare against 1.6.0 by @ajantha-bhat in #10754
Support for Flink's SpeculativeExecution in batch execution mode by @venkata91 in #10548
API: Update StatisticsFile javadoc by @rice668 in #10769
Flink: Remove JUnit4 dependency by @nastra in #10770
Docs: Add bodo to iceberg vendors by @ritwika314 in #10756
Core: Add estimateRowCount for Files and Entries Metadata Tables by @szehon-ho in #10759
Update checkstyle definition by @attilakreiner in #10681
API: Fix typo in RewriteManifestFiles java doc by @amogh-jahagirdar in #10778
Update .asf.yaml by @ajantha-bhat in #10767
Support building with Java 21 by @findepi in #10474
Infra, Docs: Publish Apache Iceberg 1.6.0 release by @jbonofre in #10752
Update iceberg version on site to 1.6.0 by @jbonofre in #10783
Hive: close the fileIO client when closing the hive catalog by @hussein-awala in #10771
Add jbonofre as collaborator on the project by @jbonofre in #10782
Docs: Make compatibility example consistent. by @emkornfield in #10781
mr：Fix issues 10639 by @lurnagao-dahua in #10661
Flink: backport PR #10331 and PR #10457 by @stevenzwu in #10757
Build: Bump software.amazon.awssdk:bom from 2.26.21 to 2.26.25 by @dependabot in #10800
Build: Bump nessie from 0.93.1 to 0.94.2 by @dependabot in #10798
Build: Bump net.snowflake:snowflake-jdbc from 3.17.0 to 3.18.0 by @dependabot in #10801
Build: Bump mkdocs-material from 9.5.29 to 9.5.30 by @dependabot in #10796
Flink: Disabling flaky test TestIcebergSourceFailover.testBoundedWithSavepoint by @pvary in #10802
Build: Bump mkdocs-awesome-pages-plugin from 2.9.2 to 2.9.3 by @dependabot in #10795
Kafka Connect: Runtime distribution with integration tests by @bryanck in #10739
Flink: improve snapshot compatibility check by comparing projected sort schema in SortKeySerializer by @stevenzwu in #10794
Flink: support limit pushdown in FLIP-27 source by @stevenzwu in #10748
Flink: Remove MiniClusterResource in Flink by @tomtongue in #10817
Docs: Use link addresses instead of descriptions in releases.md by @lurnagao-dahua in #10815
Build: Declare avro as an api dependency of iceberg-core by @devinrsmith in #10573
Flink: backport PR #10748 for limit pushdown by @stevenzwu in #10813
[DOCS] Fix header for entries metadata table by @gaborkaszab in #10826
Support Spark Column Stats by @huaxingao in #10659
Support for Flink's SpeculativeExecution in batch execution mode - Backport of PR #10548 by @venkata91 in #10776
Infra: Improve feature request template by @nastra in #10825
Core: (unit test) Replace the duplicated ALL_DATA_FILES with ALL_DELETE_FILES by @hsiang-c in #10836
Core: Adds Basic Classes for Iceberg Table Version 3 by @RussellSpitzer in #10760
Core: Allow SnapshotProducer to skip uncommitted manifest cleanup after commit by @grantatspothero in #10523
Flink: a few small fixes or tuning for range partitioner by @stevenzwu in #10823
Core: Drop support for Java 8 by @findepi in #10518
Build: Bump com.adobe.testing:s3mock-junit5 from 2.11.0 to 2.17.0 by @nastra in #10851
Core: Upgrade Jetty and Servlet API by @nastra in #10850
Build for Java 11 by @snazy in #10849
Build: Bump kafka from 3.7.1 to 3.8.0 by @dependabot in #10797
Build: Update baseline gradle plugin to 5.58.0 by @findepi in #10788
Flink: refactor sink tests to reduce the number of combinations with parameterized tests by @stevenzwu in #10777
Flink: backport PR #10823 for range partitioner fixup by @stevenzwu in #10847
Core: Remove reflection from TestParallelIterable by @findepi in #10857
Spec: Deprecate the file system table scheme. by @rdblue in #10833
Core, API: Add addNonDefaultSpec to UpdatePartitionSpec to not set the new partition spec as default by @shanielh in #10736
Build: Bump com.palantir.baseline:gradle-baseline-java from 5.58.0 to 5.61.0 by @dependabot in #10864
Build: Bump nessie from 0.94.2 to 0.94.4 by @dependabot in #10869
Build: Bump org.xerial:sqlite-jdbc from 3.46.0.0 to 3.46.0.1 by @dependabot in #10871
Build: Bump org.apache.commons:commons-compress from 1.26.0 to 1.26.2 by @dependabot in #10868
Build: Bump software.amazon.awssdk:bom from 2.26.25 to 2.26.29 by @dependabot in #10866
Build: Bump mkdocs-material from 9.5.30 to 9.5.31 by @dependabot in #10863
Build: Fix Scala compilation by @snazy in #10860
Build: Enable FormatStringAnnotation error-prone check by @findepi in #10856
Core: Use encoding/decoding methods for namespaces and deprecate Splitter/Joiner by @nastra in #10858
Aliyun: Replace assert usage with assertThat by @nastra in #10880
Core: Extract filePath comparator into it's own class by @deniskuzZ in #10664
Docs: Fix SQL in branching docs by @nakaken-churadata in #10876
API: Add SupportsRecoveryOperations mixin for FileIO by @amogh-jahagirdar in #10711
Spec: Clarify identity partition edge cases. by @emkornfield in #10835
Build: Bump org.testcontainers:testcontainers from 1.20.0 to 1.20.1 by @dependabot in #10865
Flink: add 1.20 support and remove 1.17 by @stevenzwu in #10881
Build: Add checkstyle rule to ban assert usage by @nastra in #10886
Build: Bump Apache Avro to 1.12.0 by @Fokko in #10879
Spec: Fix rendering of unified partition struct by @findepi in #10896
Docs: Nit Fix catalog name for S3 MRAP by @tomtongue in #10897
Fix stage-binaries.sh and docs after #10881 by @snazy in #10888
Flink: Remove deprecated RowDataUtil.clone method by @findepi in #10902
AWS: Fix flaky TestS3RestSigner by @nastra in #10898
AWS: Implement SupportsRecoveryOperations for S3FileIO by @amogh-jahagirdar in #10721
Remove deprecated APIs up to 1.7.0 by @nk1506 in #10818
Core, Flink: Fix build warnings by @nk1506 in #10899
Build: Bump Spark 3.5 to 3.5.2 by @manuzhang in #10918
Build: Bump com.google.errorprone:error_prone_annotations from 2.29.2 to 2.30.0 by @dependabot in #10915
Build: Bump org.awaitility:awaitility from 4.2.1 to 4.2.2 by @dependabot in #10912
Build: Bump datamodel-code-generator from 0.25.8 to 0.25.9 by @dependabot in #10917
Build: Bump nessie from 0.94.4 to 0.95.0 by @dependabot in #10910
Docs: Add Trademark symbol where appropriate by @Fokko in #10921
Build: Bump com.google.cloud:libraries-bom from 26.43.0 to 26.44.0 by @dependabot in #10916
Build: Bump org.apache.commons:commons-compress from 1.26.2 to 1.27.0 by @dependabot in #10914
Build: Bump software.amazon.awssdk:bom from 2.26.29 to 2.27.2 by @dependabot in #10913
Build: Bump org.xerial.snappy:snappy-java from 1.1.10.5 to 1.1.10.6 by @dependabot in #10911
[Docs, Infra] Mount local versioned doc branch for testing by @gaborkaszab in #10838
API: Fix JavaDoc typos in Transaction API by @hantangwangd in #10931
Core: (unit test) Set partition to the right PartitionKey by @hsiang-c in #10925
AWS, Core, Hive: Extract FileIO tracker/closer into separate class by @nastra in #10893
Build warning fix. by @nk1506 in #10938
Core: add JSON serialization for BaseFilesTable.ManifestReadTask, AllManifestsTable.ManifestListReadTask, and BaseEntriesTable.ManifestReadTask by @stevenzwu in #10735
AWS, Core: Slim down Jetty config for tests by @nastra in #10945
1 Cloudera blog in February 2023 by @SaketaChalamchala in #10947
Core: V3 Metadata Upgrade Validation and Testing by @leangjonathan in #10861
OpenAPI: Clarify in REST spec that server implementations of commit endpoints must fail with 400 for unknown requirements/updates by @amogh-jahagirdar in #10848
Build: Bump guava from 33.2.1-jre to 33.3.0-jre by @dependabot in #10960
Build: Bump org.springframework:spring-web from 5.3.37 to 5.3.39 by @dependabot in #10959
Build: Bump software.amazon.awssdk:bom from 2.27.2 to 2.27.7 by @dependabot in #10961
Docs: Update MRAP endpoint and add more docs by @singhpk234 in #9362
Flink: put everything together for range distribution in Flink sink by @stevenzwu in #10859
Flink: FLIP-27 IcebergSource builder missed a couple of configs compared to old FlinkSource: expose locality and plan parallelism by @stevenzwu in #10957
Build: Prevent implicit default locale/charset usage by @findepi in #10969
Core: Add ManifestWrite benchmark by @Fokko in #8637
AWS: Don't complete multipart upload on finalize for S3OutputStream by @jasonf20 in #10874
Build: Enable UnusedMethod error-prone check by @findepi in #10968
Upgrade to Gradle 8.10 by @jbonofre in #10976
#10668 - Support case-insensitivity for column names in PartitionSpec by @sl255051 in #10678
Core: Remove unused throws declarations by @findepi in #10974
Flink: deprecate ReaderFunction with a new Converter interface to simplify user experience by @stevenzwu in #10956
Core: Drop ParallelIterable's queue low water mark by @findepi in #10978
Check for minimal queue size in ParallelIterable by @findepi in #10977
Spark Action to Analyze table by @karuppayya in #10288
Core,AWS: Fix NPE in ResolvingFileIO when HadoopConf is not set by @munendrasn in #10872
Spark 3.5: Fix incorrect catalog loaded in TestCreateActions by @manuzhang in #10952
Flink: Maintenance - TriggerManager by @pvary in #10484
Flink: backport PR #10956 for converter interface that deprecates ReaderFunction by @stevenzwu in #10985
Flink: backport PR #10777 from 1.19 to 1.18 for sink test refactoring. by @stevenzwu in #10965
Spec: Minor modifications for v3 by @rdblue in #10948
Flink: backport PR #10859 for range distribution by @stevenzwu in #10990
Flink: Port #10484 to v1.20 by @pvary in #10989
Flink: Maintenance - TableChange refactor by @pvary in #10992
Flink: Port #10992 to v1.19 by @pvary in #10994
OpenAPI: Add endpoint field to CatalogConfig by @nastra in #10928
Spark 3.5: Add utility to load table state reliably by @aokolnychyi in #10984
AWS: Include http-auth-aws-crt module into iceberg-aws-bundle by @aajisaka in #10972
Build: Bump software.amazon.awssdk:bom from 2.27.7 to 2.27.12 by @dependabot in #11006
Build: Bump org.apache.commons:commons-compress from 1.27.0 to 1.27.1 by @dependabot in #11005
Build: Bump jetty from 11.0.22 to 11.0.23 by @dependabot in #11003
Build: Bump mkdocs-material from 9.5.31 to 9.5.33 by @dependabot in #11002
Docs: Fix wrong field name in CDC by @ebyhr in #11000
Docs: Rename Clickhouse to ClickHouse by @ebyhr in #10998
Flink: infer source parallelism for FLIP-27 source in batch execution mode by @stevenzwu in #10832
Flink: Fix duplicate data with upsert writer in case of aborted checkpoints by @zhongqishang in #10526
Introduces the new IcebergSink based on the new V2 Flink Sink Abstraction by @rodmeneses in #10179
Flink: Backport PR #10179 to Flink 1.20 for v2 sink by @rodmeneses in #11011
Core: Project data file stats only if there are equality deletes by @aokolnychyi in #11013
Build: Bump org.xerial:sqlite-jdbc from 3.46.0.1 to 3.46.1.0 by @dependabot in #11007
Docs: Add Druid docs url to sidebar by @techdocsmith in #10997
Flink: backport PR #10832 of inferring parallelism in FLIP-27 source by @stevenzwu in #11009
Add REST Compatibility Kit by @danielcweeks in #10908
Core: Generate realistic bounds in benchmarks by @aokolnychyi in #11022
OpenAPI, Build: Apply spotless to testFixtures source code by @danielcweeks in #11024
Docs: bump latest version to 1.6.1 by @cwsteinbach in #11036
OpenAPI: Add RemovePartitionSpecsUpdate REST update type by @amogh-jahagirdar in #10846
Build: Ignore benchmark output folders across all modules by @aokolnychyi in #11030
Core: Add benchmark for adding files by @aokolnychyi in #11029
Spark 3.5: Use FileGenerationUtil in PlanningBenchmark by @aokolnychyi in #11027
Flink: add unit tests for range distribution on bucket partition column by @stevenzwu in #11033
Kafka Connect: Disable publish tasks in runtime project by @bryanck in #11032
Flink: Backport PR #10526 to v1.18 and v1.20 by @zhongqishang in #11018
Build: Bump mkdocs-material from 9.5.33 to 9.5.34 by @dependabot in #11062
Build: Bump com.google.errorprone:error_prone_annotations from 2.30.0 to 2.31.0 by @dependabot in #11055
API: implement types timestamp_ns and timestamptz_ns by @jacobmarble in #9008
Core: Refactor ZOrderByteUtils by @ajantha-bhat in #10624
Initial committer guidelines and requirements for merging by @emkornfield in #10780
Flink: Fix compile warning by @ajantha-bhat in #11072
Docs: Fix Flink 1.20 support versions by @manuzhang in #11065
Build: Enable more error-prone checks by @findepi in #11078
open-api: Fix compile warnings for testFixtures by @ajantha-bhat in #11071
Spark 3.3, 3.4: Parallelize reading files in migrate procedures by @manuzhang in #11043
Spark 3.5: Mandate identifier fields when create_changelog_view for table contain unsortable columns by @dramaticlly in #11045
Core: Fix setting hasNewDataFile flag in MergingSnapshotProducer by @aokolnychyi in #11088
Docs: Document accessing instance variables by @aokolnychyi in #11087
Build: Bump jetty from 11.0.23 to 11.0.24 by @dependabot in #11096
Spec: Fix rendering of partition stats file spec by @ajantha-bhat in #11068
Build: Bump software.amazon.awssdk:bom from 2.27.12 to 2.27.21 by @dependabot in #11098
Build: Bump io.netty:netty-buffer from 4.1.112.Final to 4.1.113.Final by @dependabot in #11097
Spark 3.3, 3.4: Fix incorrect catalog loaded in TestCreateActions by @manuzhang in #11049
Core: Fix the behavior of IncrementalFileCleanup when expire a snapshot by @hantangwangd in #10983
Core, Kafka, Spark: Use AssertJ instead of JUnit assertions by @nastra in #11102
Add rmoff blogs by @rmoff in #11069
Upgrade to Gradle 8.10.1 by @jbonofre in #11104
Build: Remove unused variables, fields and parameters by @findepi in #11101
Add Scan Planning Endpoints to open api spec by @rahil-c in #9695
Kafka Connect: Docs on configuring the sink by @bryanck in #10746
Kafka Connect: Stop commits on terminated coordinator by @bryanck in #10814
Build: Upgrade google-java-format to 1.22.0 by @manuzhang in #11050
Flink: Maintenance - Lock remover by @pvary in #11010
Docs: Update Project links to include contributing and REST spec by @danielcweeks in #11114
Flink: Port #10484 to v1.19 (#11010) by @pvary in #11117
OpenAPI: Fix YAML example and value json formatting by @danielcweeks in #11119
Core: Parallelize manifest writing for many new files by @aokolnychyi in #11086
API, Core: Add manifestLocation API to ContentFile by @amogh-jahagirdar in #11044
Core: Allow servers to express supported endpoints via endpoint field in ConfigResponse by @nastra in #10929
Hive: Add View support for HIVE catalog by @nk1506 in #9852
Spark 3.4: Action to compute table stats by @karuppayya in #11106
Docs: Fix missing options for remove_orphan_files procedure by @manuzhang in #11080
Flink: Increase the number of checkpoints from 4 to 6 to fix flakiness. by @stevenzwu in #11121
Spark 3.4: Add utility to load table state reliably by @dramaticlly in #11115
Build: switch to slf4j-simple 2.x for test implementation dependency by @stevenzwu in #11001
Core: Update metadata location without updating lastUpdatedMillis by @nastra in #11151
Kafka Connect: separate CI workflow by @bryanck in #11075
Core: Move internal struct projection to SupportsIndexProjection by @rdblue in #11132
Core: Add explicit JSON parser for LoadTableResponse by @nastra in #11148
Build: Add .java-version to gitignore. by @zhongyujiang in #11167
Docs: Clarified Partition Transform by @jasonmfehr in #8337
API, Core: Enable dropping rewritten delete files in RowDelta by @aokolnychyi in #11166
Docs: field_id in name serialisation spec should read field-id by @jonaswk in #11135
AWS: Bump AWS SDK to version 2.28.5 by @sullis in #11170
Build: Bump org.xerial.snappy:snappy-java from 1.1.10.6 to 1.1.10.7 by @dependabot in #11140
Supplement test case for RollbackToTimestampProcedure by @hantangwangd in #11171
Docs: Uppercase keyword in branching by @ebyhr in #11172
Build: Bump org.apache.httpcomponents.client5:httpclient5 from 5.3.1 to 5.4 by @dependabot in #11186
Build: Bump tez010 from 0.10.3 to 0.10.4 by @dependabot in #11183
Build: Bump nessie from 0.95.0 to 0.97.1 by @dependabot in #11184
Build: Bump com.google.cloud:libraries-bom from 26.44.0 to 26.47.0 by @dependabot in #11185
Build: Bump org.roaringbitmap:RoaringBitmap from 1.2.1 to 1.3.0 by @dependabot in #11187
API: Deprecate ContentFile#path API and add location API which returns String by @amogh-jahagirdar in #11092
[Docs] Update AWS docs to add more AWS engines that supports iceberg by @singhpk234 in #11192
API, AWS: Retry S3InputStream reads by @amogh-jahagirdar in #10433
AWS: Fix AWS doc URL by @laurentgo in #11198
Build: Bump mkdocs-macros-plugin from 1.0.5 to 1.2.0 by @dependabot in #11189
Core: Add rewritten delete files to write results by @aokolnychyi in #11203
Core: Support iterating over positions in PositionDeleteIndex by @aokolnychyi in #11202
Spark: Added merge schema as spark configuration by @aleenamg21-1 in #9640
Core: Support merging in PositionDeleteIndex by @aokolnychyi in #11208
Core: Remove unused code for streaming position deletes by @wypoon in #11175
Upgrade to Gradle 8.10.2 by @jbonofre in #11212
Core: Add a util to compute partition stats by @ajantha-bhat in #11146
Parquet: update PruneColumns to inherit from TypeWithSchemaVisitor to have Iceberg type by @aihuaxu in #11179
Core: Replace use of CharSequenceMap in DeleteFileIndex with Map<String, PositionDeletes> by @amogh-jahagirdar in #11199
[Minor][Test] Fix TestFastAppend.testAddManyFiles() by @anuragmantri in #11218
Spark: Deprecate SparkAppenderFactory by @ajantha-bhat in #11076
Build: Bump guava from 33.3.0-jre to 33.3.1-jre by @dependabot in #11230
Build: Bump software.amazon.awssdk:bom from 2.28.5 to 2.28.11 by @dependabot in #11229
Build: Bump io.delta:delta-standalone_2.12 from 3.2.0 to 3.2.1 by @dependabot in #11228
Build: Bump junit-platform from 1.10.3 to 1.11.1 by @dependabot in #11227
Build: Bump io.delta:delta-spark_2.12 from 3.2.0 to 3.2.1 by @dependabot in #11225
Core: Improve error handling when parsing view representations by @nastra in #11236
REST: Handle Requests with Page Sizes Exceeding Available Number of Namespaces /Tables/Views by @rcjverhoef in #11143
Core: Support combining position deletes during writes by @aokolnychyi in #11222
Core: Add DataFileSet / DeleteFileSet by @nastra in #11195
Build: Bump nessie from 0.97.1 to 0.99.0 by @dependabot in #11224
ThreadPools introduce newExitingWorkerPool and newFixedThreadPool for clearer semantics by @fengjiajie in #11073
Core: Deprecate legacy ways for loading position deletes by @aokolnychyi in #11242
AWS: Set better defaults for S3 retry behaviour by @ookumuso in #11052
Puffin: Document stats ndv value representation by @findepi in #10793
Build: Bump mkdocs-material from 9.5.34 to 9.5.38 by @dependabot in #11233
Build: Bump org.eclipse.microprofile.openapi:microprofile-openapi-api from 3.1.1 to 3.1.2 by @dependabot in #11182
Build: Bump org.xerial:sqlite-jdbc from 3.46.1.0 to 3.46.1.3 by @dependabot in #11231
Build: Update baseline-java 5.69.0 by @findepi in #11252
Build: Forbid implicit case fall-through without a comment and enable couple more recommendable error-prone checks by @findepi in #11251
API, Core: Add default value APIs and Avro implementation by @wmoustafa in #9502
AWS: Make sure overridden configurations are applied by @hsiang-c in #11274
Build: Bump com.google.cloud:libraries-bom from 26.47.0 to 26.48.0 by @dependabot in #11271
Core: Fix UnicodeUtil#truncateStringMax returns malformed string. by @zhongyujiang in #11161
Core: Add internal Avro reader by @rdblue in #11108
Arrow: Remove dead code by @wypoon in #11276
Spec: Add v3 types and type promotion by @rdblue in #10955
Build: Bump mkdocs-material from 9.5.38 to 9.5.39 by @dependabot in #11272
Build: Bump com.google.errorprone:error_prone_annotations from 2.31.0 to 2.33.0 by @dependabot in #11270
Build: Bump junit-platform from 1.11.1 to 1.11.2 by @dependabot in #11266
Build: Bump org.testcontainers:testcontainers from 1.20.1 to 1.20.2 by @dependabot in #11265
Build: Bump io.netty:netty-buffer from 4.1.113.Final to 4.1.114.Final by @dependabot in #11269
Build: Bump software.amazon.awssdk:bom from 2.28.11 to 2.28.16 by @dependabot in #11268
Build: Bump jackson-bom from 2.14.2 to 2.18.0 by @dependabot in #11226
Build: Bump datamodel-code-generator from 0.25.9 to 0.26.1 by @dependabot in #11234
Build: Bump software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin from 2.0.0 to 2.0.1 by @dependabot in #9705
Build: Bump net.snowflake:snowflake-jdbc from 3.18.0 to 3.19.0 by @dependabot in #11057
Build: Bump org.apache.hadoop.thirdparty:hadoop-shaded-guava from 1.2.0 to 1.3.0 by @dependabot in #11061
Arrow: Deprecate unused fixed width binary reader classes by @wypoon in #11292
Api, Spark: Make StrictMetricsEvaluator not fail on nested column predicates by @zhongyujiang in #11261
Build: Use the active shadow plugin by @ajantha-bhat in #11315
Build: Bump software.amazon.awssdk:bom from 2.28.16 to 2.28.21 by @dependabot in #11311
Build: Bump org.apache.datasketches:datasketches-java from 6.0.0 to 6.1.1 by @dependabot in #11307
Core: Switch usage to DataFileSet / DeleteFileSet by @nastra in #11158
[AWS] S3FileIO - Add Cross-Region Bucket Access by @munendrasn in #11259
Core: Deprecate ContentCache.invalidateAll by @findepi in #10494
Remove unnecessary copying of FileScanTask by @huaxingao in #11319
Core: Rename DeleteFileHolder to PendingDeleteFile / Optimize duplicate data/delete file detection by @nastra in #11254
Core: Fix version number in deprecation note for invalidateAll by @findepi in #11325
Bump junit from 5.10.1 to 5.11.1 by @tomtongue in #11262
Support wasb[s] paths in ADLSFileIO by @mrcnc in #11294
Make connect compatable with kafka plugin.discovery by @joswlv in #10536
Updating SparkScan to only read Apache DataSketches by @jeesou in #11035
add checkClassName regex func by @ryanjclark in #11303
Remove repeated 'for' by @ebyhr in #11338
AWS: Fix S3InputStream retry policy by @edgarRd in #11335
Small fix to TestSerializableTypes.java by @aihuaxu in #11342
Core: fix NPE with HadoopFileIO because FileIOParser doesn't serialize Hadoop configuration by @stevenzwu in #10926
Revert "Support wasb[s] paths in ADLSFileIO" by @RussellSpitzer in #11344
Flink: make FLIP-27 default in SQL and mark the old FlinkSource as deprecated by @stevenzwu in #11345
Flink: disable the flaky range distribution bucketing tests for now by @stevenzwu in #11347
OpenAPI: Standardize credentials in loadTable/loadView responses by @nastra in #10722
Core: Add credentials to loadTable / loadView responses by @nastra in #11173
Spark: Add RewriteTablePath action interface by @laithalzyoud in #10920
AWS: Update ObjectStorageLocationProvider hash to optimize for S3 performance by @ookumuso in #11112
Flink: Add IcebergSinkBuilder interface allowed unification of most of operations on FlinkSink and IcebergSink Builders by @arkadius in #11305
Build: Bump parquet from 1.13.1 to 1.14.3 by @dependabot in #11264
Build: Bump com.palantir.baseline:gradle-baseline-java from 5.69.0 to 5.72.0 by @dependabot in #11362
Build: Bump com.google.errorprone:error_prone_annotations from 2.33.0 to 2.34.0 by @dependabot in #11360
Build: Bump software.amazon.awssdk:bom from 2.28.21 to 2.28.26 by @dependabot in #11359
Build: Bump com.google.cloud:libraries-bom from 26.48.0 to 26.49.0 by @dependabot in #11363
Core: code clean up for deleteRemovedMetadataFiles by @leesf in #11352
Arrow: Fix indexing in Parquet dictionary encoded values readers by @wypoon in #11247
Spark 3.5: Update Spark to use planned Avro reads by @rdblue in #11299
Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction by @dramaticlly in #9724
Spark: Randomize view/function names in testing by @nastra in #11381
Spark 3.4: Randomize view/function names in testing by @nastra in #11382
Spark 3.4: Action to remove dangling deletes by @dramaticlly in #11377
Reset Spark Conf for each test in TestCompressionSettings by @huaxingao in #11333
Spec: Adds Row Lineage by @RussellSpitzer in #11130
AWS: Support S3 directory bucket listing by @stubz151 in #11021
Core: Add LoadCredentialsResponse class/parser by @nastra in #11339
OpenAPI: Add endpoint to retrieve valid credentials for a given table by @nastra in #11281
AWS: Use testcontainers-minio instead of S3Mock by @sullis in #11349
Kafka Connect: Include third party licenses and notices in distribution by @bryanck in #10829
Deprecate iceberg-pig by @jbonofre in #11379
Core: Track data files by spec id instead of full PartitionSpec by @nastra in #11323
Spark 3.5: Don't change table distribution when only altering local order by @manuzhang in #10774
Spec: Fix table of content generation by @ajantha-bhat in #11067
[KafkaConnect] Fix RecordConverter for UUID and Fixed Types by @singhpk234 in #11346
Core: Snapshot summary map must have operation key by @kevinjqliu in #11354
Core: Update TableMetadataParser to ensure all streams closed by @erik-grepr in #11220
Build: Bump Spark 3.4 to 3.4.4 by @manuzhang in #11366
Build: Bump junit from 5.11.1 to 5.11.3 by @dependabot in #11401
Build: Bump software.amazon.awssdk:bom from 2.28.26 to 2.29.1 by @dependabot in #11400
Core: Remove one comment from FastAppend by @gaborkaszab in #10995
Build: Bump junit-platform from 1.11.2 to 1.11.3 by @dependabot in #11402
Build: Bump org.xerial:sqlite-jdbc from 3.46.1.3 to 3.47.0.0 by @dependabot in #11407
Build: Bump software.amazon.s3.accessgrants:aws-s3-accessgrants-java-plugin from 2.2.0 to 2.3.0 by @dependabot in #11405
Build: Bump net.snowflake:snowflake-jdbc from 3.19.0 to 3.19.1 by @dependabot in #11406
Build: Bump testcontainers from 1.20.2 to 1.20.3 by @dependabot in #11404
Build: Bump com.google.errorprone:error_prone_annotations from 2.34.0 to 2.35.1 by @dependabot in #11403
Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.7 by @dependabot in #11399
Build: Bump mkdocs-material from 9.5.39 to 9.5.42 by @dependabot in #11398
Flink: Fix disabling flaky range distribution bucketing tests by @manuzhang in #11410
Bump Azurite to the latest version by @Fokko in #11411
Build: Bump datamodel-code-generator from 0.26.1 to 0.26.2 by @dependabot in #11356
Revert "Core: Snapshot summary map must have operation key" by @kevinjqliu in #11409
Aliyun: Remove spring-boot dependency by @jbonofre in #11291
Build: Bump com.azure:azure-sdk-bom from 1.2.25 to 1.2.28 by @dependabot in #11267
Spark 3.5: Fix flaky test due to deleting temp directory failure by @manuzhang in #10811
Core: Add portable Roaring bitmap for row positions by @aokolnychyi in #11372
Flink 1.20: Update Flink to use planned Avro reads by @jbonofre in #11386
open-api: Fix testFixtures dependencies by @ajantha-bhat in #11422
Core: use ManifestFiles.open when possible by @dramaticlly in #11414
GCS: Refresh vended credentials by @nastra in #11282
New iceberg blogs nov 2024 by @AlexMercedCoder in #11424
AWS: Refresh vended credentials by @nastra in #11389
Dependencies: Bump Hadoop to 3.4.1 by @Fokko in #11428
Core: Remove credentials from LoadViewResponse by @nastra in #11432
OpenAPI: Remove credentials from LoadViewResult by @nastra in #11433
API: Add compatibility checks for Schemas with default values by @rdblue in #11434
1.7.0-RC1 Cherry-picks by @RussellSpitzer in #11466

New Contributors

@venkata91 made their first contribution in #10548
@ritwika314 made their first contribution in #10756
@shanielh made their first contribution in #10736
@nakaken-churadata made their first contribution in #10876
@SaketaChalamchala made their first contribution in #10947
@leangjonathan made their first contribution in #10861
@sl255051 made their first contribution in #10678
@techdocsmith made their first contribution in #10997
@rmoff made their first contribution in #11069
@jasonmfehr made their first contribution in #8337
@jonaswk made their first contribution in #11135
@laurentgo made their first contribution in #11198
@aleenamg21-1 made their first contribution in #9640
@aihuaxu made their first contribution in #11179
@rcjverhoef made their first contribution in #11143
@ookumuso made their first contribution in #11052
@joswlv made their first contribution in #10536
@jeesou made their first contribution in #11035
@ryanjclark made their first contribution in #11303
@laithalzyoud made their first contribution in #10920
@arkadius made their first contribution in #11305
@leesf made their first contribution in #11352
@stubz151 made their first contribution in #11021
@erik-grepr made their first contribution in #11220

Full Changelog: apache-iceberg-1.6.0...apache-iceberg-1.7.0

apache/iceberg apache-iceberg-1.7.0 Apache Iceberg 1.7.0 on GitHub

What's Changed

New Contributors

apache/iceberg apache-iceberg-1.7.0
Apache Iceberg 1.7.0

on GitHub