apache/iceberg-python pyiceberg-0.12.0 on GitHub

What's Changed

Catalog tests around nonexistent tables + namespaces by @rambleraptor in #2990
Upgrade to RAT 0.17 by @kevinjqliu in #2992
Fix: make test-coverage to properly enable coverage for all test suites by @kevinjqliu in #2989
feat: Add tell() to OutputStream writers by @geruh in #2998
infra: upgrade pytest and use pytest_lazy_fixtures by @kevinjqliu in #2978
fix: ignore local config during tests by @geruh in #3006
fix: Hive catalog test failure on CPython 3.13.12 by @geruh in #3043
docs: release docs improvements by @kevinjqliu in #3030
Add catalog properties for Catalog Tests by @rambleraptor in #2982
feat: Add Geometry & Geography Types by @SaymV in #2859
infra: set github actions max-parallel to 15 by @kevinjqliu in #3057
fix: Accept all valid http method for IRC's ConfigResponse by @sumanth-manchala in #3010
feat(cli): add --prefix option for REST catalog by @mrutunjay-kinagi in #3061
CI: Add CodeQL workflow for GitHub Actions security scanning by @kevinjqliu in #3060
Surface too many requests exception by @ms32035 in #3048
fix: add Field alias for SetPartitionStatisticsUpdate.partition_stati… by @shiwk in #3035
fix ssl deactivation from catalog properties #2985 by @MenelikV in #3012
Optimization: Prune manifest in snapshot overwrite operations by @gabeiglio in #3011
Add concurrency safety validation checks by @gabeiglio in #3049
infra: simplify Makefile and uv by @kevinjqliu in #2996
fix: Extract ADLS account_name from URI hostname in FsspecFileIO by @antonlin1 in #3005
Fix conftest dependency breaking nightly wheel build by @kevinjqliu in #3078
chore: Remove docs from uv default groups by @geruh in #3073
fix: show helpful error when hive catalog config is missing uri by @jx2lee in #3077
chore(ci): add explicit least-privilege workflow permissions by @kevinjqliu in #3082
fix: python version in release docs workflow by @geruh in #3102
chore: use ubuntu-slim for lightweight jobs by @zhjwpku in #3107
Fix pyparsing diagnostic warning on Table import by @kevinjqliu in #3081
Documentation Adjustment for s3.profile-name and client.profile-name by @thomas-pfeiffer in #3085
docs: more release docs improvements by @kevinjqliu in #3104
feat(cli): replace REST --prefix with --warehouse by @kevinjqliu in #3080
Update slate config for PRs by @nssalian in #3125
Support storage-credentials in REST catalog LoadTableResult by @rcjverhoef in #3042
feat(rest): add sigv4 retry configuration defaults by @mrutunjay-kinagi in #3063
[View Support] Add ViewMetadata read support and create_view to REST Catalog by @rambleraptor in #2154
fix: handle empty HEAD response by @AntonioYuen in #3041
Switch to uv-pre-commit for lockfile management by @kevinjqliu in #3141
Improve lockfile management documentation and enforce uv.lock freshness in CI by @kevinjqliu in #3144
Fix for ns timestamp non utc by @nssalian in #3142
Skip catalog loading for the version command by @jx2lee in #3146
docs: add type mapping tables between PyIceberg and PyArrow by @iamluan in #3098
fix(glue): Support create_table for S3 Tables federated databases by @jamesbornholt in #3058
ci: pin third-party actions to Apache-approved SHAs by @kevinjqliu in #3172
Infra: Remove GitHub Actions from Dependabot configuration by @kevinjqliu in #3187
Infra: Pin pypa/cibuildwheel version by @ebyhr in #3189
chore(ci): pin actions in PR-triggered workflows by @kevinjqliu in #3194
chore(ci): pin actions in non-PR-triggered workflows by @kevinjqliu in #3193
ci: improve github workflows by @kevinjqliu in #3204
ci: add tag as inline comment by @kevinjqliu in #3213
ci: fix nightly and release by @kevinjqliu in #3215
fix: Handle optional properties in load namespace properties response by @geruh in #3169
CI: Lock REST Fixture to 1.10.1 by @Fokko in #3240
fix zizmor ci issue by bumping codeql-action to v4.35.2 by @geruh in #3245
ci: fix zizmor workflow by @kevinjqliu in #3243
Fix ManifestEntry.snapshot_id setter writing to wrong index by @lawofcycles in #3257
Fix pa.Schema type annotations in schema_to_pyarrow by @davidbzhao in #3261
perf: build partition filter with balanced tree to avoid RecursionError by @yingjianwu98 in #3264
bug: NoopCatalog should return False, not throw error by @rambleraptor in #3284
Reading upper/lower bounds values with type promotions by @rambleraptor in #3293
Fix DELETED manifest entry snapshot_id in OverwriteFiles by @lawofcycles in #3237
infra: add python 3.14 support by @afeldman1 in #3299
fix: SaslServer busy-loop causes process hang after test suite by @kevinjqliu in #3305
perf: Hoist table_metadata at remaining repeat-access sites in snapshot update by @rynewang in #3301
Fix deepcopy for And, Or, and Not expressions by @robreeves in #3295
Add support for 'overwrite' option in register_table by @ebyhr in #3290
Document TableAlreadyExistsError in catalog.rename_table by @ebyhr in #3289
Initial work for file format writer API by @nssalian in #3119
fix:update config during table refresh by @yingjianwu98 in #3327
Remove unused code by @ebyhr in #3326
Update NOTICE year 2025 -> 2026 by @kevinjqliu in #3332
CI: Use specific patch versions in workflow action comments by @kevinjqliu in #3331
feat: add load_view to REST catalog by @spr0els in #3224
Add support for registering views by @ebyhr in #3288
fix(tests): isolate state in test_write_optional_list by @paultmathew in #3334
add papermill-based notebook tests for pyiceberg examples by @federicsp in #3330
REST: Add pagination support for list_views by @ebyhr in #3349
fix(fsspec): handle zero-byte files in len by @fallintoplace in #3353
Add @typing.override annotations to catalog and FileIO implementations by @vishnuprakaz in #3360
Fix typo in test_add_files by @ebyhr in #3378
fix: avoid TSaslClientTransport reuse to eliminate server-side SASL noise by @barking-code in #3357
feat(2152): support pa.RecordBatchReader in Table.append/overwrite by @paultmathew in #3335
REST: Rename rest-scan-planning-enabled to scan-planning-mode by @ebyhr in #3376
fix(table): avoid committing update builders after exceptions by @fallintoplace in #3354
perf(add_files): stream manifest entries for duplicate-files check by @laichunpongben in #3287
Build: Update actions/upload-artifact to v7.0.1 by @ebyhr in #3401
docs: fix New York longitude sign in partial overwrite example by @fallintoplace in #3403
Test: Replace rest-scan-planning-enabled with scan-planning-mode by @ebyhr in #3391
feat(view): View object API by @gabeiglio in #3338
REST: Add pagination support for list_namespaces by @ebyhr in #3347
REST: Add pagination support for list_tables by @ebyhr in #3348
Make View.sql_for case-insensitive and return None for unknown dialect by @ebyhr in #3407
REST: Add integration test for views by @ebyhr in #3406
REST: Add support for page-size in list_namespaces, list_tables, and list_views by @ebyhr in #3377
REST: Allow Identifier type in drop_view method by @ebyhr in #3408
feat(cli): deprecate version command in favor of --version flag by @chrisqiqiu in #3206
Fix walrus truthiness on metrics bounds and identity-partition projection by @kevinjqliu in #3412
Reject empty source-ids in PartitionField / SortField by @kevinjqliu in #3411
Fix precision loss in large integral string conversions by @fallintoplace in #3405
Chore: Updated PyArrow Dependencies by @CalebWeisgerber in #3362
Fix typo by @ebyhr in #3424
docs: add Iceberg security model by @sungwy in #3425
Add rambleraptor to collaborators list by @kevinjqliu in #3456
Make View.version return None for unknown id by @ebyhr in #3445
DOCS: Fix layout of sqlite warning by @ebyhr in #3459
Fix misleading file extension for partition statistics by @ebyhr in #3472
fix(literals): return long bounds for decimal conversion by @abnobdoss in #3470
Add defaults to ViewVersion fields by @ebyhr in #3458
Update astral-sh/setup-uv to v8.2.0 by @ebyhr in #3510
feat: add dictionary_columns to to_arrow() / to_arrow_batch_reader() for memory-efficient reads by @GayathriSrividya in #3461
Refactor: extract BaseScan and ManifestGroupPlanner by @smaheshwar-pltr in #3511
Add more examples for pyiceberg view by @MonkeyCanCode in #3414
feat: Add REST loadCredentials support by @kevinjqliu in #3499
fix: correct NOT STARTS WITH projection for truncated partitions by @GayathriSrividya in #3528
Fix NotStartsWith residual evaluation to return correct result by @ebyhr in #3503
Return overflow sentinel in LongLiteral.to(FloatType) by @ebyhr in #3502
Fix string-based starts_with and not_starts_with methods by @ebyhr in #3501
Return bounds sentinels for long date literals by @kevinjqliu in #3546
Fix duplicate filtering path in Arrow task batches by @GayathriSrividya in #3448
docs: clean OSS and COS configuration tables by @Sidra-009 in #3452
Fix strict NotEqualTo/NotIn pruning with partial nulls or NaNs by @tanmayrauth in #3521
Add strict NotEqualTo/NotIn null and NaN tests by @kevinjqliu in #3547
CI: Check ASF action allowlist on every PR by @ebyhr in #3550
fix(datetime): raise specific "Missing zone offset" error in timestamptz_to_nanos by @vishnuprakaz in #3505
Extract DeletionVector logic from PuffinFile by @ebyhr in #3491
Feature: Incremental Append Scan by @smaheshwar-pltr in #3512
Make scan_plan_helper internal (#3541) by @tanmayrauth in #3553
Make manifest cache size configurable by @kris-gaudel in #2993
Rename _scan_plan_helper method to _plan_manifest_entries by @ebyhr in #3579
Infra: Set timeout for jobs in python-ci.yml by @ebyhr in #3581
Reject unsupported identity transform types by @zhjwpku in #3517
Prune dependency upper bounds by @kevinjqliu in #3586
CI: Pass args through docker-compose by @Fokko in #3587
Fix strip spec-mandated DV blob framing before deserializing by @ebyhr in #3576
Docs: Add missing _downcast_ns_timestamp_to_us to ArrowScan docstring by @qzyu999 in #3594
fix: require pyarrow>=18.0.0 for native UUID type support by @anxkhn in #3606
Build: Add .venv/ to .gitignore by @qzyu999 in #3626
fix(ci): correct stale-issue-label and exempt security issues by @anxkhn in #3605
fix: preserve manifest min sequence number of 0 by @anxkhn in #3660
fix: raise ValidationError for decimal precision outside 1-38 range by @PLTNGM in #3585
Support zstd blob decompression in Puffin by @ebyhr in #3575
Add basic assumptions to AGENTS.md by @rambleraptor in #3552
Infra: Group github/codeql-action bumps into a single dependabot PR by @ebyhr in #3648
feat: support pyarrow float16 by widening to float on read/write by @anxkhn in #3590
fix: Pass metadata location to StaticTable FileIO for scheme inference by @kdn36 in #3592
fix: preserve write_default when applying a name mapping by @anxkhn in #3601
Only run integration tests on Python changes by @rambleraptor in #3610
Implement ParquetFormatModel and update write_file to use the format API by @nssalian in #3381
fix: preserve dictionary encoding in to_arrow_batch_reader by @avison9 in #3595
fix: add iceberg_type column for SqlCatalog by @rchowell in #3263
docs: fix undefined variable in schema evolution transaction example by @anxkhn in #3607
Make partition expression evaluation stateless by @dossett in #3664
Convert FileFormatModel from ABC to typing.Protocol by @nssalian in #3692
chore(ci): Update ASF allowlist check action dependency to track versioned releases by @kevinjqliu in #3739
build: bump pyiceberg-core from 0.9.1 to 0.10.1 by @VVKot in #3711
Remove stale VariantType transform TODO by @zhjwpku in #3742
ci: enable ASF Copilot code review by @kevinjqliu in #3740
fix: preserve empty property values in CLI by @mj006648 in #3745
CI: Add Windows unit test job by @qzyu999 in #3723
refactor: simplify CLI property lookup by @mj006648 in #3750
Update release docs for uv lock by @rambleraptor in #3749
INFRA: Use PR titles for squash commits by @kevinjqliu in #3755
Fix flaky moto_server fixture by using an OS-assigned ephemeral port by @puchengy in #3759
Docs: Update Iceberg Slack link by @ebyhr in #3768
Add --purge option to drop table CLI by @jx2lee in #3718
Release verification script by @rambleraptor in #3777
Fix/sqlcatalog iceberg type filter by @avison9 in #3709
Make metrics evaluation stateless by @dossett in #3665
Add commit retry and concurrency validation for writes by @lawofcycles in #3320
Add docstrings to pyiceberg/table/inspect.py by @daviddallakyan2005 in #3802
feat: add S3 SSE configs (FsspecFileIO only) by @achasnovskiy in #3173
Add Spark interop test for reading Puffin deletion vectors by @moomindani in #3476
fix: correct class name in Long{AboveMax,BelowMin} type-change error by @anxkhn in #3634
fix(fsspec): parse S3 virtual addressing as a boolean by @fallintoplace in #3492
fix: use removeprefix to decode DynamoDB namespace property keys by @anxkhn in #3602
fix: raise TypeError for unsupported partition source type by @anxkhn in #3617
fix: compare timestamps for partitions metadata last_updated_at by @anxkhn in #3603
test: fix operator precedence in manifest v2 assertions by @anxkhn in #3612
fix: pad sub-microsecond digits when parsing nanosecond timestamps by @anxkhn in #3614
fix: stop calling deprecated PuffinFile.to_vector in Spark DV test by @daviddallakyan2005 in #3804
bug: fix delete_data_file on partitioned tables by @gabeiglio in #3780
Validate replaced data files during commit retries by @kevinjqliu in #3811
Infra: Enable pull request auto-merge by @kevinjqliu in #3815
fix(decimal): use minimal byte length for negative powers of two by @vishnuprakaz in #3746
Fix residual NotNaN for null partition values by @BharatDeva in #3689
Fix strict evaluation for negative record counts by @kevinjqliu in #3831
Fail when explicitly deleted data file is missing by @rambleraptor in #3818

New Contributors

@SaymV made their first contribution in #2859
@mrutunjay-kinagi made their first contribution in #3061
@shiwk made their first contribution in #3035
@MenelikV made their first contribution in #3012
@antonlin1 made their first contribution in #3005
@jx2lee made their first contribution in #3077
@zhjwpku made their first contribution in #3107
@thomas-pfeiffer made their first contribution in #3085
@rcjverhoef made their first contribution in #3042
@AntonioYuen made their first contribution in #3041
@iamluan made their first contribution in #3098
@jamesbornholt made their first contribution in #3058
@lawofcycles made their first contribution in #3257
@davidbzhao made their first contribution in #3261
@afeldman1 made their first contribution in #3299
@rynewang made their first contribution in #3301
@robreeves made their first contribution in #3295
@spr0els made their first contribution in #3224
@paultmathew made their first contribution in #3334
@federicsp made their first contribution in #3330
@barking-code made their first contribution in #3357
@laichunpongben made their first contribution in #3287
@chrisqiqiu made their first contribution in #3206
@CalebWeisgerber made their first contribution in #3362
@abnobdoss made their first contribution in #3470
@GayathriSrividya made their first contribution in #3461
@MonkeyCanCode made their first contribution in #3414
@Sidra-009 made their first contribution in #3452
@tanmayrauth made their first contribution in #3521
@qzyu999 made their first contribution in #3594
@PLTNGM made their first contribution in #3585
@kdn36 made their first contribution in #3592
@avison9 made their first contribution in #3595
@dossett made their first contribution in #3664
@VVKot made their first contribution in #3711
@mj006648 made their first contribution in #3745
@daviddallakyan2005 made their first contribution in #3802
@achasnovskiy made their first contribution in #3173
@moomindani made their first contribution in #3476
@BharatDeva made their first contribution in #3689

Full Changelog: pyiceberg-0.11.1...pyiceberg-0.12.0