apache/iceberg-python pyiceberg-0.8.0rc1

pyiceberg-0.8.0-rc1

on GitHub

What's Changed

PRs

• Update PyIceberg Verify Release doc by @chinmay-bhat in #976
• DOCS: Add Github Actions Screenshots to Release Notes by @sungwy in #975
• Bump up version in dev Dockerfile and Issue Template by @ndrluis in #981
• Fix pydantic warning in the commit process by @ndrluis in #972
• Bump up Iceberg version to 1.6.0 by @ndrluis in #982
• Bug Fix: use appropriate partition spec for delete by @sungwy in #984
• [Bug Fix]Use self.table_metadata when in transaction by @HonahX in #985
• DOCS: Add more post release notes by @sungwy in #983
• Treat warning as error in CI/Dev by @ndrluis in #973
• Use 'strtobool' instead of comparing with a string. by @ndrluis in #988
• Fix: accept empty arrays in struct field lookup by @grobgl in #997
• Add ndrluis as collaborator by @sungwy in #1009
• Fix list namespace response in rest catalog by @ndrluis in #995
• Pyarrow IO property for configuring large v small types on read by @sungwy in #986
• Update metadata-log for non-rest catalogs by @soumya-ghosh in #977
• Exclude Python 3.9.7 due to import error in catalog module by @ndrluis in #526
• Deprecate rest.authorization-url in favor of oauth2-server-uri by @ndrluis in #962
• Allow setting write.parquet.row-group-limit by @Fokko in #1016
• Deprecate Redundant Identifier Support in TableIdentifier, and row_filter by @sungwy in #994
• Fix: Handle Empty RecordBatch within _task_to_record_batches, fix correctness issue with positional deletes by @sungwy in #1026
• Fix overwrite when filtering all the data by @ndrluis in #1023
• Allow setting write.parquet.page-row-limit by @Fokko in #1017
• DOCS: Remove older row for write.parquet.row-group-limit by @sungwy in #1030
• Improve test_version_format() error message for version mismatches by @laksh-krishna-sharma in #1015
• Bump version to 0.7.1 by @sungwy in #1034
• Support s3.signer.endpoint for nessie by @guitcastro in #1029
• [bug] fix reading with to_arrow_batch_reader and limit by @kevinjqliu in #1042
• Use VisitorWithPartner for name-mapping by @Fokko in #1014
• Fix tracing existing entries when there are deletes by @Fokko in #1046
• Coverage Run unit tests first before docker containers are set up by @Minfante377 in #1055
• Update "verify release" instruction by @kevinjqliu in #1064
• Fix Install Issues with docutils = 0.21.post1 and exclude 3.12 from supported python dependencies by @sungwy in #1067
• Post Release 0.7.1 version updates by @sungwy in #1073
• Update create table doc to clarify ID re-assignment by @paulcichonski in #1072
• Refactor PyArrow DataFiles Projection functions by @sungwy in #1043
• DOCS: Exclude signature files from twine upload by @sungwy in #1071
• Increase the minimal required pyarrow version to 14.0.0 by @ndrluis in #1090
• Fix table_exists behavior in REST catalog by @ndrluis in #1096
• fix: improve makefile by @TiansuYu in #1091
• fix (issue-1079): allow update_column to set doc as '' by @TiansuYu in #1083
• prevent adding duplicate files by @amitgilad3 in #1036
• Add list_views to rest catalog by @ndrluis in #817
• Emit warnings instead of failing when seeing unsupported configuration by @Fokko in #1111
• Use markdownlint instead of mdformat by @kevinjqliu in #1118
• Add drop_view to the rest catalog by @ndrluis in #820
• Support python 3.12 by @kevinjqliu in #1068
• Make commit_table public by @Fokko in #1112
• Refactoring: Break down very large table/__init__.py module by @sungwy in #1144
• fix: Invert case_sensitive logic in StructType by @AnthonyLam in #1147
• Bump duckdb to version 1.1.0 by @kevinjqliu in #1149
• Deprecate ADLFS prefix in favor of ADLS by @ndrluis in #961
• Cache Manifest files by @chinmay-bhat in #787
• Use the correct spec when rewiting existing manifests by @Fokko in #1157
• Bug Fix: Use historical partition field name by @sungwy in #1161
• fix: remove old, incorrect docstring by @dataders in #1166
• Preserve Backward compatibility in 0.8.0 for #1144 by @sungwy in #1151
• follow up for more cleanup by @dataders in #1168
• [bug] [REST] Dont remove identifier root by @kevinjqliu in #1172
• fix: support MonthTransform for partitioning by @felixscherz in #1176
• Add metadata tables for data_files and delete_files by @soumya-ghosh in #1066
• Use ArrowScan.to_table to replace project_table by @JE-Chen in #1180
• Add Docstrings to pyiceberg/table/__init__.py by @sungwy in #1189
• Support python 3.12 in poetry by @kevinjqliu in #1192
• Use cachetools's LRUCache to cache manifest list by @kevinjqliu in #1187
• HA HMS support by @awdavidson in #752
• Bug Fix: Position Deletes + row_filter yields less data when the DataFile is large by @sungwy in #1141
• Remove dead loom link by @kevinjqliu in #1213
• Drop support for Python 3.8 by @raulcd in #1221
• Add clarifying docs to transform result types by @kevinzwang in #1211
• Add flag to allow disabling creation of catalog tables by @isc-patrick in #1155
• Bug Fix: Glue and Hive catalog return only Iceberg tables by @mark-major in #1145
• Move snapshot history expire table properties to constants by @ndrluis in #1217
• abort the whole table transaction if any updates in the transaction has failed by @stevie9868 in #1246
• PyArrow: Pass in null-mask by @Fokko in #1264
• Bump PyArrow to 18.0.0 by @Fokko in #1256
• Remove numpy as a hard dependency by @Fokko in #1270
• Allow for missing operation by @Fokko in #1263
• fix: list_tables method in glue catalog now only return tables. by @omkenge in #1258
• Replace numpy usage and remove from pyproject.toml by @kevinjqliu in #1272
• Bump version to 0.8.0 by @Fokko in #1276
• Remove initial_change when CreateTableTransaction apply table updates on an empty metadata by @HonahX in #1219
• Deprecate for 0.8.0 release by @kevinjqliu in #1269
• Pass table-token to commit endpoint by @Fokko in #1278
• Updating configuration docs by @Samreay in #1292
• Allow union of {int,long}, {float,double}, etc by @Fokko in #1283
• Allow passing in ARN Role and Session name to the PyArrowFileIO by @Fokko in #1296
• Access delegation by @guitcastro in #1033
• Rename gcs.endpoint to gcs.service.host by @Fokko in #1007
• Modify S3 config naming convention by @kevinjqliu in #1301
• Add support for boolean expressions and quoted columns by @MoSheikh in #1286
• Do not deprecate botocore_session by @kevinjqliu in #1300

Dependabot PRs

• Bump tenacity from 8.5.0 to 9.0.0 by @dependabot in #978
• Bump getdaft from 0.2.31 to 0.2.33 by @dependabot in #993
• Bump mkdocstrings from 0.25.1 to 0.25.2 by @dependabot in #968
• Bump deptry from 0.17.0 to 0.18.0 by @dependabot in #990
• Bump mkdocstrings-python from 1.10.5 to 1.10.7 by @dependabot in #967
• Bump pypa/cibuildwheel from 2.19.2 to 2.20.0 by @dependabot in #999
• Bump sqlalchemy from 2.0.31 to 2.0.32 by @dependabot in #1003
• Bump cython from 3.0.10 to 3.0.11 by @dependabot in #1002
• Bump mkdocs-material from 9.5.30 to 9.5.31 by @dependabot in #1001
• Bump moto from 5.0.11 to 5.0.12 by @dependabot in #1000
• Bump coverage from 7.6.0 to 7.6.1 by @dependabot in #1011
• Bump deptry from 0.18.0 to 0.19.0 by @dependabot in #1022
• Bump mypy-boto3-glue from 1.34.143 to 1.34.156 by @dependabot in #1021
• Bump mypy-boto3-glue from 1.34.156 to 1.34.157 by @dependabot in #1027
• Bump aiohttp from 3.10.1 to 3.10.2 by @dependabot in #1035
• Bump deptry from 0.19.0 to 0.19.1 by @dependabot in #1047
• Bump pyspark from 3.5.1 to 3.5.2 by @dependabot in #1048
• Bump mkdocstrings-python from 1.10.7 to 1.10.8 by @dependabot in #1062
• Bump griffe from 0.48.0 to 0.49.0 by @dependabot in #1061
• Bump mypy-boto3-glue from 1.34.157 to 1.34.160 by @dependabot in #1059
• Bump griffe from 0.49.0 to 1.0.0 by @dependabot in #1069
• Bump griffe from 1.0.0 to 1.1.0 by @dependabot in #1076
• Bump moto from 5.0.12 to 5.0.13 by @dependabot in #1077
• Bump mkdocs-material from 9.5.31 to 9.5.32 by @dependabot in #1075
• Bump mypy-boto3-glue from 1.34.160 to 1.35.0 by @dependabot in #1070
• Bump mkdocs-autorefs from 1.0.1 to 1.1.0 by @dependabot in #1082
• Bump griffe from 1.1.0 to 1.1.1 by @dependabot in #1081
• Bump mkdocs-material from 9.5.32 to 9.5.33 by @dependabot in #1097
• Bump getdaft from 0.3.0 to 0.3.1 by @dependabot in #1098
• Bump boto3 from 1.34.131 to 1.34.162 by @dependabot in #1095
• Bump rich from 13.7.1 to 13.8.0 by @dependabot in #1103
• Bump griffe from 1.1.1 to 1.2.0 by @dependabot in #1101
• Bump pyparsing from 3.1.2 to 3.1.4 by @dependabot in #1102
• Bump deptry from 0.19.1 to 0.20.0 by @dependabot in #1107
• Bump boto3 from 1.34.162 to 1.35.7 by @dependabot in #1114
• Bump mkdocs from 1.6.0 to 1.6.1 by @dependabot in #1117
• Bump mkdocstrings-python from 1.10.8 to 1.10.9 by @dependabot in #1116
• Bump python-snappy from 0.7.2 to 0.7.3 by @dependabot in #1115
• Bump mkdocstrings from 0.25.2 to 0.26.0 by @dependabot in #1124
• Bump cryptography from 43.0.0 to 43.0.1 by @dependabot in #1130
• Bump mkdocs-autorefs from 1.1.0 to 1.2.0 by @dependabot in #1125
• Bump mkdocs-material from 9.5.33 to 9.5.34 by @dependabot in #1126
• Bump pydantic from 2.8.2 to 2.9.0 by @dependabot in #1137
• Bump mkdocstrings-python from 1.10.9 to 1.11.1 by @dependabot in #1136
• Bump getdaft from 0.3.1 to 0.3.2 by @dependabot in #1139
• Bump sqlalchemy from 2.0.32 to 2.0.34 by @dependabot in #1138
• Bump mkdocstrings from 0.26.0 to 0.26.1 by @dependabot in #1143
• Bump fastavro from 1.9.5 to 1.9.7 by @dependabot in #1142
• Bump moto from 5.0.13 to 5.0.14 by @dependabot in #1153
• Bump griffe from 1.2.0 to 1.3.0 by @dependabot in #1160
• Bump rich from 13.8.0 to 13.8.1 by @dependabot in #1158
• Bump boto3 from 1.35.7 to 1.35.16 by @dependabot in #1159
• Bump pydantic from 2.9.0 to 2.9.1 by @dependabot in #1154
• Bump griffe from 1.3.0 to 1.3.1 by @dependabot in #1170
• Bump pypa/cibuildwheel from 2.20.0 to 2.21.0 by @dependabot in #1175
• Bump mypy-boto3-glue from 1.35.3 to 1.35.23 by @dependabot in #1186
• Bump mkdocs-material from 9.5.34 to 9.5.35 by @dependabot in #1184
• Bump sqlalchemy from 2.0.34 to 2.0.35 by @dependabot in #1179
• Bump pypa/cibuildwheel from 2.21.0 to 2.21.1 by @dependabot in #1181
• Bump thrift from 0.20.0 to 0.21.0 by @dependabot in #1197
• Bump mkdocs-material from 9.5.35 to 9.5.36 by @dependabot in #1195
• Bump pypa/cibuildwheel from 2.21.1 to 2.21.3 by @dependabot in #1224
• Bump getdaft from 0.3.2 to 0.3.8 by @dependabot in #1228
• Bump moto from 5.0.14 to 5.0.17 by @dependabot in #1230
• Bump mkdocstrings from 0.26.1 to 0.26.2 by @dependabot in #1235
• Bump mypy-boto3-glue from 1.35.23 to 1.35.25 by @dependabot in #1236
• Bump werkzeug from 3.0.4 to 3.0.6 by @dependabot in #1248
• Bump boto3 from 1.35.16 to 1.35.36 by @dependabot in #1243
• Bump docutils from 0.20.1 to 0.21.2 by @dependabot in #1244
• Bump mmh3 from 4.1.0 to 5.0.1 by @dependabot in #1196
• Bump getdaft from 0.3.8 to 0.3.9 by @dependabot in #1260
• Bump psycopg2-binary from 2.9.9 to 2.9.10 by @dependabot in #1262
• Bump moto from 5.0.17 to 5.0.18 by @dependabot in #1261
• Bump mkdocs-material from 9.5.36 to 9.5.42 by @dependabot in #1266
• Bump sqlalchemy from 2.0.35 to 2.0.36 by @dependabot in #1268
• Bump pre-commit from 3.5.0 to 4.0.1 by @dependabot in #1267
• Bump rich from 13.8.1 to 13.9.3 by @dependabot in #1273
• Bump pyparsing from 3.1.4 to 3.2.0 by @dependabot in #1274
• Bump coverage from 7.6.1 to 7.6.4 by @dependabot in #1275
• Bump pyspark from 3.5.2 to 3.5.3 by @dependabot in #1280
• Bump mypy-boto3-glue from 1.35.25 to 1.35.53 by @dependabot in #1281
• Bump mkdocs-material from 9.5.42 to 9.5.43 by @dependabot in #1288
• Bump griffe from 1.3.1 to 1.5.1 by @dependabot in #1289
• Bump rich from 13.9.3 to 13.9.4 by @dependabot in #1298
• Bump duckdb from 1.1.0 to 1.1.3 by @dependabot in #1297

New Contributors

• @grobgl made their first contribution in #997
• @laksh-krishna-sharma made their first contribution in #1015
• @guitcastro made their first contribution in #1029
• @Minfante377 made their first contribution in #1055
• @paulcichonski made their first contribution in #1072
• @TiansuYu made their first contribution in #1091
• @amitgilad3 made their first contribution in #1036
• @AnthonyLam made their first contribution in #1147
• @dataders made their first contribution in #1166
• @JE-Chen made their first contribution in #1180
• @awdavidson made their first contribution in #752
• @raulcd made their first contribution in #1221
• @isc-patrick made their first contribution in #1155
• @mark-major made their first contribution in #1145
• @stevie9868 made their first contribution in #1246
• @omkenge made their first contribution in #1258
• @Samreay made their first contribution in #1292
• @MoSheikh made their first contribution in #1286

Full Changelog: pyiceberg-0.7.1...pyiceberg-0.8.0rc1