Noteworthy
⚠️ For platforms without PyArrow 7 support (e.g. MWAA, EMR, Glue PySpark Job):
➡️pip install pyarrow==2 awswrangler
New Functionalities
Enhancements
- add test infrastructure for oracle database #1274
- revisiting S3 Select performance #1287
- migrate test infra from cdk v1 to cdk v2 #1288
- to_sql() make column names quoted identifiers to allow sql keywords #1392
- throw NoFilesFound exception on 404 #1290
- fast executemany #1299
- add precombine key to upsert method for Redshift #1304
- pass precombine to redshift.copy() #1319
- use DataFrame column names in INSERT statement for UPSERT operation #1317
- add data_source param to athena.repair_table #1324
- modify athena2quicksight datatypes to allow startswith for varchar #1332
- add TagColumnOperation to quicksight.create_athena_dataset #1342
- enable list timestream databases and tables #1345
- enable s3.to_parquet to receive "zstd" compression type #1369
- create a way to perform PartiQL queries to a Dynamo DB table #1390
- s3 proxy support with data wrangler #1361
Documentation
- be more explicit about awswrangler.s3.to_parquet overwrite behavior #1300
- fix Python Version in Readme #1302
Bug Fix
- set encoding to utf-8 when no encoding is specified when reading/writing to s3 #1257
- fix Redshift Locking Behavior #1305
- specify cfn deletion policy for sqlserver and oracle instances #1378
- to_sql() make column names quoted identifiers to allow sql keywords #1392
- fix extension dtype index handling #1333
- fix issue with redshift.to_sql() method when mode set to "upsert" and schema contains a hyphen #1360
- timestream - array cols to str #1368
- read_parquet Does Not Throw Error for Missing Column #1370
Thanks
We thank the following contributors/users for their work on this release:
@bnimam, @IldarAlmakaev, @syokoysn, @IldarAlmakaev, @thomasniebler, @maxdavidson91, @takeknock, @Sleekbobby1011, @snikolakis, @willsmith28, @malachi-constant, @cnfait, @jaidisido, @kukushking
P.S. The AWS Lambda Layer file (.zip) and the AWS Glue file (.whl) are available below. Just upload it and run or use them from our S3 public bucket!