aws/aws-sdk-pandas 2.4.0-docs

AWS Data Wrangler 2.4.0 (Docs updated)

on GitHub

Caveats

⚠️ For platforms without PyArrow 3 support (e.g. EMR, Glue PySpark Job):

➡️ pip install pyarrow==2 awswrangler

Documentation

• Update to include PyArrow 3 caveats for EMR and Glue PySpark Job. #546 #547

New Functionalities

• Redshift COPY now supports the new SUPER type (i.e. SERIALIZETOJSON) #514
• S3 Upload/download files #506
• Include dataset BUCKETING for s3 datasets writing #443
• Enable Merge Upsert for existing Glue Tables on Primary Keys #503
• Support Requester Pays S3 Buckets #430
• Add botocore Config to wr.config #535

Enhancements

• Pandas 1.2.1 support #525
• Numpy 1.20.0 support
• Apache Arrow 3.0.0 support #531
• Python 3.9 support #454

Bug Fix

• Return DataFrame with unique index for Athena CTAS queries #527
• Remove unnecessary schema inference. #524

Thanks

We thank the following contributors/users for their work on this release:

@maxispeicher, @danielwo, @jiteshsoni, @igorborgest, @njdanielsen, @eric-valente, @gvermillion, @zseder, @gdbassett, @orenmazor, @senorkrabs, @Natalie-Caruana, @dragonH, @nikwerhypoport, @hwangji.

P.S. The AWS Lambda Layer file (.zip) and the AWS Glue file (.whl) are available below. Just upload it and run!