pypi awswrangler 3.18.0
AWS SDK for pandas 3.18.0
on Python PyPI

2 hours ago

Notable Changes ⚠️

  • ✨ New: Amazon Bedrock batch inference. Run a prompt over every row of a DataFrame and get the model output back joined to your rows with wr.bedrock.invoke_batch (tutorial)
  • Athena decimals keep full precision with the Arrow backend (#3497). With dtype_backend="pyarrow" and ctas_approach=False / unload_approach=False, decimal columns now come back as decimal128(p, s) instead of double[pyarrow], matching the CTAS/UNLOAD path. Previously values were silently rounded. Code that relied on floats may need updating: scalars are now Decimal, to_numpy() returns object arrays, numpy ufuncs raise, and to_json writes decimals as strings.
  • Stricter validation of caller-supplied SQL fragments (#3483, #3484, #3532). Malformed type strings passed via dtype= to to_sql, Athena Iceberg DDL column types, and PostgreSQL upsert_conflict_columns / insert_conflict_columns now raise InvalidArgumentValue instead of being passed through to the generated SQL.
  • EMR default logging bucket ownership is verified (#3480). When no log path is given, emr.create_cluster now checks that the default aws-logs-{account}-{region} bucket exists and is owned by the caller's account. This needs s3:ListBucket on that bucket and raises if the bucket is missing or owned by another account.
  • Bucketing hash for non-ASCII strings now matches Spark/Hive (#3505). Bucket assignment changes for non-ASCII values; tables bucketed by earlier versions on such columns should be rewritten.
  • Redshift COPY/UNLOAD rejects a half-specified credential pair (#3460). Passing only one of aws_access_key_id / aws_secret_access_key now raises instead of silently falling back.
  • secretsmanager.get_secret returns SecretBinary as-is (#3459). It was previously base64-decoded twice.
  • Float SQL parameters keep full precision (#3493). They were previously rounded to 6 decimal places.
  • CloudWatch default end_time is timezone-aware UTC (#3438). Results previously shifted by the local UTC offset on non-UTC hosts.
  • Oracle upsert follows DataFrame column order (#3447) and raises InvalidArgumentValue when primary_keys contains a column not in the DataFrame.
  • Stricter type hints (#3476). Several parameters (dtype_backend, mode, compression, file_format, …) are now Literal-typed, so mypy may flag plain str variables. to_parquet also accepts brotli and lz4.
  • Python 3.15 (#3511). Supported, except that the ray and modin extras are not yet available. A 3.15 Lambda layer will follow once the Lambda Python 3.15 runtime is generally available.
  • Security: multidict is bumped for CVE-2026-104874 (#3507), MySQL connections can enable TLS via Secrets Manager or the new ssl_context parameter (#3472), and Redshift COPY/UNLOAD escapes S3 paths and KMS key IDs (#3461).

Features / Enhancements 🚀

Bugfixes 🐛

  • fix: use fixed microsecond value in test_time_unit_precision to preve… by @bujjibabukatta in #3362
  • fix: omit Comment key when column_comment is None in add_column by @hsusul in #3435
  • fix: pass delete_objects to get_botocore_valid_kwargs in _delete_objects by @hsusul in #3433
  • fix(redshift): reject half-specified S3 credential pair by @kukushking in #3460
  • fix(secretsmanager): return SecretBinary without double base64 decode by @kukushking in #3459
  • fix(neptune): forward boto3_session and s3_additional_kwargs by @kukushking in #3466
  • fix(redshift): escape S3 path in COPY/UNLOAD by @kukushking in #3461
  • fix: security hardening tls region webhook by @kukushking in #3472
  • fix: roll back the connection when a chunked read_sql_query fails by @hsusul in #3437
  • fix: handle zero values for max_retries, initial_backoff, and max_backoff in OpenSearch index_documents by @hsusul in #3432
  • fix: avoid mutating the caller's common_attributes in timestream.write by @hsusul in #3439
  • fix(mysql): put UNSIGNED after the type for unsigned integer columns by @kwy404 in #3492
  • fix: align oracle upsert placeholders with dataframe column order by @knQzx in #3447
  • fix: prevent pandas_kwargs mutation during text file compression resolution by @hsusul in #3428
  • fix: preserve suffix filtering in _path2list when path is a list of S3 paths by @hsusul in #3434
  • fix: keep full precision when formatting float SQL parameters by @kwy404 in #3493
  • fix(s3): preserve s3:// inside object keys by @fallintoplace in #3501
  • fix: use timezone-aware UTC for the default cloudwatch end_time by @hsusul in #3438
  • fix(emr): prevent S3 bucket squatting on default logging path by @DG-Upwind in #3480
  • fix: stabilize integration tests by @kukushking in #3510
  • fix(athena): validate column type strings at Iceberg DDL emission by @kukushking in #3484
  • fix: hash non-ASCII strings as signed bytes when bucketing by @kwy404 in #3505
  • fix: validate caller-provided database type strings in to_sql by @kukushking in #3483
  • fix(tests): compare fixed-offset timezones by offset, not wrapper class by @kukushking in #3524
  • fix(tests): do not assert DynamoDB attribute order in read results by @kukushking in #3530
  • fix(postgresql): escape conflict column identifiers in to_sql by @kukushking in #3532
  • fix: map large_binary columns to the Athena binary type by @kwy404 in #3526
  • fix(dynamodb): return no items from read_items when a single key is missing by @sebxt in #3521
  • fix(s3): default sseType to aws:kms when only a KMS key is given for S3 Vectors by @abhishekacharya200 in #3513
  • fix: support numpy scalar types in SQL parameter formatting by @hsusul in #3431
  • fix(dynamodb): split read_items keys into BatchGetItem requests of at most 100 by @sebxt in #3518
  • fix(athena): preserve decimal precision with the Arrow backend by @rioyu123 in #3497
  • fix: correct target deletion path in merge_datasets for root dataset files by @hsusul in #3430
  • fix(neptune): apply nest_asyncio only when called from a running event loop by @kukushking in #3546

Security / Dependency Updates 🛡️

  • chore(deps): bump setuptools from 79.0.0 to 83.0.0 in /test_infra by @dependabot[bot] in #3409
  • chore(deps): bump aws-cdk-lib from 2.190.0 to 2.253.0 in /test_infra by @dependabot[bot] in #3419
  • chore(deps): bump async-timeout from 4.0.3 to 5.0.1 in the production-dependencies group by @dependabot[bot] in #3426
  • chore(deps): bump the github-actions group across 1 directory with 4 updates by @dependabot[bot] in #3425
  • chore(deps): bump cryptography from 48.0.1 to 50.0.0 by @dependabot[bot] in #3441
  • chore(deps): bump the github-actions group with 2 updates by @dependabot[bot] in #3444
  • chore(deps): bump github/codeql-action from 4.37.6 to 4.37.7 in the github-actions group by @dependabot[bot] in #3453
  • chore(deps): bump github/codeql-action from 4.37.7 to 4.37.9 in the github-actions group by @dependabot[bot] in #3462
  • chore(deps): bump the production-dependencies group across 1 directory with 7 updates by @dependabot[bot] in #3463
  • chore(deps): bump tornado from 6.5.7 to 6.5.8 by @dependabot[bot] in #3467
  • chore(deps): bump mistune from 3.3.0 to 3.3.3 by @dependabot[bot] in #3468
  • chore(deps): bump async-timeout from 4.0.3 to 5.0.1 in the production-dependencies group by @dependabot[bot] in #3470
  • chore(deps): bump the production-dependencies group with 2 updates by @dependabot[bot] in #3478
  • chore(deps): bump github/codeql-action from 4.37.9 to 4.38.0 in the github-actions group by @dependabot[bot] in #3477
  • chore(deps): bump httpx2 from 2.4.0 to 2.12.0 by @dependabot[bot] in #3474
  • chore(deps): bump the production-dependencies group with 2 updates by @dependabot[bot] in #3486
  • chore(deps): bump anyio from 4.12.1 to 4.14.2 by @dependabot[bot] in #3485
  • chore(deps): bump soupsieve from 2.8.4 to 2.9 by @dependabot[bot] in #3481
  • chore(deps): bump the production-dependencies group with 5 updates by @dependabot[bot] in #3490
  • chore(deps): bump github/codeql-action from 4.38.0 to 4.38.1 in the github-actions group by @dependabot[bot] in #3489
  • chore(deps): bump the production-dependencies group with 4 updates by @dependabot[bot] in #3495
  • chore(deps): bump github/codeql-action from 4.38.1 to 4.38.2 in the github-actions group by @dependabot[bot] in #3494
  • chore(deps): bump tornado from 6.5.8 to 6.5.9 by @dependabot[bot] in #3500
  • chore(deps): bump virtualenv from 21.2.0 to 21.7.13 by @dependabot[bot] in #3499
  • chore(deps): bump urllib3 from 2.7.0 to 2.8.0 by @dependabot[bot] in #3498
  • chore(deps): bump the production-dependencies group with 3 updates by @dependabot[bot] in #3502
  • chore(deps): bump multidict from 6.7.1 to 6.9.1 by @kukushking in #3507
  • chore(deps): bump werkzeug from 3.1.6 to 3.1.9 by @dependabot[bot] in #3508
  • chore(deps): bump the production-dependencies group with 4 updates by @dependabot[bot] in #3515
  • chore(deps): bump github/codeql-action from 4.38.2 to 4.38.3 in the github-actions group by @dependabot[bot] in #3514
  • chore(deps): bump uv from 0.11.15 to 0.13.4 by @dependabot[bot] in #3522
  • chore(deps): bump certifi from 2026.2.25 to 2026.10.19 by @dependabot[bot] in #3523
  • chore(deps): bump virtualenv from 21.7.13 to 21.14.9 by @dependabot[bot] in #3525
  • chore(deps): bump the github-actions group with 2 updates by @dependabot[bot] in #3527
  • chore(deps): bump the production-dependencies group with 4 updates by @dependabot[bot] in #3528
  • chore(deps): bump virtualenv from 21.14.9 to 21.14.13 by @dependabot[bot] in #3531
  • chore(deps): bump nbconvert from 7.17.1 to 7.17.2 by @dependabot[bot] in #3535
  • chore(deps): bump opentelemetry-api from 1.40.0 to 1.45.1 by @dependabot[bot] in #3534
  • chore(deps): bump setuptools from 83.0.0 to 84.1.0 in /test_infra by @dependabot[bot] in #3533

Housekeeping 🧹

New Contributors

Full Changelog: 3.17.1...3.18.0

Check out latest releases or
releases around awswrangler 3.18.0

Don't miss a new awswrangler release

NewReleases is sending notifications on new releases.

Get notifications