Summary
This Christmas release was heavily focused on papercuts and two must needed improvements - Batch now supports SQS, DynamoDB, and Kinesis using the new native partial response (decreased cost), and Event Handler API Gateway now supports exception handling and overriding HTTP 404 errors.
For the next release, we're going to further invest in Mypy compliance, support native serialization of Python Dataclasses and Pydantic Models in Idempotency utility, and more!
Join us on Slack #lambda-powertools to help us make the next release even better!
New Batch Processor
Docs: Infra required, SQS, Kinesis, and DynamoDB
A few weeks ago Lambda added native support for partial response. Since GA, Batch utility provided that capability to SQS by handling partial failures and deleting successful messages on your behalf.
With the new capability, we've added a new BatchProcessor that can process both SQS, Kinesis Data Streams, and DynamoDB Streams.
The new capability requires an infrastructure change in the Lambda Event Source Mapping hence why a new Batch Processor - We've made sure to add complete SAM examples for SQS, Kinesis, and DynamoDB Streams on everything you need to get going including minimal IAM permissions.
We also took this opportunity to a) provide tight integration with Event Source Data Classes for self-documented batch records, b) Parser (Pydantic) integration so you can bring your own models for data schema and validation, and c) mypy is fully compliant regardles of the event type or Pydantic model you bring.
Did I also say we refreshed the entire documentation with sample events, responses, how to unit test, and a Migration Guide?
Migration guide
If you were using SQSBatchProcessor before as a decorator or as a context manager, we've written a guide to help you transition to the new more performant and cost efficient way: https://awslabs.github.io/aws-lambda-powertools-python/develop/utilities/batch/#migration-guide
A warm start with tracing, logging, data transformation, and exception handling can run as fast as 8ms end-to-end now.
Event Handler API Gateway improvements
Overriding 404 errors
You can now override how we handle unmatched routes (404) by using a decorator not_found. The experience is similar to how you'd define routes, except your function will receive an exception - NotFoundError in this case.
You can customize the response by simply returning Response like you would in a more advanced route -- This should give you more flexibility to generate metrics, log errors, or anything you might want for unmatched routes.
Exception handling
Besides 404 errors, you might want to handle custom validation errors, or any exception you might raise as part of your route handling. You can now use exception_handler decorator and pass any valid Python Exception.
Tracer
Tracer now automatically adds a Service annotation if either service parameter or POWERTOOLS_SERVICE_NAME is set - This makes it easier to slice and dice all of your traces by the exact service you have deployed.
Moreover, we now include ColdStart=false for warm start invocations so you can more easily sample and analyze performance differences between cold and warm starts.
CloudFormation Custom Resources
As our Java Powertools colleagues added a new utility for easily writing CloudFormation Custom Resources, we added a link in our docs pointing to the official Python library for crafting custom resources.... just in case it's not widely known: https://github.com/aws-cloudformation/custom-resource-helper
Changes
🌟New features and non-breaking changes
- feat(logger): allow handler with custom kwargs signature (#913) by @heitorlessa
- feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (#886) by @heitorlessa
- feat(apigateway): add exception_handler support (#898) by @michaelbrewer
- feat(tracer): add service annotation when service is set (#861) by @heitorlessa
- feat(apigateway): access parent api resolver from router (#842) by @cakepietoast
🌟 Minor Changes
📜 Documentation updates
- docs(apigateway): add new not_found feature (#915) by @heitorlessa
- docs(nav): reference cloudformation custom resource helper (#914) by @heitorlessa
- feat(batch): new BatchProcessor for SQS, DynamoDB, Kinesis (#886) by @heitorlessa
- fix(parser): mypy support for payload type override as models (#883) by @heitorlessa
- docs(apigateway): fix sample layout provided (#864) by @michaelbrewer
- feat(tracer): add service annotation when service is set (#861) by @heitorlessa
- fix(apigateway): allow list of HTTP methods in route method (#838) by @cakepietoast
🐛 Bug and hot fixes
- fix(parser): overload parse when using envelope (#885) by @heitorlessa
- fix(parser): kinesis sequence number is str, not int (#907) by @heitorlessa
- fix(event-sources): Pass authorizer data to APIGatewayEventAuthorizer (#897) by @michaelbrewer
- fix(parser): mypy support for payload type override as models (#883) by @heitorlessa
- fix(parameters): appconfig transform and return types (#877) by @ran-isenberg
- fix(event-sources): handle Cognito claimsOverrideDetails set to null (#878) by @michaelbrewer
- fix(idempotency): include decorated fn name in hash (#869) by @michaelbrewer
- fix(tracer): add warm start annotation (ColdStart=False) (#851) by @heitorlessa
- fix(apigateway): allow list of HTTP methods in route method (#838) by @cakepietoast
🔧 Maintenance
- chore: minor housekeeping before release (#912) by @michaelbrewer
- chore(deps-dev): bump mypy from 0.910 to 0.920 (#903) by @dependabot
- chore(deps): bump fastjsonschema from 2.15.1 to 2.15.2 (#891) by @dependabot
- chore(deps-dev): bump black from 21.11b1 to 21.12b0 (#872) by @dependabot
- chore(deps): bump aws-xray-sdk from 2.8.0 to 2.9.0 (#876) by @dependabot
- chore(deps): support arm64 when developing locally (#862) by @michaelbrewer
- chore(deps-dev): bump flake8 from 3.9.2 to 4.0.1 (#789) by @dependabot
- chore(deps): bump actions/setup-python from 2.3.0 to 2.3.1 (#852) by @dependabot
- chore(deps-dev): bump black from 21.10b0 to 21.11b1 (#839) by @dependabot
- chore(deps): bump actions/setup-python from 2.2.2 to 2.3.0 (#831) by @dependabot
This release was made possible by the following contributors:
@cakepietoast, @dependabot, @dependabot[bot], @heitorlessa, @michaelbrewer, @ran-isenberg and @whardier
Full Changelog: v1.22.0...v1.22.1