Summary
LocalStack 3.1 is here with exciting new features and enhancements. It introduces initial support for new providers: EventBridge Pipes, MemoryDB, Account, WAFv2, Identity Store, AMB, ACM PCA, as well as new features for existing services: the Instance Metadata Service (IMDS). Additionally, it offers improvements in DynamoDB, SQS, CloudWatch, RDS, ElastiCache and Lambda. The release also includes enhancements in Cloud Pods, Resource Browsers, and CI Keys Usage inspection.
New Features and Services
- Initial support for a new EventBridge Pipes provider to create Pipes with SQS queues and Kinesis streams as source and target. You can filter events using EventBridge event patterns and enrich events using Lambda. (π pro)
- Initial support for a new MemoryDB provider to create and manage MemoryDB clusters locally, either in the LocalStack container itself or in separate containers for each cluster node. (π pro)
- ElastiCache now supports creating Redis cache nodes and clusters as individual Docker containers, similar to MemoryDB. (π pro)
- Initial support for a new Account provider, supporting basic CRUD operations surrounding AWS account management. (π pro)
- Initial support for a new WAFv2 provider, supporting basic CRUD operations surrounding AWS Web Application Provider. (π pro)
- Initial support for a new ACM PCA provider, supporting basic CRUD operations surrounding AWS Private Certificate Authority. (π pro)
- Initial support for the Instance Metadata Service (IMDS) that exposes an endpoint inside every EC2 instance at
http://169.254.169.254/
. (π pro) - Initial support for the Identity Store provider, supporting basic CRUD operations surrounding AWS IAM Identity Center. (π pro)
- A new LocalStack-native CloudWatch provider is available behind a feature flag:
PROVIDER_OVERRIDE_CLOUDWATCH=v2
, which includes a lot of parity fixes and is designed to be thread safe. - Initial support for a new Managed Blockchain provider, supporting basic CRUD operations surrounding Amazon Managed Blockchain. (π pro)
Enhancements
- DynamoDB now supports Time to Live (TTL) which allows you to define a per-item expiration timestamp that indicates when an item is no longer needed, and delete it automatically.
- SQS now supports dead-letter queue re-drive emulation through
StartMessageMoveTask
,CancelMessageMoveTask
, andListMessageMoveTasks
API operations. - CloudWatch now supports Lambda targets for alarm actions on LocalStack.
- RDS now supports providing admin permission to the master users on RDS for MySQL and MariaDB engines. (π pro)
- Lambda performance improvements for asynchronous invocations along with reduction in
ReadTimeoutError
occurrences during high load on LocalStack.
LocalStack Features
- Cloud Pods
save
andload
commands now show a progress status on the CLI. (π pro) - New Resource Browsers for the following AWS services are available on LocalStack Web Application: OpenSearch, CloudFront, and Application Auto Scaling. In addition, new runtimes have been added to the Lambda Resource Browser. (π pro)
- Users can now inspect CI Keys Usage on the LocalStack Web Application. (π pro)
Deprecations
DNS_LOCAL_NAME_PATTERNS
is removed in favor ofDNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM
to list domain names that should not be resolved to the LocalStack container, but forwarded to the upstream resolver.
What's Changed
Exciting New Features π
- Add option to disable AWS_ENDPOINT_URL by @joe4dev in #9792
- Introduce DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM config by @simonrw in #9692
- add support for AWS::DynamoDB::GlobalTable resource type by @pinzon in #9790
- Simplify the official docker-compose.yml by @joe4dev in #9822
- Add flag to disable kinesis-mock persistence by @dfangl in #9871
- Transcribe: Add support for transcribing video files by @viren-nadkarni in #9898
- introduce request finalizers into handler chain by @thrau in #9194
- Test lambda performance and mitigate async invoke timeout error (#9917) by @joe4dev in #9917
- add cli option to restart localstack container by @HarshCasper in #9904
- Updates deduplication ID and group ID max length to 128 by @nishadcptn in #9923
- add lambda alarm action for cloudwatch by @steffyP in #9985
- Test lambda performance and mitigate async invoke timeout error v2 by @joe4dev in #9972
- fix lambda create event source mapping duplicates by @thrau in #9996
- fix sqs multi-protocol handling and upgrade botocore by @alexrashed in #9710
- add SNS signing of messages by @bentsku in #9997
- add extension mechanism for exceptions by @thrau in #10019
- Add config option to control maximum gateway worker thread count by @dfangl in #10016
- fix support calling sqs from api ws gw on disconnect endpoint by @calvernaz in #10008
- implement apigw request validation for parameters (header/path/qs) by @bentsku in #10036
- refactor http router for better werkzeug compatibility by @thrau in #10011
- move http framework code to rolo and only keep references by @thrau in #10061
- Make the internal sqs implementation of lambda events the default by @dfangl in #10062
- Add Cloudwatch v2 Provider by @pinzon in #9347
- move gateway and handler chain framework to rolo by @thrau in #10092
Other Changes
- add APIGW PutIntegration IntegrationType validation by @bentsku in #9763
- Enable test for Fn::GetAZs and extend feature by @pinzon in #9740
- fix kms key creation for s3 encryption by @sannya-singal in #9751
- Remove thread-local region, access key and account ID by @viren-nadkarni in #9750
- added util function for time by @macnev2013 in #9736
- Enable AppSync cloudformation resources by @simonrw in #9726
- Add list to dictionary transformation function in provider_utils For CFn by @Morijarti in #9786
- Update Lambda runtime test skip condition by @simonrw in #9760
- Fix alias resource records & add missing id in AWS::Route53::RecordSet by @dominikschubert in #9788
- Add option to capture raw snapshots when running tests by @dominikschubert in #9775
- [SFN] Fix string lexing in Intrinsic Function arguments by @MEPalma in #9783
- Support creation of CloudFormation stacks with the same name after deletion by @dominikschubert in #9748
- Fix StepFunctions execution date formatting by @dominikschubert in #9702
- Fix missing "Pulling container image" message from CLI startup by @simonrw in #9778
- add missing DeadLetterConfig parameter by @baermat in #9773
- fix recursive types when generating TypedDicts by @alexrashed in #9797
- fixed account id for firehose integration by @macnev2013 in #9785
- fix AWS::SQS::Queue update missing QueueArn and QueueUrl by @bentsku in #9744
- Cfn: Fix boto client not using credentials for Lambda PublishVersion by @viren-nadkarni in #9787
- fix: fix failing dashboard lifecycle test for multi-account and region by @sannya-singal in #9799
- [SFN] Enhancements to States I/O, Support for null InputPath and OutputPath by @MEPalma in #9801
- Improve lambda multiruntime tests by @joe4dev in #9798
- Add support for conditions on CFn stack outputs by @dominikschubert in #9814
- add cross-account for CopyObject by @bentsku in #9806
- fix: handle BucketAlreadyOwnedByYou exception when uploading lambda in scenario test cases by @sannya-singal in #9805
- CFn: fix AWS::Events::Rule ref/id/name attributes by @simonrw in #9789
- Bump Postgresql CI dependency by @viren-nadkarni in #9818
- Implemented support for AWS::Lambda:* Resource Provider for CloudFormation by @Morijarti in #9508
- Implemented AWS::CDK::Metadata resource provider by @Morijarti in #9520
- fix s3 presign tests for multi account support by @macnev2013 in #9816
- Fix AWS parity for SQS list queues with empty response by @joe4dev in #9828
- Prevent error message being shown with entrypoints command by @simonrw in #9827
- Fix: lookup more specific DNS names directly by @simonrw in #9815
- APIGW: Fix tests to run with non-default account ID by @viren-nadkarni in #9690
- Install websocket-client as it is no longer included with docker client by @simonrw in #9841
- Add formatting-aware timestamp replacement via new TimestampTransformer by @dominikschubert in #9772
- Skip comment for entrypoints line by @simonrw in #9838
- Fix lambda function name validation for CreateFunction by @joe4dev in #9825
- Fix serialization of content in raw snapshots by @dominikschubert in #9820
- remove outdated documentation and improve the README files by @HarshCasper in #9850
- Fix SQS CFn updates with no change by @simonrw in #9831
- fix s3 key handling with trailing slash by @bentsku in #9856
- fix: serverless failing multi-account/region tests by @sannya-singal in #9840
- pin docker dependency to <7.0.0 by @dominikschubert in #9864
- Updated Cloudformationn Schema to latest version by @Morijarti in #9865
- fix S3 lifecycle tags filter by @bentsku in #9859
- Allow merging of config profiles by @dfangl in #9873
- Add support for partial ARNs for EventSourceMappings and fix AWS::Lambda::Version state check by @dominikschubert in #9879
- CloudWatch: Use SNS client with same region as topic by @viren-nadkarni in #9878
- Fix: Route53 populating VPC fields by @simonrw in #9885
- fix APIGW import integration method for AWS integration type by @bentsku in #9810
- Bump moto-ext to 4.2.11.post1 by @viren-nadkarni in #9852
- Update CODEOWNERS by @localstack-bot in #9835
- remove code in state locator by @giograno in #9894
- Update dynamodb snapshot to unblock pipeline by @dominikschubert in #9910
- Bump Kinesis Mock to 0.4.5 by @etspaceman in #9906
- fix S3 UploadPart logic when cancelling/error a request by @bentsku in #9901
- Allow kinesis get_records shard iterator to have surrounding quotes by @dfangl in #9884
- Added missing type conversion for CFn update of Lambda::Function MemorySize to int by @Morijarti in #9903
- add S3 object key validation by @bentsku in #9875
- Fix Lambda Kinesis event filtering by @danielpoonwj in #6826
- Refine get in touch readme by @joe4dev in #9922
- [SFN] Fix Parallel State Output Ordering by @MEPalma in #9889
- [SFN] Support for Optional ProcessorConfig Declarations by @MEPalma in #9888
- Remove error message if environment does not exist by @dfangl in #9913
- Track date of last passed validation run of tests targeting AWS by @dominikschubert in #9927
- fix CFN RequestValidator parameters being optional by @bentsku in #9915
- fix S3 service name parsing consuming the stream by @bentsku in #9874
- Fix: ACM: Additional domains missing from subject alternative names summaries by @maxhoheiser in #9897
- Fix: EventBus: match null value for anything-but rule by @maxhoheiser in #9925
- Add initial validation files for recorded snapshots by @dominikschubert in #9928
- Add support for AWS::Cognito::* in CloudFormation by @Morijarti in #9855
- Add AWS::Athena::* Resource Providers for CFn by @Morijarti in #9779
- Add AppConfig resource providers to CFn by @Morijarti in #9774
- [SFN] Enhanced Support for Exception Handling in Parallel States by @MEPalma in #9891
- Update CODEOWNERS by @localstack-bot in #9938
- Revert recent changes to async lambda invoke (#9917) by @dominikschubert in #9944
- Enable migrated SES resource providers by @dominikschubert in #9950
- Add new client types to TypedServiceClientFactory by @dominikschubert in #9953
- Cfn: Fix nested stack name generation by @viren-nadkarni in #9926
- Fix: EventBus: Exists prefix not considered for rule matching by @maxhoheiser in #9931
- Fix secret rotation in secretsmanager by @lakkeger in #9862
- fix S3 CORS headers casing matching by @bentsku in #9973
- add SQS message retention period functionality by @thrau in #9956
- fix sqs message group visibility when sending messages by @thrau in #9975
- use batches to report SQS metrics for approximate query metrics by @steffyP in #9886
- add subrange operation for ports by @thrau in #9965
- add exception module to event source mappings by @thrau in #9964
- Add integration test for SQS FIFO event source order/parallelism by @dominikschubert in #7037
- Add link-local address and init args to Docker client API by @viren-nadkarni in #9900
- fix S3 missing XML namespace by @bentsku in #9983
- fix cloudwatch alarm evaluation for SampleCount by @steffyP in #9978
- fix snapshot match order by @steffyP in #9993
- fix SQS send batch message validation for message size by @thrau in #9981
- fix CopySource unquoting of special + char by @bentsku in #9992
- add SQS message move task feature by @thrau in #9988
- snapshot test for return values on condition check failure by @giograno in #9984
- improve S3 aws-chunked detection by @bentsku in #9999
- Add events method to docker clients by @simonrw in #9932
- fix flaky sns unit test by @thrau in #10012
- add OpenSearch analysis-kuromoji plugin by @alexrashed in #10001
- [SFN] Support for Version Declarations by @MEPalma in #9948
- Revert "Add events method to docker clients (#9932)" by @dfangl in #10025
- Bump moto-ext to 5.0.0alpha1.post1 by @viren-nadkarni in #10017
- Add AWS::ApiGateway and AWS::Add AWS::ApiGatewayV2 resources by @Morijarti in #9629
- refactor Poller.handle_message and extract process_event_retry by @thrau in #10043
- [SFN] Implicit Reductions To and From botocore Types by @MEPalma in #9920
- allow headers to be set from integration definition for HTTP integrat⦠by @jimmydurkin in #10035
- fix CLI issue with outdated click version by @alexrashed in #10029
- Add new internalized sqs queue for lambda events by @dfangl in #10045
- feat: DynamoDB TTL by @giograno in #10041
- Dynamodb: Fix disable SSESpecification with UpdateTable by @viren-nadkarni in #10040
- Support dynamic port on Lambda remote debugging by @yamadayutaka in #9420
- fix source ip value for the lambda event request by @calvernaz in #10038
- Fix: ACM: Describe DomainValidationOptions for subject alternative names duplicated with additional suffix by @maxhoheiser in #10027
- patch HTTPServer fixture to spawn daemon thread by @bentsku in #10052
- refactor EventSourceAsfAdapter for easier instrumentation by @thrau in #10053
- Skip snapshot verification for CodeSha256 in lambda API test by @dominikschubert in #10055
- Make lambda sqs event source mapping interval configurable by @dfangl in #10051
- update the year in the license section by @HarshCasper in #10050
- Implement resource provider for AWS::Lambda::Function by @dominikschubert in #9321
- fix return value for AWS::Cognito::UserPoolClient by @giograno in #10060
- supports enabling S3 Object Lock on existing buckets by @bentsku in #10071
- add support for ObjectLock for CFN S3 Bucket by @bentsku in #10070
- fix S3 v-host unit tests after rolo migration by @bentsku in #10099
- use request parameter resolvers by @calvernaz in #10049
- Pin cryptography package to below 42.0.0 by @silv-io in #10105
- correctly execute host and container configuration hooks in dev.run by @thrau in #10108
- Dev mode: return correctly from raw mode by @simonrw in #10111
- Register EventBridge Pipes CloudFormation resource provider by @joe4dev in #10110
New Contributors
- @danielpoonwj made their first contribution in #6826
- @maxhoheiser made their first contribution in #9897
- @nishadcptn made their first contribution in #9923
- @jimmydurkin made their first contribution in #10035
- @yamadayutaka made their first contribution in #9420
Full Changelog: v3.0.2...v3.1.0