Summary
LocalStack 3.5 introduces significant enhancements including support for the Database Migration Service (DMS), an all-new EC2 Libvirt VM manager for emulating virtual machines, a new EventBridge provider, and much more. Other notable updates include enhanced support for container Lambda via the Kubernetes executor, custom model support for Sagemaker, several Step Functions enhancements, and new CloudFormation resource providers.
AWS Features
- LocalStack now supports Database Migration Service (DMS) and you need to set the environment variable
ENABLE_DMS=1
in order to activate it. (🌟 enterprise) - EC2 now supports Libvirt VM manager. This VM manager allows EC2 instances to be emulated as virtual machines using the KVM/QEMU on Linux via the Libvirt API. (🌟 pro)
- LocalStack now includes a native EventBridge provider accessible through the feature flag:
PROVIDER_OVERRIDE_EVENTS=v2
. Learn more about it in our Discuss post.
Enhancements
-
LocalStack has added support for container lambdas using the Kubernetes executor. The configuration option
LAMBDA_K8S_INIT_IMAGE
lets you specify the image used to download the init binary from LocalStack. (🌟 pro) -
Support for DELETE operation for Device Shadow Service over MQTT in the IoT provider. (🌟 pro)
-
Support for publishing MQTT messages with non-JSON payloads in the IoT provider. (🌟 pro)
-
Support for Apache Airflow 2.8.1 in the Managed Workflows for Apache Airflow (MWAA). (🌟 pro)
-
Support for SageMaker custom-built model inference deployment. (🌟 pro)
-
Support for named shadows in the IoT provider. (🌟 pro)
-
Custom DNS support for API Gateway REST APIs. (🌟 pro)
-
Support for ListPipes operation in the EventBridge Pipes provider. (🌟 pro)
-
Support for CloudWatch Logs logging and monitoring in Step Functions.
-
LocalStack now includes additional SNS filter operators:
- Equals-ignore-case matching for case-insensitive comparisons.
- Suffix matching for ending string patterns.
Improvements have also been made to the Anything-but matching. Furthermore, support for the $or operator now enables more complex filter policies.
-
AWS has deprecated the SQS
attribute-names
parameter for thereceive-message
operation and replaced it withmessage-system-attribute-names
. LocalStack has implemented this update. -
Previously, the Step Functions interpreter could not execute reentrant distributed map states because it did not perform necessary cleanups at the end of evaluations, preventing the activation of new workers. These issues have now been addressed.
-
Support for
CommaDelimitedList
in CloudFormation macro. -
Support for Glue optimised integrations for Step Functions.
-
Support for S3 Multipart Uploads object integrity checks.
-
Support for GZIP and bzip2 decompression for S3 Select.
-
Previously, the Step Functions interpreter did not normalize the output values of service responses when the response type was not directly inferrable as JSON, such as with StreamingBody data types. This issue has now been addressed, and normalization steps are in place for such data types.
-
Neptune default Cardinality for Vertex property changed from
Single
toSet
for better Gremlin query parity with AWS (🌟 pro) -
In CloudFormation, the deployment order now better accounts for dependencies among resources during both creation and deletion. This update should resolve issues in larger stacks with many dependent resources. To revert to the previous behavior, set
CFN_LEGACY_TEMPLATE_DEPLOYER=1
. -
The following CloudFormation resource providers are now supported:
Operation Create Update Delete AWS::ACMPCA::Certificate ✅ ✅ AWS::ACMPCA::CertificateAuthority ✅ ✅ AWS::ACMPCA::CertificateAuthorityActivation ✅ ✅ AWS::ACMPCA::Permission ✅ ✅ AWS::EC2::PrefixList ✅ ✅ AWS::EC2::VpcEndpoint ✅ ✅ AWS::Batch::ComputeEnvironment ✅ ✅ AWS::Batch::JobQueue ✅ ✅ AWS::Batch::JobDefinition ✅ ✅ AWS::Glue::Registry ✅ ✅ AWS::Glue::Schema ✅ ✅ AWS::Glue::SchemaVersion ✅ ✅ AWS::Glue::SchemaVersionMetadata ✅ ✅ AWS::Glue::Connection ✅ ✅ AWS::Cloudfront::ResponseHeadersPolicy ✅ ✅ AWS::EFS::AccessPoint ✅ ✅ AWS::EFS::MountTarget ✅ ✅ AWS::SageMaker::Model ✅ ✅ AWS::SageMaker::Endpoint ✅ ✅ AWS::SageMaker::EndpointConfig ✅ ✅ AWS::WAFv2::WebACL ✅ ✅ AWS::WAFv2::WebACLAssociation ✅ ✅ AWS::WAFv2::IPSet ✅ ✅ AWS::WAFv2::LoggingConfiguration ✅ ✅
LocalStack Features
- LocalStack's new web server implementation, which enhances performance in high-throughput scenarios and was initially enabled by a feature flag, is now the default setting.
- LocalStack CLI does not publish port
53
anymore by default. You can use the CLI flag--host-dns
to expose the port on the host. - Support for
CFN_PER_RESOURCE_TIMEOUT
environment variable, defaulting to 300 seconds. This extends LocalStack's CloudFormation deployment limit for a single resource beyond the previous 2.5-minute hard cap, accommodating larger deployments such as nested stacks. - Support for
--env-file
flag in theDOCKER_FLAGS
environment variable when using the LocalStack Docker Container, allowing you to pass the path to an environment file. - Support for environment variable placeholders in hot-reloading paths for Lambda functions.
- New Resource Browser for Database Migration Service (DMS) is now available. (🌟 enterprise)
- You can now select Athena databases and S3 paths directly from the Athena SQL editor. (🌟 pro)
- Neptune Resource Browser is now easier to use, featuring a graph browser on the main page and a dropdown menu for selecting instances. (🌟 pro)
What's Changed
Exciting New Features 🎉
- add resource providers for vpc endpoint and prefix list by @pinzon in #10735
- switch default gateway server from hypercorn to twisted by @alexrashed in #10703
- Fix: Firehose: Drop keys in destinations description not in respective return types by @maxhoheiser in #10758
- Feat: Eventbridge v2: Add pattern matching by @maxhoheiser in #10664
- Refactor: Events v2: Move existing provider code to v1 folder by @maxhoheiser in #10730
- Feature: Eventbridge v2: Add input path by @maxhoheiser in #10733
- Bump moto-ext to 5.0.6.post2 by @viren-nadkarni in #10742
- Feature/eventbridge v2 add input transformer by @maxhoheiser in #10789
- implement SNS Filter/operators $or, suffix, equals-ignore-case, anything-but by @bentsku in #10691
- Feature: Eventbridge v2: add schedule executor by @maxhoheiser in #10817
- Feature: Eventbridge v2: Add tagging by @maxhoheiser in #10840
- Add endpoint to expose lambda init binary, increase fidelity of the kubernetes dev script by @dfangl in #10828
- Add support for environment variable placeholders in hot-reloading paths by @dfangl in #10857
- add cdk utility to import docker images to ecr by @pinzon in #10861
- move source code into localstack-core by @thrau in #10800
- extract docker build / push make targets to helper script by @alexrashed in #10872
- Add support for the --env-file flag in docker flags by @dfangl in #10880
- implement S3 checksum for Multipart uploads by @bentsku in #10917
- move aws-specific patches into localstack.aws.patches by @thrau in #10958
- add DaemonAwareThreadPool for thread pools to block shutdown by @thrau in #10959
- remove remaining (transitive) usages of Quart by @alexrashed in #9754
- CFn: DAG based deploy order by @simonrw in #10849
- remove deprecated HttpRequest type alias by @alexrashed in #9757
- [SFN] Support for CloudWatch Logging and Monitoring by @MEPalma in #10975
- Step Functions: Allow describing ARNs that contain periods by @northvankiwiguy in #10985
- [SFN] Normalise Upload Values by @MEPalma in #10999
- introduce handler chain (next gen) in apigateway v1 invocation by @bentsku in #10982
- [SFN] Multi-account and Multi-region Logging by @MEPalma in #11007
- Add new option to manually expose the DNS port to the host, remove binding by default by @dfangl in #11011
- implement new runtime framework for bootstrapping localstack by @thrau in #10942
Other Changes
- update release banner in README by @HarshCasper in #10686
- fix headers for empty S3 responses by @bentsku in #10717
- Avoid concurrent pulling of identical images in lambda by @dfangl in #10720
- CFN_PER_RESOURCE_TIMEOUT env variable by @lakkeger in #10721
- add utility for resource providers by @pinzon in #10718
- Hide docker sdk error unless trace logging by @simonrw in #10711
- Fix SubtypesInstanceManager not loading subtypes lazily by @viren-nadkarni in #10728
- migrate from enhancement request to feature request by @alexrashed in #10743
- [SFN] Improve Handling of Empty SendTaskFailure Calls by @MEPalma in #10750
- Improve DDB bytes encoding by @giograno in #10740
- Parametrize test selection scripts for usage in dependent repositories by @silv-io in #10757
- fixed physical resource id for AWS::Glue::SchemaVersionMetadata by @macnev2013 in #10770
- [SFN] Fix Flaky Stop Execution Test by @MEPalma in #10771
- decrease test logging verbosity by @alexrashed in #10761
- fix serialization for sqs http calls by @baermat in #10732
- fix CircleCI test error reporting by disabling colored reporting by @alexrashed in #10786
- migrate contributing docs to core by @HarshCasper in #10775
- Fix CFn UpdateStack response on identical templates that include a transformation by @dominikschubert in #10782
- fix banner link in READMEs by @alexrashed in #10792
- Update PULL_REQUEST_TEMPLATE.md by @thrau in #10793
- [SFN] Execution of Reentrant Distributed Map States by @MEPalma in #10763
- Separate test selection env setup from tinybird setup by @dominikschubert in #10790
- fix missing href inside README.md by @gabriel-vasile in #10671
- fix SNS cross-account listing call of subscriptions by @bentsku in #10788
- enable sqs test selection by @baermat in #10784
- Remove fix to relative persistence path for Kinesis by @giograno in #10774
- fix asf update action by updating ruff fix command by @alexrashed in #10805
- [SFN] Temporary Skip of Flaky Test Cases by @MEPalma in #10807
- fix GitHub error reporting by disabling colors by @alexrashed in #10815
- fix pagination for SNS ListSubscriptions and ListSubscriptionsByTopic by @bentsku in #10810
- fix SNS cross-account issues with Subscriptions by @bentsku in #10819
- fix ListObjectVersions field order by @bentsku in #10829
- APIGW: Default to an empty dict when the provided body for an API Gateway step function is an empty string by @marcciosilva in #10816
- update test durations by @alexrashed in #10833
- Revert "APIGW: Default to an empty dict when the provided body for an API Gateway step function is an empty string" by @dfangl in #10835
- Add image sha to startup information and more details to bug report template by @dominikschubert in #9761
- skip flaky test_lambda::test_reserved_concurrency_async_queue by @alexrashed in #10834
- mitigate CI timeouts by increasing parallelism by @alexrashed in #10837
- fix SNS test with wrong base64 assumption by @bentsku in #10838
- update plux to 1.10.0 by @thrau in #10830
- validate attributes when creating SQS queues by @cloutierMat in #10820
- add support for
CommaDelimitedList
in cloudformation macro by @sannya-singal in #10836 - Bump moto-ext to 5.0.7.post1 by @viren-nadkarni in #10843
- Fix entrypoint script configurator encoding and line endings in windows by @dfangl in #10814
- Cleanup code mounting dev scripts by @simonrw in #10842
- [SFN] Support for Glue Optimised Service Integration by @MEPalma in #10802
- add small quirks for new resource providers by @pinzon in #10818
- Update CODEOWNERS by @localstack-bot in #10846
- CFn: handle secretsmanager policy BlockPublicPolicy by @simonrw in #10850
- add S3 pre-signed credentials validation by @bentsku in #10856
- skip flaky test_schedule_cron_target_sqs by @bentsku in #10863
- Replace large bytes payloads in http request logging to avoid cluttering the logs by @dfangl in #10858
- decrease runners after fixed performance degradation by @alexrashed in #10851
- fix S3 Pre-signed POST policy condition casing by @bentsku in #10862
- skip flaky test_get_free_tcp_port_range_fails_if_cannot_be_bound by @sannya-singal in #10865
- Using ARN_PARTITION_REWRITING flag causes s3api head-object to provide invalid content-length by @Morijarti in #10869
- Respect file extension setting for S3 destinations in Firehose provider by @ozgenbaris1 in #10651
- Add SSM apigateway service test by @serialbandicoot in #9097
- skip flaky tests.aws.services.firehose.test_firehose.TestFirehoseIntegration.test_kinesis_firehose_opensearch_s3_backup by @sannya-singal in #10855
- Collect analytics on error probability config options by @viren-nadkarni in #10871
- limit github token permissions, fix cli action for fork runs by @ceddy4395 in #10870
- fix Sqs fifo message group invisibility after triggering move to dlq by @baermat in #10859
- [SFN] Add Generic Resolver for Unsupported Optimised Service Integrations by @MEPalma in #10877
- fix S3 CopyObject in place deadlock for Suspended buckets by @bentsku in #10882
- Resolved issue where using CDK wouldn't generate SecretString value by @Morijarti in #10879
- Fix deduplication scope for non-high-throughput FIFO messages by @baermat in #10887
- Fix Lambda async invocation queue namespace by @joe4dev in #10831
- fix healthcheck in dockerfile by @thrau in #10894
- [SFN] Add Normalisation of Service Response Values by @MEPalma in #10883
- Update CODEOWNERS by @localstack-bot in #10900
- CFn: Parametrize Fn::GetAZs test across all regions by @simonrw in #10681
- Fix lambda persistence restore by @dfangl in #10902
- fix docker helper usage typo and push without force flag by @alexrashed in #10903
- remove unknown-markers in test_logs by @steffyP in #10892
- Remove CloudFormation moto backend from persisted state by @dominikschubert in #10901
- Opt in services to the test selection by @silv-io in #10867
- Bump moto-ext to 5.0.8.post2 by @viren-nadkarni in #10868
- fix pre-signed POST str type for content-length-range by @bentsku in #10889
- Automatically determine PR URL through branch name for test selection by @silv-io in #10913
- remove unknown-markers in test_transcribe by @sannya-singal in #10920
- Track cfn resource events by @simonrw in #10760
- validate redshift test by @steffyP in #10924
- remove unknown-markers in test_kms by @sannya-singal in #10929
- fix postgress install in ci by @cloutierMat in #10933
- fix header list parsing in ASF by @bentsku in #10935
- apigw validate test by @cloutierMat in #10926
- remove unknown markers from S3 tests by @bentsku in #10934
- validate and add S3Control tests by @bentsku in #10932
- Increase performance for first store creation per account/service by @dfangl in #10919
- Fixed scaffolding generation for CFN resources by @Morijarti in #10941
- Fix k8s dev script code locations by @simonrw in #10931
- remove unused localstack.services.messages by @thrau in #10947
- Move testing config from constants.py to its own module by @dfangl in #10922
- move cloudformation deploy ui to cloudformation provider by @thrau in #10945
- fix flaky TestServerless.test_kinesis_stream_handler_deployed by @alexrashed in #10944
- improve error reporting when docker socket is not mounted by @thrau in #10896
- Include old resolv.conf contents without nameservers by @simonrw in #10731
- Validate remaining unknown sqs tests by @baermat in #10923
- Formalize Testing Rules by @silv-io in #10878
- skip flaky TestKinesisFirehoseScenario.test_kinesis_firehose_s3 by @alexrashed in #10953
- fix Checksum validation and assumptions for S3 by @bentsku in #10954
- add code to debug monkey patches applied through localstack.utils.patch by @thrau in #10957
- SQS: Add MessageSystemAttributeNames for receive-message by @baermat in #10955
- fix S3 image publishing by @bentsku in #10960
- fix S3 Image publishing - credentials for Dockerhub by @bentsku in #10963
- validate STS tests by @bentsku in #10962
- fix S3 aws-chunked when checksum is provided by @bentsku in #10964
- fix S3 image publishing credentials by @bentsku in #10965
- validate unknown dynamodb streams test and update behavior by @bentsku in #10937
- fix S3 image healthcheck by @bentsku in #10970
- Add lambda ignore architecture flag to CLI config variables by @dfangl in #10974
- fix flaky new DDB test by @bentsku in #10973
- fix flaky APIGW test by @bentsku in #10971
- Fix sqs urlencoded parsing by @cloutierMat in #10966
- CFn: add simple template validations by @simonrw in #10938
- Refactor all pytest xfail markers to skip instead by @dominikschubert in #10967
- Refine optional Lambda dependencies by @joe4dev in #10952
- Update dev script to mount community code correctly by @simonrw in #10983
- Fix CloudFormation Fn::Transform replacing unintended values by @dominikschubert in #10978
- remove unknown markers in test_multiregion by @sannya-singal in #10987
- fix asf update action by avoiding aws cli pin by @alexrashed in #10989
- remove unknown markers in cross accounts test in lambda by @sannya-singal in #10991
- validate unknown tests - ec2 by @cloutierMat in #10977
- Fix handling None payload in Lambda Event Source Mapping by @barry-roomex in #10980
- replace unknown markers for kinesis-tests by @steffyP in #10994
- Skip flaky ElasticSearch tests by @joe4dev in #10998
- Bump moto-ext to 5.0.9.post1 by @viren-nadkarni in #10979
- Fix/events cross region bus to bus communication by @maxhoheiser in #10968
- Unify test selection scripts by @silv-io in #10996
- Add support for placeholder values in env-file flag for docker flags by @dfangl in #10972
- check AWS Support validation status by @bentsku in #11005
- validate scheduler test and check SWF validation by @bentsku in #11006
- Fix: Lambda allow usage of partial ARN and ARN in function names by @Morijarti in #10961
- Allow secondary client fixtures to take credentials from an aws profile by @dfangl in #11010
- Fix detection of docker image digest for diagnostic purposes by @dominikschubert in #10951
- Fix runtime shutdown when called triggered via signal handlers by @dominikschubert in #11018
- Skip flaky OpenSearch CloudFormation test by @joe4dev in #11017
New Contributors
- @gabriel-vasile made their first contribution in #10671
- @ozgenbaris1 made their first contribution in #10651
- @serialbandicoot made their first contribution in #9097
- @ceddy4395 made their first contribution in #10870
- @barry-roomex made their first contribution in #10980
- @northvankiwiguy made their first contribution in #10985
Full Changelog: v3.4.0...v3.5.0