Change Log

There's been quite a few changes in this release, highlights

• This is our first release to drop python 2.7 compatibility.

• lazy loading by default, greatly reduces cli and serverless cold
  start latency.

• docker images have been significantly trimmed using multi-stage
  builds with the distroless container.

• poetry is now being used for package management, existing workflows
  using pip/setuptools will work without any changes.

• pypi release artifacts are being published with fully frozen dependency graphs, to ensure
  repeatable installation over time.

Breaking Changes

Custodian strives for backwards compatibility, however in this release some planned deprecations and removals have been enacted.

• python 2.7 compatibility has been removed

• metrics and logs cli have been removed

aws

• aws - account - set password policy (#5634)
• aws - account -service-limit filter move to new trusted advisor checks (#5373)
• aws - accounts - add set glue catalog encryption action (#5539)
• aws - acm describe certs across all key types (#5363)
• aws - add additional usg region partitions (#5544)
• aws - add vpc filter to subnet and route-table (#5342)
• aws - ami policies can use server side query filters (#5570)
• aws - ami resolve template missing version key (#5458)
• aws - app-elb - metrics filter support net-elb and add example policy (#5614)
• aws - appelb - allow is-not-logging filter for network ELBs (#5612)
• aws - asg - allow msg in mark-for-op for consistency (#5399)
• aws - asg - handle null launch template versions (#5648)
• aws - backup vault resource (#5378)
• aws - cfn - add force delete to auto-disable termination protection (#5638)
• aws - cfn delete and set-protection retries (#5605)
• aws - change default lambda runtime to python3.8 (#5291)
• aws - check-permissions - respect permission boundaries by default (#5531)
• aws - cloudfront - distribution-config filter (ie. check logging) (#5577)
• aws - cloudfront add update-distribution action (#5390)
• aws - cw log delete use retry / ignore on missing (#5349)
• aws - cw log set retention action use retry (#5339)
• aws - cwe/eventbridge support passthrough on event pattern matching (#5412)
• aws - directconnect tag augment and filter/actions (#5575)
• aws - dms-endpoint add tag augments and filter/actions (#5526)
• aws - ebs - skip missing volumes in get_resources(vol_ids) (#5650)
• aws - ec2 - allow disabling implicit state filter in offhours filter (#5337)
• aws - ec2 - set-monitoring action (#5268)
• aws - ec2 - stop action - allow hibernation of instances (#5348)
• aws - ec2 ssm compliance filter (#5472)
• aws - ecr - fix lifecycle policy rule validation (#5595)
• aws - efs - configure lifecycle policy action (#5275)
• aws - efs - lifecycle-policy filter (#5302)
• aws - eks - delete nodegroups & fargate profiles (#5585)
• aws - elastic-ip alias for network-addr resource (#5541)
• aws - elasticip - use allocation id for filtering (#5630)
• aws - emr - security-configuration resource (#5643)
• aws - fix copy-related for universal/resourcegroup tags (#5607)
• aws - fix filter merge_annotation for iam access keys (#5535)
• aws - get resources sans ids returns empty set (#5545)
• aws - glue - security configuration filter (#5465)
• aws - glue dev endpoint - subnet filter (#5572)
• aws - glue-catalog - cross-account filter (#5622)
• aws - glue-catalog as its own resource with encryption actions/filters (#5573)
• aws - iam - credential report - include inactive keys w/ create dates (#5551)
• aws - iam role implement get-resources (#5384)
• aws - iam usage fix for match-operator all partial matches (#5449)
• aws - iam user & role - set permissions boundary action (#5532)
• aws - internet-gateway - delete action (#5582)
• aws - invoke-lambda - add execution context to payload (#5276)
• aws - kafka - normalize tag formatting (#5401)
• aws - kafka set-monitoring and tag actions (#5386)
• aws - lambda - kms key filter (#5624)
• aws - last-write retry on describe streams (#5600)
• aws - mu - add retries for provisioning ops on targets for cwe rules (#5656)
• aws - new glue resources (#5523)
• aws - only append region suffix if not using custom paths w/ region (#5616)
• aws - provisioning policy output contains region (#5481)
• aws - qldb resource w/ tag and delete actions (#5411)
• aws - rds - add validation for monitoring interval and monitoring role arn (#5609)
• aws - rds cluster snapshot - cross-account filter (#5640)
• aws - rds cross-account snapshot retry (#5603)
• aws - redshift pause/resume and offhours support (#5442)
• aws - revert security hub product arn change (#5444)
• aws - s3 - check-public-block fix annotation usage; add state for set-public-block (#5580)
• aws - s3 - config source normalization - include replication account id (#5381)
• aws - s3 - enhance toggle-logging and add logging filter (#5206)
• aws - s3 - set public block - refactor (#5520)
• aws - s3 - set-inventory - support additional optional fields (#5318)
• aws - s3 - set-inventory action support for additional formats (#5312)
• aws - s3 - set-replication action (#5387)
• aws - s3 individual bucket public block filter & action (#5451)
• aws - sagemaker - kms key filter (#5626)
• aws - security group - set-permissions action to enable add/remove rules (#4973)
• aws - security hub - post-finding add support for severity label (#5589)
• aws - security-group - ingress/egress filter SGReferences to filter by other group attributes (#5604)
• aws - security-group - metadata for name use GroupName instead of GroupId (#5341)
• aws - security-group used/unused take into account ecs cloud watch event targets (#4846)
• aws - securityhub - change product-arn from default to cloud-custodian (#5500)
• aws - securityhub - use assigned product arn (#5431)
• aws - sqs modify-policy action (#5546)
• aws - ssm parameter fix arn generation (#5392)
• aws - tests - replace flight recording account id swap from fake to test id (#5594)
• aws - use partition when generating lambda policy iam role arn (#5253) (#5254)
• aws - vpc - set-flow-log - create log group if non-existent (#5645)
• aws - vpc - set-flow-log - enable setting aggregation interval (#5455)
• aws - vpc endpoints - resource group tag actions (#5367)
• aws - waf - tag support (#5590)
• aws - xray usage docs and updates (#5467)
• aws.elasticache-group - support elasticache replication group resource (#5319)

azure

• Azure - Indirect Dependency (#5345)
• azure - Update max tag count to 50 (#5416)
• azure - actions - fix automatic creator tagging (#5298)
• azure - async azure functions build + helper script to wait for status (#5418)
• azure - azure functions using remote build (#5393)
• azure - docs - example gitops workflow (#4886)
• azure - fix consistent failures in live functionals (#5308)
• azure - fix lazy load output plugin regression (#5328)
• azure - fix some issues related to lazy load (#5394)
• azure - fix storage set-firewall-rules action (#5305)
• azure - nightly tests fix (#5578)
• azure - unify autotag for event resource (#5446)
• azure - fix cache key (#5447)

core

• c7n-org - azure subscription config generator checks subscription state (#5615)
• cli - fix schema output with no args, remove argcompletion around schema (#5608)
• cli - report - fix --raw output on py3 encode (#5274)
• cli - run - add dry-run alias for -dryrun (#5518)
• cli - schema command using lazy load (#5599)
• cli - version --debug shows installed depgraph versions (#5542)
• core - add resources.load_available that will load installed providers/resources (#5343)
• core - aws notify implementation only register with aws resources (#5413)
• core - examples doc test improvements (#5441)
• core - fix lazy load schema gen for empty policy set (#5504)
• core - fix value_from/resolver cache usage (#5548)
• core - generate requires fixes for python_version (#5327)
• core - lazy load resources (#5032)
• core - new data provider (#5601)
• core - policy execution conditions (#4466)
• core - remove metrics/logs infra and mark cli commands as obsolete (#5487)
• core - remove pkg_resources from custodian_archive (#5493)
• core - remove py2 support syntax (#5528)
• core - remove py2.7 compat vendored ipaddress module (#5479)
• core - serverless incorporate lazy loading (#5247)
• core - switch registry subscriber notify to on resource load (#5382)
• core - use poetry for dependency management (#5320)
• core - use set comprehensions directly when creating a set (#5651)
• core - value filter better handling of millisecond timestamps (#5369)
• core - vendored ipaddress use 3.8 compatible interpolation (#5428)

docs

• docs - add github actions ci badge to readme (#5563)
• docs - advanced usage info - fix typos and yaml errors (#5368)
• docs - document editor integration via jsonschema and yaml language server (#5488)
• docs - editor integration - fix links (#5519)
• docs - fix aws iam credential example (#5435)
• docs - fix broken links (#5496)
• docs - fix example for aws config source (#5641)
• docs - fix spelling errors (#5652)
• docs - fixed example policy iam permission typo (#5490)
• docs - generate execution mode reference docs (#5623)
• docs - quickstart - enhance the language server documentation (#5506)
• docs - remove extraneous permission (#5565)
• docs - sns kms filter example and typo fix (#5569)

gcp

• gcp - audit mode - support event actions and metrics (#5156)
• gcp - cscc handle svc breaking change w/ source properties validation (#5338)
• gcp - delete log sinks action (#5181)
• gcp - detach disks action for instances (#5102)
• gcp - docs - regex example update description, add quotes and more examples (#5263)
• gcp - reporting fields for all resources (#5629)

release engineering

• releng - 0.8 release branch github actions for ci (#5362)
• releng - add a docker build and test to ci (#5613)
• releng - docker build pipeline fix tag cli option (#5637)
• releng - docker build pipeline updates (#5617)
• releng - docker building compatible with setuptools scm sans .git dir (#5284)
• releng - docker image building refactor (#5571)
• releng - docker image includes k8s provider (#5282)
• releng - docker images retain source dirs for editable distributions (#5485)
• releng - docker images using multi-stage build with distroless base target image (#5515)
• releng - drone doc build compatible with setuptools scm (#5287)
• releng - fix azure lrucache backports dep for py<3.3 (#5331)
• releng - github actions for ci on master (#5388)
• releng - makefile use cd instead of pushd/popd for portability (#5505)
• releng - minimize syscalls/disk writes during tests (#5358)
• releng - move more ci to github actions (#5574)
• releng - pin pytest, remove xray monkey, add pytest-sugar (#5620)
• releng - remove problematic wheel pin (#5453)
• releng - remove travis config file and py27 from tox (#5557)
• releng - switch to setuptools scm for managing version.py (#5279)
• releng - temporarily disable cache (#5406)
• releng - tools/dev/changelog - support filtering by user and end date (#5432)
• releng - update base docker image to python 3.8 debian buster slim (#5433)
• releng - upgrade dependencies to resolve ci cache issue (#5550)
• releng - use dev suffix on fallback version to prevent pypi installs (#5299)
• releng - use github issue templates (#5356)
• releng- downgrade and pin sphinx, upgrade rest of dependency set (#5475)
• ci - aws test recording - auto anonymize and slim (#5561)
• ci - workaround github actions matrix/include regression (#5463)

tools

• tools/c7n-mailer - fix format util for cloudtrail (#5272)
• tools/c7n-org - aws - org account gen script allow for ignoring set of accounts (#5402)
• tools/c7n-org - python 3.8 osx compatibilty with multiprocessing spawn (#5353)
• tools/c7n_mailer - allow multiple emails to be specified via tag value ":" separators (#5448)
• tools/c7n_mailer - azure send grid delivery fix for multiple recipients (#5376)
• tools/c7n_mailer - sendgrid for AWS (#5434)
• tools/c7n_mailer - switch default runtime to python3.7 (#5543)
• tools/c7n_mailer - switch ruamel dependency to pyyaml (#5521)
• tools/c7n_org - allow account-id/project-id/subscription-id filtering in addition to name (#5311)
• tools/c7n_org - fix/remove old region condition check (#5514)
• tools/c7n_policystream - fix diffing policies in sub directories (#5372)
• tools/c7n_policystream - pin pygit2 versions for docker builds (#5410)