cloudposse/atmos v1.184.0

on GitHub

what

• Update !terraform.state YAML function
• Update docs
• Add tests

why

• Update !terraform.state YAML function
  • For s3 backend, add retry logic when reading the state from the s3 bucket (in case of an error, retry two more times before returning the error)
  • For s3 backend, increase the timeout from 5 sec to 30 sec (5 sec was not enough in some cases depending on network)
  • For s3 backend, cache the S3 client (and don't assume role many times). When a stack uses many !terraform.state functions, create just one S3 client (assume the backend role if provided) and reuse it for all !terraform.state functions. This reduces the execution time and the API calls to AWS STS

notes

• This PR improves the features implemented in:
  • #1363
  • #1374

test

The improvements in this release reduced the !terraform.state functions execution time at least 2-3 times.

For example, in a stack with 10 !terraform.output functions, the following command executed for more than a minute:

time atmos terraform plan <component> -s <stack>

real  1m13.344s
user  0m34.077s
sys   0m13.245s

When switched to the !terraform.state YAML functions, the same command executes in just 15 seconds, from which the 10 !terraform.state YAML functions took 3-4 seconds, the rest is terraform plan itself:

time atmos terraform plan <component> -s <stack>

real	0m15.796s
user	0m8.763s
sys	0m2.302s

With the improvements in this release, the same command executes in 13 seconds, from which the 10 !terraform.state YAML functions took 1-2 seconds:

time atmos terraform plan <component> -s <stack>

real	0m12.978s
user	0m8.828s
sys	0m2.133s

what

• Enhance the Atmos custom command flag handling in Atmos CLI with the following improvements:

  • Default Values for Custom Command Flags. Added support for specifying a default value for custom command flags:

    • If a default is provided, the flag will use this value when not set by the user
    • If no default is provided, string flags default to "", int flags default to 0, and boolean flags default to false

  • User Value Precedence:
    If a flag is set by the user, that value is always used, overriding any default.

  • Backward Compatibility:
    The original behavior is preserved for flags without a default value, ensuring no breaking changes for existing configurations.

Functionality added

• Custom commands can now define default values for their flags directly
• The CLI distinguishes between unset flags, user-supplied values, and default values for all supported types
• Improved robustness and flexibility for custom CLI commands in Atmos

why

• Default values are useful. Arguments cannot completely replace it and provide with the correct functionalities. The advantage of using flags is that they are prefixed (e.g., --flagname), which allows them to be specified independently and avoid ambiguity

what

• Move the logic for reading from stdin (pipe input) into a separate goroutine

why

• When Atmos reads large plan files in JSON format from stdin, the output can exceed the pipe buffer capacity. If the main thread does not consume this input concurrently, the buffer may become full, leading to a deadlock or the external command hanging. By moving the read operation into a goroutine, we ensure that the pipe is drained concurrently, preventing buffer overflow and avoiding the process getting stuck