cloudposse/atmos v1.197.0-rc.1

on GitHub

Added comprehensive support for authentication with workflows and custom commands:

• Workflows: Supports per-step identity configuration with identity field and --identity flag override
• Custom Commands: Supports identity configuration in command definition with --identity flag override
• Integration Tests: Added 8 integration tests demonstrating mock provider authentication end-to-end
• Mock Provider: Updated to set ATMOS_IDENTITY for testing
• Documentation: Added sections explaining authentication usage in workflows and custom commands

Changes

• Added identity field to WorkflowStep schema
• Added identity field to Command schema
• Implemented ExecuteWorkflow identity resolution with per-step and command-line overrides
• Added automatic --identity flag to all custom commands
• Updated mock provider to set ATMOS_IDENTITY environment variable
• Fixed stubAuthManager.PrepareShellEnvironment() to properly merge environment variables
• Created workflow integration tests with mock provider
• Created custom command integration tests with mock provider
• Updated documentation with authentication workflow examples

Test Results

• ✅ 8/8 integration tests passing
• ✅ All pkg/auth tests passing (66 tests)
• ✅ Code compiles successfully
• ✅ No regressions introduced

Testing

Run the following to verify the changes:

# Test workflows with authentication
go test -v ./internal/exec -run TestWorkflowIntegration

# Test custom commands with authentication  
go test -v ./cmd -run TestCustomCommandIntegration

# Test all auth functionality
go test -v ./pkg/auth

# Build to verify compilation
go build .

Added defer perf.Track() calls to public functions in the GCS Terraform backend implementation to comply with Atmos coding standards. Also updated the lintroller to exclude the internal/gcp package from perf.Track() linting rules to prevent an import cycle.

Changes

• Added perf.Track() to 7 public functions in internal/terraform_backend/terraform_backend_gcs.go
• Updated lintroller exclusions in tools/lintroller/rule_perf_track.go to exclude internal/gcp package
• Reason: import cycle would be created (internal/gcp → pkg/perf → pkg/schema → pkg/store → internal/gcp)

Test plan

• Lint passes without errors
• Code builds successfully
• All unit tests pass
• Pre-commit hooks pass

Summary by CodeRabbit

• Chores
  • Added performance tracking infrastructure to internal backend operations to monitor execution time.
  • Updated linter configuration to optimize dependency handling.

Note: These changes are internal infrastructure improvements with no visible impact to end-users.

Comprehensive audit and fix of unconditional test output that was creating walls of JSON/YAML and debug info in CI logs. All verbose output now respects test verbosity settings by using t.Cleanup() handlers with t.Failed() checks - output only appears when tests actually fail.

Root Issue: Tests were using fmt.Print*() and unconditional t.Log() to dump captured command output, terraform plans, schema validation results, and debug info. These bypass Go's test verbosity controls and always output to CI logs.

Solution: Wrapped all verbose output in t.Cleanup() handlers that only log when t.Failed() is true, following the pattern established in PR #1704.

What Changed

15 Test Files Fixed

Terraform Output Dumps:

• internal/exec/terraform_test.go - terraform plan output
• tests/cli_terraform_test.go - terraform apply stdout/stderr

Command Output Captures:

• cmd/root_test.go - command output
• tests/validate_schema_test.go - schema validation output

Config/Command Debugging:

• pkg/merge/merge_context_demo_test.go - error formatting demos
• pkg/config/command_merging_behavior_test.go - command structure debugging
• pkg/config/command_merge_core_test.go - command verification output

Provenance Parser Debug Output:

• pkg/provenance/yaml_parser_multiline_test.go - pathMap dumps
• pkg/provenance/yaml_parser_arrays_test.go - 5 instances of pathMap iteration logging

Pattern Applied

All unconditional output converted to:

t.Cleanup(func() {
    if t.Failed() {
        t.Logf("Debug info: %s", output)
    }
})

Impact

• ✅ CI logs dramatically quieter - no verbose output on successful test runs
• ✅ Debug info preserved - still shows when tests fail
• ✅ Consistent with PR #1704 - same pattern across entire codebase
• ✅ Fixes pre-existing issues - some code dated back to May 2025

Testing

• Code compiles without errors
• All changes follow established PR #1704 pattern
• 3 clean, focused commits

Related to PR #1704 (quiet test output on success).

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

Comprehensive MFA documentation, bug fixes, and credential precedence improvements for AWS IAM user authentication.

Problems Discovered

1. MFA ARN Ignored When !env Variables Empty (BUG)

User Report: MFA configured in YAML but no TOTP prompt appeared when !env variables were empty.

Root Cause:

• User had mfa_arn: "arn:aws:iam::...:mfa/device" in YAML
• User had access_key_id: !env AWS_ACCESS_KEY_ID (empty environment variable)
• When !env returned empty string, code fell back to keyring credentials
• Keyring credentials had no MFA ARN → MFA ARN from YAML completely ignored
• Result: No TOTP prompt despite MFA being configured

Fix: Implemented deep merge precedence that overrides keyring MFA ARN with YAML MFA ARN when using keyring credentials.

2. Excessive WARN Logs for Empty Environment Variables

User Report: 8+ WARN messages for every empty !env function result.

Root Cause: Empty environment variables are normal (not yet configured, different environment) but were logged as warnings.

Fix: Changed all empty value logs from WARN → DEBUG level.

3. Unclear Credential Precedence

Issue: Ambiguous rules for which credential source wins (YAML vs keyring).

Fix: Implemented clear per-field deep merge precedence with comprehensive tests.

Solutions Implemented

1. Deep Merge Credential Precedence

Clear Rules:

1. YAML Complete (both access_key_id and secret_access_key non-empty)

   • Use YAML entirely (access keys + MFA ARN from YAML)
   • Keyring ignored

2. YAML Empty (both keys empty or omitted)

   • Use keyring credentials as base
   • Override MFA ARN from YAML if present ← Fixes the bug
   • Allows version-controlled MFA config with keyring-stored secrets

3. YAML Partial (only one key present)

   • Error: Both keys must be provided or both empty

Recommended Pattern:

identities:
  prod-admin:
    kind: aws/user
    credentials:
      # Access keys in keyring (via atmos auth user configure)
      # MFA ARN in YAML (version controlled)
      mfa_arn: arn:aws:iam::123456789012:mfa/username
      region: us-east-1

Benefits:

• Secure local credential storage (keyring)
• Shared team MFA configuration (YAML)
• Clear separation of secrets vs config

2. Comprehensive Test Suite

Added 6 table-driven tests validating:

• ✅ All 3 precedence rules
• ✅ Error conditions (partial credentials)
• ✅ Empty !env variable handling (user's bug scenario)
• ✅ MFA ARN override behavior

3. Reduced Log Noise

Changed empty value logs in pkg/config/process_yaml.go:

• !env empty: WARN → DEBUG
• !exec empty: WARN → DEBUG
• !include empty: WARN → DEBUG
• !repo-root empty: WARN → DEBUG

4. Enhanced MFA Documentation

website/docs/cli/commands/auth/usage.mdx

• "Multi-Factor Authentication (MFA) for AWS" subsection
• Configuration examples (YAML, env var, keyring)
• Step-by-step guide to find MFA device ARN
• Authentication flow with TOTP prompt visualization
• Security model explanation
• Troubleshooting guidance

website/docs/cli/commands/auth/auth-user-configure.mdx

• Enhanced MFA section with detailed guides
• Three configuration methods explained
• Security considerations

pkg/auth/docs/PRD/PRD-Atmos-Auth.md

• Deep merge precedence rules documented
• Configuration options with examples
• 5-step authentication flow
• Implementation details with file references

Files Changed

Bug Fixes:

• pkg/auth/identities/aws/user.go: Implement deep merge precedence
• pkg/config/process_yaml.go: Change empty value warnings to debug

Tests:

• pkg/auth/identities/aws/user_test.go: Add 6 comprehensive precedence tests

Documentation:

• website/docs/cli/commands/auth/usage.mdx: MFA section added
• website/docs/cli/commands/auth/auth-user-configure.mdx: Enhanced MFA docs
• pkg/auth/docs/PRD/PRD-Atmos-Auth.md: Precedence rules + XDG path fix

Testing

✅ All 24 AWS user identity tests pass (6 new + 18 existing)
✅ Compiled successfully
✅ No breaking changes
✅ User's bug scenario validated in tests

Key Points

✅ Fixed MFA bug - YAML MFA ARN now works with keyring credentials
✅ Clear precedence - Documented and tested deep merge rules
✅ Reduced noise - Empty env vars no longer spam warnings
✅ Comprehensive docs - MFA setup, security model, troubleshooting
✅ Formalized tests - Precedence rules enforced by tests

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

Summary by CodeRabbit

• New Features

  • Optional MFA for AWS IAM users via configurable MFA device ARN with interactive TOTP and temporary session tokens.
  • Configurable session duration in multiple formats (seconds, Go duration, days); default 12h, up to 36h with MFA.
  • Credential storage migrated to XDG-style config paths and YAML/keyring deep-merge behavior.

• Bug Fixes

  • CLI flags now correctly take precedence over config/env values.
  • Identity listing preserves original name casing.

• Documentation

  • Expanded MFA, session duration, CLI guidance, examples, and troubleshooting.

• Tests

  • Added coverage for MFA flows, duration parsing, merge rules, identity casing, and YAML edge cases.

Document how to use Terraform's development overrides feature with Atmos for testing custom provider versions locally. This clarifies the distinction between provider configuration (via the providers section in Atmos stacks) and development overrides (via .terraformrc configuration).

Changes

• Enhanced Provider Documentation: Added "Local Provider Development with Dev Overrides" section to the provider configuration guide with step-by-step setup instructions, examples, and troubleshooting tips.

• New Design Pattern: Created comprehensive "Provider Development Pattern" documentation covering the workflow, best practices, team collaboration patterns, and real-world examples.

Why This Matters

This resolves the confusion from #1726 about how to test custom providers without publishing development versions to a registry.

Key Points Documented:

1. The Distinction: The providers section in Atmos stack manifests controls provider behavior (credentials, regions, etc.), while Terraform's dev_overrides controls where to find provider binaries

2. Two Separate Mechanisms:

   • providers in stack YAML → serialized to providers_override.tf.json (Atmos feature)
   • dev_overrides in .terraformrc → points to local binaries (Terraform CLI feature)

3. How They Work Together:

   • Use .terraformrc with dev_overrides to point to your local provider binary
   • Set TF_CLI_CONFIG_FILE environment variable (in component's env section)
   • Use providers section for provider configuration as usual
   • Terraform uses your local binary while Atmos configures its behavior

Test Plan

• Documentation builds successfully with npm run build
• All tests pass with go test ./...
• No regressions introduced
• Both technical reference and design pattern created for comprehensive coverage

Related Issues

Closes #1726

🤖 Generated with Claude Code

Summary by CodeRabbit

• Documentation
  • Added a comprehensive Provider Development guide describing local Terraform provider development and testing using development overrides, with end-to-end workflow, best practices, troubleshooting, and CI guidance.
  • Added a new Provider Development page.
  • Inserted an in-depth "Local Provider Development with Dev Overrides" section into core concepts for quick reference (guidance appears in two places).

Fix authentication caching bug where users with multiple identities using the same provider had to re-authenticate for each identity, even though provider credentials were already cached.

Impact: Users with multiple identities using IAM Identity Center (or other providers) no longer need to go through browser authentication for each identity switch - provider credentials are now properly reused.

Root Cause

The authenticateProviderChain function had a boundary condition error:

• It only fetched cached credentials when actualStartIndex > 0 (identity level)
• But provider credentials are cached at index 0
• This caused second/subsequent identities using the same provider to re-authenticate the provider unnecessarily

The Fix

• Changed condition from if actualStartIndex > 0 to if actualStartIndex >= 0
• Now fetches cached credentials from any level, including the provider level
• Added nil check in fetchCachedCredentials to handle edge cases

Expected Behavior

1. First identity login: Authenticate provider + cache both provider and identity credentials
2. Second identity with same provider: Reuse cached provider credentials + derive new identity credentials
3. Provider authentication only happens once per session, not per identity

Test Plan

• Added regression test TestAuth_MultipleIdentitiesSameProvider_ProviderCacheReuse to verify provider credentials are reused
• All existing auth-related tests pass
• Code compiles without errors
• Verified with mock provider that second identity authentication is instant

Fixes the issue where users with multiple identities using the same IAM Identity Center provider had to repeatedly authenticate.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests

  • Added a regression test ensuring provider credentials are cached and reused when multiple identities authenticate via the same provider.

• Bug Fixes

  • Improved authentication caching and retrieval so shared-provider logins reuse cached credentials, speeding repeat sign-ins and avoiding credential lookup errors when the cache/store is absent.

Ensure identities and providers are displayed in lexicographical (alphabetical) order across all listings and completion functions.

• Sort identities and providers in shell completion for --identity flag
• Sort identities in shell completion for identity positional arguments
• Sort providers in shell completion for --providers flag in auth list
• Sort identities in shell completion for --identities flag in auth list
• Interactive identity selector prompts already sort (unchanged)

Implementation

Go maps are inherently unordered, so we now explicitly sort identity and provider names whenever they are extracted from maps before display to users.

Changes:

• Added sort.Strings() calls to providersFlagCompletion() in cmd/auth_list.go
• Added sort.Strings() calls to identitiesFlagCompletion() in cmd/auth_list.go

Testing

Added comprehensive tests to verify sorting behavior:

• TestProvidersFlagCompletion_ReturnsSortedProviders - Creates providers in non-alphabetical order and verifies completion returns them sorted
• TestIdentitiesFlagCompletion_ReturnsSortedIdentities - Creates identities in non-alphabetical order and verifies completion returns them sorted
• TestIdentityArgCompletionSorting - Verifies identity argument completion returns sorted results
• TestIdentityArgCompletionOnlyFirstArg - Verifies completion only works for first positional argument

All existing tests pass, including the existing TestIdentityFlagCompletionSorting test.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Authentication configuration now validates provider and identity kinds on startup.
  • Shell completion for providers and identities returns consistently sorted results.

• Tests

  • Added comprehensive test coverage for authentication configuration validation.
  • Added tests for shell completion sorting behavior.

• Chores

  • Removed lintroller from pre-commit configuration.