cloudposse/atmos v1.198.0-rc.1

on GitHub

Example: atmos terraform plan vpc --stack <TAB> now only suggests
stacks that contain the vpc component.

• Modified stackFlagCompletion() to extract component from args
• Added listStacksForComponent() helper function
• Reuses existing FilterAndListStacks() logic from pkg/list
• Added comprehensive test case with fixtures
• Includes blog post documenting the feature

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com

Summary by CodeRabbit

• New Features

  • Tab completion for the --stack flag now filters stacks by a specified component, showing only relevant stacks.

• Tests

  • Added tests and snapshot updates to validate component-aware stack completion (including cases for existing, missing, and empty component).

• Documentation

  • Added a blog post describing component-aware stack completion with examples and usage across Terraform commands.

why

The Core Problem

In the main branch, developers must constantly decide:

• "Should this go to stdout or stderr?"
• "Is this data or a message?"
• "Do I need fmt.Fprintf(os.Stdout, ...) or fmt.Fprintf(os.Stderr, ...)?"

This creates inconsistency - different developers make different choices, leading to:

• Help text appearing on stderr in some commands, stdout in others
• Status messages mixed with data output
• Inconsistent use of colors and formatting

The Solution

The function name IS the destination. Developers no longer think about streams:

// ❌ OLD: Developer decides where output goes
fmt.Fprintf(os.Stderr, "Starting...\n")  // Why stderr?
fmt.Fprintf(os.Stdout, result)          // Why stdout?

// ✅ NEW: Function name encodes destination
ui.Write("Starting...\n")   // Obviously stderr (it's UI)
data.Write(result)          // Obviously stdout (it's data)

Key insight: ui.* functions always go to stderr. data.* functions always go to stdout. No decisions, no mistakes, no inconsistency.

Key Changes

Eliminates Stream Decision Making

Old pattern (main branch):

// Developer must know:
// - stdout vs stderr semantics
// - When to use which stream
// - How to format output consistently

fmt.Fprintf(os.Stderr, "Starting deployment...\n")
fmt.Fprintf(os.Stdout, "%s\n", jsonOutput)
fmt.Println("Done!")  // Wait, where does this go? stdout? Is that right?

New pattern (this PR):

// Developer thinks in terms of WHAT, not WHERE:
// - Is this a message for the user? → ui.*
// - Is this data to pipe/process? → data.*
// Function name = destination. No thinking required.

ui.Write("Starting deployment...\n")  // Message → stderr (automatic)
data.Write(jsonOutput)                // Data → stdout (automatic)
ui.Success("Done!")                   // Status → stderr with ✓ icon (automatic)

Enforces Consistency

Before: Each command could do it differently

// Command A
fmt.Fprintf(os.Stderr, "Error: %s\n", err)

// Command B
fmt.Fprintf(os.Stdout, "Error: %s\n", err)  // Wrong stream!

// Command C
fmt.Println("Error:", err)  // Wrong stream! No color!

After: All commands use the same pattern

// Command A, B, C - all identical, all correct
ui.Error(err.Error())  // Always stderr, always red, always ✗ icon

Complete API

Data Output (stdout - pipeable)

Use when: Output should be processed by other tools (jq, grep, etc.)

Available via pkg/io global writers:

• io.Data - Global stdout writer with automatic masking
• Can use with fmt.Fprintf(io.Data, ...) or pass to third-party libraries

Convenience functions via pkg/data:

• data.Write(text) - plain text → stdout
• data.Writef(format, args...) - formatted text → stdout
• data.Writeln(text) - text with newline → stdout
• data.WriteJSON(v) - JSON → stdout
• data.WriteYAML(v) - YAML → stdout

UI Output (stderr - human messages)

Use when: Output is for human eyes (status, errors, prompts)

• ui.Success(text) - ✓ text → stderr (green with icon)
• ui.Error(text) - ✗ text → stderr (red with icon)
• ui.Warning(text) - ⚠ text → stderr (yellow with icon)
• ui.Info(text) - ℹ text → stderr (cyan with icon)
• ui.Write(text) - plain text → stderr (no icon, no color)
• ui.Writef(format, args...) - formatted text → stderr
• ui.Writeln(text) - text with newline → stderr

Markdown Rendering

Function name encodes destination:

• ui.Markdown(content) - rendered markdown → stdout (help/docs are pipeable)
• ui.MarkdownMessage(content) - rendered markdown → stderr (error messages are UI)

Why two functions? Because markdown can be either:

• Documentation (help, usage) → stdout → use ui.Markdown()
• Messages (errors, warnings) → stderr → use ui.MarkdownMessage()

The function name tells you where it goes. No guessing.

Example: The Decision Elimination

Before - Developer must think about streams:

// 🤔 "Help text... is that data or UI? stdout or stderr?"
// 🤔 "Let me check what other commands do..."
// 🤔 "I think help should be pipeable, so stdout?"
fmt.Fprintf(os.Stdout, "%s\n", helpText)

// 🤔 "Status messages... definitely stderr, right?"
fmt.Fprintf(os.Stderr, "Processing...\n")

// 🤔 "JSON output... that's data, so stdout"
fmt.Fprintf(os.Stdout, "%s\n", jsonData)

// 🤔 "Success message... stderr? or stdout? should it be green?"
fmt.Fprintf(os.Stderr, "Done!\n")

After - Function name IS the answer:

// Help is documentation → Markdown → stdout
ui.Markdown(helpText)

// Status is a message → UI → stderr
ui.Write("Processing...\n")

// JSON is data → data channel → stdout
data.WriteJSON(result)

// Success is UI feedback → UI → stderr with formatting
ui.Success("Done!")

Zero cognitive overhead. The function name encodes:

• Destination (stdout vs stderr)
• Formatting (plain, colored, icons)
• Behavior (automatic masking, terminal detection)

Automatic Secret Masking

All output (stdout and stderr) is automatically masked using 8 built-in patterns covering common secrets:

• AWS credentials (AKIA*, access keys, session tokens)
• GitHub tokens (ghp_, gho_, github_pat_*)
• GitLab tokens (glpat-*)
• OpenAI API keys (sk-*)
• Bearer tokens
• Environment variables (AWS_SECRET_ACCESS_KEY, GITHUB_TOKEN, etc.)

Command line:

atmos terraform apply --mask=false  # Disable for debugging

Environment:

export ATMOS_MASK=false

Config file:

settings:
  terminal:
    mask:
      enabled: true
      replacement: "***MASKED***"

Precedence: --mask flag > ATMOS_MASK env > config > default (true)

Programmatic API:

import iolib "github.com/cloudposse/atmos/pkg/io"

// Register custom secrets
iolib.RegisterSecret("my-api-key-abc123")           // Masks literal + encodings
iolib.RegisterPattern(`COMPANY_KEY_[A-Z0-9]{32}`)   // Masks regex pattern
iolib.RegisterValue(os.Getenv("CUSTOM_SECRET"))     // Masks literal only

// Pass global writers to third-party libraries
logger := log.New(iolib.Data, "", 0)  // Automatically masked
bar := progressbar.NewOptions(100, progressbar.OptionSetWriter(iolib.UI))

Future: Config-based pattern registration (schema exists, loading not implemented yet):

settings:
  terminal:
    mask:
      patterns: ['ACME_[A-Z0-9]{32}']  # Custom regex patterns
      literals: ['my-secret']          # Literal values to mask

Benefits

1. Programmatic Consistency

• All commands use identical patterns
• ui.Success() always behaves the same way
• data.WriteJSON() always goes to stdout
• No per-command variations

2. No Stream Knowledge Required

• Don't need to understand stdout vs stderr
• Don't need to know POSIX conventions
• Function name = destination
• "Is this a message or data?" → use ui.* or data.*

3. Automatic Correctness

• Can't accidentally write data to stderr
• Can't accidentally write UI to stdout
• Can't forget to add colors/icons
• Secrets automatically masked

4. Simple Mental Model

Is it for users to read? → ui.*  (goes to stderr)
Is it for tools to process? → data.* (goes to stdout)

That's it. That's the whole model.

Architecture

• pkg/io/ - I/O context, streams, masking engine, global writers, terminal detection
• pkg/ui/ - Formatter, markdown renderer, stderr output functions
• pkg/data/ - Stdout output functions (wraps io.Data)
• pkg/terminal/ - Terminal capabilities (color, TTY, width)

All initialized automatically in cmd/root.go - commands just import and call.

Documentation

• Secret Masking Configuration - User guide for masking
• I/O and UI Output Guide - Decision tree and examples
• Logging Guidelines - UI vs logging distinction
• I/O Handling Strategy PRD - Architecture details
• Secrets Masking PRD - Implementation and future considerations
• CLAUDE.md - Developer guidelines

Testing

• 17+ test cases for I/O context and masking
• Unit tests for all output functions
• Terminal capability detection tests
• Integration tests with golden snapshots
• 42 help command snapshots verify --mask flag presence

references

• Eliminates "where should this go?" decisions
• Enforces programmatic consistency across all commands
• Makes correct output handling automatic and effortless
• Prevents secret leakage with automatic masking

Summary by CodeRabbit

• New Features

  • Automatic masking of sensitive data in command output (8 built-in patterns).
  • New global flags: --force-color, --force-tty, --mask (with env var support).
  • Clear separation of machine-readable data (stdout) and human UI output (stderr); improved UI formatting including Markdown rendering.
  • Global writers (io.Data, io.UI) for third-party library integration.
  • Programmatic API for registering custom secrets and patterns.

• Documentation

  • Added comprehensive I/O/UI, masking, terminal, and logging guides; published blog post about zero-config terminal output.

• Fixed atmos terraform generate backends to properly handle nested maps in HCL format
• Added recursive type converter GoToCty() in pkg/utils/cty_utils.go to convert Go types to cty values
• Simplified WriteTerraformBackendConfigToFileAsHcl() in pkg/utils/hcl_utils.go to use the new converter
• Added comprehensive test coverage with 4 new test functions and 13 sub-tests
• Created test fixture at tests/fixtures/scenarios/backend-nested-maps/
• Created PRD documentation at docs/prd/backend-nested-maps-support.md
• Created blog post at website/blog/2025-01-04-nested-backend-maps-support.mdx

why

• Users reported that nested maps like assume_role in S3 backend configurations were silently dropped when generating HCL format
• This prevented users from using advanced backend features like IAM role assumption, custom encryption keys, and other nested configurations
• The old implementation only handled primitive types (string, bool, int, float) and silently ignored complex types (maps, slices)
• JSON and backend-config formats worked correctly, but HCL format was broken due to incomplete type handling in the HCL generator

fix

Root Cause:
The WriteTerraformBackendConfigToFileAsHcl function used a type switch that only handled primitive types. When encountering a map[string]any or []any, it fell through the if-else chain and did nothing, effectively dropping the value.

Solution:
Created a recursive GoToCty() helper function that:

• Handles all Go types (primitives, maps, slices)
• Recursively converts nested structures
• Mirrors the existing CtyToGo() pattern for symmetry
• Supports arbitrary nesting depth

Before (40+ lines):

if v == nil { ... }
else if i, ok := v.(string); ok { ... }
else if i, ok := v.(bool); ok { ... }
// ... many more type checks
// ❌ No handler for maps or slices!

After (3 lines):

ctyVal := GoToCty(v)
backendBlockBody.SetAttributeValue(name, ctyVal)

tests

Added comprehensive test coverage to prevent regression:

1. TestBackendGenerationWithNestedMaps (3 sub-tests)

• Validates nested assume_role maps work in HCL format
• Tests JSON format via generateComponentBackendConfig helper
• Tests backend-config format with nested structures

2. TestBackendGenerationWithDifferentBackendTypes (3 sub-tests)

• S3 backend with assume_role configuration
• GCS backend with nested encryption_key configuration
• AzureRM backend with client authentication settings

3. TestBackendGenerationErrorHandling (2 sub-tests)

• Validates all three formats (hcl, json, backend-config)
• Tests graceful handling of empty backend configurations

4. TestGenerateComponentBackendConfigFunction (5 sub-tests)

• Cloud backend with {terraform_workspace} token replacement
• Cloud backend without workspace (no token replacement)
• S3 backend standard structure validation
• Critical test: preserves nested maps in backend config
• Local backend configuration

Test Results:

• All 56 backend-related sub-tests pass ✅
• Zero regressions in existing functionality ✅
• Zero linter issues ✅

Coverage Improvement:

• Before: 17 test functions, no nested map coverage
• After: 21 test functions (+4), 100% nested map coverage

real-world impact

Before (broken):

backend:
  s3:
    assume_role:
      role_arn: "arn:aws:iam::123456:role/terraform"

Generated HCL: Missing assume_role ❌

After (fixed):

terraform {
  backend "s3" {
    assume_role = {
      role_arn = "arn:aws:iam::123456:role/terraform"
    }
  }
}

Generated HCL: All nested fields present ✅

backward compatibility

✅ Fully backward compatible

• No breaking changes
• Existing configurations continue to work
• Only adds previously missing functionality
• All existing tests still pass

documentation

• Created comprehensive PRD: docs/prd/backend-nested-maps-support.md
• Created user-facing blog post: website/blog/2025-01-04-nested-backend-maps-support.mdx
  • Explains the problem with real examples
  • Shows before/after comparisons
  • Provides real-world use cases (S3 role assumption, GCS encryption, Azure configs)
  • Includes upgrade instructions
• Added inline code comments
• Test fixture serves as working example

references

• Terraform S3 Backend: https://developer.hashicorp.com/terraform/language/backend/s3
• HCL Write Library: https://pkg.go.dev/github.com/hashicorp/hcl/v2/hclwrite
• go-cty Library: https://pkg.go.dev/github.com/zclconf/go-cty/cty

Summary by CodeRabbit

• New Features

  • Preserve full nested maps in Terraform backend outputs (e.g., assume_role, encryption_key) across HCL, JSON, and backend-config formats.

• Bug Fixes

  • Fixed loss of nested backend configuration fields when generating HCL output.

• Documentation

  • Added product spec and blog post describing nested-backend-maps support and verification; updated integration docs with tooling version bump.

• Tests

  • Added extensive fixtures and tests covering nested maps, arrays, multiple backends, formats, and round-trip conversions.