cloudposse/atmos v1.229.0-rc.5

on GitHub

• Measures CPU, memory, and other resource usage for terraform plan/apply/deploy from the actual subprocess tree (the terraform/tofu process and everything it spawns, e.g. provider plugins) instead of only the Atmos CLI wrapper's own negligible usage.
• Prints a local ui.Info summary line after each terraform plan/apply/deploy run, plus one aggregate summary at the end of the whole atmos invocation covering every subprocess spawned during the run (e.g. every component in a multi-component --affected plan).
• Both new local displays are controlled by a new settings.metrics.enabled setting (default true); it never affects the Atmos Pro upload.
• The resource-usage numbers uploaded to Atmos Pro's command-execution metadata endpoint now reflect the combined terraform-subprocess + Atmos self usage instead of only Atmos's own usage; commands with no subprocess (e.g. describe affected) are unaffected.
• Adds a settings-reference docs page, a changelog post, and a CI/CD Simplification roadmap entry for the feature.

why

• The command-execution metadata feature (#2926) measured only the Atmos CLI process's own resource usage via RUSAGE_SELF, which excludes the terraform subprocess entirely — the actual expensive part of any plan/apply/deploy run. This restores the subprocess-tree measurement approach from the feature's original design (#2217, closed unmerged) at the shared execution funnel every component type goes through, so the reported numbers reflect what a run actually cost, both locally and in Atmos Pro.

references

• Related: #2217 (original subprocess resource-metrics design, closed unmerged)
• Related: #2926 (shipped the Atmos-self-only version this PR fixes)

Summary by CodeRabbit

• New Features

  • Added resource-usage metrics for Terraform runs and child processes, including elapsed time, CPU usage, and peak memory.
  • Added per-command and aggregate summaries for Atmos invocations.
  • Included metrics in Atmos Pro execution metadata and Native CI GitHub Actions job summaries.
  • Added resource-usage details to Terraform plan, apply, and deploy output, with peak memory identified as the largest observed process.

• Configuration

  • Added settings.metrics.enabled, enabled by default, to control local metric summaries.

• Documentation

  • Added guidance for metric output and disabling summaries in scripts or CI.

• Add explicit live-GitHub canaries (tests/live_github_canary_test.go): an unauthenticated vendor pull of cloudposse/terraform-null-label, its authenticated twin, an unauthenticated toolchain release-asset install (peteretelej/tree, the tool behind the recent bootstrap 404), and an unauthenticated !include of a raw GitHub file. Each drives the built atmos binary with every GitHub credential source scrubbed (GITHUB_TOKEN/ATMOS_*_TOKEN/GH_TOKEN blanked, GH_CONFIG_DIR pointed at an empty dir to defeat the gh auth token fallback) and the local git-mirror rules stripped, so they genuinely exercise the unauthenticated routes against real GitHub.
• Transient conditions (DNS/connect/timeout/TLS, 429/rate limit, 5xx) skip the canary with the matched signature; real 401/403/404 semantics or atmos bugs fail it. The classifier is unit-tested against canned stderr.
• Implement ATMOS_TEST_OFFLINE (documented in docs/prd/test-preconditions.md but never wired up): RequireGitHubAccess, RequireNetworkAccess, and the new RequireLiveGitHub/RequireLiveGitHubAuthenticated skip under it, independently of ATMOS_TEST_SKIP_PRECONDITION_CHECKS (which CI sets and which only bypasses the connectivity probes).
• Register live_github / live_github_authenticated as YAML test-case preconditions; the harness scrubs auth and removes the mirror's insteadOf rules for those cases (gitconfigenv.Without/IsInsteadOfEntry, unit-tested).
• Give .github/actions/ci-toolchain's atmos toolchain install step one bounded retry: atmos toolchain install skips tools already on disk, so the retry only re-attempts what failed (e.g. a transient release-asset download error). No shell loop.
• Docs: fix CLAUDE.md's tests/test_preconditions.go → tests/preconditions.go; document ATMOS_TEST_OFFLINE and the two preconditions accurately.

why

• Earlier PRs in this stack move the suite onto a local git mirror and (next) a local GitHub HTTP façade. We still want a small, explicit set of tests that hit real, unauthenticated GitHub — GitHub's recent unauthenticated-traffic protections are exactly the kind of upstream change those must catch — without letting a network blip fail a PR shard.
• Canaries stay in the normal PR shard matrix by decision (no scheduled workflow); the transient-skip classifier and the offline switch are what make that acceptable.
• The ci-toolchain retry addresses the actual failure that started this work: CI's own toolchain bootstrap getting a one-off HTTP 404 on a release asset and turning a shard red with no retry.

references

• Stacked on #3107 (GHES support) and #3105 (local git mirror).
• docs/fixes/2026-08-10-github-transient-error-tls-cert-flake.md — the transient-vs-real pattern the canaries follow.

Summary by CodeRabbit

• New Features

  • Added an offline testing option that skips network-dependent checks.
  • Added live GitHub canary coverage for authentication, vendoring, tool installation, and remote includes.
  • Added safeguards to isolate credentials and configuration during live GitHub tests.

• Bug Fixes

  • Improved handling of transient GitHub and network failures during canary tests.
  • CI tool installation now retries once after a failed attempt.

• Documentation

  • Updated testing and precondition guidance, including live GitHub testing instructions and offline-mode usage.

• Why now: GitHub's new protections against unauthenticated traffic (https://github.com/orgs/community/discussions/206581#discussioncomment-18269356) turn every unauthenticated fetch of a public repo into a flake or a 401; the same pressure shows up downstream in hashicorp/terraform#39130 (hashicorp/terraform#39130), where users ask for SSH module fetching because unauthenticated HTTPS to GitHub is now rate limited. Tests that hit github.com anonymously are no longer viable as a default; live access is a canary concern, not a per-PR one.

• Extend tests/testhelpers/httpmock.GitHubMockServer into a small GitHub HTTP façade: releases/tags API (with Link pagination), /api/v3/rate_limit with X-RateLimit-* headers on every API response, release-asset and archive downloads, GHES-shape raw content (/raw/{owner}/{repo}/{ref}/{path}), and an aqua-registry index + per-package files. Helpers to register tools/assets/raw files, build tiny tar.gz/zip archives, inject failures (FailWith, FailWithTimes, FailWithHeaders), set the rate limit, inspect the request log, and export the five routing env vars for subprocess tests (EnvForSubprocess/Setenv). Legacy RegisterFile/Transport/HTTPClient unchanged.
• Move tests whose subject is the install / registry / raw-fetch flow onto the façade: jq install mechanics (TestToolchainCustomCommands_InstallJQViaMock, the jq entry of TestToolchainAquaTools_InstallAllTools), TestToolchainAquaTools_NonExistentToolError, and a new !include case against a sibling fixture (atmos-include-yaml-function-mock) routed at the mock via GITHUB_SERVER_URL/GITHUB_API_URL. The original raw.githubusercontent.com !include case is byte-identical to main and stays live — atmos evaluates every manifest matched by included_paths on any stack resolution, so a co-located mock stack would have forced the live fetch anyway.
• Harness: YAML test-case env: values now expand ${VAR} against the process environment (expandTestCaseEnv, unit-tested); TestMain starts one process-wide façade exported as ATMOS_TEST_GITHUB_MOCK_URL — deliberately not the routing vars themselves, so the real toolchain bootstrap and the live canaries are unaffected.
• Deterministic coverage of rate-limit and unauthenticated routes against the façade, asserting current behavior: pkg/github GetLatestRelease/GetReleases on 401 / 404 / 429; the three live _Integration rate-limit tests in pkg/github/ratelimit_test.go become _ViaMock (+ an exhausted-but-reset case); the downloader's pre-fetch rate-limit check fires only for GitHub URLs and doesn't block on a passed reset. Aqua's 403→unauthenticated-retry fallback and the installer's 4xx/5xx handling already had deterministic httptest coverage — left as is.

why

• With the GHES endpoints resolver (#3107) every toolchain/registry/raw call site builds URLs from GITHUB_SERVER_URL / GITHUB_API_URL / ATMOS_TOOLCHAIN_*, so a plain http://127.0.0.1:<port> server can stand in for GitHub for both in-process and built-binary tests. This PR is the test-side half of that seam.
• The tests that moved never tested GitHub; they tested atmos's install/registry/fetch mechanics and paid for it with live-network flakiness. The ones that do test GitHub (kubectl naming from the real aqua registry, *_LiveNetwork, the live !include case) stay live as canaries (#3109).
• Verified: every moved test passes under HTTPS_PROXY pointed at a closed port (no GitHub reachable); the untouched live !include case fails under that same guard, proving it is still genuinely live.

references

• Stacked on #3109, #3107, #3105.
• docs/fixes/2026-08-10-github-transient-error-tls-cert-flake.md (rate-limit / transient handling background).

Summary by CodeRabbit

• Bug Fixes

  • Improved GitHub rate-limit handling so requests proceed promptly when limits have already reset.
  • Non-GitHub downloads now skip unnecessary GitHub rate-limit checks.
  • Improved handling of bracketed IPv6 hosts, including addresses with ports.

• Reliability

  • Improved reliability for GitHub release retrieval, raw-file downloads, archives, Aqua registry integrations, and tool installation.
  • Expanded validation across varied network responses, authentication states, rate limits, and supported platforms.
  • Improved consistency across supported operating systems and GitHub-compatible environments.

• Why now: GitHub's new protections against unauthenticated traffic (https://github.com/orgs/community/discussions/206581#discussioncomment-18269356) turn every unauthenticated fetch of a public repo into a flake or a 401; the same pressure shows up downstream in hashicorp/terraform#39130 (hashicorp/terraform#39130), where users ask for SSH module fetching because unauthenticated HTTPS to GitHub is now rate limited. Tests that hit github.com anonymously are no longer viable as a default; live access is a canary concern, not a per-PR one.
• Also carries one small production fix found while routing the mock case at the configured host: pkg/github Endpoints.Host was built from url.Hostname() (port stripped) while IsHost preserved non-default ports on the candidate, so a GHES on a custom port could never match its own configured URL. Now built from url.Host; covered by TestRepoEndpoints_NonDefaultPortPreserved. Default-port and portless configurations are unaffected.

• Add a single GitHub endpoints resolver (pkg/github/endpoints.go) that reads the standard GITHUB_SERVER_URL / GITHUB_API_URL variables (the ones GitHub Actions exports on both github.com and GitHub Enterprise Server) and route every place Atmos talked to your repositories through it: the CI provider, remote imports and vendoring raw fetches, the GitHub API client (releases, tags, artifacts, archived checks), the token host allowlist, and token injection for git operations.
• Keep the toolchain a separate concern: aqua-registry tools live on public github.com even when your repos are on GHES, so atmos toolchain install does not follow GITHUB_SERVER_URL. It gets its own env-only knobs — ATMOS_TOOLCHAIN_GITHUB_URL, ATMOS_TOOLCHAIN_GITHUB_API_URL, ATMOS_TOOLCHAIN_AQUA_REGISTRY_URL — for corporate release proxies/mirrors.
• Attach the GitHub token to release-asset downloads (github.com/<owner>/<repo>/releases/download/...); the allowlist previously covered only api.github.com, raw.githubusercontent.com, and uploads.github.com, so those fetches went out unauthenticated even with a token configured. Verified GitHub returns the same 302 with or without the header, and Go strips Authorization on the cross-host redirect to the storage host.
• Shorthand github.com/org/repo detection deliberately stays github.com-only (a bare hostname can't be told from a relative path); documented. No atmos.yaml changes, no schema changes — env vars only, all defaulting to today's github.com behavior.
• Docs: environment-variables reference (new GHES section), toolchain and auth notes, changelog post, roadmap entry.

why

• Atmos hardwired github.com in ~35 places in slightly different ways, so on a GitHub Enterprise Server instance private-repo auth, raw-content fetches, and CI metadata could silently fail or fall back to unauthenticated requests depending on the code path.
• One resolver consumed everywhere replaces the ad-hoc env reads that already existed in pkg/ci/providers/github and pkg/http (extend, don't fork).
• The same env seam lets the acceptance suite point the toolchain/registry/raw fetches at a local httptest server (next PR in this stack), which is how we stop the test matrix from depending on live GitHub.

references

• Stacked on #3105 (local git mirror for the acceptance suite).
• https://github.com/orgs/community/discussions/206581 (GitHub's new unauthenticated-traffic protections, one of the triggers for this work).

Summary by CodeRabbit

• New Features

  • Added GitHub Enterprise Server support across API requests, repository links, imports, vendoring, artifacts, caches, CI integrations, and Git operations.
  • Added GITHUB_SERVER_URL and GITHUB_API_URL endpoint configuration.
  • Added separate toolchain and registry mirror configuration through ATMOS_TOOLCHAIN_* variables.
  • Added GHES-aware raw content, release asset, archive, and artifact URL handling.

• Security

  • Restricted GitHub tokens to approved HTTPS hosts and removed them during unsafe redirects.

• Documentation

  • Added GHES configuration and environment-variable guidance.
  • Clarified shorthand URL and token-scoping behavior.

• Why now: GitHub's new protections against unauthenticated traffic (https://github.com/orgs/community/discussions/206581#discussioncomment-18269356) turn every unauthenticated fetch of a public repo into a flake or a 401; the same pressure shows up downstream in hashicorp/terraform#39130 (hashicorp/terraform#39130), where users ask for SSH module fetching because unauthenticated HTTPS to GitHub is now rate limited. Tests that hit github.com anonymously are no longer viable as a default; live access is a canary concern, not a per-PR one.

• Resolve real secrets for !terraform.output, !terraform.state, atmos.Component(), and custom-command execution while preserving terminal masking and credential-free inspection.
• Add regression coverage for backend credentials, Terraform plan subprocess inputs, cached lookups, missing secrets, and references consumed by Helm, Helmfile, Kubernetes, and containers.

why

• The shared component loader incorrectly enabled inspection mode during execution, returning literal <MASKED> placeholders without retrieving secrets; the new tests reproduce this behavior when the fix is removed.

references

• Root cause, affected paths, and validation
• Validation: affected exec, secrets, Terraform output, I/O, and tfvars suites; custom-command tests; repository lint and commit hooks—all passed.

Summary by CodeRabbit

• Bug Fixes
  • Component references now resolve actual secret values during execution across supported component types while keeping secrets masked in terminal output and inspection views.
  • Terraform execution receives resolved secrets without exposing them in generated variable files.
  • Inspection and provenance views use masked placeholders without retrieving or caching real secrets.
  • Missing secrets continue to produce clear errors.
• Documentation
  • Added guidance covering secret resolution, masking, inspection behavior, and supported component integrations.

• Restore Changelog / Roadmap discovery with a shared, stable header and accessible navigation, paginate curated highlights six at a time with Q2/Q3 2026 first, simplify quarter lists, and correct audited milestone statuses, progress, and links across 15 initiatives.

• Keep Terraform Maturity Journey in its Resources submenu, remove it from Get Started, and order Stage 0–10 then Nirvana without an overview self-link; replace the unrelated vendoring cast in the Version Tracker announcement with its actual dev/prod tracks demo.

• Audit Features against current docs and implementation, expand coverage from 22 to 44 cards, correct component inventories and stale claims, repair obsolete reference links, and reframe Make/Just/Task migration guides and the bundled skill around general-purpose task running.

why

• Remove jumping headers and repeated entrance fades, retain access to older highlights, reflect shipped capabilities and the Q1 2027 multi-stack plan, and make the journey easier to discover and follow.

references

• Terraform Maturity Journey
• Validation: direct Docusaurus production build; desktop/mobile, both themes, keyboard, filters, pagination, refresh/history, and journey cards; 359-milestone data audit; Version Tracker cast content and reference checks; all 44 feature cards and reference routes, mobile/theme/keyboard verification; three task-runner examples executed without stacks/components; emulator package and CastPlayer tests after merging main (remaining old roadmap target dates documented for product review).

Summary by CodeRabbit

• New Features
  • Added shared Atmos Updates navigation for Changelog and Roadmap pages with active-page highlighting.
  • Added pagination for featured roadmap items and improved initial timeline positioning.
  • Grouped task-runner migration guides and organized the Terraform Maturity Journey.
• Documentation
  • Expanded component, feature, and migration documentation, including broader infrastructure types and general task-runner adoption.
• Accessibility & UI
  • Improved roadmap status labels, keyboard behavior, focus states, responsive styling, and product navigation with an underlined tab design.

Root-causes a week of system-wide file-descriptor exhaustion that required repeated reboots.

• atmos scaffold generate walks a template's source directory verbatim, copying every file/subdirectory not explicitly declared in spec.files straight through to the target at the same relative path. A template configured with source: "." whose target lands inside that same source tree (e.g. generated/<name>, right next to atmos.yaml) therefore re-copies its own accumulated prior output into every new target -- and since sibling targets share that same container directory, any previously generated sibling leaks in too. Confirmed live in tests/fixtures/scenarios/scaffold-matrix-freetext/generated/: 47,000+ self-nested directories, 2.7GB, after repeated local atmos test runs (scaffold-matrix-computed hit the same bug at smaller scale).
• A second, independent bug let this compound indefinitely: the CLI test harness's clean: true only removed paths git status reports as Untracked -- but go-git's Status(), like plain git status, never reports a gitignored path at all, so a fixture whose own output directory is gitignored (as generated/ is here) was never actually cleaned between runs despite clean: true on every relevant test case.

Changes

• pkg/generator/templates: LoadConfigurationFromDir gains a variadic WithExcludePath(absPath) option. When the resolved generation target falls inside the template's source directory, the walk now excludes the whole shared top-level container the target lives under (not just the literal target path), so a sibling's leftover output can never leak into a fresh target either.
• cmd/scaffold/scaffold.go: threads the resolved absTargetDir through loadScaffoldTemplates -> mergeConfiguredTemplates -> convertScaffoldTemplateToConfiguration so every local-source template load knows its run's real target.
• tests/cli_test.go: cleanDirectory now also removes gitignored entries directly under workdir, using the git index (not gitignore pattern matching) as the source of truth for what's tracked, so it never deletes a directory that still holds tracked content.
• Removed the ~2.7GB of already-accumulated self-nested generated/ output from both fixtures (gitignored, never tracked).

Test plan

• go test ./pkg/generator/... -- all packages pass, including two new regression tests (TestLoadConfigurationFromDir_WithExcludePath, TestLoadConfigurationFromDir_WithExcludePath_OutsideSourceIsNoop)
• go test ./cmd/scaffold/... -- passes with existing tests updated for the new function signatures
• go test ./tests/... -run 'TestCLICommands/scaffold' -- full scaffold CLI suite passes
• Ran the scaffold-matrix-freetext CLI tests twice in a row and confirmed generated/ directory count stays flat (no compounding) across repeated runs
• ./custom-gcl run --new-from-rev=origin/main -- 0 issues

Summary by CodeRabbit

• Bug Fixes
  • Fixed scaffolding from accidentally reusing previously generated files as template content when the output directory is inside the template source.
  • Improved target-directory resolution during interactive scaffolding.
  • Improved cleanup of scaffold workspaces by removing ignored, untracked files while preserving tracked content.
  • Corrected path handling for directories with names beginning with ...
  • Improved detection and rerunning of jobs stalled during finalization.
• Documentation
  • Clarified infrastructure-failure classification and rerun behavior.
• Tests
  • Added coverage for scaffold exclusion, cleanup preservation, and stalled finalization detection.