cloudposse/atmos v1.229.1-rc.0

on GitHub

The first unchanged production rerun after #3176 did not meet the zero-write cost gate. Run 35259983114 reported Changed/new: 2654; deleted: 7 even though it built the exact same merge commit as the bootstrap deployment.

Wall-clock timestamps from three Docusaurus plugins, the generated media-kit ZIP, and the footer year changed content-hashed output. New bundle names then cascaded into thousands of otherwise unchanged HTML files, defeating the manifest-based S3 uploader and preserving much of the request cost that #3176 was intended to remove.

What changed

• derive SOURCE_DATE_EPOCH from the checked-out Git commit in production and preview builds
• use that reproducible timestamp for plugin data, the footer year, and ZIP entry metadata
• use UTC ZIP timestamps so output is stable across runner time zones
• test the timestamp helper during both website workflows

External source data can still produce legitimate content changes. A rerun with the same source and external inputs is now byte-identical.

Validation

• two fresh, cache-free website builds of bc212b2d76 with the same SOURCE_DATE_EPOCH
• 11,291 files in each build
• diff -qr: 0 byte differences
• timestamp helper: 4 tests pass; 100% line, branch, and function coverage
• actionlint passes for production and preview workflows
• repository pre-commit hooks pass

Cost impact

This restores the intended #3176 steady state: one manifest GET and zero S3 PUT/COPY/DELETE requests on an unchanged deployment, instead of rewriting approximately 2,654 objects per run.

Summary by CodeRabbit

• New Features

  • Website builds now use a consistent timestamp, producing reproducible pages and generated downloads.
  • Copyright years and build-related metadata consistently reflect the build timestamp.
  • Added validation for invalid build timestamp values.

• Bug Fixes

  • Prevented unchanged preview builds from producing unnecessary page snapshot changes.

• Tests

  • Added automated coverage for valid, missing, malformed, and out-of-range build timestamps.

• Detect ten deprecated GitHub Actions explicitly and emit warning annotations linking to the migration hub.
• Add four migration guides and vendor-update guidance, and reorganize Native CI around complete workflow examples.
• Keep existing documentation marked released when fewer than 25% of its lines changed since the latest stable tag.

why

• Make migration paths easier to find without flagging supported actions or labeling established documentation unreleased after small edits.

references

• Legacy action migration hub and Native CI guide
• Validation: GitHub provider and startup Go tests; full Docusaurus build with an 8 GB Node heap; workflow YAML, rendered headings, and internal link checks

Summary by CodeRabbit

• Bug Fixes

  • Corrected deprecated GitHub Action detection, including plan-storage and component-updater actions.
  • Added migration warnings with Native CI links and GitHub Actions annotations.

• Documentation

  • Expanded Native CI guidance for workflows, permissions, caching, SBOMs, validation, and production environment gating.
  • Added migration guidance and native Atmos replacements for deprecated GitHub Actions.
  • Documented multiline-safe settings output, built-in planfile storage, and vendor update --pull-request alternatives.
  • Improved release labeling for updates to existing documentation.

• Terraform's own Warning: diagnostics from plan/apply/deploy now become inline GitHub ::warning CI annotations, the same mechanism already used for failing terraform test assertions.
• Annotations recover file/line from the diagnostic's on <file> line <N> locator when Terraform includes one, falling back to a file-level annotation otherwise.
• Broadened the locator regex (errorLocationRe) to match the locator formats Terraform actually emits (on main.tf line 20, in resource "x" "y":, and bare on main.tf line 1 with no trailing colon), not just the narrower terraform test diagnostic format it previously matched.
• Reordered each onAfterPlan/onAfterApply/onAfterTest/onAfterDeploy hook so annotation emission runs first, immediately after parsing output, ahead of summary/output/upload/check/comment — so it survives later warn-only failures and the fatal planfile-upload early-return in onAfterPlan.

why

• Today a Terraform-level warning (e.g. a deprecated argument) is only visible buried in the PR-comment summary or raw log — it never surfaces as a PR-diff annotation the way an error or failing test does, so reviewers can easily miss it.
• Placing annotation emission first in each hook, rather than last, means a warning is still reported even if a later step in the same hook fails or returns early.

references

None.

Summary by CodeRabbit

• New Features

  • Added inline CI annotations for Terraform warnings from plan, apply, and deploy operations.
  • Added inline error annotations for failing Terraform test assertions.
  • Warning annotations include source file and line details when available.

• Bug Fixes

  • Improved source-location detection for Terraform diagnostics, including warnings and varied diagnostic formats.
  • Annotation failures no longer interrupt CI processing.

• move the remaining ordinary Linux CI jobs from RunsOn to ubuntu-latest
• pass GitHub-hosted runners into feature, nightly, and release reusable workflows
• move the Linux build matrix leg to GitHub-hosted infrastructure
• remove RunsOn-only setup and apply Harden Runner consistently to the now-hosted Linux leg

Standard GitHub-hosted runners are free for public repositories, removing the largest recurring core-auto workload while keeping macOS and Windows behavior unchanged.

Validation

• actionlint completed with only two pre-existing informational ShellCheck findings
• no RunsOn routing labels or runs-on/action steps remain in workflows
• YAML whitespace validation passes

Expected savings: approximately $200-300/month.

Summary by CodeRabbit

• Chores
  • Updated CI/CD workflows to use standard GitHub-hosted Ubuntu runners instead of custom self-hosted runners.
  • Applied security hardening consistently across build matrix jobs.
  • Preserved existing formatting, testing, release, and deployment workflow behavior.

• replace the deployment Python and shell scripts entirely with the s3:deploy Mage target
• use the existing AWS SDK for Go v2 dependency for manifest reads, typed uploads, paginated listings, and batched deletes
• use a versioned SHA-256 manifest to upload only new or content/metadata-changed objects
• delete only previously managed objects while retaining configured out-of-band paths
• preserve explicit UTF-8 content types without recursively rewriting unchanged objects
• use the existing MIME magic library as a fallback while keeping extension-based types for browser-critical assets
• add magefiles/README.md with all 20 currently exposed Mage targets and their environment controls
• record the incident, measured cost, cost characteristics, validation, and rollback in docs/fixes/2026-09-16-content-aware-s3-website-deploy.md

Why this is worth the complexity

The previous deployment was request-heavy even when website content had not
changed:

1. aws s3 sync could upload rebuilt files because generated mtimes changed.
2. The metadata pass then recursively copied every matching text object back
   onto itself with --metadata-directive REPLACE.
3. This happened on every production and preview deployment, so unchanged
   objects repeatedly incurred S3 PUT/COPY request charges.

The metadata pass was not optional: the website requires correct MIME and
charset headers. aws s3 sync can apply one --content-type value to a whole
invocation, but it cannot selectively append charset=utf-8 while preserving
each file's distinct MIME type. The workaround therefore restamped every text
object after each sync.

AWS Cost Explorer for the seven complete days from September 9-15, 2026 showed
6,433,267 PutObject/CopyObject requests costing $32.25 across the development
and production documentation-origin accounts. That is a $138/month normalized
run rate for those two request types alone; S3 Tier-1 requests overall were
running at approximately $145/month.

This PR targets the Atmos website's share of that spend, estimated at
$70-85/month. The first deployment performs a one-time bootstrap, but every
unchanged repeat deployment thereafter performs one manifest read and zero S3
PUT/COPY/DELETE writes. A changed deployment writes only the actual delta.

Why the AWS SDK

The Mage target uses Atmos's existing AWS SDK v2 dependencies rather than
wrapping the AWS CLI. This removes subprocess and temporary JSON request-file
handling, gives each upload an explicit typed Content-Type and content length,
uses the SDK's retry and context-cancellation behavior, and exposes per-object
DeleteObjects failures directly to tests. One local glob matcher now controls
protected paths during both manifest diffs and bootstrap cleanup.

Safety

The manifest is uploaded only after all object writes and deletes succeed. A failed deployment keeps the old manifest, so the next run retries the incomplete delta.

The one-time bootstrap uploads all local files before listing and deleting stale unprotected objects, so a protected local path is never accidentally omitted. Retained paths remain tracked in the manifest so removing a protection pattern later makes them eligible for cleanup again. Typed batch deletion rejects per-object S3 errors before publishing the new manifest. Existing demo media and immutable versioned schemas remain protected.

Deployments to the same bucket and prefix are serialized by the existing production and per-preview-PR workflow concurrency groups. In-flight runs are never cancelled, so manifest reads, object changes, and final manifest publication cannot interleave for one destination.

Manifest traversal accepts regular files only; symlinks, devices, pipes, and other non-regular entries fail before any source content is opened.

An unchanged deployment performs one manifest read and zero S3 write requests.

Validation

• go test -tags=mage ./magefiles
• go test -tags=mage -race -coverprofile=.context/s3-deploy-sdk-coverage.out ./magefiles/...
• S3 deployment implementation statement coverage: 90.1% (245/272)
• repository changed-file pre-commit suite, including custom golangci-lint
• go vet -tags=mage ./magefiles
• actionlint .github/workflows/website-preview-deploy.yml .github/workflows/website-deploy-prod.yml
• go tool mage -l confirms all 20 targets are represented in magefiles/README.md

Expected savings: approximately $70-85/month.

Summary by CodeRabbit

• New Features

  • Website deployments now use content-aware synchronization, updating only added or changed files and removing obsolete managed files.
  • Deployments preserve protected demo images and versioned schema assets.
  • Uploaded files receive detected content types with UTF-8 metadata where applicable.
  • Deployment manifests support faster, more efficient repeat deployments.

• Bug Fixes

  • Improved handling of deployment errors, missing manifests, partial deletions, and invalid upload metadata.

• Documentation

  • Added guidance on deployment behavior, protected content, performance, validation, and rollback procedures.

The first unchanged production rerun after #3176 did not meet the zero-write cost gate. Run 35259983114 reported Changed/new: 2654; deleted: 7 even though it built the exact same merge commit as the bootstrap deployment.

Wall-clock timestamps from three Docusaurus plugins, the generated media-kit ZIP, and the footer year changed content-hashed output. New bundle names then cascaded into thousands of otherwise unchanged HTML files, defeating the manifest-based S3 uploader and preserving much of the request cost that #3176 was intended to remove.

What changed

• derive SOURCE_DATE_EPOCH from the checked-out Git commit in production and preview builds
• use that reproducible timestamp for plugin data, the footer year, and ZIP entry metadata
• use UTC ZIP timestamps so output is stable across runner time zones
• test the timestamp helper during both website workflows

External source data can still produce legitimate content changes. A rerun with the same source and external inputs is now byte-identical.

Validation

• two fresh, cache-free website builds of bc212b2d76 with the same SOURCE_DATE_EPOCH
• 11,291 files in each build
• diff -qr: 0 byte differences
• timestamp helper: 4 tests pass; 100% line, branch, and function coverage
• actionlint passes for production and preview workflows
• repository pre-commit hooks pass

Cost impact

This restores the intended #3176 steady state: one manifest GET and zero S3 PUT/COPY/DELETE requests on an unchanged deployment, instead of rewriting approximately 2,654 objects per run.

Summary by CodeRabbit

• New Features

  • Website builds now use a consistent timestamp, producing reproducible pages and generated downloads.
  • Copyright years and build-related metadata consistently reflect the build timestamp.
  • Added validation for invalid build timestamp values.

• Bug Fixes

  • Prevented unchanged preview builds from producing unnecessary page snapshot changes.

• Tests

  • Added automated coverage for valid, missing, malformed, and out-of-range build timestamps.