gruntwork-io/terratest modules/core/v2.0.0

Terratest v2.0.0

on GitHub

Terratest v2.0.0 is the first stable release of v2. All 16 modules are tagged at v2.0.0 in lockstep, and each one installs on its own.

TL;DR: Terratest is now 16 Go modules under /v2 import paths, so a test pulls in only the SDKs of the modules it imports. Most of the upgrade from v1 is rewriting imports, and the compiler finds every other change except two Kubernetes behavior changes, listed below. v1 gets security fixes only, for 12 months after this release.

Upgrading from v1

The v2 migration guide walks through the whole move. What changes for a v1 user:

• Import paths: every module moves to github.com/gruntwork-io/terratest/modules/<name>/v2. The rewriting imports page has the sed commands, and the cases a blind find and replace gets wrong.

• Utility packages: random, files, logger, shell, retry and testing now live in core/v2.

• Package names: http-helper, dns-helper and test-structure become httphelper, dnshelper and teststructure, in the import path and in the identifier.

• Deprecated helpers removed: code still calling terraform.Apply calls terraform.ApplyContext(t, ctx, ...) instead, which is the replacement the v1 deprecation warnings already named.

• Removed packages: collections, environment, git, version-checker, slack, oci and the two cmd/ binaries are gone. v1.0.1 names the replacement for the first five in each package's deprecation notice.

• Save and load helpers moved: eight teststructure.{Save,Load}* helpers now live beside their type, in aws, k8s, packer and ssh, with the same signatures.
  Two behavior changes compile cleanly, so read them before upgrading a Kubernetes test:

• Node addresses prefer ExternalIP: k8s.FindNodeHostnameContextE, and k8s.GetServiceEndpointContextE for a NodePort service, return the node's ExternalIP when the cluster records one. On EKS that is the public IP, so the common path no longer needs ec2:DescribeInstances.

• KubectlOptions with a RestConfig refuses to serialize: MarshalJSON returns k8s.ErrRestConfigNotSerializable. v1 failed here too, with an unclear error, and dropping the config instead would have authenticated a reloaded test against the ambient kubeconfig.

The behavior changes page covers both in full.

Smaller dependency graph

v1.0.1 shipped as one module with a 675 line go.sum. In v2, a test that only runs Terraform imports terraform/v2, whose go.sum is 77 lines, and no AWS, Azure, GCP or Kubernetes SDK enters the build. A continuous integration check holds the dependencies between modules to an explicit allowlist, so a new import cannot quietly pull one module into another.

New since beta.2

The gcp module can now read back the settings of 82 kinds of Google Cloud resource, so a test can assert on what Terraform actually created rather than only that it exists. Each read comes in three forms: one that fails the test, one that returns the error, and a WithClient form that takes an injected client, for a unit test with no credentials. The additions bring no new dependencies: the go.sum of gcp/v2 is unchanged at 267 lines.

bucket := gcp.GetStorageBucketAttrs(t, ctx, bucketName)
assert.True(t, bucket.UniformBucketLevelAccess.Enabled)

The new reads cover these areas:

• Compute and networking: disks, instance templates, regional managed instance groups, VPC networks, subnetworks, firewall rules, routers, regional and global addresses, Cloud Armor policies, every piece of a global HTTPS load balancer, VPC Access connectors, private service connections and Network Connectivity Center hubs.
• Storage and data: Cloud Storage buckets, Storage Transfer agent pools, BigQuery datasets and tables, Cloud SQL instances, databases and users, Spanner, Bigtable, Firestore, Filestore, Redis, AlloyDB, Dataproc, Dataflow, Composer and Dataplex lakes.
• Containers, serverless and delivery: GKE clusters and node pools, Cloud Run services and jobs, Cloud Functions, Workflows, Cloud Scheduler, Cloud Tasks, Eventarc, Cloud Build triggers and worker pools, Cloud Deploy pipelines and targets, and Artifact Registry.
• Identity and security: service accounts, workload identity pools and providers, OAuth clients, API keys, global and regional Secret Manager secrets, KMS key rings and keys, Certificate Manager certificates and Firebase rulesets.
• Operations: log sinks, log buckets, log-based metrics, alert policies, SLOs, uptime checks, notification channels, dashboards and essential contacts.
• Messaging, DNS and AI: Pub/Sub topics, subscriptions and schemas, Cloud DNS zones, record sets and policies, Healthcare datasets, Document AI processors, Dialogflow CX agents and Contact Center Insights views.

Also since beta.2:

• Kubernetes libraries: bumped to v0.37.0 (#1889). The go.sum of k8s/v2 grows from 160 to 178 lines as a result.
• Migration guide: published on the docs site (#1884).

Fixes since v1.0.1

• random.UniqueID: no longer collides between calls made in the same instant (#1860).
• Known CVEs: dependencies bumped to versions that resolve them (#1861).
• ssh.SaveSSHKeyPair: no longer writes the private key to the test log.
• Saved test data: written with 0o600 permissions rather than 0o644.
• core/teststate: stops after reporting a fatal error, where a marshal failure used to leave an empty file behind (#1880).
• IPv6 service endpoints: built with net.JoinHostPort, so the address is bracketed correctly.
• Malformed AWS provider IDs: rejected with an error instead of a panic.

v1 maintenance

v1 is frozen at v1.0.1 and receives security fixes only, on the v1 branch, until 12 months after this release. v2 lives at different import paths, so a project pinned to v1 is unaffected and can move one module at a time.

Install

go get github.com/gruntwork-io/terratest/modules/terraform/v2@v2.0.0

All 16 modules, at github.com/gruntwork-io/terratest/modules/<name>/v2:

core ssh httphelper dnshelper docker packer database opa aws azure gcp k8s helm terraform terragrunt teststructure

Thanks

Thanks to @wjam for the v2 beta feedback in #1875 that reshaped the module dependencies in beta.2, and to @yhakbar, @maxtaran2010 and @zuwasi for contributions during the beta.

Full Changelog: v1.0.1...modules/core/v2.0.0