cloudposse/atmos v1.39.0

on GitHub

what

• Make Atmos understand Terraform configurations and dependencies
• Update atmos describe component command
• Update atmos describe affected command
• Add Atmos custom commands to atmos.yaml
• Update docs for atmos describe component and atmos describe affected commands

why

• Atmos now understands Terraform configurations and dependencies! (using a Terraform parser from HashiCorp) This is useful to detect component dependencies when a Terraform component in the components/terraform folder uses local Terraform modules defined in the local filesystem (not from the Terraform registry), and this functionality is used by the atmos describe component and atmos describe affected comamnds described below

• Update atmos describe component command. The command's output now has these additional sections:

  • atmos_cli_config - information about Atmos CLI configuration from atmos.yaml (useful in Atmos custom commands)

  • component_info - a block describing the Terraform or Helmfile components that the Atmos component manages. The component_info block has the following sections:

    • component_path - the filesystem path to the Terraform or Helmfile component

    • component_type - the type of the component (terraform or helmfile)

    • terraform_config - if the component type is terraform, this sections describes the high-level metadata about the Terraform component from its source code, including variables, outputs and child Terraform modules (using a Terraform parser from HashiCorp). The file names and line numbers where the variables, outputs and child modules are defined are also included. Invalid Terraform configurations are also detected, and in case of any issues, the warnings and errors are shows in the terraform_config.diagnostics section

   atmos describe component test/test-component-override-3 -s tenant1-ue2-dev

  atmos_cli_config:
    base_path: ./examples/complete
    components:
      terraform:
        base_path: components/terraform
        init_run_reconfigure: true
        auto_generate_backend_file: false
      helmfile:
        base_path: components/helmfile
        helm_aws_profile_pattern: '{namespace}-{tenant}-gbl-{stage}-helm'
        cluster_name_pattern: '{namespace}-{tenant}-{environment}-{stage}-eks-cluster'
    stacks:
      base_path: stacks
      included_paths:
        - orgs/**/*
      excluded_paths:
        - '**/_defaults.yaml'
      name_pattern: '{tenant}-{environment}-{stage}'
    workflows:
      base_path: stacks/workflows
  atmos_component: test/test-component-override-3
  atmos_stack: tenant1-ue2-dev
  atmos_stack_file: orgs/cp/tenant1/dev/us-east-2
  component: test/test-component
  component_info:
    component_path: examples/complete/components/terraform/test/test-component
    component_type: terraform
    terraform_config:
      path: examples/complete/components/terraform/test/test-component
      variables:
        enabled:
          name: enabled
          type: bool
          description: Set to false to prevent the module from creating any resources
          default: null
          required: false
          sensitive: false
          pos:
            filename: examples/complete/components/terraform/test/test-component/context.tf
            line: 97
        environment:
          name: environment
          type: string
          description: ID element. Usually used for region e.g. 'uw2', 'us-west-2',
            OR role 'prod', 'staging', 'dev', 'UAT'
          default: null
          required: false
          sensitive: false
          pos:
            filename: examples/complete/components/terraform/test/test-component/context.tf
            line: 115
        name:
          name: name
          type: string
          description: |
            ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'.
            This is the only ID element not also included as a `tag`.
            The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input.
          default: null
          required: false
          sensitive: false
          pos:
            filename: examples/complete/components/terraform/test/test-component/context.tf
            line: 127
        region:
          name: region
          type: string
          description: Region
          default: null
          required: true
          sensitive: false
          pos:
            filename: examples/complete/components/terraform/test/test-component/variables.tf
            line: 1
      outputs:
        service_1_id:
          name: service_1_id
          description: Service 1 ID
          sensitive: false
          pos:
            filename: examples/complete/components/terraform/test/test-component/outputs.tf
            line: 1
        service_2_id:
          name: service_2_id
          description: Service 2 ID
          sensitive: false
          pos:
            filename: examples/complete/components/terraform/test/test-component/outputs.tf
            line: 6
      requiredcore:
        - '>= 1.0.0'
      modulecalls:
        service_1_label:
          name: service_1_label
          source: cloudposse/label/null
          version: 0.25.0
          pos:
            filename: examples/complete/components/terraform/test/test-component/main.tf
            line: 1
        service_2_label:
          name: service_2_label
          source: cloudposse/label/null
          version: 0.25.0
          pos:
            filename: examples/complete/components/terraform/test/test-component/main.tf
            line: 10
      diagnostics: []

• Update atmos describe affected command

  The affected attribute in the command's output now can have the additional value component.module, which specify that the Terraform component is affected because it uses a local Terraform module (not from the Terraform registry, but from the local filesystem), and that local module has been changed.

  For example, let's suppose that we have a catalog of reusable Terraform modules in the modules folder (outside the components folder), and we have defined the following label Terraform module in modules/label:

  module "label" {
    source  = "cloudposse/label/null"
    version = "0.25.0"
    context = module.this.context
  }

  output "label" {
    value       = module.label
    description = "Label outputs"
  }

We then use the Terraform module in the components/terraform/top-level-component1 component:

  module "service_2_label" {
    source  = "../../../modules/label"
    context = module.this.context
  }

  output "service_2_id" {
    value       = module.service_2_label.label.id
    description = "Service 2 ID"
  }

The label module is not in the stack config of the top-level-component1 component (not in the YAML stack config files), but Atmos understands Terraform configurations and dependencies (using a Terraform parser from HashiCorp), and can automatically detect any changes to the module.

For example, if you make changes to any files in the folder modules/label, Atmos will detect the module changes, and since the module is a Terraform dependency of the top-level-component1 component, Atmos will mark the component as affected with the affected attribute set to component.module:

  [
    {
      "component": "top-level-component1",
      "component_type": "terraform",
      "component_path": "examples/complete/components/terraform/top-level-component1",
      "stack": "tenant1-ue2-staging",
      "stack_slug": "tenant1-ue2-staging-top-level-component1",
      "spacelift_stack": "tenant1-ue2-staging-top-level-component1",
      "atlantis_project": "tenant1-ue2-staging-top-level-component1",
      "affected": "component.module"
    },
    {
      "component": "top-level-component1",
      "component_type": "terraform",
      "component_path": "examples/complete/components/terraform/top-level-component1",
      "stack": "tenant2-ue2-staging",
      "stack_slug": "tenant2-ue2-staging-top-level-component1",
      "spacelift_stack": "tenant2-ue2-staging-top-level-component1",
      "atlantis_project": "tenant2-ue2-staging-top-level-component1",
      "affected": "component.module"
    }
  ]

• Add Atmos custom commands to atmos.yaml

  • The custom Atmos commands are examples of how to dynamically extend the Atmos CLI functionality by defining new commands in atmos.yaml without changing the Atmos binary itself

  • The command atmos list stacks is used to list all stacks in the infrastructure

  • The command atmos list components is used to all Atmos components in all stacks or in a single stack

  • The command atmos set-eks-cluster is used to download kubeconfig from an EKS cluster and set EKS cluster

  • The function set-eks-cluster is used to call the atmos set-eks-cluster command and export the KUBECONFIG ENV var into the calling (parent) process (the function itself is sourced from profile.d on a shell startup)

  • The command atmos set-eks-cluster also serves as an example on how to create Atmos custom commands that are context aware, meaning the commands "know" everything about Atmos components and stacks they operate on. This is done by using the component_config section in the custom command configuration:

    # List all stacks in the infrastructure
    atmos list stacks

    tenant1-ue2-dev
    tenant1-ue2-prod
    tenant1-ue2-staging
    tenant1-ue2-test-1
    tenant1-uw1-test-1
    tenant1-uw2-test-1
    tenant2-ue2-dev
    tenant2-ue2-prod
    tenant2-ue2-staging

    # List all components in the infrastructure
    atmos list components

    base-component-1
    base-component-2
    derived-component-1
    derived-component-2
    eks-blue/cluster
    eks-green/cluster
    infra/vpc
    infrastructure-tenant1
    infrastructure-tenant2
    mixin/test-1
    mixin/test-2
    spacelift-defaults
    test/test-component
    test/test-component-override
    test/test-component-override-2
    test/test-component-override-3
    test/test2/test-component-2
    top-level-component1
    top-level-component2
    vpc
    vpc/new

    # List all components in the stack `tenant1-ue2-prod`
    atmos list components -s tenant1-ue2-prod

    infra/vpc
    infrastructure-tenant1
    mixin/test-1
    mixin/test-2
    spacelift-defaults
    test/test-component
    test/test-component-override
    test/test-component-override-2
    test/test-component-override-3
    top-level-component1
    vpc