cloudposse/atmos v1.9.1

on GitHub

what

• Add atmos CLI config path and atmos base path parameters to the component processor to support components remote state from remote repos (Note that this does not affect atmos functionality, this is to be used in the utils provider which calls into the atmos code)

why

• The component processor's code is used by the utils provider to get the component's remote state
• We already supported the ATMOS_CLI_CONFIG_PATH and ATMOS_BASE_PATH ENV vars to specify the CLI config file (atmos.yaml) path and atmos base path to be used to get a remote state of a component from a remote repo, e.g.

module "other_repo" {
  source = "git::ssh://git@github.com/xxxx/other-repo.git"
}

locals {
  other_repo_local_path = "${path.module}/.terraform/modules/other_repo"

  env = {
    ATMOS_BASE_PATH       = local.other_repo_local_path
    ATMOS_CLI_CONFIG_PATH = "${local.other_repo_local_path}/rootfs/usr/local/etc/atmos"
  }
}

module "account_map" {
  source  = "cloudposse/stack-config/yaml//modules/remote-state"
  version = "1.0.0"

  component   = "account-map"
  env         = local.env

  context = module.always.context
}

• The problems with using the ENV vars are as follows:

  • Terraform executes a provider code in a separate process and calls it using RPC
  • But this separate process is only one per provider, so if we call the code the get the remote state of two different components from two diff repos, the same process will be called
  • When we specify the ENV vars ATMOS_BASE_PATH and ATMOS_CLI_CONFIG_PATH, the provider process gets the ENV vars set in the process space
  • Then, if we call the provider a second time from the same terraform component (e.g. to get a remote state of another component from a different repo), the initially set ENV vars ATMOS_BASE_PATH and ATMOS_CLI_CONFIG_PATH are still set in the provider process space, which prevents the provider from finding the atmos.yaml CLI config related to the current repo (since the ENV vars still point to the other/remote repo config), which in turn causes an error when searching for the component in the stack
  • Even if we unset the ENV vars in the second call to the provider, it does not help since terraform executes data sources in parallel, so one of them will get the ENV vars set, and the other call will fail during the time window when the ENV vars are still set in the same process

• We need to be able to specify atmos base path and atmos CLI config path in the utils provider w/o using ENV vars - the component processor code now supports additional parameters to specify it (and they override all other paths set by the ENV vars)