Files
opentf/website/docs/providers/terraform/d/remote_state.html.md
Martin Atkins 6bb9fa7341 website: Document alternatives to terraform_remote_state
For some time now we've been recommending explicitly passing data between
configurations using separate resource types and data sources, rather than
always using terraform_remote_state, for reasons including reducing
coupling between subsystems and allowing a configuration's state snapshots
to be under restrictive access controls.

However, those recommendations have so far not appeared directly in the
documentation for terraform_remote_state, and have instead just been
alluded to elsewhere in the documentation when discussing ways to pass
data between configurations.

This change, then, is an attempt to be clear and explicit about the
recommendation and to give a variety of specific examples of how to
implement it. The terraform_remote_state data source page is admittedly
not the most obvious place in the information architecture to put a set
of alternatives to it, but it does appear that this documentation page is
where people most commonly end up when researching options in this area
and so I've put this here in an attempt to "meet people where they are".

Possibly in a future documentation reorganization we might have an
separate page specifically about sharing data between configurations, but
we don't currently have time to do that bigger reorganization. If we do so
later, the content on this page could potentially be replaced with a
summary of the recommendation and a link to another place for the details,
but the goal here is to make this information visible in the existing
location people look for it, rather than blocking until there's a better
place for it to live.

This also includes a small amount of editing of some existing content on
the page to use terminology and style more similar to how our main
configuration language documentation is written,.
2020-11-17 09:41:54 -08:00

11 KiB

layout, page_title, sidebar_current, description
layout page_title sidebar_current description
language Terraform: terraform_remote_state docs-terraform-datasource-remote-state Retrieves the root module output values from a Terraform state snapshot stored in a remote backend.

terraform_remote_state

The terraform_remote_state data source retrieves the root module output values saved as part of the latest state snapshot from the remote backend for some other Terraform configuration.

This can be a convenient way to make use of data already generated by another Terraform configuration without publishing it explicitly elsewhere, but it's important to note that output values are only a small part of a Terraform state snapshot. Although terraform_remote_state only exposes the output values, any user of this data source must have full access to the other aspects of the state snapshot, which may be considered sensitive information.

Rather than directly sharing state snapshots between your configurations, we recommend explicitly publishing data for external consumption to a separate location than to the producing configuration's remote state backend. The shared information will then be separated from the internal details in the state snapshots, and so you can apply different access controls to each.

To share data explicitly between configurations, you can use pairs of managed resource types and data sources in various providers, including (but not limited to) the following:

System Publish with... Read with...
Alibaba Cloud DNS
(for IP addresses and hostnames)
alicloud_alidns_record resource type Normal DNS lookups, or the dns provider
Amazon Route53
(for IP addresses and hostnames)
aws_route53_record resource type Normal DNS lookups, or the dns provider
Amazon S3 aws_s3_bucket_object resource type aws_s3_bucket_object data source
Amazon SSM Parameter Store aws_ssm_parameter resource type aws_ssm_parameter data source
Azure Automation azurerm_automation_variable_string resource type azurerm_automation_variable_string data source
Azure DNS
(for IP addresses and hostnames)
azurerm_dns_a_record resource type, etc Normal DNS lookups, or the dns provider
Google Cloud DNS
(for IP addresses and hostnames)
google_dns_record_set resource type Normal DNS lookups, or the dns provider
Google Cloud Storage google_storage_bucket_object resource type google_storage_bucket_object data source and http data source
HashiCorp Consul consul_key_prefix resource type consul_key_prefix data source
Kubernetes kubernetes_config_map resource type kubernetes_config_map data source
OCI Object Storage oci_objectstorage_bucket resource type oci_objectstorage_bucket data source

-> These are some common options from the Official Terraform providers, but there are too many configuration storage options for us to list them all here, including some in partner and community providers. Any pair of managed resource type and corresponding data source can potentially be used to share data between Terraform configurations. See individual provider documentation to find other possibilities.

A key advantage of using a separate explicit configuration store instead of terraform_remote_state is that the data can potentially also be read by systems other than Terraform, such as configuration management or scheduler systems within your compute instances. For that reason, we recommend selecting a configuration store that your other infrastructure could potentially make use of. For example:

  • If you wish to share IP addresses and hostnames, you could publish them as normal DNS A, AAAA, CNAME, and SRV records in a private DNS zone and then configure your other infrastructure to refer to that zone so you can find infrastructure objects via your system's built-in DNS resolver.
  • If you use HashiCorp Consul then publishing data to the Consul key/value store or Consul service catalog can make that data also accessible via Consul Template or the HashiCorp Nomad template stanza.
  • If you use Kubernetes then you can make Config Maps available to your Pods.

Some of the data stores listed above are specifically designed for storing small configuration values, while others are generic blob storage systems. For those generic systems, you can use the jsonencode function and the jsondecode function respectively to store and retrieve structured data.

You can encapsulate the implementation details of retrieving your published configuration data by writing a data-only module containing the necessary data source configuration and any necessary post-processing such as JSON decoding. You can then change that module later if you switch to a different strategy for sharing data between multiple Terraform configurations.

Example Usage (remote Backend)

data "terraform_remote_state" "vpc" {
  backend = "remote"

  config = {
    organization = "hashicorp"
    workspaces = {
      name = "vpc-prod"
    }
  }
}

# Terraform >= 0.12
resource "aws_instance" "foo" {
  # ...
  subnet_id = data.terraform_remote_state.vpc.outputs.subnet_id
}

# Terraform <= 0.11
resource "aws_instance" "foo" {
  # ...
  subnet_id = "${data.terraform_remote_state.vpc.subnet_id}"
}

Example Usage (local Backend)

data "terraform_remote_state" "vpc" {
  backend = "local"

  config = {
    path = "..."
  }
}

# Terraform >= 0.12
resource "aws_instance" "foo" {
  # ...
  subnet_id = data.terraform_remote_state.vpc.outputs.subnet_id
}

# Terraform <= 0.11
resource "aws_instance" "foo" {
  # ...
  subnet_id = "${data.terraform_remote_state.vpc.subnet_id}"
}

Argument Reference

The following arguments are supported:

  • backend - (Required) The remote backend to use.

  • workspace - (Optional) The Terraform workspace to use, if the backend supports workspaces.

  • config - (Optional; object) The configuration of the remote backend. Although this argument is listed as optional, most backends require some configuration.

    The config object can use any arguments that would be valid in the equivalent terraform { backend "<TYPE>" { ... } } block. See the documentation of your chosen backend for details.

    -> Note: If the backend configuration requires a nested block, specify it here as a normal attribute with an object value. (For example, workspaces = { ... } instead of workspaces { ... }.)

  • defaults - (Optional; object) Default values for outputs, in case the state file is empty or lacks a required output.

Attributes Reference

In addition to the above, the following attributes are exported:

  • (v0.12+) outputs - An object containing every root-level output in the remote state.
  • (<= v0.11) <OUTPUT NAME> - Each root-level output in the remote state appears as a top level attribute on the data source.

Root Outputs Only

Only the root-level output values from the remote state snapshot are exposed for use elsewhere in your module. Resource data and output values from nested modules are not accessible.

If you wish to make a nested module output value accessible as a root module output value, you must explicitly configure a passthrough in the root module. For example:

For example:

module "app" {
  source = "..."
}

output "app_value" {
  # This syntax is for Terraform 0.12 or later.
  value = module.app.example
}

In this example, the output value named example from the "app" module is available as the app_value root module output value. If this configuration didn't include the output "app_value" block then the data would not be accessible via terraform_remote_state.

~> Warning: Although terraform_remote_state doesn't expose any other state snapshot information for use in configuration, the state snapshot data is a single object and so any user or server which has enough access to read the root module output values will also always have access to the full state snapshot data by direct network requests. Don't use terraform_remote_state if any of the resources in your configuration work with data that you consider sensitive.