diff --git a/website/docs/commands/cli-config.html.markdown b/website/docs/commands/cli-config.html.markdown index fd4e9a5521..cd0a47cdf7 100644 --- a/website/docs/commands/cli-config.html.markdown +++ b/website/docs/commands/cli-config.html.markdown @@ -51,6 +51,14 @@ disable_checkpoint = true The following settings can be set in the CLI configuration file: +- `credentials` - configures credentials for use with Terraform Cloud or + Terraform Enterprise. See [Credentials](#credentials) below for more + information. + +- `credentials_helper` - configures an external helper program for the storage + and retrieval of credentials for Terraform Cloud or Terraform Enterprise. + See [Credentials Helpers](#credentials-helpers) below for more information. + - `disable_checkpoint` — when set to `true`, disables [upgrade and security bulletin checks](/docs/commands/index.html#upgrade-and-security-bulletin-checks) that require reaching out to HashiCorp-provided network services. @@ -60,16 +68,12 @@ The following settings can be set in the CLI configuration file: id used to de-duplicate warning messages. - `plugin_cache_dir` — enables - [plugin caching](/docs/configuration/providers.html#provider-plugin-cache) + [plugin caching](#provider-plugin-cache) and specifies, as a string, the location of the plugin cache directory. -- `credentials` - configures credentials for use with Terraform Cloud or - Terraform Enterprise. See [Credentials](#credentials) below for more - information. - -- `credentials_helper` - configures an external helper program for the storage - and retrieval of credentials for Terraform Cloud or Terraform Enterprise. - See [Credentials Helpers](#credentials-helpers) below for more information. +- `provider_installation` - customizes the installation methods used by + `terraform init` when installing provider plugins. See + [Provider Installation](#provider-installation) below for more information. ## Credentials @@ -144,14 +148,199 @@ To learn how to write and install your own credentials helpers to integrate with existing in-house credentials management systems, see [the guide to Credentials Helper internals](/docs/internals/credentials-helpers.html). -## Deprecated Settings +## Provider Installation -The following settings are supported for backward compatibility but are no -longer recommended for use: +The default way to install provider plugins is from a provider registry. The +origin registry for a provider is encoded in the provider's source address, +like `registry.terraform.io/hashicorp/aws`. For convenience in the common case, +Terraform allows omitting the hostname portion for providers on +`registry.terraform.io`, so we'd normally write `hashicorp/aws` instead in +this case. + +Downloading a plugin directly from its origin registry is not always +appropriate, though. For example, the system where you are running Terraform +may not be able to access an origin registry due to firewall restrictions +within your organization or your locality. + +To allow using Terraform in these situations, there are some alternative +options for making provider plugins available to Terraform. + +### Explicit Installation Method Configuration + +A `provider_installation` block in the CLI configuration allows overriding +Terraform's default installation behaviors, so you can force Terraform to use +a local mirror for some or all of the providers you intend to use. + +The general structure of a `provider_installation` block is as follows: + +```hcl +provider_installation { + filesystem_mirror { + path = "/usr/share/terraform/providers" + include = ["example.com/*/*"] + } + direct { + exclude = ["example.com/*/*"] + } +} +``` + +Each of the nested blocks inside the `provider_installation` block specifies +one installation method. Each installation method can take both `include` +and `exclude` patterns that specify which providers a particular installation +method can be used for. In the example above, we specify that any provider +whose origin registry is at `example.com` can be installed only from the +filesystem mirror at `/usr/share/terraform/providers`, while all other +providers can be installed only directly from their origin registries. + +If you set both both `include` and `exclude` for a particular installation +method, the exclusion patterns take priority. For example, including +`registry.terraform.io/hashicorp/*` but also excluding +`registry.terraform.io/hashicorp/dns` will make that installation method apply +to everything in the `hashicorp` namespace with the exception of +`hashicorp/dns`. + +As with provider source addresses in the main configuration, you can omit +the `registry.terraform.io/` prefix for providers distributed through the +public Terraform registry, even when using wildcards. For example, +`registry.terraform.io/hashicorp/*` and `hashicorp/*` are equivalent. +`*/*` is a shorthand for `registry.terraform.io/*/*`, not for +`*/*/*`. + +The following are the two supported installation method types: + +* `direct`: request information about the provider directly from its origin + registry and download over the network from the location that registry + indicates. This method expects no additional arguments. + +* `filesystem_mirror`: consult a directory on the local disk for copies of + providers. This method requires the additional argument `path` to indicate + which directory to look in. + + Terraform expects the given directory to contain a nested directory structure + where the path segments together provide metadata about the available + providers. The following two directory structures are supported: + + * Packed layout: `HOSTNAME/NAMESPACE/TYPE/terraform-provider-TYPE_VERSION_TARGET.zip` + is the distribution zip file obtained from the provider's origin registry. + * Unpacked layout: `HOSTNAME/NAMESPACE/TYPE/VERSION/TARGET` is a directory + containing the result of extracting the provider's distribution zip file. + + In both layouts, the `VERSION` is a string like `2.0.0` and the `TARGET` + specifies a particular target platform using a format like `darwin_amd64`, + `linux_arm`, `windows_amd64`, etc. + + If you use the unpacked layout, Terraform will attempt to create a symbolic + link to the mirror directory when installing the provider, rather than + creating a deep copy of the directory. The packed layout prevents this + because Terraform must extract the zip file during installation. + + You can include multiple `filesystem_mirror` blocks in order to specify + several different directories to search. + +Terraform will try all of the specified methods whose include and exclude +patterns match a given provider, and select the newest version available across +all of those methods that matches the version constraint given in each +Terraform configuration. If you have a local mirror of a particular provider +and intend Terraform to use that local mirror exclusively, you must either +remove the `direct` installation method altogether or use its `exclude` +argument to disable its use for specific providers. + +### Implied Local Mirror Directories + +If your CLI configuration does not include a `provider_installation` block at +all, Terraform produces an _implied_ configuration. The implied configuration +includes a selection of `filesystem_mirror` methods and then the `direct` +method. + +The set of directories Terraform can select as filesystem mirrors depends on +the operating system where you are running Terraform: + +* **Windows:** `%APPDATA%/HashiCorp/Terraform/plugins` +* **Mac OS X:** `~/Library/Application Support/io.terraform/plugins` and + `/Library/Application Support/io.terraform/plugins` +* **Linux and other Unix-like systems**: Terraform implements the + [XDG Base Directory](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) + specification and appends `terraform/plugins` to all of the specified + data directories. Without any XDG environment variables set, Terraform + will use `~/.local/share/terraform/plugins`, + `/usr/local/share/terraform/plugins`, and `/usr/share/terraform/plugins`. + +Terraform will create an implied `filesystem_mirror` method block for each of +the directories indicated above that exists when Terraform starts up. + +In addition to the zero or more implied `filesystem_mirror` blocks, Terraform +also creates an implied `direct` block. Terraform will scan all of the +filesystem mirror directories to see which providers are placed there and +automatically exclude all of those providers from the implied `direct` block. +(This automatic `exclude` behavior applies only to _implicit_ `direct` blocks; +if you use explicit `provider_installation` you will need to write the intended +exclusions out yourself.) + +### Provider Plugin Cache + +By default, `terraform init` downloads plugins into a subdirectory of the +working directory so that each working directory is self-contained. As a +consequence, if you have multiple configurations that use the same provider +then a separate copy of its plugin will be downloaded for each configuration. + +Given that provider plugins can be quite large (on the order of hundreds of +megabytes), this default behavior can be inconvenient for those with slow +or metered Internet connections. Therefore Terraform optionally allows the +use of a local directory as a shared plugin cache, which then allows each +distinct plugin binary to be downloaded only once. + +To enable the plugin cache, use the `plugin_cache_dir` setting in +the CLI configuration file. For example: + +```hcl +plugin_cache_dir = "$HOME/.terraform.d/plugin-cache" +``` + +This directory must already exist before Terraform will cache plugins; +Terraform will not create the directory itself. + +Please note that on Windows it is necessary to use forward slash separators +(`/`) rather than the conventional backslash (`\`) since the configuration +file parser considers a backslash to begin an escape sequence. + +Setting this in the configuration file is the recommended approach for a +persistent setting. Alternatively, the `TF_PLUGIN_CACHE_DIR` environment +variable can be used to enable caching or to override an existing cache +directory within a particular shell session: + +```bash +export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache" +``` + +When a plugin cache directory is enabled, the `terraform init` command will +still use the configured or implied installation methods to obtain metadata +about which plugins are available, but once a suitable version has been +selected it will first check to see if the chosen plugin is already available +in the cache directory. If so, Terraform will use the previously-downloaded +copy. + +If the selected plugin is not already in the cache, Terraform will download +it into the cache first and then copy it from there into the correct location +under your current working directory. When possible Terraform will use +symbolic links to avoid storing a separate copy of a cached plugin in multiple +directories. + +The plugin cache directory _must not_ also be one of the configured or implied +filesystem mirror directories, since the cache management logic conflicts with +the filesystem mirror logic when operating on the same directory. + +Terraform will never itself delete a plugin from the plugin cache once it has +been placed there. Over time, as plugins are upgraded, the cache directory may +grow to contain several unused versions which you must delete manually. + +## Removed Settings + +The following settings are supported in Terraform 0.12 and earlier but are +no longer recommended for use: * `providers` - a configuration block that allows specifying the locations of specific plugins for each named provider. This mechanism is deprecated - because it is unable to specify a version number for each plugin, and thus - it does not co-operate with the plugin versioning mechanism. Instead, - place the plugin executable files in - [the third-party plugins directory](/docs/configuration/providers.html#third-party-plugins). + because it is unable to specify a version number and source for each provider. + See [Provider Installation](#provider-installation) above for the replacement + of this setting in Terraform 0.13 and later. diff --git a/website/docs/configuration/providers.html.md b/website/docs/configuration/providers.html.md index dff4f6cc98..4d06e20668 100644 --- a/website/docs/configuration/providers.html.md +++ b/website/docs/configuration/providers.html.md @@ -233,65 +233,35 @@ from their parents. Anyone can develop and distribute their own Terraform providers. (See [Writing Custom Providers](/docs/extend/writing-custom-providers.html) for more -about provider development.) These third-party providers must be manually -installed, since `terraform init` cannot automatically download them. +about provider development.) -Install third-party providers by placing their plugin executables in the user -plugins directory. The user plugins directory is in one of the following -locations, depending on the host operating system: +The main way to distribute a provider is via a provider registry, and the main +provider registry is +[part of the public Terraform Registry](https://registry.terraform.io/browse/providers), +along with public shared modules. -Operating system | User plugins directory -------------------|----------------------- -Windows | `%APPDATA%\terraform.d\plugins` -All other systems | `~/.terraform.d/plugins` +Providers distributed via a public registry to not require any special +additional configuration to use, once you know their source addresses. You can +specify both official and third-party source addresses in the +`required_providers` block in your module: -Once a plugin is installed, `terraform init` can initialize it normally. You must run this command from the directory where the configuration files are located. +```hcl +terraform { + required_providers { + # An example third-party provider. Not actually available. + example = { + source = "example.com/examplecorp/example" + } + } +} +``` -Providers distributed by HashiCorp can also go in the user plugins directory. If -a manually installed version meets the configuration's version constraints, -Terraform will use it instead of downloading that provider. This is useful in -airgapped environments and when testing pre-release provider builds. - -### Plugin Names and Versions - -The naming scheme for provider plugins is `terraform-provider-_vX.Y.Z`, -and Terraform uses the name to understand the name and version of a particular -provider binary. - -If multiple versions of a plugin are installed, Terraform will use the newest -version that meets the configuration's version constraints. - -Third-party plugins are often distributed with an appropriate filename already -set in the distribution archive, so that they can be extracted directly into the -user plugins directory. - -### OS and Architecture Directories - -Terraform plugins are compiled for a specific operating system and architecture, -and any plugins in the root of the user plugins directory must be compiled for -the current system. - -If you use the same plugins directory on multiple systems, you can install -plugins into subdirectories with a naming scheme of `_` (for example, -`darwin_amd64`). Terraform uses plugins from the root of the plugins directory -and from the subdirectory that corresponds to the current system, ignoring -other subdirectories. - -Terraform's OS and architecture strings are the standard ones used by the Go -language. The following are the most common: - -* `darwin_amd64` -* `freebsd_386` -* `freebsd_amd64` -* `freebsd_arm` -* `linux_386` -* `linux_amd64` -* `linux_arm` -* `openbsd_386` -* `openbsd_amd64` -* `solaris_amd64` -* `windows_386` -* `windows_amd64` +Installing directly from a registry is not appropriate for all situations, +though. If you are running Terraform from a system that cannot access some or +all of the necessary origin registries, you can configure Terraform to obtain +providers from a local mirror instead. For more information, see +[Provider Installation](../commands/cli-config.html#provider-installation) +in the CLI configuration documentation. ## Provider Plugin Cache @@ -308,51 +278,3 @@ distinct plugin binary to be downloaded only once. To enable the plugin cache, use the `plugin_cache_dir` setting in [the CLI configuration file](/docs/commands/cli-config.html). -For example: - -```hcl -# (Note that the CLI configuration file is _not_ the same as the .tf files -# used to configure infrastructure.) - -plugin_cache_dir = "$HOME/.terraform.d/plugin-cache" -``` - -This directory must already exist before Terraform will cache plugins; -Terraform will not create the directory itself. - -Please note that on Windows it is necessary to use forward slash separators -(`/`) rather than the conventional backslash (`\`) since the configuration -file parser considers a backslash to begin an escape sequence. - -Setting this in the configuration file is the recommended approach for a -persistent setting. Alternatively, the `TF_PLUGIN_CACHE_DIR` environment -variable can be used to enable caching or to override an existing cache -directory within a particular shell session: - -```bash -export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache" -``` - -When a plugin cache directory is enabled, the `terraform init` command will -still access the plugin distribution server to obtain metadata about which -plugins are available, but once a suitable version has been selected it will -first check to see if the selected plugin is already available in the cache -directory. If so, the already-downloaded plugin binary will be used. - -If the selected plugin is not already in the cache, it will be downloaded -into the cache first and then copied from there into the correct location -under your current working directory. - -When possible, Terraform will use hardlinks or symlinks to avoid storing -a separate copy of a cached plugin in multiple directories. At present, this -is not supported on Windows and instead a copy is always created. - -The plugin cache directory must _not_ be the third-party plugin directory -or any other directory Terraform searches for pre-installed plugins, since -the cache management logic conflicts with the normal plugin discovery logic -when operating on the same directory. - -Please note that Terraform will never itself delete a plugin from the -plugin cache once it's been placed there. Over time, as plugins are upgraded, -the cache directory may grow to contain several unused versions which must be -manually deleted.