Go 1.17 and earlier used a different syntax for build constraint comments,
starting with "+build". Go 1.18 changed this to the modern "go:build" form
as part of standardizing the structure of toolchain directive comments,
and so for a while it was convention to include comments in both styles
to allow building with both old and new Go compilers.
However, Go 1.17 is no longer supported, and regardless of that we only
expect OpenTofu to be built with the specific version we have selected
in "go.mod" and ".go-version" anyway, so we no longer need the legacy form
of these comments: the all supported Go toolchains now support the new
form, which this commit retains.
golangci-lint v2.6.0 includes a check for this legacy form, so removing
this will also allow us to upgrade to a newer version of that linter
aggregator in a future commit.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
These were previously settable only via environment variables. These are
now handled as part of CLI Configuration and so also settable in a new
"registry_protocols" block in a CLI configuration file, with the
environment variables now treated as if they are an additional virtual
configuration file containing the corresponding settings.
This handles our settings in our modern style where package cliconfig is
responsible for deciding the configuration and then package main reacts
to that configuration without being aware of how it is decided.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Due to some past confusion about the purpose of this package, it has grown
to include a confusing mix of currently-viable code and legacy support
code from the move to HCL 2. This has in turn caused confusion about which
parts of this package _should_ be used for new code.
To help clarify that distinction we'll move the legacy support code into
a package under the "legacy" directory, which is also where most of its
callers live.
There are unfortunately still some callers to these outside of the legacy
tree, but the vast majority are either old tests written before HCL 2
adoption or helper code used only by those tests. The one dubious exception
is the use in ResourceInstanceObjectSrc.Decode, which makes a best effort
to shim flatmap as a concession to the fact that not all state-loading
codepaths are able to run the provider state upgrade function that would
normally be responsible for the flatmap-to-JSON conversion, which is
explained in a new comment inline.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we were using a third-party library, but that doesn't have any
support for passing context.Context through its API and so isn't suitable
for our goals of adding OpenTelemetry tracing for all outgoing network
requests.
We now have our own fork that is updated to use context.Context. It also
has a slightly reduced scope no longer including various details that
are tightly-coupled to our cliconfig mechanism and so better placed in the
main OpenTofu codebase so we can evolve it in future without making
lockstep library releases.
The "registry-address" library also uses svchost and uses some of its types
in its public API, so this also incorporates v2 of that library that is
updated to use our own svchost module.
Unfortunately this commit is a mix of mechanical updates to the new
libraries and some new code dealing with the functionality that is removed
in our fork of svchost. The new code is primarily in the "svcauthconfig"
package, which is similar in purpose "ociauthconfig" but for OpenTofu's
own auth mechanism instead of the OCI Distribution protocol's auth
mechanism.
This includes some additional plumbing of context.Context where it was
possible to do so without broad changes to files that would not otherwise
have been included in this commit, but there are a few leftover spots that
are context.TODO() which we'll address separately in later commits.
This removes the temporary workaround from d079da6e9e, since we are now
able to plumb the OpenTelemetry span tree all the way to the service
discovery requests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Our handling of Docker-style configuration files as an authentication
source is intentionally a bit of a compromise because various other
software reads and writes these files despite there being no single
standard for the format, and unfortunately different software makes
different tradeoffs when the configuration is ambiguous.
One oddity that we didn't notice originally is that the "login" command of
some programs will respect the "credsStore" property for storing new
credentials using a helper program instead of storing them in cleartext
in the config file BUT will still create an empty entry in the "auths"
property for whatever domain the operator logged into. Our logic wasn't
built to tolerate an "auths" entry without an "auth" property inside it
and so we would then fail to select credentials correctly for the affected
domain.
This commit makes our handling a little more resilient against
oddly-generated configuration files by silently ignoring all three of the
following oddities:
- An "auths" entry that has no "auth" property, as described above.
- An "auths" entry that is JSON null, which would previously cause OpenTofu
to crash with a null pointer dereference.
- An "auths" entry with "auth" set to an empty string, since generating
that instead of omitting the property entirely is a relatively common
mistake when using Go's encoding/json library and forgetting to add
the special "omitempty" tag to the corresponding struct field.
(An empty string is never a valid value for this property because it's
supposed to be the base64 encoding of a string like "username:password",
and so it should always at least contain a base64-encoded colon.)
Since there is no plausible valid meaning of any of these odd constructions
we prefer to just silently ignore them without any errors and without
generating any distracting log noise, which seems to match how other
software like Docker CLI and ORAS CLI handles them.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
It's now valid to include an oci_mirror block in the provider_installation
block in the CLI configuration, specifying that OpenTofu should try to
install providers from OCI repositories based on a template that maps
from OpenTofu-style provider source addresses into OCI repository
addresses.
The getproviders.Source implementation for this was added in a previous
commit, so this is mainly just wiring it up to the cliconfig layer and
the dependency wiring code in package main.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This new method collects all of the various different settings that
describe the operator's chosen OCI credentials policy and returns a single
object that encapsulates that policy.
This is the method that will, in future commits, be used by package main
to provide the credentials policy to any OCI-registry-related subsystems
using dependency-inversion style.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We're not actually doing anything with the result of this yet, but this
covers the decoding and validation of this new block type which we'll start
making use of in subsequent commits as we start to bring together the
overall OCI credentials selection policy handling.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This was previously an unexported function used only for implementing the
Docker/Podman/etc auth format, but we'll be using the same syntax in the
labels of our CLI configuration language's "oci_credentials" blocks and
so we'll export it to allow use from package cliconfig.
The name also changes to use the "ContainersAuth" prefix rather than the
"DockerCLIStyle" prefix because we are technically implementing the
expanded form of this configuration format that supports repository path
matching, rather than the original form that Docker CLI pioneered. This
new format doesn't really have a catchy brand name, but it's documented
as "containers auth.json" and so "ContainersAuth" seems like a reasonable
compromise at following that name while keeping this concise.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we just required the slash-separated segments to match without
imposing any further constraint, but if the Docker-style config syntax
evolves to allow other syntaxes here in future it'd be better for us to
just ignore what we don't recognize rather than get confused into trying
to match it in the current way.
ParseRepositoryAddressPrefix is an exported function because package
cliconfig will use it in a future commit to deal with our OpenTofu-specific
equivalent of the "auths" objects: oci_credentials blocks in the CLI
configuration.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As of this commit we don't actually do anything with these once decoded.
In future commits we'll add decoding of repository-specific oci_credentials
blocks and then the logic for combining all of these settings together
into an ociauthconfig.CredentialsConfigs representing the overall
credentials selection policy for OCI repositories.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Our requirements for discovering credentials for OCI registries include
automatic discovery or manual specification of Docker CLI-style auth
configuration files, which is a nontrivial amount of logic in itself,
along with an OpenTofu-specific version of that configuration model
embedded in the CLI configuration.
To avoid incorporating all of this extra scope into package cliconfig,
this new package ociauthconfig helps with modeling the overall OCI registry
authentication policy and with the Docker-CLI-style auth config format.
In a future commit, package cliconfig will drive this package's behavior
based on the operator's CLI configuration settings, eventually returning
an ociauthconfig.CredentialsConfigs representing the configured auth
policy, which package main can then deliver to other components as part
of an OCI client.
This ultimately yields the ORAS Go library's credentials type, since that
module has a relatively narrow indirect dependency surface area and will
avoid us needlessly implementing and maintaining our own OCI registry
client implementations.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
* Use ~/.opentf.d instead of ~/.terraform.d
Stay backwards-compatible, though.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix imports.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Add tests.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Use util function.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix windows directories.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Add a comment to the tests.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
---------
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Rename module name from "github.com/hashicorp/terraform" to "github.com/placeholderplaceholderplaceholder/opentf".
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Gofmt.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Regenerate protobuf.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comments.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo issue and pull request link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo comment changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comment.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo some link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* make generate && make protobuf
Signed-off-by: Jakub Martin <kubam@spacelift.io>
---------
Signed-off-by: Jakub Martin <kubam@spacelift.io>
Since it's already possible to activate the dependency lock file using an
environment variable, we should allow opting in to it having broken
behavior using the environment too.
It's kinda odd in retrospect that TF_PLUGIN_CACHE_DIR is the only setting
we allow to be configured both in the environment and the CLI
configuration. That means that the infrastructure for dealing with that
situation was relatively immature here and so I did some light refactoring
to make it unit-testable without actually modifying the test program's
environment.
Currently Terraform will use an entry from the global plugin cache only if
it matches a checksum already recorded in the dependency lock file. This
allows Terraform to produce a complete lock file entry on the first
encounter with a new provider, whereas using the cache in that case would
cause the lock file to only cover the single package in the cache and
thereefore be unusable on any other operating system or CPU architecture.
This temporary CLI config option is a pragmatic exception to support those
who cannot currently correctly use the dependency lock file but who still
want to benefit from the plugin cache. With this setting enabled,
Terraform has permission to produce a dependency lock file that is only
suitable for the current system if that would allow use of an existing
entry in the plugin cache.
We are introducing this option to resolve a conflict between the needs of
folks who are using the dependency lock file as expected and the needs of
folks who cannot use the dependency lock file for some reason. The hope
then is to give respite to those who need this exception in the meantime
while we understand better why they cannot use the dependency lock file
and improve its design so that everyone will be able to use it
successfully in a future version of Terraform. This option will become a
silent no-op in a future version of Terraform, once the dependency lock
file behavior is sufficient for all supported Terraform development
workflows.
Go 1.19's "fmt" has some awareness of the new doc comment formatting
conventions and adjusts the presentation of the source comments to make
it clearer how godoc would interpret them. Therefore this commit includes
various updates made by "go fmt" to acheve that.
In line with our usual convention that we make stylistic/grammar/spelling
tweaks typically only when we're "in the area" changing something else
anyway, I also took this opportunity to review most of the comments that
this updated to see if there were any other opportunities to improve them.
Hyphen characters are allowed in environment variable names, but are not valid POSIX variable names. Usually, it's still possible to set variable names with hyphens using utilities like env or docker. But, as a fallback, host names may encode their hyphens as double underscores in the variable name. For the example "café.fr", the variable name "TF_TOKEN_xn____caf__dma_fr" or "TF_TOKEN_xn--caf-dma_fr"
may be used.
