This adds a new context.Context argument to the Backend.Configure method,
updates all of the implementations to match, and then updates all of the
callers to pass in a context.
A small number of callers don't yet have context plumbed to them so those
use context.TODO() as a placeholder for now, so we can more easily find
and fix them in later commits once we have contexts more thoroughly
plumbed.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This test uses a temporary file as an overridden CLI configuration to
force using a specific plugin cache directory, but temporary file paths
contain backslashes on Windows and the CLI configuration syntax is HCL
so would require backslashes to be escaped.
Since Windows will accept forward-slash paths as a supported variation,
and we typically recommend that folks write paths that way in HCL for
portability anyway, this uses filepath.ToSlash to force consistent use
of slashes on all platforms. It would also have been reasonable to use
a %q format verb to use Go's string quoting syntax, but that's not the
way we typically recommend folks hand-write their CLI configurations and
so this way is just-so-slightly more "realistic".
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Our handling of Docker-style configuration files as an authentication
source is intentionally a bit of a compromise because various other
software reads and writes these files despite there being no single
standard for the format, and unfortunately different software makes
different tradeoffs when the configuration is ambiguous.
One oddity that we didn't notice originally is that the "login" command of
some programs will respect the "credsStore" property for storing new
credentials using a helper program instead of storing them in cleartext
in the config file BUT will still create an empty entry in the "auths"
property for whatever domain the operator logged into. Our logic wasn't
built to tolerate an "auths" entry without an "auth" property inside it
and so we would then fail to select credentials correctly for the affected
domain.
This commit makes our handling a little more resilient against
oddly-generated configuration files by silently ignoring all three of the
following oddities:
- An "auths" entry that has no "auth" property, as described above.
- An "auths" entry that is JSON null, which would previously cause OpenTofu
to crash with a null pointer dereference.
- An "auths" entry with "auth" set to an empty string, since generating
that instead of omitting the property entirely is a relatively common
mistake when using Go's encoding/json library and forgetting to add
the special "omitempty" tag to the corresponding struct field.
(An empty string is never a valid value for this property because it's
supposed to be the base64 encoding of a string like "username:password",
and so it should always at least contain a base64-encoded colon.)
Since there is no plausible valid meaning of any of these odd constructions
we prefer to just silently ignore them without any errors and without
generating any distracting log noise, which seems to match how other
software like Docker CLI and ORAS CLI handles them.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The "tofu show" command has historically been difficult to extend to meet
new use-cases, such as showing the current configuration without creating
a plan, because it was designed to take zero or one arguments and then try
to guess what the one specified argument was intended to mean.
This commit introduces a new style where the type of object to inspect is
specified using command line option syntax, using one of two
mutually-exclusive options:
-state Show the latest state snapshot.
-plan=FILE Show the plan from the given saved plan file.
We expect that a future commit will extend this with a new "-config" option
to inspect the configuration rooted in the current working directory, and
possibly with "-module=DIR" to shallowly inspect a single module without
necessarily having to fully initialize it with all of its dependencies
first. However, both of those use-cases (and any others) are not in scope
for this commit, which is focused only on refactoring to make those future
use-cases possible.
The old mode of specifying neither option and providing zero or one
positional arguments is still supported for backward compatibility.
Notably, the legacy style is the only way to access the legacy behavior of
inspecting a specific state snapshot file from the local filesystem, which
has not often been used since Terraform v0.9 as we've moved away
from manual management of state files to the structure of state backends.
Those who _do_ still need that old behavior can still access it in the
old way, but there will be no new-style equivalent of it unless we learn
of a compelling use case for it.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Following the lead of similar earlier work on testing the installation of
provider packages from OCI repositories, this new test exercises the new
OCI-based module source address syntax in an end-to-end fashion by directly
running "tofu init".
For the reasons described inline, this test uses a local test server as its
target OCI Registry and therefore needs to rely on a Go standard library
feature for overriding the trusted TLS certs which only works on Unix
systems other than macOS, and therefore this test will only run when the
e2etest suite is run on Linux systems. This matches the same compromise we
previously made for the provider installation flavor of this test, with
the same assumption that our module installer isn't doing anything
particularly platform-specific and that we're doing this in e2etest only
because that's an effective way to test that "package main" is wiring all
of the internal components together correctly.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We previously had two tests of how the module installer responds to
cancellation (e.g. SIGINT) which were flakey because they tried to rely
on the cancellation being detected at some arbitrary point before the
module installer attempted to make a request, which isn't guaranteed in
practice because our interrupt mechanism only aims to cause OpenTofu to
exit "soon", with no guarantee about how much ongoing progress it will
make before it does.
To make these tests more robust, we'll now instead tell the module
installer to install from a real HTTP server that is intentionally designed
to stall the client by accepting its request but then just leaving the
connection open without responding.
This means that we can now test the more realistic situation of the cancel
signal being triggered after a slow request is already in progress, and
be sure that we're definitely sending the cancel signal at a moment that
matches that intention.
This is similar to a strategy we previously took to improve the reliability
of the tests for cancellation of the _provider_ installer, in
TestInit_cancelProviders. However, our provider installer version of this
used an intentionally-stalling implementation of getproviders.Source
instead of running a real server because the provider installer is designed
to support configurable installation methods, while the module installer
is not: its policy about what module source types are accepted is
hard-coded in package getproviders, at least for now.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our work to follow the dependency inversion style for the
"package fetcher" component of the module installer.
Mimicking the existing pattern for providers, package main is now
responsible for instantiating the PackageFetcher and providing it to
the "command" package as a field of command.Meta.
We could potentially go further here and follow dependency inversion style
for _all_ of the special clients needed by the various go-getter getters,
but our primary concern for now is preparing to add a new "getter" for
installation from an OCI Distribution repository, and so we'll leave the
other already-working code unchanged to reduce the risk of this initial
work.
Future commits will actually wire in the implementation details for OCI
Repository access. This commit focuses only on plumbing the necessary
objects through the API layers.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Earlier work started to reshape this API to follow the dependency inversion
style, but didn't go so far as treating the package fetcher as an argument
because so far it hasn't offered any customizable policy anyway.
In future commits we will be introducing some policy arguments for the
package fetcher, and so this is some preparation work where we move the
responsibility for calling getmodules.NewPackageFetcher() out into the
caller of initwd.NewModuleInstaller().
This changes the API consumed by a bunch of unit testing helpers, so
splitting this out into its own commit should hopefully make future
commits more focused. The module installer now explicitly supports being
instantiated without a registry client or a remote package fetcher and
will in that case return an error if it's asked to install from anywhere
other than local relative directories. Most of our existing tests are
comfortable running under that constraint and so will not need any further
work in later commits that will change the signature of
getmodules.NewPackageFetcher.
However, a couple tests in package initwd _itself_ were making use of the
esoteric legacy support for treating an absolute filesystem path as a funny
sort of remote source, and so for now those will instantiate their own
package fetcher. Future commits that change the NewPackageFetcher signature
will need to offer a concession for those two tests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As discussed in opentofu/opentofu#2656, this consolidates the two concerns
of the PackageAuthentication interface into a single function that deals
both with package authentication _and_ with reporting all of the package
hashes that were used to make the authentication decision.
This means that any .zip archive that OpenTofu directly verifies during
installation can now have its hash recorded in the dependency lock file
even if that package didn't come from the provider's origin registry, which
is beneficial when the first installation of a provider comes from a
secondary ("mirror") source because it creates an additional hook by which
that dependency lock file entry can be "upgraded" to be complete in a
future "tofu init" run against the origin registry, or by the
"tofu providers lock" command.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Most of the OCI registry interactions are unit tested in the most relevant
packages, but the overall system will only work correctly if all of the
components are correctly wired together by "package main", and that's one
part of the system that needs to be tested concretely rather than via
test doubles.
Therefore this adds an end-to-end test in our existing e2etest package
that runs "tofu init" with a CLI configuration that forces using an OCI
mirror with a TLS server provided locally by our test program. It exercises
the main happy path of provider installation in the same way that an
end-user would interact with it, to help avoid accidentally regressing
the interactions between these packages in future versions.
Unfortunately the technique this test uses to force the OpenTofu CLI
binary to trust the test server doesn't work on macOS or Windows and so
for now this test is Linux-specific. That's certainly non-ideal, but
pragmatic since we'll be relying mainly on the platform-agnostic unit tests
to cover this behavior, and we're unlikely to ever stop running the
e2etests on Linux as part of our pull request checks so even those
developing on macOS or Windows can still notice if this test becomes
broken before merging a change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
It's now valid to include an oci_mirror block in the provider_installation
block in the CLI configuration, specifying that OpenTofu should try to
install providers from OCI repositories based on a template that maps
from OpenTofu-style provider source addresses into OCI repository
addresses.
The getproviders.Source implementation for this was added in a previous
commit, so this is mainly just wiring it up to the cliconfig layer and
the dependency wiring code in package main.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a minimal implementation of the read-only parts of the OCI
Distribution spec that OpenTofu uses for dependency installation. It's not
intended as a complete registry implementation.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This new method collects all of the various different settings that
describe the operator's chosen OCI credentials policy and returns a single
object that encapsulates that policy.
This is the method that will, in future commits, be used by package main
to provide the credentials policy to any OCI-registry-related subsystems
using dependency-inversion style.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We're not actually doing anything with the result of this yet, but this
covers the decoding and validation of this new block type which we'll start
making use of in subsequent commits as we start to bring together the
overall OCI credentials selection policy handling.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This was previously an unexported function used only for implementing the
Docker/Podman/etc auth format, but we'll be using the same syntax in the
labels of our CLI configuration language's "oci_credentials" blocks and
so we'll export it to allow use from package cliconfig.
The name also changes to use the "ContainersAuth" prefix rather than the
"DockerCLIStyle" prefix because we are technically implementing the
expanded form of this configuration format that supports repository path
matching, rather than the original form that Docker CLI pioneered. This
new format doesn't really have a catchy brand name, but it's documented
as "containers auth.json" and so "ContainersAuth" seems like a reasonable
compromise at following that name while keeping this concise.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we just required the slash-separated segments to match without
imposing any further constraint, but if the Docker-style config syntax
evolves to allow other syntaxes here in future it'd be better for us to
just ignore what we don't recognize rather than get confused into trying
to match it in the current way.
ParseRepositoryAddressPrefix is an exported function because package
cliconfig will use it in a future commit to deal with our OpenTofu-specific
equivalent of the "auths" objects: oci_credentials blocks in the CLI
configuration.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As of this commit we don't actually do anything with these once decoded.
In future commits we'll add decoding of repository-specific oci_credentials
blocks and then the logic for combining all of these settings together
into an ociauthconfig.CredentialsConfigs representing the overall
credentials selection policy for OCI repositories.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Our requirements for discovering credentials for OCI registries include
automatic discovery or manual specification of Docker CLI-style auth
configuration files, which is a nontrivial amount of logic in itself,
along with an OpenTofu-specific version of that configuration model
embedded in the CLI configuration.
To avoid incorporating all of this extra scope into package cliconfig,
this new package ociauthconfig helps with modeling the overall OCI registry
authentication policy and with the Docker-CLI-style auth config format.
In a future commit, package cliconfig will drive this package's behavior
based on the operator's CLI configuration settings, eventually returning
an ociauthconfig.CredentialsConfigs representing the configured auth
policy, which package main can then deliver to other components as part
of an OCI client.
This ultimately yields the ORAS Go library's credentials type, since that
module has a relatively narrow indirect dependency surface area and will
avoid us needlessly implementing and maintaining our own OCI registry
client implementations.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We seem to have inherited an incomplete implementation of something from
the predecessor project here: a "tofu cloud" command that just tries to
immediately delegate any invocation to another executable called
"terraform-cloudplugin" in the current working directory, used as a
go-plugin style plugin.
This has some TODO comments suggesting that it was intended to change to
download a plugin from some remote place before executing it, but our
stubby version doesn't do that. I was also hidden behind an experimental
feature guard and so has never been accessible in any released version of
OpenTofu; we don't currently produce any releases with experimental
features enabled.
Therefore this commit just deletes it so that we don't have this dead code
to potentially worry about. Perhaps one day we'll offer some extension
point for adding extra subcommands through plugins, but if we do that then
we'll presumably design our own mechanism for doing it rather than
extending this dead code that was added for reasons unknown to us.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Go 1.24 introduces stricter checks for format string validation.
This commit fixes instances where non-constant format strings were
used in calls to functions like `fmt.Errorf`, `fmt.Printf`, and similar.
Changes include:
- Replacing dynamically constructed strings passed as format strings
with constant format strings.
- Refactoring `fmt.Sprintf` calls to ensure the format string matches
the number of arguments provided.
- Simplifying redundant formatting and ensuring compliance with Go
1.24's stricter `vet` tool checks.
This update ensures compatibility with Go 1.24 and prevents potential
runtime errors caused by misinterpreted dynamic format strings.
Resolves#2389
Signed-off-by: Mikel Olasagasti Uranga <mikel@olasagasti.info>
Co-authored-by: Martin Atkins <mart@degeneration.co.uk>
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This function was previously quite long and complex, so this commit splits
it into a number of smaller functions.
The previous code structure was made more awkward by having to work around
all being together in one function -- particularly the part iterating over
the values used in an expression -- and so the new layout is quite
different and thus the diff is hard to read. However, there are
intentionally no test changes in this commit to help us be confident that
this has not regressed anything, and the existing unit tests for this
component seem quite comprehensive.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We're intending to gradually improve all of the existing functions that
fail these checks as a separate project from other work, because fixing
for these particular lint rules tends to be too invasive to be safe or
sensible to combine with other work.
Therefore we'll temporarily disable these lints from the main lint run
and add a separate .golangci-complexity.yml that we can use to track our
progress towards eliminating those lint failures without continuing to
litter the code with nolint comments in the meantime.
This also removes all of the existing nolint comments for these linters so
that we can start fresh and review each one as part of our improvement
project.
We'll re-enable these linters (and remove .golangci-complexity.yml) once
each example has either been rewritten to pass the checks or we've
concluded that further decomposition would hurt readability and so added
"nolint" comments back in so we can review whether our lint rules are too
strict once we've got a bunch of examples to consider together.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
