As discussed in opentofu/opentofu#2656, this consolidates the two concerns
of the PackageAuthentication interface into a single function that deals
both with package authentication _and_ with reporting all of the package
hashes that were used to make the authentication decision.
This means that any .zip archive that OpenTofu directly verifies during
installation can now have its hash recorded in the dependency lock file
even if that package didn't come from the provider's origin registry, which
is beneficial when the first installation of a provider comes from a
secondary ("mirror") source because it creates an additional hook by which
that dependency lock file entry can be "upgraded" to be complete in a
future "tofu init" run against the origin registry, or by the
"tofu providers lock" command.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Most of the OCI registry interactions are unit tested in the most relevant
packages, but the overall system will only work correctly if all of the
components are correctly wired together by "package main", and that's one
part of the system that needs to be tested concretely rather than via
test doubles.
Therefore this adds an end-to-end test in our existing e2etest package
that runs "tofu init" with a CLI configuration that forces using an OCI
mirror with a TLS server provided locally by our test program. It exercises
the main happy path of provider installation in the same way that an
end-user would interact with it, to help avoid accidentally regressing
the interactions between these packages in future versions.
Unfortunately the technique this test uses to force the OpenTofu CLI
binary to trust the test server doesn't work on macOS or Windows and so
for now this test is Linux-specific. That's certainly non-ideal, but
pragmatic since we'll be relying mainly on the platform-agnostic unit tests
to cover this behavior, and we're unlikely to ever stop running the
e2etests on Linux as part of our pull request checks so even those
developing on macOS or Windows can still notice if this test becomes
broken before merging a change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
It's now valid to include an oci_mirror block in the provider_installation
block in the CLI configuration, specifying that OpenTofu should try to
install providers from OCI repositories based on a template that maps
from OpenTofu-style provider source addresses into OCI repository
addresses.
The getproviders.Source implementation for this was added in a previous
commit, so this is mainly just wiring it up to the cliconfig layer and
the dependency wiring code in package main.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a minimal implementation of the read-only parts of the OCI
Distribution spec that OpenTofu uses for dependency installation. It's not
intended as a complete registry implementation.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This new method collects all of the various different settings that
describe the operator's chosen OCI credentials policy and returns a single
object that encapsulates that policy.
This is the method that will, in future commits, be used by package main
to provide the credentials policy to any OCI-registry-related subsystems
using dependency-inversion style.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We're not actually doing anything with the result of this yet, but this
covers the decoding and validation of this new block type which we'll start
making use of in subsequent commits as we start to bring together the
overall OCI credentials selection policy handling.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This was previously an unexported function used only for implementing the
Docker/Podman/etc auth format, but we'll be using the same syntax in the
labels of our CLI configuration language's "oci_credentials" blocks and
so we'll export it to allow use from package cliconfig.
The name also changes to use the "ContainersAuth" prefix rather than the
"DockerCLIStyle" prefix because we are technically implementing the
expanded form of this configuration format that supports repository path
matching, rather than the original form that Docker CLI pioneered. This
new format doesn't really have a catchy brand name, but it's documented
as "containers auth.json" and so "ContainersAuth" seems like a reasonable
compromise at following that name while keeping this concise.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we just required the slash-separated segments to match without
imposing any further constraint, but if the Docker-style config syntax
evolves to allow other syntaxes here in future it'd be better for us to
just ignore what we don't recognize rather than get confused into trying
to match it in the current way.
ParseRepositoryAddressPrefix is an exported function because package
cliconfig will use it in a future commit to deal with our OpenTofu-specific
equivalent of the "auths" objects: oci_credentials blocks in the CLI
configuration.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As of this commit we don't actually do anything with these once decoded.
In future commits we'll add decoding of repository-specific oci_credentials
blocks and then the logic for combining all of these settings together
into an ociauthconfig.CredentialsConfigs representing the overall
credentials selection policy for OCI repositories.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Our requirements for discovering credentials for OCI registries include
automatic discovery or manual specification of Docker CLI-style auth
configuration files, which is a nontrivial amount of logic in itself,
along with an OpenTofu-specific version of that configuration model
embedded in the CLI configuration.
To avoid incorporating all of this extra scope into package cliconfig,
this new package ociauthconfig helps with modeling the overall OCI registry
authentication policy and with the Docker-CLI-style auth config format.
In a future commit, package cliconfig will drive this package's behavior
based on the operator's CLI configuration settings, eventually returning
an ociauthconfig.CredentialsConfigs representing the configured auth
policy, which package main can then deliver to other components as part
of an OCI client.
This ultimately yields the ORAS Go library's credentials type, since that
module has a relatively narrow indirect dependency surface area and will
avoid us needlessly implementing and maintaining our own OCI registry
client implementations.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We seem to have inherited an incomplete implementation of something from
the predecessor project here: a "tofu cloud" command that just tries to
immediately delegate any invocation to another executable called
"terraform-cloudplugin" in the current working directory, used as a
go-plugin style plugin.
This has some TODO comments suggesting that it was intended to change to
download a plugin from some remote place before executing it, but our
stubby version doesn't do that. I was also hidden behind an experimental
feature guard and so has never been accessible in any released version of
OpenTofu; we don't currently produce any releases with experimental
features enabled.
Therefore this commit just deletes it so that we don't have this dead code
to potentially worry about. Perhaps one day we'll offer some extension
point for adding extra subcommands through plugins, but if we do that then
we'll presumably design our own mechanism for doing it rather than
extending this dead code that was added for reasons unknown to us.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Go 1.24 introduces stricter checks for format string validation.
This commit fixes instances where non-constant format strings were
used in calls to functions like `fmt.Errorf`, `fmt.Printf`, and similar.
Changes include:
- Replacing dynamically constructed strings passed as format strings
with constant format strings.
- Refactoring `fmt.Sprintf` calls to ensure the format string matches
the number of arguments provided.
- Simplifying redundant formatting and ensuring compliance with Go
1.24's stricter `vet` tool checks.
This update ensures compatibility with Go 1.24 and prevents potential
runtime errors caused by misinterpreted dynamic format strings.
Resolves#2389
Signed-off-by: Mikel Olasagasti Uranga <mikel@olasagasti.info>
Co-authored-by: Martin Atkins <mart@degeneration.co.uk>
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This function was previously quite long and complex, so this commit splits
it into a number of smaller functions.
The previous code structure was made more awkward by having to work around
all being together in one function -- particularly the part iterating over
the values used in an expression -- and so the new layout is quite
different and thus the diff is hard to read. However, there are
intentionally no test changes in this commit to help us be confident that
this has not regressed anything, and the existing unit tests for this
component seem quite comprehensive.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We're intending to gradually improve all of the existing functions that
fail these checks as a separate project from other work, because fixing
for these particular lint rules tends to be too invasive to be safe or
sensible to combine with other work.
Therefore we'll temporarily disable these lints from the main lint run
and add a separate .golangci-complexity.yml that we can use to track our
progress towards eliminating those lint failures without continuing to
litter the code with nolint comments in the meantime.
This also removes all of the existing nolint comments for these linters so
that we can start fresh and review each one as part of our improvement
project.
We'll re-enable these linters (and remove .golangci-complexity.yml) once
each example has either been rewritten to pass the checks or we've
concluded that further decomposition would hurt readability and so added
"nolint" comments back in so we can review whether our lint rules are too
strict once we've got a bunch of examples to consider together.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Validate doesn't yet do anything with its new context, but we'll
plumb this deeper in future.
Since the local backend's implementation of backend.Local.LocalRun calls
Validate on the given configuration before returning, it this also extends
that interface method to take a context, and so the various commands that
directly create "local runs" (rather than going through the backend
operation API) now all pass in a context derived from the one created
in "package main".
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Eval doesn't yet do anything with its new context, but we'll
plumb this deeper in future.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Import doesn't yet do anything with its new context, but we'll
plumb this deeper in future.
OpenTofu has some historical situational private uses of context.Context
to handle the graceful shutdown behaviors. Those use context.Context as
a private implementation detail rather than public API, and so this commit
leaves them as-is and adds a new "primary context" alongside. Hopefully
in future refactoring we can simplify this to use the primary context also
as the primary cancellation signal, but that's too risky a change to bundle
in with this otherwise-mostly-harmless context plumbing.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Apply does not yet do anything with its new context, but this gets
the context plumbed in enough that we should be able to pass values like
telemetry spans all the way from the top-level in future.
OpenTofu has some historical situational private uses of context.Context
to handle the graceful shutdown behaviors. Those use context.Context as
a private implementation detail rather than public API, and so this commit
leaves them as-is and adds a new "primary context" alongside. Hopefully
in future refactoring we can simplify this to use the primary context also
as the primary cancellation signal, but that's too risky a change to bundle
in with this otherwise-mostly-harmless context plumbing.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing effort to get a coherent chain of
context.Context all the way from "package main" to all of our calls to
external components.
Context.Plan does not yet do anything with its new context, but this gets
the context plumbed in enough that we should be able to pass values like
telemetry spans all the way from the top-level in future.
OpenTofu has some historical situational private uses of context.Context
to handle the graceful shutdown behaviors. Those use context.Context as
a private implementation detail rather than public API, and so this commit
leaves them as-is and adds a new "primary context" alongside. Hopefully
in future refactoring we can simplify this to use the primary context also
as the primary cancellation signal, but that's too risky a change to bundle
in with this otherwise-mostly-harmless context plumbing.
All of the _test.go file updates here are purely mechanical additions of
the extra argument. No test is materially modified by this change, which
is intentional to get some assurance that isn't a breaking change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is part of an ongoing effort to plumb a properly-connected series of
contexts through all of the layers where we might want to generate
telemetry (or similar) in future.
This is _just enough_ to connect the top-level context created by package
main with the various child contexts created by the local backend, so
that they could in principle access the root span that package main
generates.
This is not yet sufficient to propagate the context all the way into the
language runtime. More plumbing to follow in later commits!
This intentionally does not introduce any new OpenTelemetry-specific
context: the goal is only to get the context chain in place so that we
can use it for telemetry delivery in future.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The "backendConfigNeedsMigration" helper evaluates the backend
configuration inline to compare it with the object previously saved in the
.terraform/terraform.tfstate file.
However, this wasn't updated to use the new "static eval" functionality
and so was treating any references to variables or function calls as
invalid, causing a spurious "backend configuration changed" error when
re-initializing the working directory with identical backend configuration
settings.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
