This introduces the concept of "backend aliases", which are alternative
names that can be used to refer to a given backend.
Each backend type has one canonical name and zero or more alias names. The
"backend" block in the root module can specify either a canonical backend
type or an alias, but internally OpenTofu will always track the backend
type using its canonical name.
In particular, the following are all true when the configuration specifies
an alias instead of a canonical backend type:
- The "tofu init" output includes a brief extra message saying which
backend type OpenTofu actually used, because that is the name that we'd
prioritize in our documentation and so an operator can use the canonical
type to find the relevant docs when needed.
- The .terraform/terraform.tfstate file that tracks the working directory's
currently-initialized backend settings always uses the canonical backend
type, and so it's possible to freely switch between aliases and canonical
without "tofu init" thinking that a state migration might be needed.
- Plan files similarly use the canonical backend type to track which
backend was active when the plan was created, which doesn't have any
significant user-facing purpose, but is consistent with the previous
point since the settings in the plan file effectively substitute for
the .terraform/terraform.tfstate file when applying a saved plan.
- The terraform_remote_state data source in the provider
terraform.io/builtin/terraform accepts both canonical and alias in its
backend type argument, treating both as equivalent for the purpose of
fetching the state snapshot for the configured workspace.
The primary motivation for this new facility is to allow the planned
"oracle_oci" backend to have an alias "oci" to allow writing configurations
that are cross-compatible with HashiCorp Terraform, since that software
has chosen to have unqualified OCI mean Oracle's system, whereas OpenTofu
has previously established that unqualified OCI means "Open Container
Initiative" in our ecosystem.
In particular, this design makes it possible in principle to bring an
existing Terraform configuration specifying backend "oci" over to OpenTofu
without modifications, and then to optionally switch it to specifying
backend "oracle-oci" at a later time without a spurious prompt to migrate
state snapshots to the same physical location where they are already
stored.
This commit doesn't actually introduce any aliases and therefore doesn't
have any tests for the new mechanism because our backend system uses a
global table that isn't friendly to mocking for testing purposes. I've
tested this manually using a placeholder alias to have confidence that it
works, and I expect that a subsequent commit introducing the new
"oracle_oci" backend will also introduce its "oci" alias and will include
tests that cover use of the alias and migration from the alias to the
canonical name and vice-versa.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This completes some of the missing connections for contexts in the provider
source codepaths by introducing context.Context parameters and wiring them
through so we can eliminate a few more context.TODO() placeholders.
For consistency's sake this adds context.Context to all four of the
getproviders.Source implementations that directly interact with stuff
outside of OpenTofu (network services or filesystem), even though not
all of them currently make use of it, just because interactions with
outside stuff tends to encourage cross-cutting concerns like logging and
tracing and so this ensures we have contexts propagated in there for such
future uses.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we were using a third-party library, but that doesn't have any
support for passing context.Context through its API and so isn't suitable
for our goals of adding OpenTelemetry tracing for all outgoing network
requests.
We now have our own fork that is updated to use context.Context. It also
has a slightly reduced scope no longer including various details that
are tightly-coupled to our cliconfig mechanism and so better placed in the
main OpenTofu codebase so we can evolve it in future without making
lockstep library releases.
The "registry-address" library also uses svchost and uses some of its types
in its public API, so this also incorporates v2 of that library that is
updated to use our own svchost module.
Unfortunately this commit is a mix of mechanical updates to the new
libraries and some new code dealing with the functionality that is removed
in our fork of svchost. The new code is primarily in the "svcauthconfig"
package, which is similar in purpose "ociauthconfig" but for OpenTofu's
own auth mechanism instead of the OCI Distribution protocol's auth
mechanism.
This includes some additional plumbing of context.Context where it was
possible to do so without broad changes to files that would not otherwise
have been included in this commit, but there are a few leftover spots that
are context.TODO() which we'll address separately in later commits.
This removes the temporary workaround from d079da6e9e, since we are now
able to plumb the OpenTelemetry span tree all the way to the service
discovery requests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The primary reason for this change is that registry.NewClient was
originally imposing its own decision about service discovery request
policy on every other user of the shared disco.Disco object by modifying
it directly.
We have been moving towards using a dependency inversion style where
package main is responsible for deciding how everything should be
configured based on global CLI arguments, environment variables, and the
CLI configuration, and so this commit moves to using that model for the
HTTP clients used by the module and provider registry client code.
This also makes explicit what was previously hidden away: that all service
discovery requests are made using the same HTTP client policy as for
requests to module registries, even if the service being discovered is not
a registry. This doesn't seem to have been the intention of the code as
previously written, but was still its ultimate effect: there is only one
disco.Disco object shared across all discovery callers and so changing its
configuration in any way changes it for everyone.
This initial rework is certainly not perfect: these components were not
originally designed to work in this way and there are lots of existing
test cases relying on them working the old way, and so this is a compromise
to get the behavior we now need (using consistent HTTP client settings
across all callers) without disrupting too much existing code.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We previously had two tests of how the module installer responds to
cancellation (e.g. SIGINT) which were flakey because they tried to rely
on the cancellation being detected at some arbitrary point before the
module installer attempted to make a request, which isn't guaranteed in
practice because our interrupt mechanism only aims to cause OpenTofu to
exit "soon", with no guarantee about how much ongoing progress it will
make before it does.
To make these tests more robust, we'll now instead tell the module
installer to install from a real HTTP server that is intentionally designed
to stall the client by accepting its request but then just leaving the
connection open without responding.
This means that we can now test the more realistic situation of the cancel
signal being triggered after a slow request is already in progress, and
be sure that we're definitely sending the cancel signal at a moment that
matches that intention.
This is similar to a strategy we previously took to improve the reliability
of the tests for cancellation of the _provider_ installer, in
TestInit_cancelProviders. However, our provider installer version of this
used an intentionally-stalling implementation of getproviders.Source
instead of running a real server because the provider installer is designed
to support configurable installation methods, while the module installer
is not: its policy about what module source types are accepted is
hard-coded in package getproviders, at least for now.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Earlier work started to reshape this API to follow the dependency inversion
style, but didn't go so far as treating the package fetcher as an argument
because so far it hasn't offered any customizable policy anyway.
In future commits we will be introducing some policy arguments for the
package fetcher, and so this is some preparation work where we move the
responsibility for calling getmodules.NewPackageFetcher() out into the
caller of initwd.NewModuleInstaller().
This changes the API consumed by a bunch of unit testing helpers, so
splitting this out into its own commit should hopefully make future
commits more focused. The module installer now explicitly supports being
instantiated without a registry client or a remote package fetcher and
will in that case return an error if it's asked to install from anywhere
other than local relative directories. Most of our existing tests are
comfortable running under that constraint and so will not need any further
work in later commits that will change the signature of
getmodules.NewPackageFetcher.
However, a couple tests in package initwd _itself_ were making use of the
esoteric legacy support for treating an absolute filesystem path as a funny
sort of remote source, and so for now those will instantiate their own
package fetcher. Future commits that change the NewPackageFetcher signature
will need to offer a concession for those two tests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
* Rename module name from "github.com/hashicorp/terraform" to "github.com/placeholderplaceholderplaceholder/opentf".
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Gofmt.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Regenerate protobuf.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comments.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo issue and pull request link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo comment changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comment.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo some link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* make generate && make protobuf
Signed-off-by: Jakub Martin <kubam@spacelift.io>
---------
Signed-off-by: Jakub Martin <kubam@spacelift.io>
This is a mostly mechanical refactor with a handful of changes which
are necessary due to the semantic difference between earlyconfig and
configs.
When parsing root and descendant modules in the module installer, we now
check the core version requirements inline. If the Terraform version is
incompatible, we drop any other module loader diagnostics. This ensures
that future language additions don't clutter the output and confuse the
user.
We also add two new checks during the module load process:
* Don't try to load a module with a `nil` source address. This is a
necessary change due to the move away from earlyconfig.
* Don't try to load a module with a blank name (i.e. `module ""`).
Because our module loading manifest uses the stringified module path
as its map key, this causes a collision with the root module, and a
later panic. This is the bug which triggered this refactor in the
first place.
