Previously we interpreted a "required_version" argument in a "terraform"
block as if it were specifying an OpenTofu version constraint, when in
reality most modules use this to represent a version constraint for
OpenTofu's predecessor instead.
The primary effect of this commit is to introduce a new top-level block
type called "language" which describes language and implementation
compatibility metadata in a way that intentionally differs from what's used
by OpenTofu's predecessor.
This also causes OpenTofu to ignore the required_version argument unless
it appears in an OpenTofu-specific file with a ".tofu" suffix, and makes
OpenTofu completely ignore the language edition and experimental feature
opt-in options from OpenTofu's predecessor on the assumption that those
could continue to evolve independently of changes in OpenTofu.
We retain support for using required_versions in .tofu files as a bridge
solution for modules that need to remain compatible with OpenTofu versions
prior to v1.12. Module authors should keep following the strategy of
having both a versions.tf and a versions.tofu file for now, and wait until
the OpenTofu v1.11 series is end-of-life before adopting the new "language"
block type.
I also took this opportunity to simplify how we handle these parts of the
configuration, since the OpenTofu project has no immediate plans to use
either multiple language editions or language experiments and so for now
we can reduce our handling of those language features to just enough that
we'd return reasonable error messages if today's OpenTofu is exposed to
a module that was written for a newer version of OpenTofu that extends
these language features. The cross-cutting plumbing for representing the
active experiments for a module is still present so that we can reactivate
it later if we need to, but for now that set will always be empty.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We don't typically just broadly run automatic rewriting tools like "go fix"
across our codebase because that tends to cause annoying and unnecessary
merge conflicts when we're backporting to earlier release branches.
But all of the files in this commit were changed in some non-trivial way
already during the OpenTofu v1.11 development period anyway, and so the
likelyhood we'd be able to successfully backport from them is reduced and
therefore this seems like a good opportunity to do some focused
modernization using "go fix".
My rules for what to include or not are admittedly quite "vibes-based", but
the general idea was:
- Focusing on files under the "command" directory only, because that's
already been an area of intentional refactoring during this development
period.
- If the existing diff in a file is already significantly larger than
the changes the fixer proposed to make, or if the fixer is proposing
to change a line that was already changed in this development period.
- More willing to include "_test.go" files than non-test files, even if
they hadn't changed as much already, just because backports from test
files for bug fixes tend to be entirely new test cases more than they
are modifications to existing test cases, and so the risk of conflicts
is lower there.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we were using a third-party library, but that doesn't have any
support for passing context.Context through its API and so isn't suitable
for our goals of adding OpenTelemetry tracing for all outgoing network
requests.
We now have our own fork that is updated to use context.Context. It also
has a slightly reduced scope no longer including various details that
are tightly-coupled to our cliconfig mechanism and so better placed in the
main OpenTofu codebase so we can evolve it in future without making
lockstep library releases.
The "registry-address" library also uses svchost and uses some of its types
in its public API, so this also incorporates v2 of that library that is
updated to use our own svchost module.
Unfortunately this commit is a mix of mechanical updates to the new
libraries and some new code dealing with the functionality that is removed
in our fork of svchost. The new code is primarily in the "svcauthconfig"
package, which is similar in purpose "ociauthconfig" but for OpenTofu's
own auth mechanism instead of the OCI Distribution protocol's auth
mechanism.
This includes some additional plumbing of context.Context where it was
possible to do so without broad changes to files that would not otherwise
have been included in this commit, but there are a few leftover spots that
are context.TODO() which we'll address separately in later commits.
This removes the temporary workaround from d079da6e9e, since we are now
able to plumb the OpenTelemetry span tree all the way to the service
discovery requests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The primary reason for this change is that registry.NewClient was
originally imposing its own decision about service discovery request
policy on every other user of the shared disco.Disco object by modifying
it directly.
We have been moving towards using a dependency inversion style where
package main is responsible for deciding how everything should be
configured based on global CLI arguments, environment variables, and the
CLI configuration, and so this commit moves to using that model for the
HTTP clients used by the module and provider registry client code.
This also makes explicit what was previously hidden away: that all service
discovery requests are made using the same HTTP client policy as for
requests to module registries, even if the service being discovered is not
a registry. This doesn't seem to have been the intention of the code as
previously written, but was still its ultimate effect: there is only one
disco.Disco object shared across all discovery callers and so changing its
configuration in any way changes it for everyone.
This initial rework is certainly not perfect: these components were not
originally designed to work in this way and there are lots of existing
test cases relying on them working the old way, and so this is a compromise
to get the behavior we now need (using consistent HTTP client settings
across all callers) without disrupting too much existing code.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As part of our ongoing work to plumb cross-cutting concerns like tracing
spans into the core language runtime, here we change the exported API
of the context.Schemas method to take a context.Context, and trivially
update all of the callers to pass in a suitable context.
Earlier work on this means that we don't have fix up too many call stack
levels before we already have a suitable context.Context value to use.
The Schemas method doesn't yet make any use of its new context.Context, but
that will follow in subsequent PRs.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our work to follow the dependency inversion style for the
"package fetcher" component of the module installer.
Mimicking the existing pattern for providers, package main is now
responsible for instantiating the PackageFetcher and providing it to
the "command" package as a field of command.Meta.
We could potentially go further here and follow dependency inversion style
for _all_ of the special clients needed by the various go-getter getters,
but our primary concern for now is preparing to add a new "getter" for
installation from an OCI Distribution repository, and so we'll leave the
other already-working code unchanged to reduce the risk of this initial
work.
Future commits will actually wire in the implementation details for OCI
Repository access. This commit focuses only on plumbing the necessary
objects through the API layers.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is part of an ongoing effort to plumb a properly-connected series of
contexts through all of the layers where we might want to generate
telemetry (or similar) in future.
This is _just enough_ to connect the top-level context created by package
main with the various child contexts created by the local backend, so
that they could in principle access the root span that package main
generates.
This is not yet sufficient to propagate the context all the way into the
language runtime. More plumbing to follow in later commits!
This intentionally does not introduce any new OpenTelemetry-specific
context: the goal is only to get the context chain in place so that we
can use it for telemetry delivery in future.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
* Rename module name from "github.com/hashicorp/terraform" to "github.com/placeholderplaceholderplaceholder/opentf".
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Gofmt.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Regenerate protobuf.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comments.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo issue and pull request link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo comment changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comment.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo some link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* make generate && make protobuf
Signed-off-by: Jakub Martin <kubam@spacelift.io>
---------
Signed-off-by: Jakub Martin <kubam@spacelift.io>
Several times over the years we've considered adding tracing
instrumentation to Terraform, since even when running in isolation as a
CLI program it has a "distributed system-like" structure, with lots of
concurrent internal work and also some work delegated to provider plugins
that are essentially temporarily-running microservices.
However, it's always felt a bit overwhelming to do it because much of
Terraform predates the Go context.Context idiom and so it's tough to get
a clean chain of context.Context values all the way down the stack without
disturbing a lot of existing APIs.
This commit aims to just get that process started by establishing how a
context can propagate from "package main" into the command package,
focusing initially on "terraform init" and some other commands that share
some underlying functions with that command.
OpenTelemetry has emerged as a de-facto industry standard and so this uses
its API directly, without any attempt to hide it behind an abstraction.
The OpenTelemetry API is itself already an adapter layer, so we should be
able to swap in any backend that uses comparable concepts. For now we just
discard the tracing reports by default, and allow users to opt in to
delivering traces over OTLP by setting an environment variable when
running Terraform (the environment variable was established in an earlier
commit, so this commit builds on that.)
When tracing collection is enabled, every Terraform CLI run will generate
at least one overall span representing the command that was run. Some
commands might also create child spans, but most currently do not.
Currently Terraform will use an entry from the global plugin cache only if
it matches a checksum already recorded in the dependency lock file. This
allows Terraform to produce a complete lock file entry on the first
encounter with a new provider, whereas using the cache in that case would
cause the lock file to only cover the single package in the cache and
thereefore be unusable on any other operating system or CPU architecture.
This temporary CLI config option is a pragmatic exception to support those
who cannot currently correctly use the dependency lock file but who still
want to benefit from the plugin cache. With this setting enabled,
Terraform has permission to produce a dependency lock file that is only
suitable for the current system if that would allow use of an existing
entry in the plugin cache.
We are introducing this option to resolve a conflict between the needs of
folks who are using the dependency lock file as expected and the needs of
folks who cannot use the dependency lock file for some reason. The hope
then is to give respite to those who need this exception in the meantime
while we understand better why they cannot use the dependency lock file
and improve its design so that everyone will be able to use it
successfully in a future version of Terraform. This option will become a
silent no-op in a future version of Terraform, once the dependency lock
file behavior is sufficient for all supported Terraform development
workflows.
