To facilitate early development and testing of the new language runtime
we're introducing a temporary mechanism to opt in to using the new codepaths
based on an environment variable. This environment variable is effective
only for experiment-enabled builds of OpenTofu, and so it will be
completely ignored by official releases of OpenTofu.
This commit just deals with the "wiring" of this new mechanism, without
actually connecting it with the new language runtime yet. The goal here
is to disturb existing codepaths as little as possible to minimize both
the risk of making this change and the burden this causes for ongoing
maintenance unrelated to work on the new language runtime.
This strategy of switching at the local backend layer means that we will
have some duplicated logic in the experimental functions compared to the
non-experimental functions, which is an intentional tradeoff to allow us
to isolate what we're doing so we don't churn existing code while we're
still in this early exploration phase. In a later phase of the language
runtime project we may pivot to a different approach which switches at
a deeper point in the call stack, but for now we're keeping this broad
to give us flexibility.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Unfortunately the Go team has unilaterally decided that all programs built
with Go 1.24 and later always allow enabling FIPS-140-3 mode -- both in
its "on" and "only" configurations -- regardless of whether the authors
of that software intend to support running in that restricted mode, or
whether they are even testing their application in that configuration.
We have not yet made a final decision on how and whether we intend to
support this mode in our official builds, but we _do_ know that OpenTofu
cannot currently function correctly with this mode enabled because it
relies on standard library features and external libraries that are not
available in that case.
Therefore in the meantime we'll mention explicitly in both the internal
logs and in the "tofu version" output if we appear to be running in that
mode, meaning that if someone tries to use it and finds that it doesn't
work properly then if they open a GitHub issue and share those two
artifacts (as requested by our bug report template) then we can know that
we might need to turn on the special mode in order to reproduce the
reported problem, rather than wasting time trying to reproduce it in the
standard mode.
We do still need to make a final decision about what we want to do with
this in the long run, but this is intended as an short-term compromise
that allows folks to experiment with this unsupported mode if they wish
while hopefully making it clearer that in the mean time we may
deprioritize fixing problems that only occur when this unusual mode is
enabled.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
By design the "-raw" option to "tofu output" writes the literal output
value directly to stdout without any quoting or escaping, and so it's
risky to use it with an output value that could be controlled by an
attacker when stdout is a terminal.
This risk is inherent in the purpose of this option and is part of the
reason why this is not the default behavior (OpenTofu returns a quoted
representation of an output string by default) so here we just make that
risk explicit in the documentation, in the hope that operators will use
this operation mindfully.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Since this is replacing C0 control characters with other control characters
rather than just removing them completely, "replace" is probably the more
intuitive name for this function.
This also removes the preallocation of the output buffer in the case where
control characters were present in the input, letting the strings.Builder
implementation manage the buffer growth automatically itself.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
From reading the special-case login code for the HCP Terraform host it
appears that the intention of the special "motd.v1" protocol was that it
should return a message that uses the syntax defined by the "colorstring"
library to make use of a limited set of control characters, and that it
was not intended to allow the text to include arbitrary control characters
that might cause more significant effects on a terminal and would not be
filtered out properly when running in "no color" mode.
Therefore we'll make this slightly more robust by filtering out any control
characters, using format.FilterControlChars. Note that this behavior is
exclusive to the HCP Terraform hostname "app.terraform.io", which is
unlikely to be used in OpenTofu anyway (since that service is presumably
offered for Terraform's use) and so this is not a particularly significant
change but is just part of some work to avoid situations where remote
network services can potentially cause OpenTofu to emit arbitrary control
characters to a terminal.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
A diagnostic message can potentially include substrings derived from data
fetched from untrusted sources, such as if a network request fails in a
way that causes part of the response data to be included verbatim in the
error message.
This new filtering ensures that if any such data is included then any
C0 control characters in the string cannot affect the state of a terminal
that stdout/stderr might be connected to, by replacing them with their
corresponding printable representations from Unicode's "Control Pictures"
block.
The filtering of source snippets and source filenames is not technically
necessary because those are under control of module authors only and
operators are already expected to review modules they use to ensure that
they can cause only desirable behavior, since modules are arbitrary code.
However, it's included here for defense-in-depth because there is little
reason for such characters to appear legitimately in either of those
contexts in practice.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This utility function replaces C0 control characters in a given string with
their corresponding symbols from the "Control Pictures" Unicode block.
As of this commit nothing is using this, but in future commits we will use
this when preparing terminal UI output that may contain arbitrary strings
that are not subject to any other quoting/escaping to ensure that it is
not possible to affect virtual terminal state in sitations where that is
not intentionally allowed by OpenTofu.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Prevously OpenTofu delegated browser launching entirely to the third-party
module github.com/cli/browser, which consists of a number of
platform-specific lists of executable commands to try to run to launch
a web browser.
On Unix systems there is also a de-facto convention of using an environment
variable called BROWSER to explicitly specify what to launch. That variable
can either point directly to a browser, or can point to a script which
implements some more complex policy for choosing a browser, such as
detecting whether the command is running in a GUI context and launching
either a GUI or textmode browser.
The BROWSER variable has been most commonly implemented with similar
treatment to earlier variables like EDITOR and PAGER where it's expected
to be set to just a single command to run, with the URL given as the first
and only argument. There was also an attempt to define a more complex
interpretation of this variable at http://www.catb.org/~esr/BROWSER/ , but
that extended treatment was only implemented in a small amount of software,
and those which implemented it did so slightly inconsistently due to the
specification being ambiguous.
OpenTofu's implementation therefore follows the common simpler convention,
but will silently ignore variable values it cannot use so that OpenTofu
won't fail when run in an environment that has that variable set in a way
that's intended for use by some other software. In that case OpenTofu
will continue to perform the default behavior as implemented in the
third-party library.
Because this convention is Unix-specific, OpenTofu will check for and use
this environment variable only on operating systems that the Go toolchain
considers to be "unix". This means that in particular on Windows systems
OpenTofu will continue to follow the Windows convention of specifying
the default browser via an entry in the Windows Registry.
As usual with this sort of system-integration mechanism it isn't really
viable to test this end-to-end in a portable way, but the main logic is
separated out into testable functions, and I manually tested this on my
own Linux system to verify that it works in a real OpenTofu executable.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Go 1.17 and earlier used a different syntax for build constraint comments,
starting with "+build". Go 1.18 changed this to the modern "go:build" form
as part of standardizing the structure of toolchain directive comments,
and so for a while it was convention to include comments in both styles
to allow building with both old and new Go compilers.
However, Go 1.17 is no longer supported, and regardless of that we only
expect OpenTofu to be built with the specific version we have selected
in "go.mod" and ".go-version" anyway, so we no longer need the legacy form
of these comments: the all supported Go toolchains now support the new
form, which this commit retains.
golangci-lint v2.6.0 includes a check for this legacy form, so removing
this will also allow us to upgrade to a newer version of that linter
aggregator in a future commit.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
OpenTelemetry has various Go packages split across several Go modules that
often need to be carefully upgraded together. And in particular, we are
using the "semconv" package in conjunction with the OpenTelemetry SDK's
"resource" package in a way that requires that they both agree on which
version of the OpenTelemetry Semantic Conventions are being followed.
To help avoid "dependency hell" situations when upgrading, this centralizes
all of our direct calls into the OpenTelemetry SDK and tracing API into
packages under internal/tracing, by exposing a few thin wrapper functions
that other packages can use to access the same functionality indirectly.
We only use a relatively small subset of the OpenTelemetry library surface
area, so we don't need too many of these reexports and they should not
represent a significant additional maintenance burden.
For the semconv and resource interaction in particular this also factors
that out into a separate helper function with a unit test, so we should
notice quickly whenever they become misaligned. This complements the
end-to-end test previously added in opentofu/opentofu#3447 to give us
faster feedback about this particular problem, while the end-to-end test
has the broader scope of making sure there aren't any errors at all when
initializing OpenTelemetry tracing.
Finally, this also replaces the constants we previously had in package
traceaddrs with functions that return attribute.KeyValue values directly.
This matches the API style used by the OpenTelemetry semconv packages, and
makes the calls to these helpers from elsewhere in the system a little
more concise.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The Go team uses automation to generate unnecessary version bumps across
all of these that make it impossible to upgrade them individually because
they all mutually depend on the latest versions of each other, so
unfortunately we have to accept the risk of updating all of these at once
in order to update any one of them.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously the Go toolchain had no explicit support for "tools" and so we
used the typical Go community workaround of adding "tools.go" files (two,
for some reason) that existed only to trick the Go toolchain into
considering the tools as dependencies we could track in go.mod.
Go 1.24 introduced explicit support for tracking tools as part of go.mod,
and the ability to run those using "go tool" instead of "go run", and so
this commit switches us over to using that strategy for everything we were
previously managing in tools.go.
There are some intentional exceptions here:
- The protobuf-compile script can't use "go tool" or "go run" because the
tools in question are run only indirectly through protoc. However, we
do still use the "tool" directive in go.mod to tell the Go toolchain that
we depend on those tools, so that it'll track which versions we are
currently using as part of go.mod.
- Our golangci-lint Makefile target uses "go run" to run a specific
version of golangci-lint. We _intentionally_ don't consider that tool
to be a direct dependency of OpenTofu because it has a lot of indirect
dependencies that would pollute our go.mod file. Therefore that continues
to use "go run" after this commit.
- Both of our tools.go files previously referred to
github.com/nishanths/exhaustive , but nothing actually appears to be
using that tool in the current OpenTofu tree, so it's no longer a
dependency after this commit.
All of the dependencies we have _only_ for tools are now classified as
"indirect" in the go.mod file. This is the default behavior of the Go
toolchain and appears to be motivated by making it clearer that these
modules do not contribute anything to the runtime behavior of OpenTofu.
This also corrected a historical oddity in our go.mod where for some reason
the "indirect" dependencies had been split across two different "require"
directives; they are now all grouped together in a single directive.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We were previously using this module to access the then-experimental
"slices" package, but equivalent functionality is now available in a
standard library package so we no longer need to use the experimental
version.
This remains as an indirect dependency just because some of the tools we
use depend on it.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This introduces the concept of "backend aliases", which are alternative
names that can be used to refer to a given backend.
Each backend type has one canonical name and zero or more alias names. The
"backend" block in the root module can specify either a canonical backend
type or an alias, but internally OpenTofu will always track the backend
type using its canonical name.
In particular, the following are all true when the configuration specifies
an alias instead of a canonical backend type:
- The "tofu init" output includes a brief extra message saying which
backend type OpenTofu actually used, because that is the name that we'd
prioritize in our documentation and so an operator can use the canonical
type to find the relevant docs when needed.
- The .terraform/terraform.tfstate file that tracks the working directory's
currently-initialized backend settings always uses the canonical backend
type, and so it's possible to freely switch between aliases and canonical
without "tofu init" thinking that a state migration might be needed.
- Plan files similarly use the canonical backend type to track which
backend was active when the plan was created, which doesn't have any
significant user-facing purpose, but is consistent with the previous
point since the settings in the plan file effectively substitute for
the .terraform/terraform.tfstate file when applying a saved plan.
- The terraform_remote_state data source in the provider
terraform.io/builtin/terraform accepts both canonical and alias in its
backend type argument, treating both as equivalent for the purpose of
fetching the state snapshot for the configured workspace.
The primary motivation for this new facility is to allow the planned
"oracle_oci" backend to have an alias "oci" to allow writing configurations
that are cross-compatible with HashiCorp Terraform, since that software
has chosen to have unqualified OCI mean Oracle's system, whereas OpenTofu
has previously established that unqualified OCI means "Open Container
Initiative" in our ecosystem.
In particular, this design makes it possible in principle to bring an
existing Terraform configuration specifying backend "oci" over to OpenTofu
without modifications, and then to optionally switch it to specifying
backend "oracle-oci" at a later time without a spurious prompt to migrate
state snapshots to the same physical location where they are already
stored.
This commit doesn't actually introduce any aliases and therefore doesn't
have any tests for the new mechanism because our backend system uses a
global table that isn't friendly to mocking for testing purposes. I've
tested this manually using a placeholder alias to have confidence that it
works, and I expect that a subsequent commit introducing the new
"oracle_oci" backend will also introduce its "oci" alias and will include
tests that cover use of the alias and migration from the alias to the
canonical name and vice-versa.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
These were previously settable only via environment variables. These are
now handled as part of CLI Configuration and so also settable in a new
"registry_protocols" block in a CLI configuration file, with the
environment variables now treated as if they are an additional virtual
configuration file containing the corresponding settings.
This handles our settings in our modern style where package cliconfig is
responsible for deciding the configuration and then package main reacts
to that configuration without being aware of how it is decided.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
