To support the workflow of saving a plan to disk and applying it on some
other machine we need to be able to represent the execution graph as a byte
stream and then reload it later to produce an equivalent execution graph.
This is an initial implementation of that, based on the way the execgraph
package currently represents execution graphs. We may change that
representation more in future as we get more experience working in the new
architecture, but this is intended as part of our "walking skeleton" phase
where we try to get the new architecture working end-to-end with simple
configurations as soon as possible to help verify that we're even on the
right track with this new approach, and try to find unknown unknowns that
we ought to deal with before we get too deep into this.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is primarily to clear naive security scanner reports for GO-2025-4135,
which is a potential denial of service if attacker-controlled software can
send malformed packets back to OpenTofu through the SSH Agent proxy
channel.
We are not considering this a significant vulnerability for OpenTofu
because the SSH Agent forwarding pattern already assumes that software on
the remote system is trusted not to misuse the keys that are exposed though
the proxy channel.
Due to the Go team's policy of ratcheting upgrades between all of the
golang.org/x/* modules, this also requires upgrading three other modules.
I have reviewed the changes in those, and most appear to not affect
OpenTofu at all. There are some performance improvements to the HTTP2 and
QUIC implementations in x/net, but they don't seem to be a big concern for
us.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is some minimal glue to help the new runtime use the providers that
were gathered up by the existing logic in the "command" package.
This is cheating a little because this is relying on "tofu init" still
using the old approach just enough to find out which providers are needed
and get them installed, but our current focus is on the main plan and
apply phases and so it's convenient to be able to leave that part untouched
for now and return to improve it later, once we have more of the
fundamentals in place.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a relatively uninteresting milestone where it's possible to load
and plan a root module that contains nothing except local values and
output values.
The module loader currently supports only local sources and the plugin
APIs just immediately return errors, so configurations more complicated
than that are likely to just fail immediately with one or more errors.
We'll gradually improve on this in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
None of these actually work yet, but this satisfies the new-style config
loader enough for it to return a real error instead of immediately
panicking.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We currently re-export evalglue.ExternalModules, but the method in that
interface requires returning evalglue.UncompiledModule and so we need to
export that too or else it's impossible to actually implement the interface
from outside this family of packages.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This doesn't actually work yet. It's just to sketch out a minimal overall
sequence of steps to make this behave somewhat like the main implementation
of "tofu plan", and then we'll make it work better in subsequent commits.
The main omission as of this commit is that we don't yet pass module,
provider, and provisioner dependency access objects in the EvalContext,
and so config loading immediately fails trying to request the root module
from a nil object.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
To facilitate early development and testing of the new language runtime
we're introducing a temporary mechanism to opt in to using the new codepaths
based on an environment variable. This environment variable is effective
only for experiment-enabled builds of OpenTofu, and so it will be
completely ignored by official releases of OpenTofu.
This commit just deals with the "wiring" of this new mechanism, without
actually connecting it with the new language runtime yet. The goal here
is to disturb existing codepaths as little as possible to minimize both
the risk of making this change and the burden this causes for ongoing
maintenance unrelated to work on the new language runtime.
This strategy of switching at the local backend layer means that we will
have some duplicated logic in the experimental functions compared to the
non-experimental functions, which is an intentional tradeoff to allow us
to isolate what we're doing so we don't churn existing code while we're
still in this early exploration phase. In a later phase of the language
runtime project we may pivot to a different approach which switches at
a deeper point in the call stack, but for now we're keeping this broad
to give us flexibility.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The GODEBUG environment variable is a side-channel that allows dynamically
opting in to a million billion different variations of Go runtime and
standard library behavior, and we can obviously not routinely test
OpenTofu's behavior across all of those different variations.
Just in case someone encounters a problem caused by running OpenTofu with
a combination of settings that are not enabled in our default build
configuration, we'll include an explicit note in the logs so that we can
tell when we're investigating a bug report that it might only be
reproducible when the Go runtime behavior has been overridden in this way,
and so that someone debugging their own problem can notice that they are
using OpenTofu in an unsupported way and could potentially solve their own
problem by unsetting that environment variable.
This is a generalization of the previous addition of a log message when
running in FIPS 140-3 mode, which is also updated here to use [WARN]
instead of [WARNING] as the prefix because our logging system does not
actually understand "WARNING" as a valid prefix. Keeping the separate
specialized message for FIPS 140-3 mode is warranted because we _know_
that OpenTofu does not behave as intended when that mode is enabled, while
we've not tested with any other combination of settings so we cannot
predict whether they will cause misbehavior or not.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
At some point after this documentation was originally written the schema
structure grew to include the possibility of structural attribute types,
represented using "nested_type" instead of "type" in the attribute
definition, but it seems that the documentation was not updated to mention
that.
This is just a minimal extra note about that focused mainly on just
acknowledging that this possibility exists at all, in case anyone is
relying on these docs to build something to parse this format. It would
probably be helpful to expand both this and the existing documentation to
specify the format more precisely, but my focus here is just on quickly
filling in this missing piece so that the documentation is complete, even
if not detailed and precise.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously the ReferenceTransformer just had a broad rule that any node
which implements GraphNodeDestroyer has its expression references
completely ignored.
Now that we're starting to allow dynamic expressions for destroy-related
settings like "prevent_destroy", we need to be able to represent the
dependencies implied by those expressions.
However, the assumption that configuration is mostly ignored when planning
destroy is load-bearing for minimizing awkward dependency problems in the
destroy-planning graph, so this introduces a new concept of "destroy
references" which means that we can implement only a small, curated subset
of references -- for now, just the ones from prevent_destroy -- that get
considered for any node type that implements GraphNodeDestroyer.
Having GraphNodeDestroyer effectively take priority over
GraphNodeReferencer seems like the least disruptive way to retrofit this
idea surgically as a small change to the previous unilateral rule against
any references at all, because in practice all of the destroy nodes embed
NodeAbstractResourceInstance and therefore implement GraphNodeReferencer,
so it is important that we continue to ignore that type's
GraphNodeReferencer implementation whenever it's embedded in something
that is also a GraphNodeDestroyer.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The Go team tends to ratchet all of these libraries forward together even
if there have been no significant changes since the last release, and so
these three only include some fixes to typos in comments and test code
and some internal modernization to use the generic reflect.TypeFor instead
of the interface-based reflect.TypeOf.
Upgrading these should not affect OpenTofu's behavior in any material way,
and so this is just to get these easy ones out of the way before we deal
with the more significant changes in the other related modules that will
likely require closer review.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Unfortunately the Go team has unilaterally decided that all programs built
with Go 1.24 and later always allow enabling FIPS-140-3 mode -- both in
its "on" and "only" configurations -- regardless of whether the authors
of that software intend to support running in that restricted mode, or
whether they are even testing their application in that configuration.
We have not yet made a final decision on how and whether we intend to
support this mode in our official builds, but we _do_ know that OpenTofu
cannot currently function correctly with this mode enabled because it
relies on standard library features and external libraries that are not
available in that case.
Therefore in the meantime we'll mention explicitly in both the internal
logs and in the "tofu version" output if we appear to be running in that
mode, meaning that if someone tries to use it and finds that it doesn't
work properly then if they open a GitHub issue and share those two
artifacts (as requested by our bug report template) then we can know that
we might need to turn on the special mode in order to reproduce the
reported problem, rather than wasting time trying to reproduce it in the
standard mode.
We do still need to make a final decision about what we want to do with
this in the long run, but this is intended as an short-term compromise
that allows folks to experiment with this unsupported mode if they wish
while hopefully making it clearer that in the mean time we may
deprioritize fixing problems that only occur when this unusual mode is
enabled.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The fake time implementation in package synctest does not create a
synchronization point itself, so the previous version of this test was
technically incorrect, though it was passing in practice due to other
effective constraints on the execution behavior.
A synctest.Wait after each fake time.Sleep creates a synchronization point
that helps ensure the correct execution order without relying on unrelated
implementation details such as the mutexes around the planned changes
and in-memory state data.
This does not actually change the current test behavior in any significant
way, but would avoid this test becoming more flaky if implementation
details of the language runtime were to change significantly in future.
(That seems pretty unlikely in practice -- we're intending to write a new
one instead of evolving this one significantly -- but I think it's still
worth making the effort to use the synctest API correctly as part of
learning to make effective use of it.)
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we evaluated prevent_destroy expressions immediately inside the
config loader, thereby forcing it to always be a constant expression
producing a bool value.
Now the config loader just saves whatever expression it was given and we
let the language runtime deal with it instead, which means we can allow
references to dynamically-chosen values from elsewhere in the same module.
The language runtime's "validate" phase still performs a type check for
bool that's equivalent to what we used to do during config loading to make
sure that the "tofu validate" command can catch a similar subset of
problems as it used to be able to catch, but we have more information
available during the plan phase that allows us to produce more complete and
relevant error messages, so for any expression that we can't evaluate
with a nil evaluation context we'll now let the plan phase deal with the
checks instead.
The policy for handling annoying cases such as unknown values, ephemeral
values, sensitive values, and references to local symbols like count.index
is intentionally the most conservative choice to start, because future
versions of OpenTofu can allow more once we've got more experience but
cannot permit less if we find that we've made a mistake. Future changes
could potentially make these rules a little more liberal, once we have
learned from feedback on this initial functionality.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
By design the "-raw" option to "tofu output" writes the literal output
value directly to stdout without any quoting or escaping, and so it's
risky to use it with an output value that could be controlled by an
attacker when stdout is a terminal.
This risk is inherent in the purpose of this option and is part of the
reason why this is not the default behavior (OpenTofu returns a quoted
representation of an output string by default) so here we just make that
risk explicit in the documentation, in the hope that operators will use
this operation mindfully.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Since this is replacing C0 control characters with other control characters
rather than just removing them completely, "replace" is probably the more
intuitive name for this function.
This also removes the preallocation of the output buffer in the case where
control characters were present in the input, letting the strings.Builder
implementation manage the buffer growth automatically itself.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
From reading the special-case login code for the HCP Terraform host it
appears that the intention of the special "motd.v1" protocol was that it
should return a message that uses the syntax defined by the "colorstring"
library to make use of a limited set of control characters, and that it
was not intended to allow the text to include arbitrary control characters
that might cause more significant effects on a terminal and would not be
filtered out properly when running in "no color" mode.
Therefore we'll make this slightly more robust by filtering out any control
characters, using format.FilterControlChars. Note that this behavior is
exclusive to the HCP Terraform hostname "app.terraform.io", which is
unlikely to be used in OpenTofu anyway (since that service is presumably
offered for Terraform's use) and so this is not a particularly significant
change but is just part of some work to avoid situations where remote
network services can potentially cause OpenTofu to emit arbitrary control
characters to a terminal.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
A diagnostic message can potentially include substrings derived from data
fetched from untrusted sources, such as if a network request fails in a
way that causes part of the response data to be included verbatim in the
error message.
This new filtering ensures that if any such data is included then any
C0 control characters in the string cannot affect the state of a terminal
that stdout/stderr might be connected to, by replacing them with their
corresponding printable representations from Unicode's "Control Pictures"
block.
The filtering of source snippets and source filenames is not technically
necessary because those are under control of module authors only and
operators are already expected to review modules they use to ensure that
they can cause only desirable behavior, since modules are arbitrary code.
However, it's included here for defense-in-depth because there is little
reason for such characters to appear legitimately in either of those
contexts in practice.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This utility function replaces C0 control characters in a given string with
their corresponding symbols from the "Control Pictures" Unicode block.
As of this commit nothing is using this, but in future commits we will use
this when preparing terminal UI output that may contain arbitrary strings
that are not subject to any other quoting/escaping to ensure that it is
not possible to affect virtual terminal state in sitations where that is
not intentionally allowed by OpenTofu.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We have a number of trace attributes that we use across all of our OCI
Distribution-based functionality, so this centralizes their definitions
in package traceattrs.
This intentionally ignores a few additional attribute names that are used
only in the code that interacts with Docker-style credential helpers,
because all of those are used only in a single function and so adding
indirection for those doesn't have enough benefit to offset the cost of
additional indirection.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We previously added re-exports for some of the functions we'd previously
been importing directly from semconv elsewhere in this codebase. For this
one we'd previously just hard-coded the standardized attribute name, but
for consistency we'll also use a re-export of a semconv function for this
one too.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Prevously OpenTofu delegated browser launching entirely to the third-party
module github.com/cli/browser, which consists of a number of
platform-specific lists of executable commands to try to run to launch
a web browser.
On Unix systems there is also a de-facto convention of using an environment
variable called BROWSER to explicitly specify what to launch. That variable
can either point directly to a browser, or can point to a script which
implements some more complex policy for choosing a browser, such as
detecting whether the command is running in a GUI context and launching
either a GUI or textmode browser.
The BROWSER variable has been most commonly implemented with similar
treatment to earlier variables like EDITOR and PAGER where it's expected
to be set to just a single command to run, with the URL given as the first
and only argument. There was also an attempt to define a more complex
interpretation of this variable at http://www.catb.org/~esr/BROWSER/ , but
that extended treatment was only implemented in a small amount of software,
and those which implemented it did so slightly inconsistently due to the
specification being ambiguous.
OpenTofu's implementation therefore follows the common simpler convention,
but will silently ignore variable values it cannot use so that OpenTofu
won't fail when run in an environment that has that variable set in a way
that's intended for use by some other software. In that case OpenTofu
will continue to perform the default behavior as implemented in the
third-party library.
Because this convention is Unix-specific, OpenTofu will check for and use
this environment variable only on operating systems that the Go toolchain
considers to be "unix". This means that in particular on Windows systems
OpenTofu will continue to follow the Windows convention of specifying
the default browser via an entry in the Windows Registry.
As usual with this sort of system-integration mechanism it isn't really
viable to test this end-to-end in a portable way, but the main logic is
separated out into testable functions, and I manually tested this on my
own Linux system to verify that it works in a real OpenTofu executable.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
For anything we run as PR checks it should always be straightforward to
recreate any failure locally in a development environment, so instead of
using the golangci/golangci-lint-action GitHub action (whose behavior
is nontrivial and annoying to reproduce outside of GitHub Actions) we'll
just use our existing Makefile target as part of the check run, matching
how we deal with most other similar concerns in this GitHub Actions
workflow.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade, keeping us up-to-date with the latest
linter changes. This already passes when run on Linux.
We know that currently the lint checks aren't passing on Windows even with
v2.4.0, and this doesn't do anything about that but it does mean that once
we make the linters pass on Windows we'll be dealing with the full set
supported in v2.6.0, rather than potentially having to do this across
two commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Go 1.17 and earlier used a different syntax for build constraint comments,
starting with "+build". Go 1.18 changed this to the modern "go:build" form
as part of standardizing the structure of toolchain directive comments,
and so for a while it was convention to include comments in both styles
to allow building with both old and new Go compilers.
However, Go 1.17 is no longer supported, and regardless of that we only
expect OpenTofu to be built with the specific version we have selected
in "go.mod" and ".go-version" anyway, so we no longer need the legacy form
of these comments: the all supported Go toolchains now support the new
form, which this commit retains.
golangci-lint v2.6.0 includes a check for this legacy form, so removing
this will also allow us to upgrade to a newer version of that linter
aggregator in a future commit.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
OpenTelemetry has various Go packages split across several Go modules that
often need to be carefully upgraded together. And in particular, we are
using the "semconv" package in conjunction with the OpenTelemetry SDK's
"resource" package in a way that requires that they both agree on which
version of the OpenTelemetry Semantic Conventions are being followed.
To help avoid "dependency hell" situations when upgrading, this centralizes
all of our direct calls into the OpenTelemetry SDK and tracing API into
packages under internal/tracing, by exposing a few thin wrapper functions
that other packages can use to access the same functionality indirectly.
We only use a relatively small subset of the OpenTelemetry library surface
area, so we don't need too many of these reexports and they should not
represent a significant additional maintenance burden.
For the semconv and resource interaction in particular this also factors
that out into a separate helper function with a unit test, so we should
notice quickly whenever they become misaligned. This complements the
end-to-end test previously added in opentofu/opentofu#3447 to give us
faster feedback about this particular problem, while the end-to-end test
has the broader scope of making sure there aren't any errors at all when
initializing OpenTelemetry tracing.
Finally, this also replaces the constants we previously had in package
traceaddrs with functions that return attribute.KeyValue values directly.
This matches the API style used by the OpenTelemetry semconv packages, and
makes the calls to these helpers from elsewhere in the system a little
more concise.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We got this new functionality from an upgrade of the upstream cty library,
but we didn't update the docs to mention it.
The examples in this doc were also evidently generated with a much older
version of OpenTofu's predecessor, because the illustrated output was not
shown as a quoted string. The example output now matches how the current
version of "tofu console" would describe these results.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
