Previously we evaluated prevent_destroy expressions immediately inside the
config loader, thereby forcing it to always be a constant expression
producing a bool value.
Now the config loader just saves whatever expression it was given and we
let the language runtime deal with it instead, which means we can allow
references to dynamically-chosen values from elsewhere in the same module.
The language runtime's "validate" phase still performs a type check for
bool that's equivalent to what we used to do during config loading to make
sure that the "tofu validate" command can catch a similar subset of
problems as it used to be able to catch, but we have more information
available during the plan phase that allows us to produce more complete and
relevant error messages, so for any expression that we can't evaluate
with a nil evaluation context we'll now let the plan phase deal with the
checks instead.
The policy for handling annoying cases such as unknown values, ephemeral
values, sensitive values, and references to local symbols like count.index
is intentionally the most conservative choice to start, because future
versions of OpenTofu can allow more once we've got more experience but
cannot permit less if we find that we've made a mistake. Future changes
could potentially make these rules a little more liberal, once we have
learned from feedback on this initial functionality.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
OpenTelemetry has various Go packages split across several Go modules that
often need to be carefully upgraded together. And in particular, we are
using the "semconv" package in conjunction with the OpenTelemetry SDK's
"resource" package in a way that requires that they both agree on which
version of the OpenTelemetry Semantic Conventions are being followed.
To help avoid "dependency hell" situations when upgrading, this centralizes
all of our direct calls into the OpenTelemetry SDK and tracing API into
packages under internal/tracing, by exposing a few thin wrapper functions
that other packages can use to access the same functionality indirectly.
We only use a relatively small subset of the OpenTelemetry library surface
area, so we don't need too many of these reexports and they should not
represent a significant additional maintenance burden.
For the semconv and resource interaction in particular this also factors
that out into a separate helper function with a unit test, so we should
notice quickly whenever they become misaligned. This complements the
end-to-end test previously added in opentofu/opentofu#3447 to give us
faster feedback about this particular problem, while the end-to-end test
has the broader scope of making sure there aren't any errors at all when
initializing OpenTelemetry tracing.
Finally, this also replaces the constants we previously had in package
traceaddrs with functions that return attribute.KeyValue values directly.
This matches the API style used by the OpenTelemetry semconv packages, and
makes the calls to these helpers from elsewhere in the system a little
more concise.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This caused a bunch of mechanical changes to callers, of course. Expression
evaluation is a very cross-cutting concern, so updating everything all at
once would be a lot and so this stops at a mostly-arbitrary point wiring
a bunch of callers to pass in contexts without changing anything that has
lots of callers.
We'll continue pulling on this thread in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Because of the support for provider-contributed functions, expression
evaluation can potentially cause provider gRPC requests to happen, and so
we'll need to be able to plumb OpenTelemetry trace information through to
those calls.
This initial commit focuses mainly on just getting the functions in
lang.Scope set up to take context.Context, along with their companions in
configs.StaticEvaluator, while leaving most of the callers just passing
context.TODO() for now so we can gradually deal with the rest of the
plumbing in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Continuing our work to gradually plumb context.Context to everywhere that
we want to generate OpenTelemetry traces, this completes the call path
for most (but not all) of the gRPC requests to provider plugins, so that
we can add OpenTelemetry trace instrumentation in a future commit.
Unfortunately there are still a few providers.Interface callers left in
functions that don't have context.Context plumbed to them yet, and so
those are temporarily stubbed as context.TODO() here so we can more easily
find and complete them later.
The two gRPC implementations of providers.Interface were previously making
provider requests using a single context.Context established at the time
the provider process was started, but that isn't an appropriate context
to use for per-request concerns like tracing, so that context is now
unused and could potentially be removed in a future commit, but this change
already got pretty large and so I intend to deal with that separately
later.
This now exposes the gRPC provider calls to potential context cancellation
that they would previously observe only indirectly though the Stop method.
Since Stop is primarily used for graceful shutdown of ApplyResourceChange,
the changes here explicitly disconnect the cancellation signal for
ApplyResourceChange in particular, while letting the others get canceled
in the normal way since they are expected to be free of significant
side-effects. In future work we could consider removing Stop from the
internal API entirely and keeping it only as an implementation detail of
the gRPC implementation of this interface, with ApplyResourceChange
directly reacting to context cancellation and sending the gRPC Stop call
itself, but again that's too much change for this already-large commit.
The internal/legacy package currently contains some legacy code preserved
for the benefit of the backends, and unfortunately contains more than is
strictly necessary to support those callers, and so there was some dead
code there that also needed updating. provider_mock.go is removed entirely
because it's just an older copy of the similar file in package tofu. The
few calls to providers in schemas.go are updated to use
context.Background() rather than context.TODO() because we have no
intention of plumbing context.Context into that legacy code, and will
hopefully just delete it wholesale one day.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The graph node types related to resources and resource instances use a
bunch of helper functions in different combinations, rather than calling
directly into the provider API.
This commit plumbs context.Context through to the functions that _do_
eventually call methods directly on the provider object, leaving us just
one more step away from plumbing the context through to the actual gRPC
calls. The next step (in a future commit) will be to update the
providers.Interface methods to take context.Context arguments and then
have the gRPC-based implementations of that interface pass the context
through to the gRPC client stub calls, and then we should be pretty close
to being able to turn on OTel tracing instrumentation for our gRPC
client requests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing standardization on using evalCtx for parameters
of type EvalContext, so that we can use ctx for parameters of type
context.Context.
This commit is just a bunch of mechanical renames with no substantive
changes. Future commits will introduce additional context.Context params
to many of these functions.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This commit begins the introduction of OpenTelemetry tracing into the
implementations of individual graph node types in the language runtime.
It's tempting to just plumb this in centrally in ContextGraphWalker for all
graph nodes, but our intention is for the trace output to correspond as
much as possible to the OpenTofu user experience rather than its
implementation details, and the execution graph is _very much_ an
implementation detail with many aspects that have no direct manifestation
in the UI.
With that in mind, this focuses only on capturing traces for graph node
types that fit into one or more of the following categories:
- Corresponds to something we already announce in the UI output today.
- Can cause at least one provider gRPC (or other similar) request to occur
and so makes a good parent for later-added traces of those requests.
- Has an execution time that depends on something outside of OpenTofu's
direct control, such as interactive input which takes as long as the
user takes to answer the prompts. (Including these helps account for
this time separately from the time when OpenTofu was doing useful work.)
As long as our focus continues to be on echoing concepts we expect users
will already be familiar with from their use of OpenTofu, this probably
represents the deepest level of detail we ought to represent for the
purposes of describing OpenTofu's conceptual process. In a future commit
we'll add one extra level of nested spans beneath this one representing
directly the gRPC requests we're making to providers, since those spans
will represent the handoff to a potential sub-trace generated by the
provider plugin itself once we've established a mechanism for providers to
discover their trace parent.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
In order to generate OpenTelemetry traces of the main work that OpenTofu
Core does we'll need to be able to propagate the active trace context into
the main "Execute" method of each graph node, since that's where we
typically make requests to providers, and other such work that could take
a noticeable amount of time.
Changing these frequently-used interfaces is always a noisy diff, so this
commit intentionally focuses only on changing the signature of that
interface and its one caller, and then dealing with all of the fallout of
that on existing unit test code.
For any use of Execute that was affected by this change we'll also switch
to our newer naming scheme of using "evalCtx" as the name of the
tofu.EvalContext variable, in preparation for using "ctx" idiomatically to
refer to context.Context. However, the implementations currently don't yet
name their context.Context argument because the method bodies don't yet
make use of it. We'll name each of those arguments to "ctx" individually
as we gradually add tracing support to each graph node type.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we made a very generic suggestion to use -target to split a
change into two parts as a workaround for the fact that count and for_each
must be known during planning. That works, but we didn't have enough
information available to tell the operator exactly what to target and so
anyone who is not an expert on the configuration they're working with tends
to get stuck unable to figure out exactly what they need to do.
The new -exclude option gives us an opportunity to do better here: we tend
to know for which object we're currently evaluating count or for_each, and
so we can mention that object directly in the error message when if we
recommend to use -exclude instead of -target.
Not all objects that support count/for_each will necessarily be directly
targetable, so we can still potentially recommend -target when we're
dealing with one of those objects. For example, as of this commit that
is true for for_each in a provider block, because there is not currently
any syntax for specifying a provider configuration as an addrs.Targetable.
Perhaps we'll introduce such a thing in the future, but that's outside the
scope of this change that's primarily focused on improving the messaging
for resource and module count/for_each.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
