This caused a bunch of mechanical changes to callers, of course. Expression
evaluation is a very cross-cutting concern, so updating everything all at
once would be a lot and so this stops at a mostly-arbitrary point wiring
a bunch of callers to pass in contexts without changing anything that has
lots of callers.
We'll continue pulling on this thread in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Continuing the ongoing work of getting context.Context wired in everywhere
we might want to generate OpenTelemetry traces, this completes all of the
provider-related methods of EvalContext.
Unfortunately there is still one remaining path not included here: the
EvalContext.EvaluationScope method needs to somehow arrange for contexts
to reach the provider-defined functions so that we can pass the context
to providers.Interface.CallFunction, which is tricky because that has to
get through the cty function API that wasn't designed for functions that
are backed by network calls. We'll deal with that in a subsequent commit
because it's likely to be a more invasive change than the
relatively-mechanical wiring updates included here.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
It seems that a small number of providers are now able to return a special
signal when they find that they are unable to perform an operation due to
unknown values in the provider or resource configuration.
This is a uses that new signal to recommend a workaround in that situation,
giving a more actionable error message than would've been returned by the
provider otherwise.
We've not yet decided how OpenTofu might make use of these new signals in
the long term, and so this is intentionally implemented in a way where
most of the logic is centralized in the provider-related packages rather
than sprawled all over "package tofu".
It's likely that a future incarnation of this will plumb this idea in more
deeply, but this is just a temporary stop-gap to give slightly better
error messages in the meantime and so it's better to keep it relatively
contained for now until we have a longer-term plan for what OpenTofu Core
might do with this information.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Continuing our work to gradually plumb context.Context to everywhere that
we want to generate OpenTelemetry traces, this completes the call path
for most (but not all) of the gRPC requests to provider plugins, so that
we can add OpenTelemetry trace instrumentation in a future commit.
Unfortunately there are still a few providers.Interface callers left in
functions that don't have context.Context plumbed to them yet, and so
those are temporarily stubbed as context.TODO() here so we can more easily
find and complete them later.
The two gRPC implementations of providers.Interface were previously making
provider requests using a single context.Context established at the time
the provider process was started, but that isn't an appropriate context
to use for per-request concerns like tracing, so that context is now
unused and could potentially be removed in a future commit, but this change
already got pretty large and so I intend to deal with that separately
later.
This now exposes the gRPC provider calls to potential context cancellation
that they would previously observe only indirectly though the Stop method.
Since Stop is primarily used for graceful shutdown of ApplyResourceChange,
the changes here explicitly disconnect the cancellation signal for
ApplyResourceChange in particular, while letting the others get canceled
in the normal way since they are expected to be free of significant
side-effects. In future work we could consider removing Stop from the
internal API entirely and keeping it only as an implementation detail of
the gRPC implementation of this interface, with ApplyResourceChange
directly reacting to context cancellation and sending the gRPC Stop call
itself, but again that's too much change for this already-large commit.
The internal/legacy package currently contains some legacy code preserved
for the benefit of the backends, and unfortunately contains more than is
strictly necessary to support those callers, and so there was some dead
code there that also needed updating. provider_mock.go is removed entirely
because it's just an older copy of the similar file in package tofu. The
few calls to providers in schemas.go are updated to use
context.Background() rather than context.TODO() because we have no
intention of plumbing context.Context into that legacy code, and will
hopefully just delete it wholesale one day.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The graph node types related to resources and resource instances use a
bunch of helper functions in different combinations, rather than calling
directly into the provider API.
This commit plumbs context.Context through to the functions that _do_
eventually call methods directly on the provider object, leaving us just
one more step away from plumbing the context through to the actual gRPC
calls. The next step (in a future commit) will be to update the
providers.Interface methods to take context.Context arguments and then
have the gRPC-based implementations of that interface pass the context
through to the gRPC client stub calls, and then we should be pretty close
to being able to turn on OTel tracing instrumentation for our gRPC
client requests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This continues our ongoing standardization on using evalCtx for parameters
of type EvalContext, so that we can use ctx for parameters of type
context.Context.
This commit is just a bunch of mechanical renames with no substantive
changes. Future commits will introduce additional context.Context params
to many of these functions.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This commit begins the introduction of OpenTelemetry tracing into the
implementations of individual graph node types in the language runtime.
It's tempting to just plumb this in centrally in ContextGraphWalker for all
graph nodes, but our intention is for the trace output to correspond as
much as possible to the OpenTofu user experience rather than its
implementation details, and the execution graph is _very much_ an
implementation detail with many aspects that have no direct manifestation
in the UI.
With that in mind, this focuses only on capturing traces for graph node
types that fit into one or more of the following categories:
- Corresponds to something we already announce in the UI output today.
- Can cause at least one provider gRPC (or other similar) request to occur
and so makes a good parent for later-added traces of those requests.
- Has an execution time that depends on something outside of OpenTofu's
direct control, such as interactive input which takes as long as the
user takes to answer the prompts. (Including these helps account for
this time separately from the time when OpenTofu was doing useful work.)
As long as our focus continues to be on echoing concepts we expect users
will already be familiar with from their use of OpenTofu, this probably
represents the deepest level of detail we ought to represent for the
purposes of describing OpenTofu's conceptual process. In a future commit
we'll add one extra level of nested spans beneath this one representing
directly the gRPC requests we're making to providers, since those spans
will represent the handoff to a potential sub-trace generated by the
provider plugin itself once we've established a mechanism for providers to
discover their trace parent.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
In order to generate OpenTelemetry traces of the main work that OpenTofu
Core does we'll need to be able to propagate the active trace context into
the main "Execute" method of each graph node, since that's where we
typically make requests to providers, and other such work that could take
a noticeable amount of time.
Changing these frequently-used interfaces is always a noisy diff, so this
commit intentionally focuses only on changing the signature of that
interface and its one caller, and then dealing with all of the fallout of
that on existing unit test code.
For any use of Execute that was affected by this change we'll also switch
to our newer naming scheme of using "evalCtx" as the name of the
tofu.EvalContext variable, in preparation for using "ctx" idiomatically to
refer to context.Context. However, the implementations currently don't yet
name their context.Context argument because the method bodies don't yet
make use of it. We'll name each of those arguments to "ctx" individually
as we gradually add tracing support to each graph node type.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we made a very generic suggestion to use -target to split a
change into two parts as a workaround for the fact that count and for_each
must be known during planning. That works, but we didn't have enough
information available to tell the operator exactly what to target and so
anyone who is not an expert on the configuration they're working with tends
to get stuck unable to figure out exactly what they need to do.
The new -exclude option gives us an opportunity to do better here: we tend
to know for which object we're currently evaluating count or for_each, and
so we can mention that object directly in the error message when if we
recommend to use -exclude instead of -target.
Not all objects that support count/for_each will necessarily be directly
targetable, so we can still potentially recommend -target when we're
dealing with one of those objects. For example, as of this commit that
is true for for_each in a provider block, because there is not currently
any syntax for specifying a provider configuration as an addrs.Targetable.
Perhaps we'll introduce such a thing in the future, but that's outside the
scope of this change that's primarily focused on improving the messaging
for resource and module count/for_each.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
