Apparently I had the wrong noun in my head when I wrote this changelog description, since this change _externally_ uses "exclude" but _internally_ uses a feature called "deferred". 🙄
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This caused a bunch of mechanical changes to callers, of course. Expression
evaluation is a very cross-cutting concern, so updating everything all at
once would be a lot and so this stops at a mostly-arbitrary point wiring
a bunch of callers to pass in contexts without changing anything that has
lots of callers.
We'll continue pulling on this thread in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Because of the support for provider-contributed functions, expression
evaluation can potentially cause provider gRPC requests to happen, and so
we'll need to be able to plumb OpenTelemetry trace information through to
those calls.
This initial commit focuses mainly on just getting the functions in
lang.Scope set up to take context.Context, along with their companions in
configs.StaticEvaluator, while leaving most of the callers just passing
context.TODO() for now so we can gradually deal with the rest of the
plumbing in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This addresses several security advisories. None of them impact OpenTofu
significantly, and so the primary goal here is just to clear false-positive
security scanner reports for future releases per our typical policy for
non-impactful advisories in our upstream dependencies.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Continuing the ongoing work of getting context.Context wired in everywhere
we might want to generate OpenTelemetry traces, this completes all of the
provider-related methods of EvalContext.
Unfortunately there is still one remaining path not included here: the
EvalContext.EvaluationScope method needs to somehow arrange for contexts
to reach the provider-defined functions so that we can pass the context
to providers.Interface.CallFunction, which is tricky because that has to
get through the cty function API that wasn't designed for functions that
are backed by network calls. We'll deal with that in a subsequent commit
because it's likely to be a more invasive change than the
relatively-mechanical wiring updates included here.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
It seems that a small number of providers are now able to return a special
signal when they find that they are unable to perform an operation due to
unknown values in the provider or resource configuration.
This is a uses that new signal to recommend a workaround in that situation,
giving a more actionable error message than would've been returned by the
provider otherwise.
We've not yet decided how OpenTofu might make use of these new signals in
the long term, and so this is intentionally implemented in a way where
most of the logic is centralized in the provider-related packages rather
than sprawled all over "package tofu".
It's likely that a future incarnation of this will plumb this idea in more
deeply, but this is just a temporary stop-gap to give slightly better
error messages in the meantime and so it's better to keep it relatively
contained for now until we have a longer-term plan for what OpenTofu Core
might do with this information.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This completes some of the missing connections for contexts in the provider
source codepaths by introducing context.Context parameters and wiring them
through so we can eliminate a few more context.TODO() placeholders.
For consistency's sake this adds context.Context to all four of the
getproviders.Source implementations that directly interact with stuff
outside of OpenTofu (network services or filesystem), even though not
all of them currently make use of it, just because interactions with
outside stuff tends to encourage cross-cutting concerns like logging and
tracing and so this ensures we have contexts propagated in there for such
future uses.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This introduces a new testing helper to package tracing which we can use
to make sure that context.Context values are able to reach all the way
from a test caller to some specific function elsewhere in the system.
We then use that helper in package tofu to test whether context values are
able to reach calls to providers, which tend to be the deepest place that
package tofu is responsible for routing them to.
As of this test the providers.Interface.Configure method does not seem to
receive values, so that is currently commented out in this test and we'll
deal with that properly in a subsequent commit.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we were using a third-party library, but that doesn't have any
support for passing context.Context through its API and so isn't suitable
for our goals of adding OpenTelemetry tracing for all outgoing network
requests.
We now have our own fork that is updated to use context.Context. It also
has a slightly reduced scope no longer including various details that
are tightly-coupled to our cliconfig mechanism and so better placed in the
main OpenTofu codebase so we can evolve it in future without making
lockstep library releases.
The "registry-address" library also uses svchost and uses some of its types
in its public API, so this also incorporates v2 of that library that is
updated to use our own svchost module.
Unfortunately this commit is a mix of mechanical updates to the new
libraries and some new code dealing with the functionality that is removed
in our fork of svchost. The new code is primarily in the "svcauthconfig"
package, which is similar in purpose "ociauthconfig" but for OpenTofu's
own auth mechanism instead of the OCI Distribution protocol's auth
mechanism.
This includes some additional plumbing of context.Context where it was
possible to do so without broad changes to files that would not otherwise
have been included in this commit, but there are a few leftover spots that
are context.TODO() which we'll address separately in later commits.
This removes the temporary workaround from d079da6e9e, since we are now
able to plumb the OpenTelemetry span tree all the way to the service
discovery requests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This text was from earlier on in the development period when we hadn't yet
settled on a strategy, but now we are intending to work with the ORAS team
so that ORAS can construct index manifests in the form that OpenTofu
expects, so we'll talk about that instead of making a comment about
prereleases that will become confusing once OpenTofu v1.10.0 final is
released.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Continuing our work to gradually plumb context.Context to everywhere that
we want to generate OpenTelemetry traces, this completes the call path
for most (but not all) of the gRPC requests to provider plugins, so that
we can add OpenTelemetry trace instrumentation in a future commit.
Unfortunately there are still a few providers.Interface callers left in
functions that don't have context.Context plumbed to them yet, and so
those are temporarily stubbed as context.TODO() here so we can more easily
find and complete them later.
The two gRPC implementations of providers.Interface were previously making
provider requests using a single context.Context established at the time
the provider process was started, but that isn't an appropriate context
to use for per-request concerns like tracing, so that context is now
unused and could potentially be removed in a future commit, but this change
already got pretty large and so I intend to deal with that separately
later.
This now exposes the gRPC provider calls to potential context cancellation
that they would previously observe only indirectly though the Stop method.
Since Stop is primarily used for graceful shutdown of ApplyResourceChange,
the changes here explicitly disconnect the cancellation signal for
ApplyResourceChange in particular, while letting the others get canceled
in the normal way since they are expected to be free of significant
side-effects. In future work we could consider removing Stop from the
internal API entirely and keeping it only as an implementation detail of
the gRPC implementation of this interface, with ApplyResourceChange
directly reacting to context cancellation and sending the gRPC Stop call
itself, but again that's too much change for this already-large commit.
The internal/legacy package currently contains some legacy code preserved
for the benefit of the backends, and unfortunately contains more than is
strictly necessary to support those callers, and so there was some dead
code there that also needed updating. provider_mock.go is removed entirely
because it's just an older copy of the similar file in package tofu. The
few calls to providers in schemas.go are updated to use
context.Background() rather than context.TODO() because we have no
intention of plumbing context.Context into that legacy code, and will
hopefully just delete it wholesale one day.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
