This adds a cache for resource values that is evicted by changes
to state or plan sync objects. In theory the eviction is not needed,
but there are hints from tests that that assumption is not fully
correct.
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
When given a resource m that depends on resource n, if for_each or count
were used in any significant way a massive lock contention was
encountered.
In pseudocode:
```
for mi in m {
for ni in n {
lock()
for c in {m, n} {
...
}
unlock()
}
}
// Iterations: m * n * (m+n)
// Locking: m * n
```
To:
```
for mi in m {
lock()
for c in {m, n} {
...
}
unlock()
for ni in n {
...
}
for ni in n {
...
}
}
// Iterations: m * (m+n+n+n)
// Locking: m
```
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
This minor release includes a security update that does not directly affect
OpenTofu, but we'll adopt the fix for future releases anyway since the
risk seems low and this will likely help quiet false positive reports from
security scanners moving forward.
This also includes various non-security bugfixes which may improve
OpenTofu's reliability.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The disco.Disco API isn't yet set up to pass through context.Context, and
so if we give it a HTTP client that has the OpenTelemetry instrumentation
on it then any HTTP request causes an orphan trace span disconnected from
the main trace, which causes annoying noise in some trace viewers.
As a temporary solution so we can ship v1.10 soon without making large
changes to the svchost library, we'll prevent the HTTP client constructor
function from detecting that tracing is enabled by passing it
context.TODO() instead of the actual context. This would not be acceptable
in the long run but is safe for this temporary workaround because currently
httpclient.New doesn't use the given context for anything except detecting
whether tracing is enabled.
We will address this in a more complete way during the v1.11 development
period by modernizing svchost to take context.Context arguments on all
functions that can potentially make external requests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The primary reason for this change is that registry.NewClient was
originally imposing its own decision about service discovery request
policy on every other user of the shared disco.Disco object by modifying
it directly.
We have been moving towards using a dependency inversion style where
package main is responsible for deciding how everything should be
configured based on global CLI arguments, environment variables, and the
CLI configuration, and so this commit moves to using that model for the
HTTP clients used by the module and provider registry client code.
This also makes explicit what was previously hidden away: that all service
discovery requests are made using the same HTTP client policy as for
requests to module registries, even if the service being discovered is not
a registry. This doesn't seem to have been the intention of the code as
previously written, but was still its ultimate effect: there is only one
disco.Disco object shared across all discovery callers and so changing its
configuration in any way changes it for everyone.
This initial rework is certainly not perfect: these components were not
originally designed to work in this way and there are lots of existing
test cases relying on them working the old way, and so this is a compromise
to get the behavior we now need (using consistent HTTP client settings
across all callers) without disrupting too much existing code.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As part of our continuing effort to plumb context.Context everywhere we
might want to generate tracing spans, this is a first tentative step to
retrofitting the EvalContext interface.
This is a _huge_ interface, so a single commit updating all of the methods
would be too much to deal with. Instead, we'll start small and focus only
on the provider schema lookup because at this point that's the one
remaining part of the core language runtime that makes
potentially-expensive external requests but isn't yet able to generate
tracing spans, and so currently the time spent on this gets misattributed
to whichever caller is unfortunate enough to ask for the schema first and
thus becomes responsible for populating the schema cache.
The callers of this method are not yet plumbed for context themselves, so
a subsequent commit will replace these new context.TODO() calls with
something more useful.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We're moving away from using "ctx" to refer to a variable of any type
other than context.Context, to match with the modern Go idiom. Our current
conventions would normally call for an EvalContext variable to be called
evalCtx, but we also tend not to use such long names for reciever symbols
and instead use a one or two letter initialism from nouns in the type
name, and so in this case we'll adopt just "c" which also happens to match
the naming convention used on the other implementation of this type,
MockEvalContext.
In future commits we'll start adding "ctx context.Context" arguments to
many of the methods of this type, so the motivation here is to just get
these conflicting symbols out of the way in one ugly-but-boring commit
so that this systematic change is separate from any other changes.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Earlier work made GraphBuilder.Build take a context, but we had a missing
link in the context-passing chain due to the plan and apply graph
construction being factored out into a separate unexported function.
This commit just adds the remaining few steps of plumbing so that a useful
upstream context can reach all of the graph builders, so that it'd be
okay for downstream code to generate trace spans in future.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Most of our transformers are pure compute and so don't really have a strong
need to generate trace spans under our current focus of only exposing
user-facing concepts and external requests in our traces, but unfortunately
some of them indirectly depend on provider schema, which in turn means that
they can potentially be unlucky enough to be the trigger for making all
of the provider requests needed to fill the schema cache and therefore
would end up with provider request spans being reported beneath them.
As usual with these interface updates, this initial change focuses only
on changing the interface and updating its direct callers and implementers
to match, without any further refactoring or attempts to plumb contexts
to or from other functions that don't have them yet. That means there are
a few new context.TODO() calls here that we'll tidy up in a later commit
that hopefully won't involve all of the noise that is caused by changing
an interface API.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This, along with its supporting interface GraphVertexTransformer, were not
used by any code except their own tests, but we hadn't previously noticed
that because (like many symbols in this package, unfortunately) they had
exported names despite being implementation details.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
In preparation for propagating cross-cutting concerns like tracing spans
to the plugin calls we make to load schemas, this adds context.Context
parameters to the schema lookup functions.
Unfortunately we call these from many different locations inside the
package today, and not all of them yet have a suitable parent context to
pass in, and so those locations currently use context.TODO() placeholders
to make it easier for us to track those down and improve them in subsequent
commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As part of our ongoing work to plumb cross-cutting concerns like tracing
spans into the core language runtime, here we change the exported API
of the context.Schemas method to take a context.Context, and trivially
update all of the callers to pass in a suitable context.
Earlier work on this means that we don't have fix up too many call stack
levels before we already have a suitable context.Context value to use.
The Schemas method doesn't yet make any use of its new context.Context, but
that will follow in subsequent PRs.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
