Commit Graph

73 Commits

Author SHA1 Message Date
RoseSecurity
6e2250e050 chore: typo fixes (#3957)
Signed-off-by: RoseSecurity <michael@rosesecurity.dev>
2026-03-27 18:25:56 -03:00
Martin Atkins
1db4c38ce6 tofu2024: Unit tests for the handling of "enabled" meta-arguments
As with the previous tests added for for_each, this is some basic coverage
under the assumption that the rules for count are not likely to change
so much that it would be arduous to update these tests.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-16 10:58:40 -07:00
Martin Atkins
febfa260b2 tofu2024: Unit tests for the handling of "count" meta-arguments
As with the previous tests added for for_each, this is some basic coverage
under the assumption that the rules for count are not likely to change
so much that it would be arduous to update these tests.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-16 10:58:40 -07:00
Martin Atkins
29ddba7783 tofu2024: Some extra checks for the range of "count"
The previous implementation was rejecting most out-of-range errors, but
it was not rejecting negative numbers, and it was also generating
misleading error messages in other cases because cty's own error messages
for numeric range are pretty generic.

Now we'll do our own numeric range checks before we attempt to extract
the number into an int variable, so we can return more tailored error
messages and can reject negative numbers properly.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-16 10:58:40 -07:00
Martin Atkins
1e3cddd948 tofu2024: Initial unit test for singleton instance selector
This case is not particularly interesting -- it just always produces a
hard-coded result -- but is tested anyway for completeness.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-16 10:58:40 -07:00
Martin Atkins
4f0ba5a6cb tofu2024: Initial unit tests for for_each handling
So far we've been pretty light on testing in these new codepaths because
they've been churning quite a lot during our "walking skeleton" phase of
the new runtime development.

But this instance-selector-related code seems relatively self-contained and
settled, so this introduces an initial set of unit tests for the handling
of for_each expressions, primarily just to document what the intended
behavior of the current implementation was and to illustrate that it is
indeed unit-testable in (relative) isolation.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-16 10:58:40 -07:00
Martin Atkins
6fc3a03198 tofu2024: Properly handle unknown object-typed values in for_each
The comment here was correct that we can accept unknown values of object
types because the attributes are known as part of the type, but we do still
need to handle that in a special way because we need to produce the result
based only on the type information in this case, without trying to access
the value.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-16 10:58:40 -07:00
Martin Atkins
906e35cff1 tofu2024: Do typechecking of for_each even when it's unknown
In earlier work we added some more detailed typechecking rules further
down that can reject incorrectly-typed values even when the value is
unknown, but forgot to remove this early return that made that other code
unreachable whenever the value is unknown.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-16 10:58:40 -07:00
Martin Atkins
d437a65797 lang/eval: Propagate provider instances between modules
Earlier work dealt with the problem of evaluating the references in a
"providers" argument in a module call, but we didn't actually plumb it
through the code that "compiles" the config representation into our
internal configgraph representation.

Filling in this gap means that it's now possible to declare a provider
instance in the root module and then use it in a child module, at the
expense of unfortunately spreading around this "providers side-channel"
special case to some more parts of the system. That seems inevitable
though, since that special case is part of the user-facing language.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-03-02 09:16:44 -08:00
Martin Atkins
d59266dec1 eval/tofu2024: Support "enabled" for resources and module calls
The underlying code for this was already there, but because this was being
written concurrently with implementing that new feature the configs package
part wasn't available yet.

Passing in the enabled expressions here makes enabled work in a similar
way to how count and for_each already work.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-26 16:19:12 -08:00
Martin Atkins
1261d29d41 eval/tofu2024: Handle for_each for resources, etc
In earlier work I skipped implementing for_each because I was just using
count for initial testing, so now it's time to resolve that TODO so that
all of the usual instance-selection arguments are now implemented.

This also includes a few small retroactive fixes of details missed in the
implementations of the other arguments before, which I discovered during
some manual testing.

As usual with these codepaths that are still quite in flux, there aren't
any unit tests here yet but this code is written with the goal of being
relatively easy to unit test in isolation later.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-26 16:19:12 -08:00
Martin Atkins
6981b9f8c6 planning: Dependencies between resource instance object subgraphs
The previous commit arranged for each resource instance object with a
planned change to have an execution subgraph generated for it, but didn't
honor the dependencies between those objects.

There's now a followup loop that adds all of the needed "waiter" edges
after the fact, including both the "forward" dependencies between
create/update changes and the "reverse" dependencies between delete
changes.

The shape of the leaf code here got quite messy. In future commits I intend
to work on cleaning up the details more, but the main focus here was to
restore the execgraph building functionality just enough to prove that this
new two-pass planning approach gives us enough information to insert
all of the needed dependency edges.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-23 10:25:42 -08:00
Martin Atkins
1e0a920a34 planning: Sketch of even more managed resource instance actions
This provides at least a partial implementation of every resource instance
action except the ones involving "forget" actions.

However, we don't really quite have all of the building blocks needed to
properly model "delete" yet, because properly handling those actions means
we need to generate "backwards" dependency edges to preserve the guarantee
that destroying happens in reverse order to creating. Therefore the main
outcome of this commit is to add a bunch of FIXME and TODO comments
explaining where the known gaps are, with the intention of then filling
those gaps in later commits once we devise a suitable strategy to handle
them.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-10 07:07:01 -08:00
Martin Atkins
6ec04c1227 planning: Further simplify provider instance items in execgraph
The previous commit split the handling of provider instance items into
separate dependency-analysis and execgraph construction steps, with the
intention of avoiding the need for the execGraphBuilder to directly
interact with the planning oracle and thus indirectly with the evaluator.
Overall the hope is that execGraphBuilder will be a self-contained object
that doesn't depend on anything else in the planning engine so that it's
easier to write unit tests for it that don't require creating an entire
fake planning context.

However, on reflection that change introduced a completely unnecessary
extra handoff from the execGraphBuilder to _another part of itself_, which
is obviously unnecessary complexity because it doesn't serve to separate
any concerns.

This is therefore a further simplification that returns to just doing the
entire handling of a provider instance's presence in the execution graph
only once we've decided that at least one resource instance will
definitely use the provider instance during the apply phase.

There is still a separation of concerns where the planGlue type is
responsible for calculating the provider dependencies and then the
execGraphBuilder is only responsible for adding items to the execution
graph based on that information. That separation makes sense because
planGlue's job is to bridge between the planning engine and the evaluator,
and it's the evaluator's job to calculate the dependencies for a provider
instance, whereas execGraphBuilder is the component responsible for
deciding exactly which low-level execgraph operations we'll use to describe
the situation to the apply engine.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-05 13:06:39 -08:00
Martin Atkins
22e5168af1 planning: Split responsibility for provider instance execgraph building
Previously we did _all_ of the work related to preparing execgraph items
for provider instances as a side-effect of the ProviderInstance method
of execGraphBuilder.

Now we'll split it into two parts: the first time we encounter each
provider instance during planning we'll proactively gather its dependencies
and stash them in the graph builder. However, we'll still wait until the
first request for the execution subgraph of a provider instance before
we actually insert that into the graph, because that then effectively
excludes from the apply phase any provider instances that aren't needed for
the actual planned side-effects. In particular, if all of the resource
instances belonging to a provider instance turn out to have "no-op" plans
then that provider instance won't appear in the execution graph at all.

An earlier draft of this change did the first dependency capturing step via
a new method of planGlue called by the evaluator, but after writing that
I found it unfortunate to introduce yet another inversion of control
situation where readers of the code just need to know and trust that the
evaluator will call things in a correct order -- there's already enough
of that for resource instances -- and so I settled on the compromise of
having the ensureProviderInstanceDependencies calls happen as part of the
linear code for handling each resource instance object, which makes it far
easier for a future maintainer to verify that we're upholding the contract
of calling ensureProviderInstanceDependencies before asking for an
execgraph result, while still allowing us to handle that step generically
instead of duplicating it into each resource-mode-specific handler.

While working on this I noticed a slight flaw in our initial approach to
handling ephemeral resource instances in the execution graph, which is
described inline as a comment in planDesiredEphemeralResourceInstance.
We'll need to think about that some more and deal with it in a future
commit.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-05 13:06:39 -08:00
Christian Mesh
8ce780b4e0 engine/ephemeral: Wire together basic ephemeral functionality (#3710)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2026-02-04 16:44:20 -05:00
Martin Atkins
0e422bc546 engine/applying: Managed resource plan and apply
Here we ask the provider to produce a plan based on the finalized
configuration taken from the "desired state", and return it as a final plan
object only if the provider request succeeds and returns something that is
valid under our usual plan consistency rules.

We then ask the provider to apply that plan, and deal with whatever it
returns. The apply part of this is not so complete yet, but there's enough
here to handle the happy path where the operation succeeds and the provider
returns something valid.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-02 10:27:07 -08:00
Martin Atkins
9497e6eab3 eval: ApplyOracle.DesiredResourceInstance returns config value
Previously this was calling "Value", which is incorrect because that
function resolves only after we've already planned and applied the resource
instance and so this was causing a promise self-dependency error.

Instead, we need to have the configgraph package expose the configuration
value directly and then use just that as part of this result.

(We do still want to eventually unify the two codepaths that produce these
DesiredResourceInstance objects, but this commit is focused only on fixing
this bug as directly as possible.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-02 10:27:07 -08:00
Martin Atkins
9b35d84dd1 engine/applying: Track grapheval/workgraph requests
There is currently a bug in the apply engine that's causing a
self-dependency error during promise resolution, which was a good reminder
that we'd not previously finished connecting all of the parts to be able
to unwind such problems into a useful error message.

Due to how the work is split between the apply engine, the evaluator, and
the execgraph package it takes some awkward back-and-forth to get all of
the needed information together into one place. This compromise aims to
do as little work as possible in the happy path and defer more expensive
analysis until we actually know we're going to report an error message.

In this case we can't really avoid proactively collecting the request IDs
because we don't know ahead of time what (if anything) will be involved in
a promise error, but only when actually generating an error message will
we dig into the original source execution graph to find out what each of
the affected requests was actually trying to do and construct a
human-friendly summary of each one.

This was a bit of a side-quest relative to the current goal of just getting
things basically working with a simple configuration, but this is useful
in figuring out what's going on with the current bug (which will be fixed
in a future commit) and will probably be useful when dealing with future
bugs too.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-02 10:27:07 -08:00
Martin Atkins
2af3eff7b5 execgraph: A more complete set of supported opcodes
The first pass of this was mainly just to illustrate how the overall model
of execution graphs might work, using just a few "obvious" opcodes as a
placeholder for the full set.

Now that we're making some progress towards getting this to actually work
as part of a real apply phase this reworks the set of opcodes in a few
different ways:

- Some concepts that were previously handled as special cases in the
  execution graph executor are now just regular operations. The general
  idea here is that anything that is fallible and/or has externally-visible
  side-effects should be modelled as an operation.
- The set of supported operations for managed resources should now be
  complete enough to describe all of the different series of steps that
  result from different kinds of plan. In particular, this now has the
  building blocks needed to implement a "create then destroy" replace
  operation, which includes the step of "deposing" the original object
  until we've actually destroyed it.
- The new package "exec" contains vocabulary types for data flowing between
  operations, and an interface representing the operations themselves.
  The types in this package allow us to carry addressing information along
  with objects that would not normally carry that directly, which means
  we don't need to propagate that address information around the execution
  graph through side-channels.

  (A notable gap as of this commit is that resource instance objects don't
  have address information yet. That'll follow in a subsequent commit just
  because it requires some further rejiggering.)
- The interface implemented by the applying engine and provided to the
  execution graph now more directly relates to the operations described
  in the execution graph, so that it's considerably easier to follow how
  the graph built by the planning phase corresponds to calls into that
  interface in the applying phase.
- We're starting to consider OpenTofu's own tracking identifiers for
  resource instances as separate from how providers refer to their resource
  types, just so that the opinions about how those two related can be
  more centralized in future, vs. the traditional runtime where the rules
  about how those related tend to be spread haphazardly throughout the
  codebase, and in particular implementing migration between resource
  types was traditionally challenging because it creates a situation where
  the desired resource type and the current resource type _intentionally_
  differ.

This is a large commit because the execution graph concepts are
cross-cutting between the plan and apply engines, but outside of execgraph
the changes here are mainly focused on just making things compile again
with as little change as possible, and then subsequent commits will get
the other packages into a better working shape again.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-02-02 10:27:07 -08:00
Martin Atkins
2e09323694 engine/applying: First somewhat-working implementation
This is just enough to connect all of the apply engine parts together into
a functional whole which can at least attempt to apply a plan. There are
various gaps here that make this not fully work in practice, but the goal
of this commit is to just get all of the components connected together in
their current form and then subsequent commits will gradually build on this
to make it work better.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-01-16 09:50:36 -08:00
Martin Atkins
4e07f8949c lang/eval: Populate DesiredResourceInstance.RequiredResourceInstances
The dependencies of a resource instance are based primarily on the marks
in the configuration value, and so we'll decide them as part of the same
function that decides the configuration value.

We then need to propagate that result through all of the intermediate
layers to the code that actually constructs the DesiredResourceInstance
object, at which point it's then available to the planning engine. In a
future commit the planning engine will use it to record the resource
instance dependencies explicitly in the generated execution graph.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-01-16 09:50:36 -08:00
Martin Atkins
1792e85e0b configgraph: ResourceInstance.Value should use grapheval.Once
In e879e9060f I introduced the manually-written memoization that was
previously here, as a quick fix for a problem I noticed while working on
something else.

However, using sync.Mutex directly here is not correct because it makes it
possible for this function to deadlock if used incorrectly elsewhere in the
system. The "grapheval" utilities are there to avoid exactly that by making
a call fail immediately with an error if a self-dependency problem arises.

Therefore this will now use grapheval.Once to better encapsulate the
memoization. This both ensures that we cooperate well with the rest of the
system using grapheval and requires less boilerplate inside the Value
method because grapheval.Once is already tailored for exactly this sort of
memoization.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2026-01-16 07:45:10 -08:00
Martin Atkins
e879e9060f configgraph: Only ask ResourceInstanceGlue once for each value
Most of what we interact with in configgraph is other parts of the
evaluator that automatically get memoized by patterns like OnceValuer, but
the value for a resource instance is always provided by something outside
of the evaluator that won't typically be able to use those mechanisms, and
so the evaluator's ResourceInstance.Value implementation will now provide
memoization on behalf of that external component, to ensure that we end
up with only one value for each resource instance regardless of how that
external component behaves.

In the case of the current planning phase, in particular this means that
we'll now only try to plan each resource instance once, whereas before
we would ask it to make a separate plan for each call to Value.

For now this is just retrofitted in an minimally-invasive way as part of
our "walking skeleton" phase where we're just trying to wire the existing
parts together end-to-end and then decide at the end whether we want to
refactor things more. If this need for general-purpose memoization ends
up appearing in other places too then maybe we'll choose to structure this
a little differently.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-12-12 07:03:52 -08:00
Christian Mesh
ffc9c4d556 Split out provider schemas vs instances in new engine (#3530)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2025-12-01 13:09:58 -05:00
Martin Atkins
102449c9ec backend/local: Use experimental runtime for planning, when enabled
This is a relatively uninteresting milestone where it's possible to load
and plan a root module that contains nothing except local values and
output values.

The module loader currently supports only local sources and the plugin
APIs just immediately return errors, so configurations more complicated
than that are likely to just fail immediately with one or more errors.
We'll gradually improve on this in later commits.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-11-21 09:20:59 -08:00
Martin Atkins
02f48f7b32 lang/eval: Re-export evalglue.UncompiledModule
We currently re-export evalglue.ExternalModules, but the method in that
interface requires returning evalglue.UncompiledModule and so we need to
export that too or else it's impossible to actually implement the interface
from outside this family of packages.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-11-21 09:20:59 -08:00
Martin Atkins
639dff15c3 lang/eval: Clear some linter warnings
Based on discussion so far it seems that we have consensus about merging
the lang/eval and engine-related commits that precede this one as a
starting point for ongoing work in future (hopefully smaller) PRs.

This commit therefore mostly just acknowledges a small amount of unused
code in a way that it won't cause linter noise, since the future work is
likely to make these be used and so it'll be helpful to have them around as
examples to build on. (Which is, after all, the whole point of merging this
as dead code in the first place!)

This does _actually_ remove an old, now-redundant declaration of the
"compiled module instance" interface though, since that eventually ended
up in a different place and I just forgot to clean up this initial form
of it in later refactoring.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
4218231438 execgraph: Most of the "compiler" machinery
This covers most of the logic required to turn a source graph into a
compiled graph ready for execution. There's currently only support for one
of the opcodes though, so subsequent commits will sketch those out more
and then add some tests and fix any problems that inevitably exist here but
aren't yet visible because there are no tests.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
b6cdfd1afa lang/eval: Bind PlanningOracle to PlanGlue
Previously the PlanGlue methods all took PlanningOracle pointers as one
of their arguments, which is annoying since all of them should end up with
pointers to the same object and it makes it hard for the PlanGlue
implementation to do any work outside of and between the PlanGlue method
calls.

Instead then we'll have DrivePlanning take a function for building a
PlanGlue implementation given a PlanningOracle pointer, and then the
planning engine returns an implementation that binds a planContext to a
PlanningOracle it can then use to do all of its work.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
4772944e91 engine/planning: Provider instances and stub of data resource planning
This is a sketch of starting and configuring provider instances once they
are needed and then closing them once they are no longer needed. The
stubby implementation of data resource instance planning is there mainly
just to illustrate how this is intended to work, but it's not yet complete
enough to actually function.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
4ea51e4faf lang/eval: Provider instance to ephemeral resource instance dependencies
This completes a previously-missing piece of the "prepareToPlan" result,
tracking which provider instances are relying on each ephemeral resource
instance.

This is important because the planning engine can "see"
resource-instance-to-provider relationships in the state that the eval
system isn't aware of, and so the planning engine must be able to keep
a provider instance open long enough to deal with both config-driven and
state-driven uses of it, which in turn means keeping open any ephemeral
resource instances that those provider instances depend on.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
6b9cebaf8b engine/planning: Find "orphan" resource instances
The eval system gradually reports the "desired" declarations at various
different levels of granularity, and so the planning engine should compare
that with the instances in the previous run state to notice when any
existing resource instance is no longer in the desired state.

This doesn't yet include a real implementation of planning the deletion of
such resource instances. Real planning behaviors depend on us having some
sort of provider instance and ephemeral resource instance manager to be
able to make requests to configured providers, so that will follow in
subsequent commits before we can implement the actual planning behaviors.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
a82da0f674 lang/eval: Call PlanGlue.Plan*Orphans methods
In a new async recursive tree walk we'll gradually traverse the tree of
module calls and announce the desired state objects at each level, so that
the PlanGlue implementation can then compare that with its own record of
the prior state to determine by omission which resource instances are
"orphans" that should be planned for deletion.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
450fe5c7ab lang/eval: More stubbing for PlanGlue.Plan*Orphans calls
This puts the infrastructure in place to allow us to notify the PlanGlue
implementation about what's declared in the configuration so it can detect
orphaned resource instances by omission, but stops short of actually doing
that because it'll require another concurrent recursive tree walk which
is likely to be complex enough to deserve to live in its own later commit.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
557026c5db lang/eval: A different approach to "orphan" resource instances
Previously this had the idea that the planning engine would work in two
subphases where first it would deal with the desired state (decided by
evaluating the configuration) and then it would do a second pass to plan
to delete any resource instances that didn't show up in the desired state.

Although that would technically work, we need to keep configured provider
instances open for the whole timespan between planning the first
desired resource instance and the final orphan instance using each
provider, and so splitting into two phases effectively means we'd have
provider instances sitting open but idle waiting for the desired state
subphase to complete to learn if they are needed for the orphan subphase.

This commit stubs out (but does not yet implement) a new approach where
the orphan planning can happen concurrently with the desired state
planning. This works by having the evaluator gradually describe to the
planning engine which module calls, module call instances, resources, and
resource instances _are_ present in the desired state -- as soon as that
information becomes available -- so that the planning engine can notice
by omission what subset of the prior state resource instances are no longer
desired and are therefore orphans.

This means that the planning engine should now have all of the information
it needs to recognize when a provider can be closed: the PlanningOracle
reports which _desired_ resource instances (or placeholders thereof) are
expected to need each provider instance, and the prior state tracks
exactly which provider instance each prior resource instance was associated
with, and so a provider's work is finished once all instances in the union
of those two sets have had their action planned.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
7e5378bea2 lang/eval: Expose EvalContext for caller use
Although our EvalContext API is primarily focused on what the eval system
needs, it's desirable for higher-level subsystems that wrap the evaluator
to be able to guarantee they are using the same provider and other
dependencies, so we'll expose the EvalContext that each ConfigInstance is
associated with in the public API.

The Providers interface now also grows to include NewConfiguredProvider as
a way to instantiate a configured provider instance. The eval system
doesn't need that itself, but exposing it here means that a caller can be
confident that they are definitely instantiating and configuring the same
provider that the eval system had previously used to validate the
configuration.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
eb1d686806 lang/eval: PlanningOracle methods have their own workgraph workers
As was already noted in a comment in the PlanningOracle declaration, the
PlanningOracle API is an exported entrypoint usable from outside lang/eval
and so each exported method should establish its own workgraph worker to
make sure it's handling those concerns correctly regardless of how the
caller chooses to use them.

For example, a caller might choose to make concurrent calls into the oracle
from multiple goroutines, which would violate the assumption that each
concurrently-running codepath is associated with a separate workgraph
worker.

Even if a caller is managing their own workgraph workers in the expected
way, this is functionally harmless and relatively cheap.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
7cab34815a lang/eval: PlanningOracle.AwaitResourceInstancesCompletion
This is a building block for use by a planning engine implemented elsewhere
in OpenTofu so that it can be notified (by the completion of a call to the
function) when all resource instances that use a particular provider
instance or ephemeral resource instance have completed their plan-time
evaluation work.

Existing methods of PlanningOracle already expose sets of addresses to
wait for, but this method is separate from those because the planning
engine also needs the dependency information as data to know when an
ephemeral resource instance might need to be left open beyond the part of
the planning process driven by lang/eval in order to plan the delete
actions for any "orphan" resource instances that are in the prior state
but no longer declared in the configuration.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
ab6af47df7 lang/eval: Move PlanningOracle to its own file
Continuing to split up some of the source files that have grown long over
ongoing experimentation, this is the start of splitting the config_plan.go
file into smaller parts.

For now only PlanningOracle gets its own file, but other types related to
planning might follow in later commits.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
52a27d1865 lang/eval: PlanningOracle can now return a provider instance config
This was the big known missing piece for making a "real" implementation
of planning using this new package, so that the planning engine can know
what configuration to use when preparing a configured provider instance
for planning requests.

I'm sure there are other missing pieces that I've just not learned about
yet, but this is still a good checkpoint to stop iterating on lang/eval
for a little while and start stubbing out the planning engine.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
db8af25531 lang/eval: Rework how module calls work
The previous approach had the problem that we only had the
CompiledModuleInstance object for a child module briefly in a local
variable while evaluating the child call, which is sufficient for just
evaluation but is not enough to support full config tree traversal for
questions like "what resource instances are declared throughout the
configuration?" and "what configgraph node corresponds to this provider
configuration instance address?"

This moves the separation of concerns around a little to be perhaps more
like how resource instances work, where the ModuleCallInstance.Value
method just wraps a function provided by the "compile" layer which then
takes the config value and compiles the child module instance.

This means that the "compile" layer can then hold on to the
CompiledModuleInstance object for the child module instance as part of
the "glue" in the ModuleCallInstance object and then use it to deal with
the config-tree-traversing methods in its own CompiledModuleInstance
implementation.

The new test of ConfigInstance.PrepareToPlan in lang/eval illustrates that
the system is now finally able to walk the whole configuration tree to find
resource instances and the provider instances they depend on.

(The handling of inheritance of providers between parent and child module
instances is still not working like the current system does, because the
"providers sidechannel" mechanism remains incomplete.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
372501516c lang/eval/internal/tofu2024: Split up compile.go
This file was originally in lang/eval directly and so it had all of the
compile-related functions grouped together to separate them from everything
else in that package.

Now that the "compile" process has its own package we have more room to
have many smaller files instead of one huge file, making this a little
easier to navigate now that the logic has grown.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
af3bd79a03 lang/eval: Work-in-progress changes to modeling of module tree
Exposing only "ResourceInstancesDeep" on CompiledModuleInstance seemed
attractive at first when it was only partially implemented anyway, but
trying to finish implementing it revealed that the current design couldn't
actually support that API because we don't currently keep the
CompiledModuleInstance object for a child module instance after we've used
it to evaluate the child module instance's output values.

This commit doesn't actually solve that problem, but it does rework the
CompiledModuleInstance API so that the problem of finding and enumerating
child module instances is separated from the problem of finding resource
instances in just the current module instance, and then I'll try to find
a good way to satisfy this new, slightly-generalized API in future commits.

(The ultimate goal here is to be able to enumerate all of the resource
instances throughout the configuration for dependency-tracking purposes,
and to be able to find a specific ProviderInstance object given its
absolute address so that the plan or apply engine can request the
configuration value to configure the provider with.)

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
a532c43eb1 lang/eval: Evaluation of provider configurations
This introduces the "compiler" changes and the configgraph changes to let
us evaluate the configuration for a particular provider instance.

A future commit will expose this through the public API of lang/eval so
that other systems like the planning engine will be able to ask for the
configuration for whatever provider instance a given resource instance
belongs to.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
ffe8a79658 lang/eval: Slightly more complete handling of provider configurations
This is still not completely true to how current OpenTofu behaves, but it's
at least a little closer and avoids hard-coding a specific fixed provider
address in favor of actually following what's declared in the
configuration.

The system doesn't actually support evaluating the configuration for a
provider instance yet, so this remains just a placeholder for the
relatively simple tests in this package. Support for actually evaluating
the provider configurations will follow in later commits.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
635b444c8e lang/eval: Some deeper stubbing out of the DrivePlanning design
This is a little too busy/complicated for my liking but it's a place to
start and hopefully we'll be able to cut this down a little after we see
how the planning engine implementation turns out.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
64ad4cc66a lang/eval: Some initial support for child module calls
The way this all fits together doesn't feel quite right yet since there's
a bunch of wrapping layers that aren't really adding anything except
indirection between the packages, but this is at least a starting point
for child module support.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
b06287111b lang/eval: Temporary shimmed support for our core functions
This borrows the function table preparation from our previous lang.Scope
API (which this experiment would, if successful, ultimately replace) and
also intentionally implements the exprs.Evalable interface a little
improperly for now in the hope that we'll update our HCL patch for
function inspections to use hcl.StaticCall instead of misusing
hcl.Traversal at some point in the future.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00
Martin Atkins
1a7dd21fc2 lang/evalglue: Different approach to EvalContext defaults for tests
In other packages we've tried being pragmatic by making the main code
compensate for missing arguments in older tests, so that we don't always
need to update every test across the whole system every time we add a new
feature.

However, that compromise unfortunately tends to lead to non-test code
inadvertently relying on the compensations too, so this is an attempt at
a slightly different compromise where the main code will panic if called
incorrectly but there's an extra helper function for tests to fill in
reasonable "inert" defaults for the fields they leave unpopulated.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-10-27 10:15:41 -07:00