Provider and Provisioner Managers encapsulate all of
the logic nessesary to start, manage, and stop plugins.
They also have the advantage of sharing a schema cache between
managers started in different parts of the application.
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
This mostly mechanical change defines a central location
for providing installed plugins as a cohesive unit to the
rest of the application. This also provides a location
for future caching of schemas and other potential optimizations.
This is part of the overall initiative to start stripping
functionality out of the tofu package into re-usable components
which can be used by the new engine.
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
This is a _very_ early, minimal plumbing of the execution graph builder
into the experimental planning engine.
The goal here is mainly just to prove the idea that the planning engine can
build execution graphs using the execgraph.Builder API we currently have.
This implementation is not complete yet, and also we are expecting to
rework the structure of the planning engine later on anyway so this initial
work focuses on just plumbing it in to what we had as straightforwardly
as possible.
This is enough to get a serialized form of _an_ execution graph included
in the generated plan, though since we don't have the apply engine
implemented we don't actually use it for anything yet.
In subsequent commits we'll continue building out the graph-building logic
and then arrange for the apply phase to unmarshal the saved execution graph
and attempt to execute it, so we can hopefully see a minimal version of all
of this working end-to-end for the first time. But for now, this was mainly
just a proof-of-concept of building an execution graph and capturing it
into its temporary home in the plans.Plan model.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
ResultRef[*states.ResourceInstanceObjectFull] is our current canonical
representation of the "final result" of applying changes to a resource
instance, and so it is going to appear a bunch in the plan and apply
engines.
The generic type is clunky to read and write though, so for this one in
particular we'll offer a more concise type alias that we can use for parts
of the API that are representing results from applying.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a minimal set of changes to introduce uses of the new
states.ResourceInstanceObjectFull to all of the leaf functions related to
planning managed and data resource instances.
The main goal here was just to prove that we'd reasonably be able to look
up objects with the new type in all of the places we'd need to. We're
planning some more substantial changes to the planning engine in future
commits (e.g. to generate execution graphs instead of traditional plans)
and so we'll plumb this in better as part of that work.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we had a gap here where we only knew the resource type to use
when there was a "desired state" object to get it from, which isn't true
when we're planning to destroy an undesired object.
The new extended model for resource instance object state gives us direct
access to the resource type name, so we can now handle that properly.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Instead of states.ResourceInstanceObject, we'll now use the "full"
representation added in a recent commit. This new representation is a
better fit for execgraph because it tracks all of the relevant information
about a resource instance object inline without assuming that the object
is always used in conjunction with a pointer to the entire state.
We're not really making any special use of those new capabilities yet. This
is mostly just mechanical updates to refer to the new type, and to fill
in values for the extra fields as far as that's possible without doing
further refactoring.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
To support the workflow of saving a plan to disk and applying it on some
other machine we need to be able to represent the execution graph as a byte
stream and then reload it later to produce an equivalent execution graph.
This is an initial implementation of that, based on the way the execgraph
package currently represents execution graphs. We may change that
representation more in future as we get more experience working in the new
architecture, but this is intended as part of our "walking skeleton" phase
where we try to get the new architecture working end-to-end with simple
configurations as soon as possible to help verify that we're even on the
right track with this new approach, and try to find unknown unknowns that
we ought to deal with before we get too deep into this.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a relatively uninteresting milestone where it's possible to load
and plan a root module that contains nothing except local values and
output values.
The module loader currently supports only local sources and the plugin
APIs just immediately return errors, so configurations more complicated
than that are likely to just fail immediately with one or more errors.
We'll gradually improve on this in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This now seems to more-or-less work, at least as far as the new
compile-and-execute is concerned.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This covers most of the logic required to turn a source graph into a
compiled graph ready for execution. There's currently only support for one
of the opcodes though, so subsequent commits will sketch those out more
and then add some tests and fix any problems that inevitably exist here but
aren't yet visible because there are no tests.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Doesn't actually do anything yet. Just sketching how it might be
structured, with a temporary object giving us somewhere to keep track of
the relationships between nodes that we can discard once compilation is
complete, keeping only the information required to execute.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
So far this is mainly just the mechanism for building a graph piecemeal
from multiple callers working together as part of the planning engine.
The end goal is for it to be possible to "compile" an assembled graph into
something that can then be executed, and to be able to marshal/unmarshal an
uncompiled graph to save as part of a plan file, but those other
capabilities will follow in later commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This sketches the main happy path of planning for a desired managed
resource instance, though doesn't include all of the possible variations
that authors can cause with different configuration settings.
The main goal here is just to illustrate how the planning step might
interact with the work done in the eval system. If we decide to go in this
direction then we'll need to bring over all of the complexity from the
traditional runtime into this codepath, or possibly find some way to reuse
that behavior directory from "package tofu" in the short term.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously the PlanGlue methods all took PlanningOracle pointers as one
of their arguments, which is annoying since all of them should end up with
pointers to the same object and it makes it hard for the PlanGlue
implementation to do any work outside of and between the PlanGlue method
calls.
Instead then we'll have DrivePlanning take a function for building a
PlanGlue implementation given a PlanningOracle pointer, and then the
planning engine returns an implementation that binds a planContext to a
PlanningOracle it can then use to do all of its work.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a sketch of starting and configuring provider instances once they
are needed and then closing them once they are no longer needed. The
stubby implementation of data resource instance planning is there mainly
just to illustrate how this is intended to work, but it's not yet complete
enough to actually function.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a similar idea to a sync.WorkGroup except that it tracks the
completion of individual items represented by comparable values, rather
than only a count of expected items to complete.
My intention for this is for the planning engine to use it to track when
all of the expected work for a provider instance or ephemeral resource
instance has been completed, so that it can be closed shortly afterwards
instead of waiting for the entire planning operation to run to completion.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The eval system gradually reports the "desired" declarations at various
different levels of granularity, and so the planning engine should compare
that with the instances in the previous run state to notice when any
existing resource instance is no longer in the desired state.
This doesn't yet include a real implementation of planning the deletion of
such resource instances. Real planning behaviors depend on us having some
sort of provider instance and ephemeral resource instance manager to be
able to make requests to configured providers, so that will follow in
subsequent commits before we can implement the actual planning behaviors.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously this had the idea that the planning engine would work in two
subphases where first it would deal with the desired state (decided by
evaluating the configuration) and then it would do a second pass to plan
to delete any resource instances that didn't show up in the desired state.
Although that would technically work, we need to keep configured provider
instances open for the whole timespan between planning the first
desired resource instance and the final orphan instance using each
provider, and so splitting into two phases effectively means we'd have
provider instances sitting open but idle waiting for the desired state
subphase to complete to learn if they are needed for the orphan subphase.
This commit stubs out (but does not yet implement) a new approach where
the orphan planning can happen concurrently with the desired state
planning. This works by having the evaluator gradually describe to the
planning engine which module calls, module call instances, resources, and
resource instances _are_ present in the desired state -- as soon as that
information becomes available -- so that the planning engine can notice
by omission what subset of the prior state resource instances are no longer
desired and are therefore orphans.
This means that the planning engine should now have all of the information
it needs to recognize when a provider can be closed: the PlanningOracle
reports which _desired_ resource instances (or placeholders thereof) are
expected to need each provider instance, and the prior state tracks
exactly which provider instance each prior resource instance was associated
with, and so a provider's work is finished once all instances in the union
of those two sets have had their action planned.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a problem that has been on my mind for the past little while as
I've been building this out, so I just wanted to write it down in a place
where others reading this code could find it, to acknowledge that there is
a gap in this new design idea that prevents it from having the similar
provider open/close behavior to the current implementation.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We already have a few situations where the evaluation phase can produce
an incomplete placeholder for zero or more resource instances that might
appear in a future plan/apply round, and the plan implementation itself
will introduce some more based on replies from provider requests.
Our goal is to always plan as much as possible with the information we
have, including possibly returning errors for partially-evaluated objects
when we're confident that they could possibly become valid in the presence
of more information, and so in some cases we will end up visiting a
resource instance that would not need to be deferred if considered in
isolation but nonetheless its configuration is depending on an outcome of
an action that was already deferred and so it must therefore also be
deferred.
This currently-unused cty mark will allow us to use dynamic analysis to
track when parts of a resource instance whose action was deferred are used
as part of another resource instance. This is superior to a static analysis
approach because it will allow us to notice situations such as when only
one arm of a conditional relies on a deferred result.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
All of this still leads to a TODO panic but we now have a separate method
to implement for each of the resource modes, since the handling of each
is pretty independent of the others.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This doesn't actually work yet because the main implementation functions
are not written, but this is a sketch of the general layout of things that
the lang/eval package was designed to support: the planning phase first
lets lang/eval drive based on the desired state it discovers gradually in
the configuration, but afterwards it then does followup work for any
resource instances that didn't appear in the desired state but yet are
present in the previous round state.
This overall structure should allow the planning engine to discover all
of the changes that need to be made to resource instances without any
direct knowledge of how the OpenTofu language works and how the module
author chose to describe the desired state as code.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
