This connects up various work done in earlier commits so that it's now
possible to install modules from source addresses that start with "oci:",
which will each get interpreted as a reference to an artifacts in an OCI
Distribution repository.
For the first time we now have a getter that needs to be configured
dynamically based on the CLI configuration, so this slightly reworks the
"reusingGetter" type so that each instance has its own map of getters
that's based on the statically-configured one. Currently "oci" is the only
getter that needs this dynamic configuration, but perhaps in future we'll
adopt a similar dependency inversion style for some of the other getters
so that we can centralize concerns such as allowing operators to configure
additional TLS certificates for OpenTofu to trust.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Originally reusingGetter was just a map directly, since it had no other
data to track beyond "remembering" what it had previously installed.
In a future commit we'll be adding a dynamically-constructed set of
"getters" so that we can configure the new "OCI" getter dynamically based
on the active CLI configuration, so we'll need this to be a struct type
instead.
Since we are now able to do so, this commit introduces a mutex to make it
safe to call the reusingGetter concurrently from multiple goroutines,
although this initial implementation is intentionally simplistic and just
forces all of the install requests to be handled sequentially, since we
don't _actually_ currently attempt concurrent module installs and so this
addition is primarily for robustness against incorrect use.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a new implementation of go-getter's "Getter" interface intended
to support installing OpenTofu module packages from OCI Distribution
repositories.
Since this is currently only intended for OpenTofu's use it makes some
simplifying assumptions that would not be acceptable for an upstream
getter, but are okay for the limited way that OpenTofu's module installer
uses go-getter, which is already intentionally constrained and hidden
behind a simpler API so we can treat go-getter as purely an implementation
detail.
This commit only introduces the getter, without actually registering it
as available for use in the module package fetcher used by "tofu init",
and so this is effectively just a bunch of dead code. A later commit will
wire this in properly and introduce an end-to-end test to demonstrate that
it's properly integrated.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
* Rename module name from "github.com/hashicorp/terraform" to "github.com/placeholderplaceholderplaceholder/opentf".
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Gofmt.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Regenerate protobuf.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comments.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo issue and pull request link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo comment changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comment.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo some link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* make generate && make protobuf
Signed-off-by: Jakub Martin <kubam@spacelift.io>
---------
Signed-off-by: Jakub Martin <kubam@spacelift.io>
There was an unintended regression in go-getter v1.5.9's GitGetter which
caused us to temporarily fork that particular getter into Terraform to
expedite a fix. However, upstream v1.5.10 now includes a
functionally-equivalent fix and so we can heal that fork by upgrading.
We'd also neglected to update the Module Sources docs when upgrading to
go-getter v1.5.9 originally and so we were missing documentation about the
new "depth" argument to enable shadow cloning, which I've added
retroactively here along with documenting its restriction of only
supporting named refs.
This new go-getter release also introduces a new credentials-passing
method for the Google Cloud Storage getter, and so we must incorporate
that into the Terraform-level documentation about module sources.
This is a pragmatic temporary solution to allow us to more quickly resolve
an upstream regression in go-getter locally within Terraform, so that the
work to upstream it for other callers can happen asynchronously and with
less time pressure.
This commit doesn't yet include any changes to address the bug, and
instead aims to be functionally equivalent to getter.GitGetter. A
subsequent commit will then address the regression, so that the diff of
that commit will be easier to apply later to the upstream to get the same
effect there.
Earlier work to make "terraform init" interruptible made the getproviders
package context-aware in order to allow provider installation to be cancelled.
Here we make a similar change for module installation, which is now also
cancellable with SIGINT. This involves plumbing context through initwd and
getmodules. Functions which can make network requests now include a context
parameter whose cancellation cancels those requests.
Since the module installation code is shared, "terraform get" is now
also interruptible during module installation.
It's been a long while since we gave close attention to the codepaths for
module source address parsing and external module package installation.
Due to their age, these codepaths often diverged from our modern practices
such as representing address types in the addrs package, and encapsulating
package installation details only in a particular location.
In particular, this refactor makes source address parsing a separate step
from module installation, which therefore makes the result of that parsing
available to other Terraform subsystems which work with the configuration
representation objects.
This also presented the opportunity to better encapsulate our use of
go-getter into a new package "getmodules" (echoing "getproviders"), which
is intended to be the only part of Terraform that directly interacts with
go-getter.
This is largely just a refactor of the existing functionality into a new
code organization, but there is one notable change in behavior here: the
source address parsing now happens during configuration loading rather
than module installation, which may cause errors about invalid addresses
to be returned in different situations than before. That counts as
backward compatible because we only promise to remain compatible with
configurations that are _valid_, which means that they can be initialized,
planned, and applied without any errors. This doesn't introduce any new
error cases, and instead just makes a pre-existing error case be detected
earlier.
Our module registry client is still using its own special module address
type from registry/regsrc for now, with a small shim from the new
addrs.ModuleSourceRegistry type. Hopefully in a later commit we'll also
rework the registry client to work with the new address type, but this
commit is already big enough as it is.
This new package aims to encapsulate all of our interactions with
go-getter to fetch remote module packages, to ensure that the rest of
Terraform will only use the small subset of go-getter functionality that
our modern module installer uses.
In older versions of Terraform, go-getter was the entire implementation
of module installation, but along the way we found that several aspects of
its design are poor fit for Terraform's needs, and so now we're using it
as just an implementation detail of Terraform's handling of remote module
packages only, hiding it behind this wrapper API which exposes only
the services that our module installer needs.
This new package isn't actually used yet, but in a later commit we will
change all of the other callers to go-getter to only work indirectly
through this package, so that this will be the only package that actually
imports the go-getter packages.
