Previously we were using a mixture of old and new, with our code generation
using the plugin from the old github.com/golang/protobuf library but
our callers using the modern google.golang.org/protobuf . We were also
using pretty ancient version of protoc.
This brings us up to the current latest releases and consistently using
the new Go protobuf library. There have been some notable changes to these
tools in the meantime:
Previously the protoc-gen-go plugin handled grpc by having its own
additional level of Go-specific "plugins" of which the gRPC codegen was
an example.
Now the protobuf generator and the gRPC generator are separate plugins
handled directly by protoc, which means the command line arguments are
a different shape and the gRPC stubs get generated in a separate file
from the main protobuf messages, rather than all being in one .pb.go file
as before.The results are otherwise similar, though.
The grpc codegen now also defaults to requiring that implementations embed
the generated "unimplemented" server, which is an implementation of each
service where the methods just immediately return the "unimplemented"
error. This is not super important for us because we maintain the generated
interfaces and their implementations together in the same repository
anyway, but adding the "unimplemented" server embeds was not a big change
and so seems better to follow the prevailing convention.
Using these new versions means that we could in principle now switch to
using protobuf edition 2024 and the new "sealed" style for Go code
generation, but this commit does not include any such changes and focuses
only on getting things upgraded with as few other changes as possible. We
can discuss using different codegen style later and deal with that in
separate commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Continuing our work to gradually plumb context.Context to everywhere that
we want to generate OpenTelemetry traces, this completes the call path
for most (but not all) of the gRPC requests to provider plugins, so that
we can add OpenTelemetry trace instrumentation in a future commit.
Unfortunately there are still a few providers.Interface callers left in
functions that don't have context.Context plumbed to them yet, and so
those are temporarily stubbed as context.TODO() here so we can more easily
find and complete them later.
The two gRPC implementations of providers.Interface were previously making
provider requests using a single context.Context established at the time
the provider process was started, but that isn't an appropriate context
to use for per-request concerns like tracing, so that context is now
unused and could potentially be removed in a future commit, but this change
already got pretty large and so I intend to deal with that separately
later.
This now exposes the gRPC provider calls to potential context cancellation
that they would previously observe only indirectly though the Stop method.
Since Stop is primarily used for graceful shutdown of ApplyResourceChange,
the changes here explicitly disconnect the cancellation signal for
ApplyResourceChange in particular, while letting the others get canceled
in the normal way since they are expected to be free of significant
side-effects. In future work we could consider removing Stop from the
internal API entirely and keeping it only as an implementation detail of
the gRPC implementation of this interface, with ApplyResourceChange
directly reacting to context cancellation and sending the gRPC Stop call
itself, but again that's too much change for this already-large commit.
The internal/legacy package currently contains some legacy code preserved
for the benefit of the backends, and unfortunately contains more than is
strictly necessary to support those callers, and so there was some dead
code there that also needed updating. provider_mock.go is removed entirely
because it's just an older copy of the similar file in package tofu. The
few calls to providers in schemas.go are updated to use
context.Background() rather than context.TODO() because we have no
intention of plumbing context.Context into that legacy code, and will
hopefully just delete it wholesale one day.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This commit adds the definitions of provider protocol 5.6, 5.7, 6.6, and
6.7 to our archive of the historical protocol versions and then adopts
5.7 as the current version of major version 5 and 6.7 as the current
version of major version 6.
These MPL-licensed schema definitions are from the plugin protocol server
implementation in this repository, copyright HashiCorp:
https://github.com/hashicorp/terraform-plugin-go
The only modifications made are to change the "option go_package" directive
to match where the stubs need to be generated for OpenTofu, and to claim
copyright for that change and thus make the copyright header consistent
with what our pre-commit rules require.
The regeneration of the Go API stubs for the two major protocol versions
introduces some new fields and messages that OpenTofu does not yet support
but will happily ignore. Future work might make some use of these new
additions, but that's out of scope of this change that is intended only
to synchronize our protocol definition with what new plugin server releases
are likely to be linked against.
This commit continues the existing precedent of having the stubs for the
newly-added interface methods in package grpcwrap being just a panicking
stub, which is how they will remain until a future project begins using
those methods in a way which requires them to be implemented, since
implementing the wrappers would require a deeper understanding of the
desired behavior of those methods.
It appears that we previously accepted a pull request to correct a typo
that originated in the older versions of the upstream protocol definitions,
but I have intentionally not forward-ported that here because it seems
clearer to keep these definitions as close as possible to their source
of truth from upstream, given that our current intention is to follow the
protocol as documented and not to change it.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
* Rename module name from "github.com/hashicorp/terraform" to "github.com/placeholderplaceholderplaceholder/opentf".
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Gofmt.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Regenerate protobuf.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comments.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo issue and pull request link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo comment changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Fix comment.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* Undo some link changes.
Signed-off-by: Jakub Martin <kubam@spacelift.io>
* make generate && make protobuf
Signed-off-by: Jakub Martin <kubam@spacelift.io>
---------
Signed-off-by: Jakub Martin <kubam@spacelift.io>
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
This is part of a general effort to move all of Terraform's non-library
package surface under internal in order to reinforce that these are for
internal use within Terraform only.
If you were previously importing packages under this prefix into an
external codebase, you could pin to an earlier release tag as an interim
solution until you've make a plan to achieve the same functionality some
other way.
* providers.Interface: rename ValidateDataSourceConfig to
ValidateDataResourceConfig
This PR came about after renaming ValidateResourceTypeConfig to
ValidateResourceConfig: I now understand that we'd called it the former
instead of the latter to indicate that the function wasn't necessarily
operating on a resource that actually exists. A possibly-more-accurate
renaming of both functions might then be ValidateManagedResourceConfig
and ValidateDataResourceConfig.
The next commit will update the protocol (v6 only) as well; these are in
separate commits for reviewers and will get squashed together before
merging.
* extend renaming to protov6
* Add support for plugin protocol v6
This PR turns on support for plugin protocol v6. A provider can
advertise itself as supporting protocol version 6 and terraform will
use the correct client.
Todo:
The "unmanaged" providers functionality does not support protocol
version, so at the moment terraform will continue to assume that
"unmanaged" providers are on protocol v5. This will require some
upstream work on go-plugin (I believe).
I would like to convert the builtin providers to use protocol v6 in a
future PR; however it is not necessary until we remove protocol v6.
* add e2e test for using both plugin protocol versions
- copied grpcwrap and made a version that returns protocol v6 provider
- copied the test provider, provider-simple, and made a version that's
using protocol v6 with the above fun
- added an e2etest
* providers.Interface: huge renamification
This commit renames a handful of functions in the providers.Interface to
match changes made in protocol v6. The following commit implements this
change across the rest of the codebase; I put this in a separate commit
for ease of reviewing and will squash these together when merging.
One noteworthy detail: protocol v6 removes the config from the
ValidateProviderConfigResponse, since it's never been used. I chose to
leave that in place in the interface until we deprecate support for
protocol v5 entirely.
Note that none of these changes impact current providers using protocol
v5; the protocol is unchanged. Only the translation layer between the
proto and terraform have changed.
