During the "refresh" and "plan" steps we build the marks for a managed
resource object's value as a combination of the marks from the input
(prior state or configuration, respectively) and the marks implied by the
provider schema.
However, the apply step was previously relying only on the marks from the
planned new state, without considering marks from the provider schema. That
meant that a sensitive attribute contained within a container that is
unknown during planning could not be marked as sensitive once the container
became known, because the corresponding value did not exist at all in the
planned new state and therefore could not carry a sensitive mark.
To fix this problem, this changes the apply step to match the strategy
already used in the refresh and plan steps: using combinePathValueMarks
to blend the dynamic marks with the static information from the schema,
so that the final value saved in the new state snapshot will have a full
set of sensitive markings for the next plan/apply round to rely on. Without
this the next plan/apply round would produce a spurious diff due to the
sensitivity of the nested attributes appearing to have changed.
This introduces a new test TestContext2Apply_sensitiveInsideUnknown which
covers the case where the sensitivity information comes from schema. The
preexisting test TestContext2Apply_additionalSensitiveFromState already
covered the case of dynamically-tracked sensitivity information, and
remains passing without modification after this change.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Over time the discussion about "lifecycle" blocks in the documentation
became confusing because the docs originally written for managed resource
lifecycle got partially generalized for resources of other modes and for
module calls, even though each of those has a completely different
lifecycle and thus a different set of lifecycle settings.
This is a first pass at trying to reorganize that so that the "lifecycle"
page is really just an index of all of the different kinds of lifecycle
block that exist in the language, while the main documentation for each
use of that block type now belongs with the documentation of the block
it's nested within.
While working on this I also found that there was some duplication inside
the "data sources" page where the same information was described multiple
times, and a few other cases where things had become inconsistent over
time. This also includes a little extra content to try to clarify the
difference between managed, data, and ephemeral resources and to make it
explicit that the "Resources" section is focused only on managed resources
because that is the primary resource mode.
As usual there's lots more that could be done here -- this documentation
has gradually evolved over time and is full of weird quirks due to that
evolution -- but I decided to draw a line here so that the diff wouldn't
get too large.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
As usual we've been gradually adding things to this in a decentralized and
uncoordinated fashion throughout the v1.11 development period, and so this
is a first pass of copyediting to try to present the information in a more
useful order (which is, of course, highly subjective), to describe the
changes in a more consistent writing style, to consolidate entries for
work that happened over multiple PRs but yet represents only one
end-user-facing feature, and to remove a few entries for changes that
are really just implementation details that don't affect the externally-
visible behavior of OpenTofu in any significant way.
We'll probably edit this some more throughout the v1.11 prerelease period,
but this is intended as a starting point for use with the v1.11.0-beta1
release.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
For some reason we had a few direct dependencies included in the "require"
clause for indirect dependencies. "go mod tidy" considers any directive
that contains a mixture of direct and indirect dependencies as
manually-edited and so avoids adding any new items to it, which was the
root cause of the repeated addition of a third "require" directive whenever
we made a change that caused a new indirect dependency to appear in this
file.
Moving these Azure-related direct dependencies into the first "require"
directive along with all of the other direct dependencies should hopefully
avoid that "go mod tidy" quirk occurring again under future maintenance.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Implicit move statement for modules previously using `count` and
then moving to use `enabled`, or vice versa.
Signed-off-by: Diogenes Fernandes <diofeher@gmail.com>
This library is a dependency of our "postgresql" state storage
implementation.
This is just a routine upgrade to the latest available patch release. This
is mainly just a collection of minor upstream bugfixes and performance
improvements.
Notably this adds support for context.Context-based cancellation, though
that won't do anything in OpenTofu because we intentionally prevent
cancellation from reaching any of the state storage implementations so that
we can save new state snapshots even when other parts of OpenTofu are
shutting themselves down in response to a cancellation.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade, with no significant change in
functionality. It caused several indirect dependency changes too, but
they don't affect OpenTofu's main behavior.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade, bringing this last remaining OpenTelemetry
dependency up-to-date to match with the others we've upgraded recently.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The Go 1.25.2 release included some new-strict validation of TLS
certificates that might potentially cause breakage for folks using private
PKI with OpenTofu.
Go 1.25.3 was an emergency patch release to re-weaken that validation back
to accepting what it used to accept and addressing the original motivating
vulnerabilities in a different way.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Although we usually prefer to upgrade in small units, in practice these
dependencies are all coupled together and relatively expensive to test, so
in this case it seems better to upgrade them all together and test them
all in a single set of acceptance test runs.
This is intended only as a routine upgrade, although upgrading the SDK is
often all that's needed to support any newly-added AWS regions or support
for existing services in additional regions, so this may expand the set
of regions that our AWS-integrated features can interact with.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade. We use this dependency only in our tests,
so this upgrade does not risk changing OpenTofu's behavior. There do not
seem to be any concerning changes upstream.
There are some systematic changes to the shape of the generated mock code,
with the results also included in this commit.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
My original intention was just to reduce our number of dependencies by
standardizing on a single comparison library, but in the process of doing
so I found various examples of the kinds of problems that caused this
codebase to begin adopting go-cmp instead of go-test/deep in the first
place, which make it easy to accidentally write a false-positive test that
doesn't actually check what the author thinks is being checked:
- deep.Equal silently ignores unexported fields, so comparing two values
that differ only in data in unexported fields succeeds even when it ought
not to.
TestContext2Apply_multiVarComprehensive in package tofu was an excellent
example of this problem: it had various test assertions that were
actually checking absolutely nothing, despite appearing to compare
pairs of cty.Value.
- deep.Equal also silently ignores anything below a certain level of
nesting, and so comparison of deep data structures can appear to succeed
even though they don't actually match.
There were a few examples where that problem had already been found and
fixed by temporarily overriding the package deep global settings, but
with go-cmp the default behavior already visits everything, or panics
if it cannot.
This does mean that in a few cases this needed some more elaborate options
to cmp.Diff to align with the previous behavior, which is a little annoying
but overall I think better to be explicit about what each test is relying
on. Perhaps we can rework these tests to need fewer unusual cmp options
in future, but for this commit I want to keep focused on the smallest
possible changes to remove our dependency on github.com/go-test/deep .
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The changes in the previous commit confirmed that this test was passing
only as a false-positive when running on Windows, because the test was
previously only checking that the provisioner was stopped shortly after
asking it to stop, but that wasn't accounting for the possibility that it
stopped due to an unrelated error.
Windows Command Interpreter does not support semicolon as a command
separator, so on Windows we need to use an ampersand instead.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously this test was just assuming that the provisioner run would
succeed and only requiring that it run for more than 50ms before exiting.
That meant that it could potentially false-positive succeed if the
provisioner happened to return an error but take more than 50ms to do so.
Now we'll test for failure before we ask the provisioner to stop, which
narrows the false-positive window. This still isn't completely robust
because we don't have any way to test whether the provisioner failed due
to being canceled or for some other reason. The error message returned on
cancellation varies depending on what state the provisioner was in when
it got the cancellation message, so it's not currently feasible to write
a robust check that would definitely distinguish between the expected error
vs. unexpected errors.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The changes to the core functionality since v1.9.3 seem to largely just
be internal changes such as adopting more modern functions from the Go
standard library. Most of the externally-visible changes were to
vendor-specific filesystem implementations that OpenTofu does not make any
use of, and so don't affect OpenTofu's behavior at all.
One notable change is that the upstream project has switched to a
multi-module structure where the vendor-specific filesystem implementations
that have special dependencies are each split into a separate Go module,
and so the root module now has far fewer direct dependencies, and that's
the cause of all of the deletions from our go.sum in this commit. The Go
toolchain is smart enough to know that we were not depending on the parts
of Afero that used those dependencies anyway and so they were not listed
in our go.mod, but it's still nice to remove a few items from the set
of dependencies the toolchain needs to probe when making installation
decisions.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We were previously running a very old untagged commit of this module, and
the code in this module has been rearranged a lot in the meantime making
it hard to compare, but it seems like it's still implementing essentially
the same main functionality for tracing setup, just now extended for more
elaborate forms of the standard OpenTelemetry environment variables.
Unfortunately, the purpose of this module is to make dynamic runtime
decisions about exactly which transports to use and so this new version
brings in many additional indirect dependencies than before, most of which
are irrelevant for OpenTofu's current purposes because they related to
the logging and tracing parts of OpenTelemetry, whereas OpenTofu currently
focuses only on tracing. The logging and metrics part of OpenTelemetry were
still very early and experimental at the time of the commit we previously
depended on, and so it seems that the main upstream change here has been
improved support for those other use-cases. Most of these new indirect
dependencies are therefore unreachable at runtime by OpenTofu in practice.
For OpenTofu's purposes the only significant new capability here is to
set OTEL_TRACES_EXPORTER to "console" to write trace information to stdout,
but that remains unreachable in OpenTofu in practice because we currently
ignore that environment variable unless it is set to "otlp" exactly. We
could consider supporting "console" in a future release, but this mode
is questionable for OpenTofu because writing to stdout would interfere
with OpenTofu's main UI or machine-readable output. We'll wait for another
day to discuss the tradeoffs of that.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The Go team uses automation to generate unnecessary version bumps across
all of these that make it impossible to upgrade them individually because
they all mutually depend on the latest versions of each other, so
unfortunately we have to accept the risk of updating all of these at once
in order to update any one of them.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously the Go toolchain had no explicit support for "tools" and so we
used the typical Go community workaround of adding "tools.go" files (two,
for some reason) that existed only to trick the Go toolchain into
considering the tools as dependencies we could track in go.mod.
Go 1.24 introduced explicit support for tracking tools as part of go.mod,
and the ability to run those using "go tool" instead of "go run", and so
this commit switches us over to using that strategy for everything we were
previously managing in tools.go.
There are some intentional exceptions here:
- The protobuf-compile script can't use "go tool" or "go run" because the
tools in question are run only indirectly through protoc. However, we
do still use the "tool" directive in go.mod to tell the Go toolchain that
we depend on those tools, so that it'll track which versions we are
currently using as part of go.mod.
- Our golangci-lint Makefile target uses "go run" to run a specific
version of golangci-lint. We _intentionally_ don't consider that tool
to be a direct dependency of OpenTofu because it has a lot of indirect
dependencies that would pollute our go.mod file. Therefore that continues
to use "go run" after this commit.
- Both of our tools.go files previously referred to
github.com/nishanths/exhaustive , but nothing actually appears to be
using that tool in the current OpenTofu tree, so it's no longer a
dependency after this commit.
All of the dependencies we have _only_ for tools are now classified as
"indirect" in the go.mod file. This is the default behavior of the Go
toolchain and appears to be motivated by making it clearer that these
modules do not contribute anything to the runtime behavior of OpenTofu.
This also corrected a historical oddity in our go.mod where for some reason
the "indirect" dependencies had been split across two different "require"
directives; they are now all grouped together in a single directive.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We tend to get scared off from routine dependency upgrades because it's
hard to know where to start when we want to avoid upgrading too many things
at once and thus making it hard for us to understand the impact.
This tool makes a best effort to suggest an order of upgrades that lets us
upgrade one thing at a time when possible, and if not possible then at
least tries to minimize how many things get upgraded at once.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
We were previously using this module to access the then-experimental
"slices" package, but equivalent functionality is now available in a
standard library package so we no longer need to use the experimental
version.
This remains as an indirect dependency just because some of the tools we
use depend on it.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade, and doesn't change any functionality that
OpenTofu depends on. It includes a potential performance improvement for
comparisons between versions, and implementation of some
irrelevant-to-OpenTofu interfaces from the database/sql/driver package.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This upgrade includes some behavior changes that will affect OpenTofu's
handling of the TF_ARG_* environment variables, but in a generally positive
way: the parsing of those is now closer to how Unix shells typically
behave.
Although it's unlikely that this would affect anyone, it is technically a
behavior change that could potentially change the handling of some unusual
patterns such as empty arguments represented as pairs of quotes with
nothing between them.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This fixes some bugs that could potentially affect Windows users, though
they all appear to be unlikely problems that we've never had any reports
of from OpenTofu users, so this is mostly just a routine upgrade.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade. The upstream library contains no relevant
functional changes, just some different implementation details.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This release introduces an optional new backoff policy implementation, but
that is entirely new functionality that no OpenTofu caller uses and so
this should not cause any behavior changes.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This patch release includes an improvement to the error messages for
certain syntax errors involving "v" prefixes, which were previously
returning confusing error messages that misdiagnosed the problem.
There are no other behavior changes.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is a no-changes release that was generated by the Go team's upgrade
bot, which we're adopting here only because some of our other dependencies
require it but I want to upgrade those alone so we're not changing many
things all at once.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is an utterly pointless upgrade because it only modifies this module's
go.mod to require Go 1.24 instead of Go 1.23, but I'm upgrading it here
anyway because some of our other pending upgrades require this newer
version and I want to keep each of these upgrade PRs as small as possible.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
The 1.25.1 and 1.25.2 patch releases both include various security updates
that don't appear to directly affect OpenTofu, but nonetheless we'll
upgrade in preparation for our forthcoming v1.11.0 release so that these
advisories won't cause false-positives for imprecise security scanners
throughout our v1.11 series.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This includes a fix to a bug introduced in an earlier version of go-getter:
https://github.com/hashicorp/go-getter/pull/560
We did not actually use an affected version of go-getter in any stable
release yet, and so this change does not require a changelog update for
OpenTofu.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade, which doesn't cause any significant changes
to OpenTofu's behavior.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This also upgrades two indirect dependencies, but we have those primarily
because of go-plugin and don't rely on them for any other significant
functionality.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
Previously we were using a mixture of old and new, with our code generation
using the plugin from the old github.com/golang/protobuf library but
our callers using the modern google.golang.org/protobuf . We were also
using pretty ancient version of protoc.
This brings us up to the current latest releases and consistently using
the new Go protobuf library. There have been some notable changes to these
tools in the meantime:
Previously the protoc-gen-go plugin handled grpc by having its own
additional level of Go-specific "plugins" of which the gRPC codegen was
an example.
Now the protobuf generator and the gRPC generator are separate plugins
handled directly by protoc, which means the command line arguments are
a different shape and the gRPC stubs get generated in a separate file
from the main protobuf messages, rather than all being in one .pb.go file
as before.The results are otherwise similar, though.
The grpc codegen now also defaults to requiring that implementations embed
the generated "unimplemented" server, which is an implementation of each
service where the methods just immediately return the "unimplemented"
error. This is not super important for us because we maintain the generated
interfaces and their implementations together in the same repository
anyway, but adding the "unimplemented" server embeds was not a big change
and so seems better to follow the prevailing convention.
Using these new versions means that we could in principle now switch to
using protobuf edition 2024 and the new "sealed" style for Go code
generation, but this commit does not include any such changes and focuses
only on getting things upgraded with as few other changes as possible. We
can discuss using different codegen style later and deal with that in
separate commits.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
This is just a routine upgrade. Several of our other dependencies also
depend on these modules and so upgrading those others tends to ratchet up
these libraries. Therefore I'm upgrading them separately here really just
to allow subsequent upgrades of other dependencies without changing too
many dependencies at once in a single commit.
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
