Commit Graph

109 Commits

Author SHA1 Message Date
Diógenes Fernandes
35a3100cfe Fix http test server errors on TestBackendConfig_Authentication by removing usage of os.Clearenv (#3221)
Signed-off-by: Diogenes Fernandes <diofeher@gmail.com>
2025-09-02 16:15:30 -03:00
Andrei Ciobanu
48c55a4bfe Add logging for dynamodb client in s3 backend (#3103)
Signed-off-by: Andrei Ciobanu <andrei.ciobanu@opentofu.org>
2025-08-20 11:28:02 +03:00
Aaron George
e802c63f58 Add tag options for S3 state backend objects (#3038)
Signed-off-by: Aaron George <aarongeorge1994+github@gmail.com>
Co-authored-by: Andrei Ciobanu <andreic9203@gmail.com>
2025-08-07 10:39:19 +03:00
Martin Atkins
868dc2f01b hcl2shim: Split out legacy subset
Due to some past confusion about the purpose of this package, it has grown
to include a confusing mix of currently-viable code and legacy support
code from the move to HCL 2. This has in turn caused confusion about which
parts of this package _should_ be used for new code.

To help clarify that distinction we'll move the legacy support code into
a package under the "legacy" directory, which is also where most of its
callers live.

There are unfortunately still some callers to these outside of the legacy
tree, but the vast majority are either old tests written before HCL 2
adoption or helper code used only by those tests. The one dubious exception
is the use in ResourceInstanceObjectSrc.Decode, which makes a best effort
to shim flatmap as a concession to the fact that not all state-loading
codepaths are able to run the provider state upgrade function that would
normally be responsible for the flatmap-to-JSON conversion, which is
explained in a new comment inline.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-07-10 08:13:25 -07:00
Martin Atkins
67a5cd0911 statemgr+remote: context.Context parameters
This extends statemgr.Persistent, statemgr.Locker and remote.Client to
all expect context.Context parameters, and then updates all of the existing
implementations of those interfaces to support them.

All of the calls to statemgr.Persistent and statemgr.Locker methods outside
of tests are consistently context.TODO() for now, because the caller
landscape of these interfaces has some complications:

1. statemgr.Locker is also used by the clistate package for its state
   implementation that was derived from statemgr.Filesystem's predecessor,
   even though what clistate manages is not actually "state" in the sense
   of package statemgr. The callers of that are not yet ready to provide
   real contexts.

   In a future commit we'll either need to plumb context through to all of
   the clistate callers, or continue the effort to separate statemgr from
   clistate by introducing a clistate-specific "locker" API for it
   to use instead.

2. We call statemgr.Persistent and statemgr.Locker methods in situations
   where the active context might have already been cancelled, and so we'll
   need to make sure to ignore cancellation when calling those.

   This is mainly limited to PersistState and Unlock, since both need to
   be able to complete after a cancellation, but there are various
   codepaths that perform a Lock, Refresh, Persist, Unlock sequence and so
   it isn't yet clear where is the best place to enforce the invariant that
   Persist and Unlock must not be called with a cancelable context. We'll
   deal with that more in subsequent commits.

Within the various state manager and remote client implementations the
contexts _are_ wired together as best as possible with how these subsystems
are already laid out, and so once we deal with the problems above and make
callers provide suitable contexts they should be able to reach all of the
leaf API clients that might want to generate OpenTelemetry traces.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-07-10 08:11:39 -07:00
Andrei Ciobanu
a183f8ac6d Adjust one of the tests to ensure that the customer managed sse is configured correctly for the request to get the lock file (#2978)
Signed-off-by: Andrei Ciobanu <andrei.ciobanu@opentofu.org>
2025-06-30 11:26:47 -04:00
Andrei Ciobanu
482042e3f1 Unify encryption calls s3 backend (#2976)
Signed-off-by: Andrei Ciobanu <andrei.ciobanu@opentofu.org>
2025-06-30 17:19:43 +03:00
Andrei Ciobanu
0e8d32e556 Add encryption headers for the s3 native locking file (#2975)
Signed-off-by: Andrei Ciobanu <andrei.ciobanu@opentofu.org>
2025-06-30 16:51:12 +03:00
Martin Atkins
0afbaae42d backend/s3: Use APNInfo instead of UserAgent (#2955)
Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-06-25 09:15:40 -04:00
Larry Bordowitz
9b6a8fae60 refactor: implement all simple static checks (#2895)
Signed-off-by: Larry Bordowitz <laurence.bordowitz@gmail.com>
2025-06-10 15:48:43 -04:00
Larry Bordowitz
839392d52c refactor: remove unused func in s3 backend (#2879)
Signed-off-by: Larry Bordowitz <laurence.bordowitz@gmail.com>
2025-06-10 10:28:17 -04:00
James Humphries
dd8acbb113 Added otel tracing to show command (#2734)
Signed-off-by: James Humphries <james@james-humphries.co.uk>
2025-05-22 15:10:23 +01:00
Martin Atkins
b3ab138799 backend: Backend.DeleteWorkspace takes context.Context
This adds a new context.Context argument to the Backend.DeleteWorkspace
method, updates all of the implementations to match, and then updates all
of the callers to pass in a context.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-05-07 14:14:34 -07:00
Martin Atkins
601e84ee71 backend: Backend.StateMgr takes context.Context
This adds a new context.Context argument to the Backend.StateMgr method,
updates all of the implementations to match, and then updates all of the
callers to pass in a context.

A small number of callers don't yet have context plumbed to them so those
use context.TODO() as a placeholder for now, so we can more easily find
and fix them in later commits once we have contexts more thoroughly
plumbed.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-05-07 14:14:34 -07:00
Martin Atkins
b9573139ab backend: Backend.Workspaces takes context.Context
This adds a new context.Context argument to the Backend.Workspaces method,
updates all of the implementations to match, and then updates all of the
callers to pass in a context.

A small number of callers don't yet have context plumbed to them so those
use context.TODO() as a placeholder for now, so we can more easily find
and fix them in later commits once we have contexts more thoroughly
plumbed.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-05-07 14:14:34 -07:00
Martin Atkins
2922059ff3 backend: Backend.Configure takes context.Context
This adds a new context.Context argument to the Backend.Configure method,
updates all of the implementations to match, and then updates all of the
callers to pass in a context.

A small number of callers don't yet have context plumbed to them so those
use context.TODO() as a placeholder for now, so we can more easily find
and fix them in later commits once we have contexts more thoroughly
plumbed.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-05-07 14:14:34 -07:00
Martin Atkins
45e3a7bcf4 various: Remove context.TODO() usage from all tests
When context.Context was new, APIs using it arrived sporadically and so
the Go team introduced context.TODO() as an explicit way to say "I need a
context but I don't yet have a useful one to provide".

It took quite a while for there to be an established pattern for contexts
in tests, but now there is finally testing.T.Context which returns a
context that gets cancelled once the test is complete, and so that's a good
parent context to use for all contexts belonging to a test case.

This commit therefore mechanically replaces every use of context.TODO in
our test cases throughout the codebase with a call to t.Context instead.
There were a small number of tests that were using a mixture of
context.TODO and context.Background as placeholders and so those are also
updated to use t.Context consistently. There are probably still some
remaining uses of context.Background in our tests, but we'll save those
for another day.

As of this commit there are still various uses of context.TODO left in
_non-test_ code, but we need to take more care in how we update those so
those are intentionally excluded here.

Signed-off-by: Martin Atkins <mart@degeneration.co.uk>
2025-05-07 10:33:03 -07:00
Andrei Ciobanu
44d07cea4b Lock object writing is configured the same with the state object writing (#2606)
Signed-off-by: yottta <andrei.ciobanu@opentofu.org>
2025-03-18 12:34:27 +02:00
Andrei Ciobanu
1b103f3ac4 Bump aws sdk - mx-central-1 included and new default checksum behaviour changed for 3rd party s3 providers (#2596)
Signed-off-by: yottta <andrei.ciobanu@opentofu.org>
2025-03-18 10:36:57 +02:00
Christian Mesh
b2bf39802a Implement the first part of RFC 20250303-linter-policy (#2577)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2025-03-10 13:16:27 -04:00
Andrei Ciobanu
eba25e2fed Add the initial implementation for s3 locking (#2521)
Signed-off-by: yottta <andrei.ciobanu@opentofu.org>
2025-02-25 14:17:30 +02:00
Andrei Ciobanu
a3276028ad Upgrade aws sdk to a version that will give us access to s3 conditional writes arguments (#2528)
Signed-off-by: yottta <andrei.ciobanu@opentofu.org>
2025-02-18 15:58:17 +02:00
Haider Bari
6614782e6d Improved error messages for lock-related problems in the s3 backend (#2410)
Signed-off-by: Bari, Haider <haider.bari@fmr.com>
Co-authored-by: Bari, Haider <haider.bari@fmr.com>
2025-01-29 11:20:28 -05:00
Christian Mesh
87cd9227dc Fix S3 acceptance test panic, introduced in 1.7 encryption (#2420)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2025-01-24 09:59:43 -05:00
Alexandru Trendler
48abc52e46 bug: add error handling for missing state lock config in S3, HTTP and OSS backends (#1977)
Signed-off-by: g0dfl3sh <alex1trendler@gmail.com>
Signed-off-by: Alexandru Trendler <117138249+g0dfl3sh@users.noreply.github.com>
Co-authored-by: Oleksandr Levchenkov <ollevche@gmail.com>
2024-09-20 18:29:36 +02:00
Nathan Baulch
9b7bec31b4 Another batch of minor typos (#1953)
Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
2024-09-09 07:51:39 -04:00
Nathan Baulch
ea558d9d4b Fix typos (#1905)
Signed-off-by: Nathan Baulch <nathan.baulch@gmail.com>
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
Co-authored-by: Christian Mesh <christianmesh1@gmail.com>
2024-08-29 13:20:33 -04:00
Janos
261b966562 Fixes #1594: Support lower-case http/https_proxy env variables in S3 backend (#1736)
Signed-off-by: Janos <86970079+janosdebugs@users.noreply.github.com>
2024-06-20 11:11:19 -04:00
Arel Rabinowitz
927d1a5759 Remove use_legacy_workflow from the S3 backend configuration (#1730)
Signed-off-by: RLRabinowitz <rlrabinowitz2@gmail.com>
2024-06-17 13:33:10 -04:00
Oleksandr Levchenkov
02e12d054e ignore AccessDenied error on listing workspaces in S3 (#1445)
Signed-off-by: ollevche <ollevche@gmail.com>
2024-03-29 14:08:34 +02:00
Christian Mesh
07a9185767 Initial implementation of aws_kms encryption.key_provider (#1349)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
Signed-off-by: James Humphries <james@james-humphries.co.uk>
Co-authored-by: James Humphries <james@james-humphries.co.uk>
2024-03-13 13:19:20 -04:00
Christian Mesh
2f5dcd5c0a Integrate Encryption into State Backends (#1288)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2024-03-04 09:25:14 -05:00
namgyalangmo
cb2e9119aa Update copyright notice (#1232)
Signed-off-by: namgyalangmo <75657887+namgyalangmo@users.noreply.github.com>
2024-02-08 09:48:59 +00:00
James Humphries
199ec898e6 Resolve more inconsistencies in 32-bit systems (#1214)
Signed-off-by: James Humphries <james@james-humphries.co.uk>
Co-authored-by: Isaac Parker <parrotmac@gmail.com>
2024-02-02 13:46:38 +00:00
Dmitry Kisler
7d73f2bbe6 Refactor tests to follow a safer way of setting envvars (#1215)
Signed-off-by: Dmitry Kisler <admin@dkisler.com>
2024-01-31 11:25:02 +01:00
Ronny Orot
55651dca33 Backend/S3: Change default of use_legacy_workflow to false and deprecate (#1049)
Signed-off-by: Ronny Orot <ronny.orot@gmail.com>
2023-12-24 10:19:01 +02:00
Kuba Martin
5880b49645 Update aws-sdk-go-base and fix resulting type errors. (#957)
Signed-off-by: Jakub Martin <kubam@spacelift.io>
2023-12-01 16:37:31 +01:00
Dmitry Kisler
70dd385136 Add no_proxy and https_proxy attributes to s3 backend schema (#896)
Signed-off-by: Dmitry Kisler <admin@dkisler.com>
2023-11-20 14:37:59 +01:00
Christian Mesh
14b0765cb4 Fix S3 Remote State Backend Validation (#876)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2023-11-15 10:19:42 -05:00
Christian Mesh
637c65f6f2 Update s3 remote state tests (#865)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2023-11-14 10:40:25 -05:00
Christian Mesh
49f631d88b S3 Client HeadObject requires SSE parameters (#864)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2023-11-13 13:16:04 -05:00
Christian Mesh
e2d5a17e56 Address s3 compatible remote state issues + logging (#840)
Signed-off-by: Christian Mesh <christianmesh1@gmail.com>
2023-11-09 11:36:20 -05:00
James Humphries
9c24b6183a Revert PRs that introduced propogating contexts (#835)
Co-authored-by: Dmitry Kisler <admin@dkisler.com>
2023-11-08 21:09:14 +00:00
Tomas
b2069bb0bb Backend/S3: Update the Put method (#795)
Signed-off-by: tomasmik <tomasmik@protonmail.com>
2023-11-06 09:49:19 +02:00
Marcin Białoń
16c6432c3a Backend/S3: Add support for skip_requesting_account_id (#788)
Signed-off-by: Marcin Białoń <mbialon@spacelift.io>
2023-11-03 15:24:33 +01:00
Marcin Białoń
a1e110c679 Backend/S3: Custom Service Endpoint Configuration (#794)
Signed-off-by: Marcin Białoń <mbialon@spacelift.io>
2023-10-31 10:02:58 +01:00
Marcin Białoń
c633b24824 Backend/S3: Add support for use_path_style (#787)
Signed-off-by: Marcin Białoń <mbialon@spacelift.io>
2023-10-26 11:14:50 +02:00
Marcin Wyszynski
772ac1fc35 Pass context to all statemgr.Locker operations (#789)
Signed-off-by: Marcin Wyszynski <marcin.pixie@gmail.com>
2023-10-25 14:22:11 +02:00
Marcin Wyszynski
275dd116f9 Pass context to all remote.Client operations (#786)
Signed-off-by: Marcin Wyszynski <marcin.pixie@gmail.com>
2023-10-25 12:37:58 +02:00
Marcin Wyszynski
da1ba56b41 Make backend.Workspaces accept a context (#784)
Signed-off-by: Marcin Wyszynski <marcin.pixie@gmail.com>
2023-10-24 19:59:28 +02:00