Commit Graph

668 Commits

Author SHA1 Message Date
Alisdair McDiarmid
c5c1f31db3 backend: Validate remote backend Terraform version
When using the enhanced remote backend, a subset of all Terraform
operations are supported. Of these, only plan and apply can be executed
on the remote infrastructure (e.g. Terraform Cloud). Other operations
run locally and use the remote backend for state storage.

This causes problems when the local version of Terraform does not match
the configured version from the remote workspace. If the two versions
are incompatible, an `import` or `state mv` operation can cause the
remote workspace to be unusable until a manual fix is applied.

To prevent this from happening accidentally, this commit introduces a
check that the local Terraform version and the configured remote
workspace Terraform version are compatible. This check is skipped for
commands which do not write state, and can also be disabled by the use
of a new command-line flag, `-ignore-remote-version`.

Terraform version compatibility is defined as:

- For all releases before 0.14.0, local must exactly equal remote, as
  two different versions cannot share state;
- 0.14.0 to 1.0.x are compatible, as we will not change the state
  version number until at least Terraform 1.1.0;
- Versions after 1.1.0 must have the same major and minor versions, as
  we will not change the state version number in a patch release.

If the two versions are incompatible, a diagnostic is displayed,
advising that the error can be suppressed with `-ignore-remote-version`.
When this flag is used, the diagnostic is still displayed, but as a
warning instead of an error.

Commands which will not write state can assert this fact by calling the
helper `meta.ignoreRemoteBackendVersionConflict`, which will disable the
checks. Those which can write state should instead call the helper
`meta.remoteBackendVersionCheck`, which will return diagnostics for
display.

In addition to these explicit paths for managing the version check, we
have an implicit check in the remote backend's state manager
initialization method. Both of the above helpers will disable this
check. This fallback is in place to ensure that future code paths which
access state cannot accidentally skip the remote version check.
2020-11-19 13:19:40 -05:00
Alisdair McDiarmid
eee57280f6 backend: Faster remote backend tests
The remote backend tests spent most of their execution time sleeping in
various polling and backoff waits. This is unnecessary when testing
against a mock server, so reduce all of these delays when under test to
much lower values.

Only one remaining test has an artificial delay: verifying the discovery
of services against an unknown hostname. This times out at DNS
resolution, which is more difficult to fix than seems worth it at this
time.
2020-11-18 16:00:05 -05:00
James Bardin
8e7a9b6312 output test for plan with no root output changes
Module outputs do not show up in the plan, and are not rendered in the
UI.
2020-11-17 16:11:57 -05:00
Upo
3b9c5e5bbb backend/gcs: remove deprecated "path" config argument (#26841) 2020-11-06 16:35:40 -05:00
Upo
4ccc63d79d backend: Add service account impersonation to GCS Backend and update the docs (#26700) 2020-11-06 09:06:07 -05:00
Alisdair McDiarmid
b335918c3c backend: Only show root module output changes
When rendering planned output changes, we need to filter the plan's
output changes to ensure that only root module outputs which have
changed are rendered. Otherwise we will render changes for submodule
outputs, and (with concise diff disabled) render unchanged outputs also.
2020-11-02 10:24:22 -05:00
Paul Thrasher
fc8a76a2a3 Stop printing tfce error message log 2020-10-28 16:03:55 -07:00
Kristin Laemmert
b8e3b8036a backend: remove deprecated atlas backend 2020-10-26 14:05:18 -04:00
Paul Thrasher
960042ad42 Merge pull request #26618 from hashicorp/pault/tfce-continue-on-error
Allow runs to continue on tfce error
2020-10-20 13:23:09 -07:00
James Bardin
0b31ffa587 use a single log writer
Use a single log writer instance for all std library logging.

Setup the std log writer in the logging package, and remove boilerplate
from test packages.
2020-10-19 14:29:54 -04:00
James Bardin
6ca477f042 move helper/logging to internal
remove a dead code file too
2020-10-19 14:27:53 -04:00
Paul Thrasher
d380f5873c Allow runs to continue on tfce error
A cost estimation error does not actually stop a run, so the run was continuing in the background after the cli exits, causing confusion. This change matches the UI behavior.
2020-10-16 17:27:04 -07:00
James Bardin
5e9425b562 unreachable 2020-10-14 14:06:00 -04:00
James Bardin
b8df47c9ac add struct field names 2020-10-14 14:05:41 -04:00
James Bardin
657dd33008 Merge pull request #26557 from remilapeyre/skip-ddl-commands
Add skip_table_creation and skip_index_creation options to the pg backend
2020-10-14 13:36:26 -04:00
James Bardin
5eca0788c6 rely solely on the plan changes for outputs
Now that outputs changes are tracked in full, we can remove the
comparisons with the prior state and use the planned changes directly.
2020-10-12 18:59:14 -04:00
Rémi Lapeyre
12a0a21c0b Add skip_table_creation and skip_index_creation options to the pg backend
Closes https://github.com/hashicorp/terraform/issues/25708
2020-10-12 22:47:19 +02:00
James Bardin
57a47d0b82 Merge pull request #26519 from hashicorp/jbardin/FIXME
Cleanup and remove legacy types from test mocks
2020-10-08 14:54:40 -04:00
James Bardin
103a6cf2db update mock provider call 2020-10-08 13:52:04 -04:00
Kristin Laemmert
6f4f6fbc72 helper: deprecate pathorcontents
pathorcontents was solely used by the gcs backend. I moved the function
into the backend package so it could still be used by other backends for
good measure.
2020-10-08 08:42:16 -04:00
Kristin Laemmert
d2e999ba1f remove unused code (#26503)
* remove unused code

I've removed the provider-specific code under registry, and unused nil
backend, and replaced a call to helper from backend/oss (the other
callers of that func are provisioners scheduled to be deprecated).

I also removed the Dockerfile, as our build process uses a different
file.

Finally I removed the examples directory, which had outdated examples
and links. There are better, actively maintained examples available.

* command: remove various unused bits

* test wasn't running

* backend: remove unused err
2020-10-07 11:00:06 -04:00
James Bardin
826ccdd123 re-enable test 2020-10-07 10:44:41 -04:00
Pam Selle
f84a7c1d57 Merge pull request #26476 from remilapeyre/postgres-backend-escape-schema_name
Properly quote schema_name in the pg backend configuration
2020-10-05 14:57:53 -04:00
James Bardin
70652814e3 add missing Context argument 2020-10-05 14:43:46 -04:00
James Bardin
5e13d57f4e Merge pull request #26463 from hashicorp/kt/azurerm-secret-backend-fix
AzureRM backend: correctly lookup environment from metadata host
2020-10-05 14:26:32 -04:00
Alisdair McDiarmid
35d4926bf7 Merge pull request #25837 from remilapeyre/consul-force-unlock
Fix `terraform force-unlock <lock_id>` for Consul backend
2020-10-05 13:11:08 -04:00
Kristin Laemmert
3933cbd491 remove LegacyProvider (#26433) 2020-10-05 08:33:49 -04:00
Rémi Lapeyre
42be5854a2 Properly quote schema_name in the pg backend configuration 2020-10-03 18:02:13 +02:00
kt
b66364f7aa AzureRM backend: correctly lookup environment from metadata host 2020-10-01 16:34:51 -07:00
Kristin Laemmert
d76cfc8c0c Merge pull request #26440 from hashicorp/mildwonkey/remove-older-state-vs
refactor tests to use modern states.State in favor of terraform.State where possible
2020-10-01 08:11:27 -04:00
Pam Selle
3a99405b8b Merge pull request #26420 from remilapeyre/postgres-default-backend
Always have the default workspace in the pg backend
2020-09-30 17:54:24 -04:00
Kristin Laemmert
479655ad47 refactor tests to use modern states.State in favor of terraform.State where possible 2020-09-30 16:07:54 -04:00
James Bardin
ab6d6f99ae fix races in remote backend mock 2020-09-30 13:36:04 -04:00
Rémi Lapeyre
9acfe30ca6 Always have the default workspace in the pg backend
Closes https://github.com/hashicorp/terraform/issues/23121
2020-09-29 23:18:34 +02:00
Rémi Lapeyre
5c7008bd89 Merge remote-tracking branch 'origin/master' into consul-force-unlock 2020-09-29 10:59:36 +02:00
James Bardin
37569f5cc3 insert PlanRefresh into the context 2020-09-24 09:34:49 -04:00
James Bardin
bc82347a04 fix tests
Update tests to match the new behavior. Some were incorrect, some no
longer make sense, and some just weren't setup to handle th plan api
calls.
2020-09-21 16:17:46 -04:00
James Bardin
312317abd0 wrong instance key in test state
This was never picked up by the tests until now
2020-09-17 09:55:00 -04:00
James Bardin
8658424059 skip plan with no refresh test
We still need to determine if `-refresh=false` is even useful with the
new planning strategy.
2020-09-17 09:55:00 -04:00
James Bardin
f52d836e0a fix local backend tests to match new behavior
Leaving plan with -refresh=false tests failing for now.
2020-09-17 09:55:00 -04:00
James Bardin
be757bd416 Refresh instances during plan
This change refreshes the instance state during plan, so a complete
Refresh no longer needs to happen before Plan.
2020-09-17 09:54:59 -04:00
Alisdair McDiarmid
e183163739 Merge pull request #25856 from remilapeyre/consul-split-state
Split the state in chunks when they outgrow the limit of Consul KV store
2020-09-15 10:00:25 -04:00
Alisdair McDiarmid
92abaadc02 Merge pull request #25842 from remilapeyre/consul-path-slash
Sanitize lock path for the Consul backend when it ends with a /
2020-09-11 11:14:49 -04:00
Alisdair McDiarmid
09d8355f43 command: Add experimental concise diff renderer
When rendering a diff between current state and projected state, we only
show resources and outputs which have changes. However, we show a full
structural diff for these values, which includes all attributes and
blocks for a changed resource or output. The result can be a very long
diff, which makes it difficult to verify what the changed fields are.

This commit adds an experimental concise diff renderer, which suppresses
most unchanged fields, only displaying the most relevant changes and
some identifying context. This means:

- Always show all identifying attributes, initially defined as `id`,
  `name`, and `tags`, even if unchanged;
- Only show changed, added, or removed primitive values: `string`,
  `number`, or `bool`;
- Only show added or removed elements in unordered collections and
  structural types: `map`, `set`, and `object`;
- Show added or removed elements with any surrounding unchanged elements
  for sequence types: `list` and `tuple`;
- Only show added or removed nested blocks, or blocks with changed
  attributes.

If any attributes, collection elements, or blocks are hidden, a count
is kept and displayed at the end of the parent scope. This ensures that
it is clear that the diff is only displaying a subset of the resource.

The experiment is currently enabled by default, but can be disabled by
setting the TF_X_CONCISE_DIFF environment variable to 0.
2020-09-10 10:35:55 -04:00
Kirill Zaborsky
fbd3f191bd Minor typo 2020-09-03 11:25:55 +03:00
Matt Kasa
c2e35ae3b5 backend/http: add support for configuration by environment variable (#25439) 2020-09-02 08:35:08 -04:00
Kristin Laemmert
196c183dda terraform: remove state from validate graph walk (#26063)
This pull reverts a recent change to backend/local which created two context, one with and one without state. Instead I have removed the state entirely from the validate graph (by explicitly passing a states.NewState() to the validate graph builder).

This changed caused a test failure, which (ty so much for the help) @jbardin discovered was inaccurate all along: the test's call to `Validate()` was actually what was removing the output from state. The new expected test output matches terraform's actual behavior on the command line: if you use -target to destroy a resource, an output that references only that resource is *not* removed from state even though that test would lead you to believe it did.

This includes two tests to cover the expected behavior:

TestPlan_varsUnset has been updated so it will panic if it gets more than one request to input a variable
TestPlan_providerArgumentUnset covers #26035

Fixes #26035, #26027
2020-08-31 15:45:39 -04:00
James Bardin
0c5430f88d Merge pull request #25585 from chushenmeshile/master
fix ossbackend only returned 100 workspaces
2020-08-19 16:55:31 -04:00
James Bardin
bd5ff98ca7 Merge pull request #25830 from hashicorp/kt/secret-metadata-host
backend/azurerm: support loading environments from a metadata host
2020-08-19 13:57:10 -04:00
James Bardin
d46e9a4198 Merge pull request #25770 from hashicorp/f/azure-backend-spcert
backend/azurerm: support for authenticating using a Client Certificate
2020-08-19 13:56:30 -04:00