opentf/TSC/2024-07-10_NOTES.md

2024-07-10

Attendees

• Christan Mesh (@cam72cam) (OpenTofu Tech Lead)
• Igor Savchenko (@DiscyDel)
• Roger Simms (@allofthesepeople)
• Wojciech Barczynski (@wojciech12)
• Zach Goldberg (@ZachGoldberg)

Agenda

We dedicated this meeting to agree on the initial plan for 1.9. Christian – OpenTofu Tech Lead – presented the plan to the TSC.

OpenTofu 1.9 Planning

Summary

Although a long list, the corresponding terraform 1.9 changes consist of a lot of small tasks (as well as quite a few items catching up to OpenTofu). This leaves us a fair amount of room to innovate in ways that will drive adoption. I'd like to specifically focus on provider for_each iteration (top issue in both tofu and terraform), as well as paying down technical debt related to testing. Additionally, I'd like to propose a stretch goal of adding the -exclude flag to tofu as it helps round off quite a few sharp UX edges.

Leftover from 1.7

• Tofu test changes, nearly ready to merge: https://github.com/opentofu/opentofu/issues/1185

Leftover from 1.8

• Bug in provider dev overrides, investigating at a low priority: https://github.com/opentofu/opentofu/issues/1715

Terraform 1.9

• Improved variable validation: small task, mostly testing, https://github.com/opentofu/opentofu/issues/1336
• Multiline console support: small task, already discussed by community, https://github.com/opentofu/opentofu/issues/1307
• Breaking change on providers in terraform test: unknown, https://github.com/hashicorp/terraform/issues/35160
• Bugfix sensitive templatefile: small task, https://github.com/hashicorp/terraform/issues/31119
• Improved version constraint calculations: small task, https://github.com/hashicorp/terraform/issues/33452
• Bugfix conflict between import and destroy: unknown, https://github.com/hashicorp/terraform/issues/35151
• Fix crash with tofu providers mirror: small task, https://github.com/hashicorp/terraform/issues/35318
• Fix conflict between create_before_destroy and -refresh=false, https://github.com/hashicorp/terraform/issues/35218

Proposed Goals:

• Implement provider iteration (for_each support): https://github.com/opentofu/opentofu/issues/300
  • Top requested feature from the community
  • Builds on top of static evaluation in 1.8
  • RFC has been reviewed and accepted: https://github.com/opentofu/opentofu/blob/main/rfc/20240513-static-evaluation-providers.md
  • Is a big migration incentive
• Improve test confidence and stability
  • Large quantity of testing exists in the codebase, of varying quality
  • We need to take time to understand it and plan improvements
  • Also need to define where and how we should be testing as a team going forward
  • Igor: what clean-up can we do to reduce complexity?
  • Roger: Potential for opt-in profiling
• Add -exclude flag for targeted plan/apply (stretch goal)
  • Currently in RFC process: https://github.com/opentofu/opentofu/pull/1717
  • Makes multi-stage applies much simpler
  • Allows selective excluding of portions of infrastructure without a massive -include list
  • Is a frequently requested feature.
  • Igor: How do we handle new features and experiments? Feature flag? Canary release?
  • Christian: To come back to this discussion in future meeting

Adjacent work (not locked to 1.9 milestone):

• Registry UI, good progress is being made, but it is a large undertaking
• Maintain official fork of HCL as HashiCorp ignores pull requests from our team.
  • There is a strong indication that we’re being ignored
  • Igor: Let’s upvote it to try to get it in
  • https://github.com/hashicorp/hcl/pull/676
• Registry should lock tags to a single commit to prevent supply chain attacks
• License check for mirrored providers (aws, gcp, etc...)
  • Not under CLA so lower risk
  • Janos: dual licensing new features
• Update OpenTofu.org:
  • Update landing page to reflect state of project, proposals already in progress
  • Clean up sponsorships and create job postings page
  • Include official support contracts from TACOS and similar
    • Igor: Needs a real support offering, how do we maintain this list?
    • Roger: Potentially ranked by level of OpenTofu support
  • Create quick start guides, could be defined by core team and implemented by the community

Looking ahead to 1.10:

• Some interesting core changes in terraform that may be tricky to mirror (ephemeral values)
• Community Requests:
  • OCI Registry support https://github.com/opentofu/opentofu/issues/308
  • Conditional single instance resources: https://github.com/opentofu/opentofu/issues/1306
  • Backends as plugins or start to add new/updated backends
• Technical Debt:
  • Introduce internal concept of "immutable state" to allow refactoring and more efficient operation
  • Refactor and clean up command package
  • Supply chain (go.mod) review, with focus on removing hashicorp dependencies

Decision

High level approval of split between terraform mirroring features and future development.

Approval given to turn the 1.9 plan above into a public milestone and to start breaking down the issues involved.