opentf/docs/tracing.md

# OpenTofu Tracing Guide

This document describes how to use and implement tracing in OpenTofu Core using OpenTelemetry.

> [!NOTE]  
> For background on the design decisions and motivation behind OpenTofu's tracing implementation, see the [OpenTelemetry Tracing RFC](https://github.com/opentofu/opentofu/blob/main/rfc/20250129-Tracing-For-Extra-Context.md).

## Overview

OpenTofu provides distributed tracing capabilities via OpenTelemetry to help end users understand the execution flow and performance characteristics of OpenTofu operations. Tracing is particularly useful for:

- Debugging performance issues (e.g., "Why is my plan taking so long?")
- Understanding time spent in different operations
- Visualizing the execution flow across providers and modules
- Diagnosing issues in CI/CD pipelines

Tracing in OpenTofu is **strictly opt-in** and disabled by default. It's designed to have minimal overhead when disabled and to provide valuable insights when enabled.

> [!IMPORTANT]  
> OpenTofu's tracing functionality refers only to OpenTelemetry traces for local debugging and analysis.
> No telemetry or usage data is sent to external servers, and no data leaves your environment unless you explicitly configure an external collector.

## Enabling Tracing

To enable tracing in OpenTofu:

1. Set the environment variable `OTEL_TRACES_EXPORTER=otlp`
2. Configure the OpenTelemetry exporter using standard OpenTelemetry environment variables

Example configuration for a local Jaeger collector:

```bash
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_EXPORTER_OTLP_INSECURE=true
```

For a complete list of configuration options, refer to the [OpenTelemetry Documentation](https://opentelemetry.io/docs/specs/otel/protocol/exporter/).

## Quick Start with Jaeger

To quickly spin up a local Jaeger instance with OTLP support:

```bash
docker run -d --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 5778:5778 \
  -p 9411:9411 \
  jaegertracing/jaeger:2.5.0
```

Then configure OpenTofu as shown above and access the Jaeger UI at http://localhost:16686.

## Adding Tracing to OpenTofu Code

> [!NOTE]  
> **For Contributors**: When adding tracing to OpenTofu, remember that the primary audience is **end users** who need to understand performance, not developers. Add spans sparingly to avoid polluting traces with too much detail.

### Basic Span Creation

```go
import (
    "github.com/opentofu/opentofu/internal/tracing"
    "github.com/opentofu/opentofu/internal/tracing/traceattrs"
    otelAttr "go.opentelemetry.io/otel/attribute"  // Note the alias
)

func SomeFunction(ctx context.Context) error {
    // Create a new span
    ctx, span := tracing.Tracer().Start(ctx, "Human readable operation name")
    defer span.End()
    
    // Add attributes to provide context
    span.SetAttributes(otelAttr.String("opentofu.some.attribute", "value"))
    
    // Using predefined attributes from traceattrs package
    span.SetAttributes(otelAttr.String(traceattrs.ProviderAddress, "hashicorp/aws"))
    
    // Your function logic here...
    
    // If an error occurs
    if err != nil {
        tracing.SetSpanError(span, err)
        return err
    }
    
    return nil
}
```

> [!TIP]  
> We should use the `otelAttr` alias for OpenTelemetry's attribute package to clearly distinguish it from OpenTofu's trace attribute constants in the `traceattrs` package.
> This convention makes the code more readable and prevents import conflicts.

### Tracing Conventions

#### Span Naming

- Use human-readable, action-oriented names that describe operations from a user perspective
- Prefer names like "Provider installation" over internal function names like "InstallProvider"
- Use consistent terminology from the OpenTofu CLI and documentation
- Span names should represent UX-level concepts, not internal code structure

#### Attributes

- Prefer standard [OpenTelemetry semantic conventions](https://opentelemetry.io/docs/specs/semconv/) where applicable
- Follow the [OpenTelemetry attribute naming convention](https://opentelemetry.io/docs/specs/semconv/general/naming/)
- Cross-cutting attributes are defined in `internal/tracing/traceattrs`

```go
// Good attribute names
"opentofu.provider.address"      // For provider addresses
"opentofu.module.source"         // For module sources
"opentofu.operation.target_count" // For operation-specific counts
```


#### Error Handling

Use the `tracing.SetSpanError` helper to consistently record errors:

```go
if err != nil {
    tracing.SetSpanError(span, err)
    return err
}
```

This helper supports various error types including standard errors, strings, and OpenTofu diagnostics.

### Instrumentation Guidelines

1. **Focus on Key Operations**: Instrument high-level operations that are meaningful to end users rather than every internal function.
2. **Include Valuable Context**: Add attributes that help identify resources, modules, or operations.
3. **Respect Performance**: Avoid expensive computations solely for tracing.