1
0
mirror of synced 2025-12-21 11:01:41 -05:00
Files
airbyte/docs/ai-agents/embedded/api/authentication.md
Teal Larson b50994c6f3 docs: Add comprehensive Sonar Embedded API documentation (#67580)
## Overview

This PR addresses critical documentation drift between the Sonar
implementation and the Airbyte documentation repository. Based on a
comprehensive drift analysis performed on 2025-10-08, this update adds
missing API documentation for HIGH PRIORITY features that are fully
implemented in Sonar but completely undocumented.

Reference: docs-drift-analysis-2025-10-08.md (in sonar repo)

## Changes Summary

### New Documentation Files (3)

1. **workspaces.md** - Workspace Management API
   - List workspaces with cursor-based pagination
   - Get, update, and delete workspaces
   - Workspace statistics and sync operations
   - Complete pagination guide and filtering examples

2. **authentication.md** - Complete Authentication Guide
   - All three token types (Operator Bearer, Scoped, Widget)
   - Region selection (US/EU)
   - Template filtering via tags during token generation
   - Token lifecycle management and security best practices
   - 20+ code examples

3. **tags.md** - Template Tag Management
   - Tag CRUD operations
   - Tag selection modes (ANY vs ALL)
   - Tagging source and connection templates
   - Tier-based access control examples

### Updated Documentation Files (3)

4. **source-templates.md** - Enhanced with stream management, PATCH
endpoint, tag operations, source definition catalogs

5. **connection-templates.md** - Added sync_on_create,
non_breaking_changes_preference, cron validation, PATCH endpoint, tag
operations

6. **configuring-sources.md** - Added region selection, stream
management, JMESPath querying, JSON Patch operations

## Impact

- 25+ previously undocumented API endpoints now documented
- ~2,500 lines of documentation added
- 100+ code examples
- 12 HIGH PRIORITY documentation gaps addressed

## Validation

All documentation validated against:
- Sonar codebase at commit 350a75fe73
- FastAPI route definitions
- Pydantic schemas
- Existing documentation style

## Breaking Changes

None - Documentation only

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
2025-10-13 12:41:54 -07:00

15 KiB

Authentication

The Airbyte Embedded API uses a hierarchical authentication system with three types of tokens, each designed for specific use cases and security requirements.

Token types overview

Token Type Use Case Scope Access Level
Operator Bearer Token Organization management, template creation Organization-wide Full access to organization resources
Scoped Token API integration, programmatic workspace access Single workspace Limited to specific workspace
Widget Token Embedded widget integration Single workspace + origin validation Limited to specific workspace with CORS protection

Operator bearer token

The Operator Bearer Token provides full organization-level access and is used for administrative operations.

Use cases

  • Creating and managing source templates
  • Creating and managing connection templates
  • Managing workspaces across the organization
  • Generating scoped tokens and widget tokens
  • Administrative API operations

Usage

Include the operator token in the Authorization header:

curl https://api.airbyte.ai/api/v1/integrations/templates/sources \

  -H 'Authorization: Bearer <your_operator_token>' \
  -H 'Content-Type: application/json'

Security best practices

  • Never expose operator tokens in client-side code
  • Store securely in secrets management system
  • Use scoped tokens for end-user operations
  • Rotate tokens periodically
  • Limit token distribution to trusted administrators only

Scoped token

Scoped tokens provide workspace-level access and are designed for allowing end-users to create and edit sources in their workspace.

Use cases

  • API integrations for managing sources within a specific workspace
  • Multi-tenant applications with isolated workspaces

Features

  • Workspace-scoped access (cannot access other workspaces)
  • Automatically creates workspace from a workspace name if it doesn't exist
  • Region selection support
  • Embedded in JWT with io.airbyte.auth.workspace_scope claim

Generate scoped token

Endpoint

POST https://api.airbyte.ai/api/v1/embedded/scoped-token

Authentication

Requires Operator Bearer Token

Request body

Field Type Required Description
workspace_name string Yes Name of the workspace to create or use
region_id UUID No Region where workspace should be created (defaults to US)

Region ids

Region Region ID
US (Default) 645a183f-b12b-4c6e-8ad3-99e165603450
EU b9e48d61-f082-4a14-a8d0-799a907938cb

Request example

Create scoped token for US workspace:

curl -X POST https://api.airbyte.ai/api/v1/embedded/scoped-token \

  -H 'Authorization: Bearer <your_operator_token>' \
  -H 'Content-Type: application/json' \
  -d '{

    "workspace_name": "customer_workspace_123"
  }'

Create scoped token for EU workspace:

curl -X POST https://api.airbyte.ai/api/v1/embedded/scoped-token \

  -H 'Authorization: Bearer <your_operator_token>' \
  -H 'Content-Type: application/json' \
  -d '{

    "workspace_name": "eu_customer_workspace",
    "region_id": "b9e48d61-f082-4a14-a8d0-799a907938cb"
  }'

Response example

{
  "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
}

Behavior notes

  • If the workspace already exists, returns a token for the existing workspace
  • If the workspace doesn't exist:
    • Checks if a workspace with the same name exists in Airbyte Cloud
    • If found in cloud, creates local database record
    • If not found, creates new workspace in both Airbyte Cloud and local database
  • The region_id is only used when creating a new workspace

Using scoped tokens

Once generated, use scoped tokens to access workspace-specific endpoints:

# List sources in the workspace
curl https://api.airbyte.ai/api/v1/embedded/sources \

  -H 'Authorization: Bearer <scoped_token>'

# Create a source
curl -X POST https://api.airbyte.ai/api/v1/embedded/sources \

  -H 'Authorization: Bearer <scoped_token>' \
  -H 'Content-Type: application/json' \
  -d '{

    "source_template_id": "template-123",
    "name": "My Data Source"
  }'

Get scoped token information

Retrieve organization and workspace information from a scoped token.

Endpoint

GET https://api.airbyte.ai/api/v1/embedded/scoped-token/info

Authentication

Requires Scoped Token

Request example

curl https://api.airbyte.ai/api/v1/embedded/scoped-token/info \

  -H 'Authorization: Bearer <scoped_token>'

Response example

{
  "organization_id": "12345678-1234-1234-1234-123456789012",
  "workspace ID": "a1b2c3d4-e5f6-7890-ab12-cd34ef567890"
}

Widget token

Widget tokens are specialized tokens designed for embedded widget integration with enhanced security features.

Use cases

  • Embedding the Airbyte configuration widget in your application
  • Providing end-users with a UI to configure data sources
  • Multi-tenant applications with isolated workspaces

Features

  • All features of scoped tokens
  • Origin validation for CORS protection via allowed_origin
  • Template filtering via tags (both source and connection templates)
  • Base64-encoded payload containing token and pre-configured widget URL
  • Tag selection modes: any (at least one tag) or all (all tags required)

Generate widget token

Endpoint

POST https://api.airbyte.ai/api/v1/embedded/widget-token

Authentication

Requires Operator Bearer Token

Request body

Field Type Required Default Description
workspace_name string Yes - Name of the workspace to create or use
allowed_origin string Yes - The allowed origin for CORS (for example, https://yourapp.com)
region_id UUID No US region Region where workspace should be created
selected_source_template_tags array of strings No [] Tags to filter which source templates are available
selected_source_template_tags_mode string No any Tag selection mode: any or all
selected_connection_template_tags array of strings No [] Tags to filter which connection templates are available
selected_connection_template_tags_mode string No any Tag selection mode: any or all

Tag selection modes

Mode Behavior Example
any Template must have at least one of the specified tags Tags: ["crm", "sales"] matches templates with either "crm" OR "sales"
all Template must have all of the specified tags Tags: ["crm", "sales"] matches only templates with both "crm" AND "sales"

Request examples

Basic widget token:

curl -X POST https://api.airbyte.ai/api/v1/embedded/widget-token \

  -H 'Authorization: Bearer <your_operator_token>' \
  -H 'Content-Type: application/json' \
  -d '{

    "workspace_name": "customer_workspace_123",
    "allowed_origin": "https://yourapp.com"
  }'

Widget token with source template filtering:

curl -X POST https://api.airbyte.ai/api/v1/embedded/widget-token \

  -H 'Authorization: Bearer <your_operator_token>' \
  -H 'Content-Type: application/json' \
  -d '{

    "workspace_name": "customer_workspace_123",
    "allowed_origin": "https://yourapp.com",
    "selected_source_template_tags": ["crm", "sales"],
    "selected_source_template_tags_mode": "any"
  }'

Widget token with both source and connection template filtering:

curl -X POST https://api.airbyte.ai/api/v1/embedded/widget-token \

  -H 'Authorization: Bearer <your_operator_token>' \
  -H 'Content-Type: application/json' \
  -d '{

    "workspace_name": "customer_workspace_123",
    "allowed_origin": "https://yourapp.com",
    "selected_source_template_tags": ["crm"],
    "selected_source_template_tags_mode": "any",
    "selected_connection_template_tags": ["standard-sync"],
    "selected_connection_template_tags_mode": "all",
    "region_id": "b9e48d61-f082-4a14-a8d0-799a907938cb"
  }'

Response example

{
  "token": "eyJ0b2tlbiI6ImV5SmhiR2NpT2lKSVV6STFOaUlzSW5SNWNDSTZJa3BYVkNKOS4uLiIsIndpZGdldFVybCI6Imh0dHBzOi8vYXBwLmFpcmJ5dGUuYWkvcXVpY2stc3RhcnQvd2lkZ2V0P3dvcmtzcGFjZUlkPWExYjJjM2Q0LWU1ZjYtNzg5MC1hYjEyLWNkMzRlZjU2Nzg5MCZhbGxvd2VkT3JpZ2luPWh0dHBzOi8veW91cmFwcC5jb20ifQ=="
}

Using widget tokens

The widget token is a base64-encoded JSON object containing:

  1. Scoped token - For API authentication
  2. Widget URL - Pre-configured URL with workspace ID, origin, and template filters

Decode widget token

// Example: Decode widget token in JavaScript
const decodedToken = JSON.parse(atob(widgetToken));

console.log(decodedToken.token);      // Scoped token for API calls
console.log(decodedToken.widgetUrl);  // URL to load the widget

Embed widget

<!-- Embed the Airbyte widget in an iframe -->
<iframe
  id="airbyte-widget"
  src=""
  width="100%"
  height="600px"
  frameborder="0">
</iframe>

<script>
  // Decode the widget token received from your backend
  const widgetToken = "eyJ0b2tlbiI6..."; // From API response
  const decoded = JSON.parse(atob(widgetToken));

  // Set the iframe src to the pre-configured widget URL
  document.getElementById('airbyte-widget').src = decoded.widgetUrl;
</script>

Template filtering with tags

Widget tokens support filtering both source templates and connection templates using tags. This allows you to customize which connectors and sync configurations are available to specific users or customer tiers.

Use cases

Tier-based access:

{
  "workspace_name": "free_tier_customer",
  "allowed_origin": "https://yourapp.com",
  "selected_source_template_tags": ["free-tier"],
  "selected_source_template_tags_mode": "any"
}

Industry-specific connectors:

{
  "workspace_name": "healthcare_customer",
  "allowed_origin": "https://yourapp.com",
  "selected_source_template_tags": ["healthcare", "hipaa-compliant"],
  "selected_source_template_tags_mode": "all"
}

Feature gating:

{
  "workspace_name": "beta_customer",
  "allowed_origin": "https://yourapp.com",
  "selected_source_template_tags": ["stable", "beta"],
  "selected_source_template_tags_mode": "any",
  "selected_connection_template_tags": ["premium-features"],
  "selected_connection_template_tags_mode": "any"
}

Authentication flow patterns

Pattern 1: direct API integration

For backend services or API integrations:

  1. Store Operator Bearer Token securely in your backend
  2. Generate Scoped Token for each customer workspace
  3. Use scoped token for all workspace-specific API calls
  4. Refresh scoped tokens when they expire (they expire after 20 minutes)
# 1. Generate scoped token (once per workspace)
SCOPED_TOKEN=$(curl -X POST https://api.airbyte.ai/api/v1/embedded/scoped-token \

  -H "Authorization: Bearer $OPERATOR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"workspace_name": "customer_123"}' | jq -r '.token')

# 2. Use scoped token for operations
curl https://api.airbyte.ai/api/v1/embedded/sources \

  -H "Authorization: Bearer $SCOPED_TOKEN"

Pattern 2: embedded widget integration

For embedding the Airbyte UI in your application:

  1. Store Operator Bearer Token in your backend
  2. Create API endpoint in your backend to generate widget tokens
  3. Generate Widget Token with appropriate tags and origin
  4. Return widget token to frontend
  5. Frontend decodes token and loads widget
// Backend endpoint (Node.js example)
app.post('/api/airbyte/widget-token', async (req, res) => {
  const { customerId, tier } = req.body;

  // Determine tags based on customer tier
  const tags = tier === 'premium' ? ['all'] : ['free-tier'];

  const response = await fetch('https://api.airbyte.ai/api/v1/embedded/widget-token', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${process.env.AIRBYTE_OPERATOR_TOKEN}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      workspace_name: `customer_${customerId}`,
      allowed_origin: 'https://yourapp.com',
      selected_source_template_tags: tags,
      selected_source_template_tags_mode: 'any'
    })
  });

  const data = await response.json();
  res.json({ widgetToken: data.token });
});

// Frontend code
async function loadAirbyteWidget(customerId, tier) {
  // Get widget token from your backend
  const response = await fetch('/api/airbyte/widget-token', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ customerId, tier })
  });

  const { widgetToken } = await response.json();

  // Decode and load widget
  const decoded = JSON.parse(atob(widgetToken));
  document.getElementById('airbyte-widget').src = decoded.widgetUrl;
}

Pattern 3: multi-region support

For applications serving users in different regions:

# Determine region based on customer location
REGION_ID="645a183f-b12b-4c6e-8ad3-99e165603450"  # US
# REGION_ID="b9e48d61-f082-4a14-a8d0-799a907938cb"  # EU

curl -X POST https://api.airbyte.ai/api/v1/embedded/scoped-token \

  -H "Authorization: Bearer $OPERATOR_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{

    \"workspace_name\": \"customer_workspace\",
    \"region_id\": \"$REGION_ID\"
  }"

Token lifecycle management

Token expiration

  • Operator Bearer Tokens: Short-lived, expires after 15 minutes
  • Scoped Tokens: Short-lived, expires after 20 minutes
  • Widget Tokens: Contain scoped tokens, same lifetime

Security considerations

CORS and origin validation

The allowed_origin parameter in widget tokens enforces CORS policies:

// Correct origin format
allowed_origin: "https://yourapp.com"       // ✓ Correct
allowed_origin: "https://yourapp.com:443"   // ✓ Correct with port
allowed_origin: "http://localhost:3000"     // ✓ For development

// Incorrect formats
allowed_origin: "https://yourapp.com/"      // ✗ No trailing slash
allowed_origin: "yourapp.com"               // ✗ Missing protocol
allowed_origin: "*.yourapp.com"             // ✗ No wildcards

Error responses

401 unauthorized

{
  "detail": "Invalid authentication credentials"
}

Causes:

  • Missing Authorization header
  • Invalid or expired token
  • Wrong token type for endpoint

Solutions:

  • Verify token is included in Authorization header
  • Ensure using correct token type (operator vs scoped)
  • Generate new token if expired

403 forbidden

{
  "detail": "Access denied to this resource"
}

Causes:

  • Scoped token trying to access resources in different workspace
  • Insufficient permissions for operation

Solutions:

  • Verify token scope matches target workspace
  • Use operator token for organization-level operations

422 validation error

{
  "detail": [
    {
      "loc": ["body", "allowed_origin"],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}

Cause: Missing or invalid request parameters

Solution: Verify all required fields are included and properly formatted

Next steps