jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-19 18:14:56 -05:00
Code Issues Packages Projects Releases Wiki Activity

docs: Reorganize MCP server documentation (#68205)

Browse Source 
## What

Reorganizes the MCP (Model Context Protocol) server documentation to
clearly distinguish between multiple Airbyte MCP servers and makes all
MCP documentation discoverable through the sidebar.

Requested by AJ Steers (@aaronsteers) in [this Devin
session](https://app.devin.ai/sessions/7f61ccb7731f4009b19076fa2b5df36d).

**Problem**: The existing MCP documentation only covered one remote MCP
server (the Heroku-hosted "fast-coder" server), but there are actually
four MCP servers:
1. The new PyAirbyte MCP (local STDIO server) - the recommended option
2. The PyAirbyte Fast-Coder MCP (remote Heroku server) - for rapid code
generation
3. The Connector Builder MCP (in development)
4. The Embedded Operator MCP (existed but wasn't in the sidebar)

## How

- **Renamed** `docs/ai-agents/pyairbyte-mcp/` →
`docs/ai-agents/pyairbyte-fast-coder-mcp/` (the old remote server)
- **Created** new `docs/ai-agents/pyairbyte-mcp/README.md` with
documentation for the new local STDIO MCP server (copied from the
docstring in `airbyte/mcp/__init__.py` in the PyAirbyte repo)
- **Created** `docs/ai-agents/connector-builder-mcp/README.md` as a
stub/"coming soon" page
- **Updated** `docusaurus/sidebar-ai-agents.js` to:
  - Group all four MCP servers under a "MCP Servers" category
  - Add the previously hidden Embedded Operator MCP to the sidebar
- **Updated** `docs/ai-agents/README.md` to describe all four MCP
servers with clear distinctions
- **Fixed** typo: "embedded source" → "embedded search" in the
Fast-Coder MCP docs

## Review guide

1. **`docs/ai-agents/README.md`** - Verify the high-level descriptions
of each MCP server are clear and accurate
2. **`docs/ai-agents/pyairbyte-mcp/README.md`** - Review the setup
instructions for the new PyAirbyte MCP (local server). This content was
copied from the PyAirbyte repo docstring - please verify it's complete
and correct.
3. **`docs/ai-agents/pyairbyte-fast-coder-mcp/README.md`** - Verify the
renamed "Fast-Coder" MCP docs are still accurate
4. **`docusaurus/sidebar-ai-agents.js`** - Check that the sidebar
structure makes sense with all four MCP servers grouped together
5. **`docs/ai-agents/connector-builder-mcp/README.md`** - Confirm the
stub page is appropriate for the in-development server

**Key distinction to verify**: The new "PyAirbyte MCP" (local,
recommended) vs "PyAirbyte Fast-Coder MCP" (remote, for code generation)
are clearly differentiated in the docs.

## User Impact

**Positive:**
- Users can now discover all available Airbyte MCP servers in one place
- The new PyAirbyte MCP (local STDIO server) is documented and
positioned as the recommended option
- The Embedded Operator MCP is now visible in the documentation sidebar
- Clear guidance on which MCP server to use for different use cases

**Potential confusion:**
- Having two PyAirbyte-related MCP servers might initially confuse
users, but the docs clearly explain the differences

## Can this PR be safely reverted and rolled back?

- [x] YES 💚

This is purely documentation changes with no code or infrastructure
changes. Rolling back would just restore the previous documentation
structure.

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This commit is contained in:
Aaron ("AJ") Steers
2025-10-20 20:30:24 -07:00
committed by GitHub
parent 7bae93473e
commit 3b879abd1f
5 changed files with 183 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
docs/ai-agents/README.md
View File

@@ -3,12 +3,16 @@ products: embedded
---
# AI Agents
Airbyte provides multiple tools to help you build data applications.
- **Airbyte Embedded Widget**: App development teams who have signed up for Airbyte Embedded and are looking to get started onboarding customers using the Embedded Widget can follow the get started guide at the bottom of this page, which will step you through a complete sample onboarding app.
- **MCP Server**: App developers looking to utilize PyAirbyte to generate pipelines in code can follow the steps below to utilize the PyAirbyte MCP server.
- **Authentication Proxies**: Connect safely to 3rd party APIs using Airbyte's Authentication Proxies.
- **Authentication Proxies**: Connect safely to third party APIs using Airbyte's Authentication Proxies.
- **MCP Servers**: Airbyte provides multiple MCP (Model Context Protocol) servers for different use cases:
- [**PyAirbyte MCP**](#pyairbyte-mcp): Local MCP server for managing Airbyte connectors through AI assistants.
- [**Connector Builder MCP**](#connector-builder-mcp): AI-assisted connector development - _**coming soon!**_
- [**Embedded Operator MCP**](#embedded-operator-mcp): Manage embedded configurations and pipelines.
- [**PyAirbyte Fast-Coder MCP**](#pyairbyte-fast-coder-mcp) _(deprecated)_: Remote MCP server for rapid pipeline code generation.
## Prerequisites
@@ -28,10 +32,25 @@ Airbyte Embedded creates isolated workspaces for each of your customers, allowin
Once your Organization is enabled via Airbyte Embedded, you can begin onboarding customers via the Embedded Widget. You can download the code for the onboarding app [via GitHub](https://github.com/airbytehq/embedded-demoapp).
## MCP Servers
## PyAirbyte MCP
Airbyte provides multiple MCP (Model Context Protocol) servers to enable AI-assisted data integration workflows:
[The PyAirbyte remote MCP server](./pyairbyte-mcp/README.md) provides the ability for application developers to generate a data pipeline in Python using a single prompt. It is currently designed to work within Cursor, with broader support coming in the near future.
### PyAirbyte MCP
[The PyAirbyte MCP server](./pyairbyte-mcp.md) is a local MCP server that provides a standardized interface for managing Airbyte connectors through MCP-compatible clients. It allows you to list connectors, validate configurations, and run sync operations using the MCP protocol. This is the recommended MCP server for most use cases.
### Connector Builder MCP
[The Connector Builder MCP server](./connector-builder-mcp.md) (coming soon) will provide AI-assisted capabilities for building and testing Airbyte connectors using the Model Context Protocol.
### Embedded Operator MCP
[The Embedded Operator MCP](./embedded/operator-mcp/README.md) is a remote MCP server providing tools that enable managing embedded configurations and the resulting pipelines. Users can create connection and source templates, securely create sources, query API and File Storage sources, monitor connections and jobs, and more.
### PyAirbyte Fast-Coder MCP
[The PyAirbyte Fast-Coder MCP](./pyairbyte-fast-coder-mcp.md) is a remote MCP server that provides the ability for data engineers to generate a data pipeline in Python using a single prompt. It is currently designed to work within Cursor, with broader support coming in the near future.
## Proxy Requests

18
docs/ai-agents/connector-builder-mcp.md Normal file
View File

@@ -0,0 +1,18 @@
---
products: embedded
---
# Connector Builder MCP Server
> **NOTE:**
> The Connector Builder MCP server is currently in development. This documentation will be updated as the server becomes available.
The Connector Builder MCP server provides AI-driven connector building experience for building and testing Airbyte connectors using the [Model Context Protocol](https://modelcontextprotocol.io/). This enables AI assistants to help developers create, configure, and validate custom connectors through a standardized interface.
## Coming Soon
This MCP server is under active development. Check back for updates.
## Early Access Preview
If you are an active Airbyte customer interested in early access to the **Connector Builder MCP** server, please reach out to your sales representative to request early access.

12
docs/ai-agents/pyairbyte-mcp/README.md → docs/ai-agents/pyairbyte-fast-coder-mcp.md
View File

@@ -2,9 +2,13 @@
products: embedded
---
# Overview
# PyAirbyte Fast-Coder MCP (Deprecated)
The PyAirbyte remote MCP server provides the ability for application developers to generate a data pipeline in Python using a single prompt. It is currently designed to work within Cursor, and broader support in the near future.
::warning
The PyAirbyte Fast-Coder MCP has been deprecated and is no longer actively maintained. If you would like to see continued development on this tool, please let us know by adding feedback on the related GitHub Discussion.
::
The PyAirbyte Fast-Coder MCP is a remote MCP server that provides the ability for data engineers to generate a data pipeline in Python using a single prompt. It is currently designed to work within Cursor, with broader support coming in the near future.
To add the PyAirbyte MCP open Cursor and navigate to Settings > Tools & Integrations, and tap New MCP Sever. Add the following json snippet. This file tells Cursor which remote MCP servers to connect to and what credentials to pass along.
@@ -33,7 +37,7 @@ Within your Cursor project, start a new chat. In the input box, type the followi
create a data pipeline from source-faker to destination-snowflake
```
The MCP server will process your prompt and respond by generating all the necessary Python code to extract data from `faker` and load it into `Snowflake`. We suggest you prefix your source and destination with `source-` and `destination-` to ensure specificity when the MCP server performs a embedded source on the Airbyte Connector registry. Connectors for sources and destinations may have the same name, but different configuration parameters.
The MCP server will process your prompt and respond by generating all the necessary Python code to extract data from `faker` and load it into `Snowflake`. We suggest you prefix your source and destination with `source-` and `destination-` to ensure specificity when the MCP server performs a search on the Airbyte Connector registry. Connectors for sources and destinations may have the same name, but different configuration parameters.
In a few moments, your pipeline will be created, typically in a file called `pyairbyte_pipeline.py`. In addition, the MCP server will generate complete instructions on how to use the server and configure required parameters using a `.env` file that includes environment variables you’ll need to fill in.
@@ -46,4 +50,4 @@ AIRBYTE_DESTINATION__SNOWFLAKE__PASSWORD=your_password
AIRBYTE_DESTINATION__SNOWFLAKE__DATABASE=your_db
AIRBYTE_DESTINATION__SNOWFLAKE__SCHEMA=your_schema
AIRBYTE_DESTINATION__SNOWFLAKE__WAREHOUSE=your_warehouse
```
```

109
docs/ai-agents/pyairbyte-mcp.md Normal file
View File

@@ -0,0 +1,109 @@
---
products: embedded
---
# PyAirbyte MCP Server
> **NOTE:**
> This MCP server implementation is experimental and may change without notice between minor versions of PyAirbyte. The API may be modified or entirely refactored in future versions.
The PyAirbyte MCP (Model Context Protocol) server provides a standardized interface for managing Airbyte connectors through MCP-compatible clients. This experimental feature allows you to list connectors, validate configurations, and run sync operations using the MCP protocol.
## Getting Started with PyAirbyte MCP
To get started with the PyAirbyte MCP server, follow these steps:
1. Create a Dotenv secrets file.
2. Register the MCP server with your MCP client.
3. Test the MCP server connection using your MCP client.
### Step 1: Generate a Dotenv Secrets File
To get started with the PyAirbyte MCP server, you will need to create a dotenv file containing your Airbyte Cloud credentials, as well as credentials for any third-party services you wish to connect to via Airbyte.
Create a file named `~/.mcp/airbyte_mcp.env` with the following content:
```ini
# Airbyte Project Artifacts Directory
AIRBYTE_PROJECT_DIR=/path/to/any/writeable/project-dir
# Airbyte Cloud Credentials (Required for Airbyte Cloud Operations)
AIRBYTE_CLOUD_CLIENT_ID=your_api_key
AIRBYTE_CLOUD_CLIENT_SECRET=your_api_secret
AIRBYTE_CLOUD_WORKSPACE_ID=your_workspace_id
# API-Specific Credentials (Optional, depending on your connectors)
# For example, for a PostgreSQL source connector:
# POSTGRES_HOST=your_postgres_host
# POSTGRES_PORT=5432
# POSTGRES_DB=your_database_name
# POSTGRES_USER=your_database_user
# POSTGRES_PASSWORD=your_database_password
# For example, for a Stripe source connector:
# STRIPE_API_KEY=your_stripe_api_key
# STRIPE_API_SECRET=your_stripe_api_secret
# STRIPE_WEBHOOK_SECRET=your_stripe_webhook_secret
```
Note:
1. You can add more environment variables to this file as needed for different connectors. To start, you only need to create the file and pass it to the MCP server.
2. Ensure that this file is kept secure, as it contains sensitive information. Your LLM *should never* be given direct access to this file or its contents.
3. The MCP tools will give your LLM the ability to view *which* variables are available, but it does not give access to their values.
4. The `AIRBYTE_PROJECT_DIR` variable specifies a directory where the MCP server can store temporary project files. Ensure this directory is writable by the user running the MCP server.
### Step 2: Registering the MCP Server
First install `uv` (`brew install uv`).
Then, create a file named `server_config.json` (or the file name required by your MCP client) with the following content. This uses `uvx` (from `brew install uv`) to run the MCP server. If a matching version Python is not yet installed, a `uv`-managed Python version will be installed automatically. This will also auto-update to use the "latest" Airbyte MCP release at time of launch. You can alternatively pin to a specific version of Python and/or of the Airbyte library if you have special requirements.
```json
{
"mcpServers": {
"airbyte": {
"command": "uvx",
"args": [
"--python=3.11",
"--from=airbyte@latest",
"airbyte-mcp"
],
"env": {
"AIRBYTE_MCP_ENV_FILE": "/path/to/my/.mcp/airbyte_mcp.env"
}
}
}
}
```
Note:
- Replace `/path/to/my/.mcp/airbyte_mcp.env` with the absolute path to your dotenv file created in Step 1.
### Step 3: Testing the MCP Server Connection
You can test the MCP server connection using your MCP client.
Helpful prompts to try:
1. "Use your MCP tools to list all available Airbyte connectors."
2. "Use your MCP tools to get information about the Airbyte Stripe connector."
3. "Use your MCP tools to list all variables you have access to in the dotenv secrets file."
4. "Use your MCP tools to check your connection to your Airbyte Cloud workspace."
5. "Use your MCP tools to list all available destinations in my Airbyte Cloud workspace."
## Contributing to the Airbyte MCP Server
- [PyAirbyte Contributing Guide](https://github.com/airbytehq/PyAirbyte/blob/main/docs/CONTRIBUTING.md)
### Additional resources
- [Model Context Protocol Documentation](https://modelcontextprotocol.io/)
- [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk)
For issues and questions:
- [PyAirbyte GitHub Issues](https://github.com/airbytehq/pyairbyte/issues)
- [PyAirbyte Discussions](https://github.com/airbytehq/pyairbyte/discussions)

27
docusaurus/sidebar-ai-agents.js
View File

@@ -45,9 +45,30 @@ export default {
]
},
{
type: "doc",
id: "pyairbyte-mcp/README",
label: "PyAirbyte MCP",
type: "category",
label: "MCP Servers",
items: [
{
type: "doc",
id: "pyairbyte-mcp",
label: "PyAirbyte MCP",
},
{
type: "doc",
id: "pyairbyte-fast-coder-mcp",
label: "PyAirbyte Fast-Coder MCP",
},
{
type: "doc",
id: "connector-builder-mcp",
label: "Connector Builder MCP",
},
{
type: "doc",
id: "embedded/operator-mcp/README",
label: "Embedded Operator MCP",
}
]
}
]
}