1
0
mirror of synced 2025-12-19 18:14:56 -05:00
Files
airbyte/docs/integrations/sources/slack.md
octavia-bot-hoard[bot] 04fef1e52a 🐙 source-slack: run up-to-date pipeline [2025-10-29] (#68772)
Co-authored-by: octavia-bot-hoard[bot] <230633153+octavia-bot-hoard[bot]@users.noreply.github.com>
2025-10-28 22:56:40 -04:00

26 KiB
Raw Permalink Blame History

Slack

This page contains the setup guide and reference information for the Slack source connector.

Prerequisites

Before you begin, have the following ready:

  • Administrator access to an active Slack Workspace
  • Slack App OAuth (preferred) or API Key

Setup guide

Step 1: Set up Slack

The following instructions guide you through creating a Slack app. Airbyte can only replicate messages from channels that the app has been added to.

:::info If you are using a legacy Slack API Key, you can skip this section. :::

:::warning OAuth-only rate limit note: When authenticating via OAuth, the Slack source applies a stricter API budget for the Channel Messages and Threads streams (effectively one request per minute). To avoid slowing down other Slack streams, we recommend creating a separate connection for these two streams. See the Rate limiting section for details.

If you authenticate using a legacy Slack API token, this OAuth-specific limit does not apply. :::

To create a Slack App, read this tutorial on how to create an app, or follow these instructions.

  1. Go to your Apps
  2. Click Create New App. Select From Scratch.
  3. Choose a name for your app and select the name of your Slack workspace. Click Create App.
  4. In the navigation menu, select OAuth & Permissions.
  5. Navigate to Scopes. In Bot Token Scopes, select the following scopes:
 channels:history
 channels:join
 channels:read
 files:read
 groups:read
 links:read
 reactions:read
 remote_files:read
 team:read
 usergroups:read
 users:read
 users.profile:read
  1. At the top of the "OAuth & Permissions" page, click Install to Workspace. This will generate a Bot User OAuth Token. Copy this for later if you are using the API token for authentication.
  2. Go to your Slack instance. For any public channel, go to Info, More, and select Add Apps.
  3. Search for your newly created app. (If you are using the desktop version of Slack, you may need to restart Slack for it to pick up the new App). Add the App to all channels you want to sync data from.

:::note If you are using an API key to authenticate to Slack, a refresh token is not required, as acccess tokens never expire. You can learn more about refresh tokens here. :::

Step 2: Set up the Slack connector in Airbyte

For Airbyte Cloud:

  1. In the navigation bar, click Sources.
  2. Click New source.
  3. Find and click Slack.
  4. Click Authenticate your Slack account. Log in and authorize Airbyte to access your Slack account.
  5. Toggle join_channels, if you want to join all public channels or to sync data only from channels the bot is already in. If not set, you'll need to manually add the bot to all the channels from which you'd like to sync messages.
  6. Start Date: Any data before this date will not be extracted.
  7. Threads Lookback window (Days). This corresponds to the number of days in the past from which you want to sync data.
  8. (Optional) Channel filter the list of channel names (without leading '#' char) that limits the channels from which you'd like to sync. If no channels are specified, Airbyte will replicate data from all channels.
  9. (Optional) Include_private_channels Toggle on to sync data from private channels. You will need to manually add the bot to private channels even if join_channels is toggled on.
  10. Click Set up source. You must add the App created in Step 1 to the channels with the data that you want to sync.

For Airbyte Open Source:

  1. In the navigation bar, click Sources.
  2. Click New source.
  3. Find and click Slack.
  4. Click Sign in via Slack (OAuth). Enter the Access Token, Client ID, and Client Secret. Alternatively, enter the API Token from Step 1.
  5. Toggle join_channels, if you want to join all public channels or to sync data only from channels the bot is already in. If not set, you'll need to manually add the bot to all the channels from which you'd like to sync messages.
  6. Start Date: Any data before this date will not be extracted.
  7. Threads Lookback window (Days). This corresponds to the number of days in the past from which you want to sync data.
  8. (Optional) Channel filter the list of channel names (without leading '#' char) that limits the channels from which you'd like to sync. If no channels are specified, Airbyte will replicate data from all channels.
  9. (Optional) Include_private_channels Toggle on to sync data from private channels. You will need to manually add the bot to private channels even if join_channels is toggled on.
  10. Click Set up source. You must add the App created in Step 1 to the channels with the data that you want to sync.

Supported sync modes

The Slack source connector supports the following sync modes:

Feature Supported?
Full Refresh Sync Yes
Incremental Sync Yes
Namespaces No

Supported Streams

For most of the streams, the Slack source connector uses the Conversations API under the hood.

Performance considerations

The connector is restricted by Slack rate limits.

We highly recommend only syncing required channels. This can be done by specifying the channel_filter in the Slack configuration setings.

If you expect to sync a large amount of data (like historical data) you can try to increase the number of workers, default is 1. This could cause to being rate limited by Slack, so you should monitor the logs for rate limit errors.

Data type map

Integration Type Airbyte Type
string string
number number
array array
object object

Limitations & Troubleshooting

Expand to see details about Slack connector limitations and troubleshooting.

Connector limitations

Rate limiting

Slack has rate limit restrictions.

Rate Limits for Channel Messages and Threads streams:

OAuth authentication: For apps authenticated via OAuth, the connector enforces a stricter budget on:

These two streams are effectively limited to one request per minute. Consider creating a separate connection for them so other streams (Users, Channels, Channel Members, etc.) are not slowed down.

Legacy API token authentication: When using a legacy Slack API token, this OAuth-specific one-per-minute policy does not apply; only Slacks general rate limits apply.

Troubleshooting

  • Check out common troubleshooting issues for the Slack source connector on our Airbyte Forum here.

Changelog

Expand to review
Version Date Pull Request Subject
3.1.7 2025-10-29 68772 Update dependencies
3.1.6 2025-10-21 68282 Update dependencies
3.1.5 2025-10-14 67767 Update dependencies
3.1.4 2025-10-07 67449 Update dependencies
3.1.3 2025-10-06 67084 Update dependencies
3.1.2 2025-09-30 66566 Update to CDK v7
3.1.1 2025-09-24 66640 Update dependencies
3.1.0 2025-09-18 66501 Promoting release candidate 3.1.0-rc.1 to a main version.
3.1.0-rc.1 2025-09-10 64160 Migrate to manifest-only.
3.0.0 2025-09-10 65937 Add migration guide for missing state issue
2.2.0 2025-09-10 66155 Promoting release candidate 2.2.0-rc.7 to a main version.
2.2.0-rc.7 2025-08-21 65132 Update API budget to depend on auth method (rate limits apply only with OAuth).
2.2.0-rc.6 2025-08-14 64553 Add API budget for Threads and Channel Messages streams.
2.2.0-rc.5 2025-08-06 64530 Set use_cache = true for Channels and Channel Messages streams.
2.2.0-rc.4 2025-08-04 64486 Add backoff strategy for Channels stream.
2.2.0-rc.3 2025-07-29 64107 Add custom partition router.
2.2.0-rc.2 2025-07-23 63732 Enable progressive rollout.
2.2.0-rc.1 2025-07-23 63278 Migrate Threads stream to manifest.
2.1.0 2025-07-11 62930 Promoting release candidate 2.1.0-rc.1 to a main version.
2.1.0-rc.1 2025-07-07 62110 Bump cdk v6
2.0.2 2025-07-05 62709 Update dependencies
2.0.1 2025-06-28 51965 Update dependencies
2.0.0 2025-06-25 62055 Add breaking change notification for migrating to the new Slack Marketplace application to retain higher rate limits.
1.3.2 2025-01-11 43812 Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64
1.3.1 2024-07-24 42485 Fix MRO error for IncrementalMessageStream
1.3.0 2024-07-17 41994 Migrate to CDK v3.5.1
1.2.0 2024-07-16 41970 Migrate to CDK v2.4.0
1.1.13 2024-07-13 41863 Update dependencies
1.1.12 2024-07-10 41485 Update dependencies
1.1.11 2024-07-09 41231 Update dependencies
1.1.10 2024-07-06 40839 Update dependencies
1.1.9 2024-06-25 40347 Update dependencies
1.1.8 2024-06-22 40166 Update dependencies
1.1.7 2025-06-14 39343 Update state handling for threads Python stream
1.1.6 2024-06-12 39132 Respect include_private_channels option in threads stream
1.1.5 2024-06-10 39132 Convert string state to float for threads stream
1.1.4 2024-06-06 39271 [autopull] Upgrade base image to v1.2.2
1.1.3 2024-06-05 39121 Change cursor format for channel_messages stream to %s_as_float
1.1.2 2024-05-23 38619 Fix cursor granularity for the channel_messages stream
1.1.1 2024-05-02 36661 Schema descriptions
1.1.0 2024-04-18 37332 Add the capability to sync from private channels
1.0.0 2024-04-02 35477 Migration to low-code CDK
0.4.1 2024-03-27 36579 Upgrade airbyte-cdk version to emit record counts as floats
0.4.0 2024-03-19 36267 Pin airbyte-cdk version to ^0
0.3.9 2024-02-12 35157 Manage dependencies with Poetry
0.3.8 2024-02-09 35131 Fixed the issue when schema discovery fails with 502 due to the platform timeout
0.3.7 2024-01-10 1234 Prepare for airbyte-lib
0.3.6 2023-11-21 32707 Threads: do not use client-side record filtering
0.3.5 2023-10-19 31599 Base image migration: remove Dockerfile and use the python-connector-base image
0.3.4 2023-10-06 31134 Update CDK and remove non iterable return from records
0.3.3 2023-09-28 30580 Add bot_id field to threads schema
0.3.2 2023-09-20 30613 Set default value for channel_filters during discover
0.3.1 2023-09-19 30570 Use default availability strategy
0.3.0 2023-09-18 30521 Add unexpected fields to streams channel_messages, channels, threads, users
0.2.0 2023-05-24 26497 Fixed lookback window value limitations
0.1.26 2023-05-17 26186 Limited the lookback window range for input configuration
0.1.25 2023-03-20 22889 Specified date formatting in specification
0.1.24 2023-03-20 24126 Increase page size to 1000
0.1.23 2023-02-21 21907 Do not join channels that not gonna be synced
0.1.22 2023-01-27 22022 Set AvailabilityStrategy for streams explicitly to None
0.1.21 2023-01-12 21321 Retry Timeout error
0.1.20 2022-12-21 20767 Update schema
0.1.19 2022-12-01 19970 Remove OAuth2.0 broken refresh_token support
0.1.18 2022-09-28 17315 Always install latest version of Airbyte CDK
0.1.17 2022-08-28 16085 Increase unit test coverage
0.1.16 2022-08-28 16050 Fix SATs
0.1.15 2022-03-31 11613 Add 'channel_filter' config and improve performance
0.1.14 2022-01-26 9575 Correct schema
0.1.13 2021-11-08 7499 Remove base-python dependencies
0.1.12 2021-10-07 6570 Implement OAuth support with OAuth authenticator
0.1.11 2021-08-27 5830 Fix sync operations hang forever issue
0.1.10 2021-08-27 5697 Fix max retries issue
0.1.9 2021-07-20 4860 Fix reading threads issue
0.1.8 2021-07-14 4683 Add float_ts primary key
0.1.7 2021-06-25 3978 Release Slack CDK Connector