jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-20 10:32:35 -05:00
Code Issues Packages Projects Releases Wiki Activity
Files
yarden/source-mongodb-support-dynamic-arrays
airbyte/docs/integrations/sources/amazon-ads.md
devin-ai-integration[bot] 9ce132dfd2 feat(source-amazon-ads): Increase max concurrency to 20 and add rate limits documentation (#69836) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2025-12-11 17:16:25 +02:00

41 KiB
Raw Permalink Blame History

Amazon Ads

This page contains the setup guide and reference information for the Amazon Ads source connector.

Prerequisites

  • Client ID
  • Client Secret
  • Refresh Token
  • Region
  • Start Date (Optional)
  • Profile IDs (Optional)
  • Marketplace IDs (Optional)

Setup guide

Step 1: Set up Amazon Ads

Create an Amazon user with access to an Amazon Ads account.

For Airbyte Open Source: To use the Amazon Ads API, you must first complete the onboarding process. The onboarding process has several steps and may take several days to complete. After completing all steps you will have to get the Amazon client application's Client ID, Client Secret and Refresh Token.

Step 2: Set up the Amazon Ads connector in Airbyte

For Airbyte Cloud:

  1. Log into your Airbyte Cloud account.
  2. Click Sources and then click + New source.
  3. On the Set up the source page, select Amazon Ads from the Source type dropdown.
  4. Enter a name for the Amazon Ads connector.
  5. Click Authenticate your Amazon Ads account.
  6. Log in and Authorize to the Amazon account.
  7. Select Region to pull data from North America (NA), Europe (EU), Far East (FE). See docs for more details.
  8. Start Date (Optional) is used for generating reports starting from the specified start date. This should be in YYYY-MM-DD format and not more than 90 days in the past. If a date is not specified, today's date is used. The date is treated in the timezone of the processed profile.
  9. Profile IDs (Optional) you want to fetch data for. The Amazon Ads source connector supports only profiles with seller and vendor type, profiles with agency type will be ignored. See docs for more details.
  10. Marketplace IDs (Optional) you want to fetch data for. Note: If Profile IDs are also selected, profiles will be selected if they match the Profile ID OR the Marketplace ID.
  11. Click Set up source.

For Airbyte Open Source:

  1. Navigate to the Airbyte Open Source dashboard.
  2. Set the name for your source.
  3. Client ID of your Amazon Ads developer application. See onboarding process for more details.
  4. Client Secret of your Amazon Ads developer application. See onboarding process for more details.
  5. Refresh Token. See onboarding process for more details.
  6. Select Region to pull data from North America (NA), Europe (EU), Far East (FE). See docs for more details.
  7. Start Date (Optional) is used for generating reports starting from the specified start date. This should be in YYYY-MM-DD format and not more than 90 days in the past. If a date is not specified, yesterday's date is used. The date is treated in the timezone of the processed profile.
  8. Profile IDs (Optional) you want to fetch data for. The Amazon Ads source connector supports only profiles with seller and vendor type, profiles with agency type will be ignored. See docs for more details.
  9. Marketplace IDs (Optional) you want to fetch data for. Note: If Profile IDs are also selected, profiles will be selected if they match the Profile ID OR the Marketplace ID.
  10. Click Set up source.

For Airbyte Open Source:

  1. Client ID of your Amazon Ads developer application. See onboarding process for more details.
  2. Client Secret of your Amazon Ads developer application. See onboarding process for more details.
  3. Refresh Token. See onboarding process for more details.
  4. Select Region to pull data from North America (NA), Europe (EU), Far East (FE). See docs for more details.
  5. Start Date (Optional) is used for generating reports starting from the specified start date. This should be in YYYY-MM-DD format and not more than 90 days in the past. If a date is not specified, today's date is used. The date is treated in the timezone of the processed profile.
  6. Profile IDs (Optional) you want to fetch data for. The Amazon Ads source connector supports only profiles with seller and vendor type, profiles with agency type will be ignored. See docs for more details.
  7. Marketplace IDs (Optional) you want to fetch data for. Note: If Profile IDs are also selected, profiles will be selected if they match the Profile ID OR the Marketplace ID.

:::note The Amazon Ads source connector uses Sponsored Products, Sponsored Brands, and Sponsored Display APIs which are not compatible with agency account type. See docs for more details. If you have only agency profile, please use accounts associated with the profile of seller/vendor type. :::

Supported sync modes

The Amazon Ads source connector supports the following sync modes:

  • Full Refresh
  • Incremental

Supported Streams

This source is capable of syncing the following streams:

:::note As of connector version 5.0.0, the Sponsored Products Ad Group Bid Recommendations stream provides bid recommendations and impact metrics for an existing automatic targeting ad group. The stream returns bid recommendations for match types CLOSE_MATCH, LOOSE_MATCH, SUBSTITUTES, and COMPLEMENTS per theme. For more detail on theme-based bid recommendations, review Amazon's Theme-base bid suggestions - Quick-start guide. :::

Connector-specific features and highlights

All the reports are generated relative to the target profile's timezone.

Campaign reports may sometimes have no data or may not be presenting in records. This can occur when there are no clicks or views associated with the campaigns on the requested day - details.

Report data synchronization only covers the last 90 days - details.

:::note The 'Reports' stream(s) by default will have timeUnit set to SUMMARY. If you would like more granularity, use the _daily versions of the report streams, which have timeUnit set to DAILY. More info about this can be found here.

Important limitation: Amazon may incorrectly detect duplicate report requests when syncing both summary and daily versions of the same report type simultaneously (e.g., sponsored_brands_v3_report_stream and sponsored_brands_v3_report_stream_daily). If you encounter this issue, create a separate source with only the needed report streams and set the "Number of concurrent workers" to 2 to ensure sequential processing. :::

Performance considerations

Information about expected report generation waiting time can be found here.

Rate Limits

The Amazon Ads API uses dynamic rate limiting that varies by region and system load. Rate limits are not publicly documented with specific numbers. For more details, see the Amazon Ads API Rate Limiting documentation.

Adjusting Concurrency Settings:

If you experience rate limiting errors (429 status codes) during syncs, decrease the "Number of concurrent workers" setting in your connector configuration to reduce the load on the API.

If you need better sync performance and are not experiencing rate limiting errors, you can increase the "Number of concurrent workers" setting (up to a maximum of 20) to improve throughput.

Data type map

Integration Type Airbyte Type
string string
int, float, number number
date date
datetime datetime
array array
object object

Changelog

Expand to review
Version Date Pull Request Subject
7.3.9 2025-12-11 69836 Increase max concurrency to 20 and add rate limits documentation
7.3.8 2025-11-25 69911 Update dependencies
7.3.7 2025-10-29 69040 Update dependencies
7.3.6 2025-10-21 68579 Raise report creation errors as customer config errors instead of system errors.
7.3.5 2025-09-30 66760 Update dependencies
7.3.4 2025-09-08 65992 Update to CDK v7
7.3.3 2025-09-09 66032 Update dependencies
7.3.2 2025-08-09 64654 Update dependencies
7.3.1 2025-07-19 60633 Update dependencies
7.3.0 2025-07-07 62839 Promoting release candidate 7.3.0-rc.1 to a main version.
7.3.0-rc.1 2025-06-26 61707 Convert to manifest-only format
7.2.3 2025-06-23 61652 Add Error Hanlding for 425 Errors With Report Streams & Configurable Concurrency Levels
7.2.2 2025-05-10 59332 Update dependencies
7.2.1 2025-04-28 55745 Enable max concurrent async job count configurability
7.2.0 2025-04-28 59121 Promoting release candidate 7.2.0-rc.2 to a main version & 7.2.0-rc.1 since they were merged at the same time.
7.2.0-rc.2 2025-04-23 58610 Used latest version of Python cdk (6.45.10)
7.2.0-rc.1 2025-04-22 55217 Add daily versions of reports streams & Auto-update start_date if past lookback window
7.1.7 2025-04-12 57591 Update dependencies
7.1.6 2025-04-05 57138 Update dependencies
7.1.5 2025-03-29 56554 Update dependencies
7.1.4 2025-03-22 56155 Update dependencies
7.1.3 2025-03-08 55352 Update dependencies
7.1.2 2025-02-24 53692 Added handling 401 error for Polling Requester
7.1.1 2025-03-01 54895 Update dependencies
7.1.0 2025-02-24 53692 Migrate stream Portfolios to Portfolios API v3
7.0.1 2025-02-22 52918 Update dependencies
7.0.0 2025-02-06 48449 Migrate *-report streams to low-code
6.2.7 2025-01-25 52210 Update dependencies
6.2.6 2025-01-18 51722 Update dependencies
6.2.5 2025-01-11 51239 Update dependencies
6.2.4 2025-01-04 50902 Update dependencies
6.2.3 2024-12-28 50478 Update dependencies
6.2.2 2024-12-21 50202 Update dependencies
6.2.1 2024-12-14 48229 Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64
6.2.0 2024-11-12 48116 Migrate REST streams to low-code
6.1.4 2024-11-12 48471 Bumped automatically in the pull request, please see PR description
6.1.3 2024-11-05 48343 Set is_resumable only for FullRefresh streams
6.1.2 2024-11-04 48138 Add error message for TooManyRequests exception
6.1.1 2024-11-04 48128 Fix date parse in report streams
6.1.0 2024-11-01 47940 Bump CDK to ^5
6.0.0 2024-10-28 47366 Migrate stream SponsoredDisplayReportStream to Amazon Ads Reports v3
5.0.20 2024-10-29 47032 Update dependencies
5.0.19 2024-10-12 46860 Update dependencies
5.0.18 2024-10-05 46451 Update dependencies
5.0.17 2024-09-28 45794 Update dependencies
5.0.16 2024-09-14 45548 Update dependencies
5.0.15 2024-09-07 45308 Update dependencies
5.0.14 2024-08-31 45051 Update dependencies
5.0.13 2024-08-24 44648 Update dependencies
5.0.12 2024-08-17 43845 Update dependencies
5.0.11 2024-08-12 43354 Fix download request for sponsored_products_report_stream
5.0.10 2024-08-10 42162 Update dependencies
5.0.9 2024-07-13 41876 Update dependencies
5.0.8 2024-07-10 41487 Update dependencies
5.0.7 2024-07-09 41143 Update dependencies
5.0.6 2024-07-06 40798 Update dependencies
5.0.5 2024-06-25 40403 Update dependencies
5.0.4 2024-06-21 39926 Update dependencies
5.0.3 2024-06-04 38962 [autopull] Upgrade base image to v1.2.1
5.0.2 2024-05-29 38737 Update authenticator to requests_native_auth package
5.0.1 2024-04-29 37655 Update error messages and spec with info about agency profile type.
5.0.0 2024-03-22 36169 Update SponsoredBrand and SponsoredProduct streams due to API endpoint deprecation
4.1.0 2024-03-19 36267 Pin airbyte-cdk version to ^0
4.0.4 2024-02-23 35481 Migrate source to YamlDeclarativeSource with custom check_connection
4.0.3 2024-02-12 35180 Manage dependencies with Poetry
4.0.2 2024-02-08 35013 Add missing field to sponsored_display_budget_rules stream
4.0.1 2023-12-28 33833 Updated oauth spec to put region, so we can choose oauth consent url based on it
4.0.0 2023-12-28 33817 Fix schema for streams: SponsoredBrandsAdGroups and SponsoredBrandsKeywords
3.4.2 2023-12-12 33361 Fix unexpected crash when handling error messages which don't have requestId field
3.4.1 2023-10-19 31599 Base image migration: remove Dockerfile and use the python-connector-base image
3.4.0 2023-06-09 25913 Add Stream DisplayCreatives
3.3.0 2023-09-22 30679 Fix unexpected column for SponsoredProductCampaigns and SponsoredBrandsKeywords
3.2.0 2023-09-18 30517 Add suggested streams; fix unexpected column issue
3.1.2 2023-08-16 29233 Add filter for Marketplace IDs
3.1.1 2023-08-28 29900 Add 404 handling for no associated with bid ad groups
3.1.0 2023-08-08 29212 Add T00030 tactic support for sponsored_display_report_stream
3.0.0 2023-07-24 27868 Fix attribution report stream schemas
2.3.1 2023-07-11 28155 Bugfix: validation error when record values are missing
2.3.0 2023-07-06 28002 Add sponsored_product_ad_group_suggested_keywords, sponsored_product_ad_group_bid_recommendations streams
2.2.0 2023-07-05 27607 Add stream for sponsored brands v3 purchased product reports
2.1.0 2023-06-19 25412 Add sponsored_product_campaign_negative_keywords, sponsored_display_budget_rules streams
2.0.0 2023-05-31 25874 Type portfolioId as integer
1.1.0 2023-04-22 25412 Add missing reporting metrics
1.0.6 2023-05-09 25913 Small schema fixes
1.0.5 2023-05-08 25885 Improve error handling for attribution_report(s) streams
1.0.4 2023-05-04 25792 Add availability strategy for basic streams (not including report streams)
1.0.3 2023-04-13 25146 Validate pk for reports when expected pk is not returned
1.0.2 2023-02-03 22355 Migrate products_report stream to API v3
1.0.1 2022-11-01 18677 Add optional config report_record_types
1.0.0 2023-01-30 21677 Fix bug with non-unique primary keys in report streams. Add asins_keywords and asins_targets
0.1.29 2023-01-27 22038 Set AvailabilityStrategy for streams explicitly to None
0.1.28 2023-01-18 19491 Add option to customize look back window value
0.1.27 2023-01-05 21082 Fix bug with handling: "Report date is too far in the past." - partial revert of #20662
0.1.26 2022-12-19 20662 Fix bug with handling: "Report date is too far in the past."
0.1.25 2022-11-08 18985 Remove "report_wait_timeout", "report_generation_max_retries" from config
0.1.24 2022-10-19 17475 Add filters for state on brand, product and display campaigns
0.1.23 2022-09-06 16342 Add attribution reports
0.1.22 2022-09-28 17304 Migrate to per-stream state.
0.1.21 2022-09-27 17202 Improved handling if known reporting errors
0.1.20 2022-09-08 16453 Increase report_wait_timeout 30 -> 60 minutes
0.1.19 2022-08-31 16191 Improved connector's input configuration validation
0.1.18 2022-08-25 15951 Skip API error "Tactic T00020 is not supported for report API in marketplace A1C3SOZRARQ6R3."
0.1.17 2022-08-24 15921 Skip API error "Report date is too far in the past."
0.1.16 2022-08-23 15822 Set default value for region if needed
0.1.15 2022-08-20 15816 Update STATE of incremental sync if no records
0.1.14 2022-08-15 15637 Generate slices by lazy evaluation
0.1.12 2022-08-09 15469 Define primary_key for all report streams
0.1.11 2022-07-28 15031 Improve report streams date-range generation
0.1.10 2022-07-26 15042 Update additionalProperties field to true from schemas
0.1.9 2022-05-08 12541 Improve documentation for Beta
0.1.8 2022-05-04 12482 Update input configuration copy
0.1.7 2022-04-27 11730 Update fields in source-connectors specifications
0.1.6 2022-04-20 11659 Add adId to products report
0.1.5 2022-04-08 11430 Add support OAuth2.0
0.1.4 2022-02-21 10513 Increasing REPORT_WAIT_TIMEOUT for supporting report generation which takes longer time
0.1.3 2021-12-28 8388 Add retry if recoverable error occurred for reporting stream processing
0.1.2 2021-10-01 6367 Add option to pull data for different regions. Add option to choose profiles we want to pull data. Add lookback
0.1.1 2021-09-22 6367 Add seller and vendor filters to profiles stream
0.1.0 2021-08-13 5023 Initial version

master

Reference in New Issue View Git Blame Copy Permalink