docs(source-iterable): improve documentation clarity and fix broken issue link (#68168)
## What This PR improves the Iterable source connector documentation by: - Removing a broken link to internal issue #24968 that is not accessible to end users - Clarifying prerequisites and API key requirements - Adding detailed documentation about incremental sync behavior and error handling The changes were requested by ian.alton@airbyte.io to verify and improve the connector documentation. **Link to Devin run**: https://app.devin.ai/sessions/4570bbfaf450476782d6f5dfe1e30cab ## How The documentation updates include: 1. **Fixed broken issue reference** (line 77): Replaced link to non-existent issue #24968 with a clear explanation of the List Users error handling behavior 2. **Enhanced Prerequisites section** (lines 5-9): Clarified that only Server-side API keys with Standard permissions are supported (not Mobile or Browser keys) 3. **Added Performance and data retrieval section** (lines 77-91): - Documents 90-day interval slicing for Users stream with `profileUpdatedAt` cursor - Explains adaptive slicing for Event streams - Documents Templates stream incremental sync with `updatedAt` cursor - Details retry logic with exponential backoff (10 retries, 20-400 second delays) Changes were based on code review of: - `source_iterable/manifest.yaml` (Users stream configuration) - `source_iterable/streams.py` (retry logic, error handling) ## Review guide 1. **docs/integrations/sources/iterable.md** - Focus on: - Lines 83-84: Verify the 90-day interval and `profileUpdatedAt` cursor field are accurate for Users stream - Lines 91: Confirm retry delays (20-400 seconds) and max retries (10) match the implementation - Lines 89-90: Ensure List Users error handling description is accurate - Overall: Check if the new technical details are helpful for end users vs too implementation-specific ## User Impact **Positive:** - End users now have clearer guidance on API key requirements - Better understanding of how incremental syncs work and how much data will be requested at once - No longer confused by a broken link to an inaccessible internal issue **Neutral:** - More detailed technical information may be helpful for troubleshooting but could also be overwhelming ## Can this PR be safely reverted and rolled back? - [x] YES 💚 This is a documentation-only change with no code modifications. Reverting would simply restore the previous documentation state. Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: ian.alton@airbyte.io <ian.alton@airbyte.io>
This commit is contained in:
committed by
GitHub
parent
f4e0437c12
commit
1dbd573f50
@@ -4,7 +4,9 @@ This page contains the setup guide and reference information for the Iterable so
|
||||
|
||||
## Prerequisites
|
||||
|
||||
To set up the Iterable source connector, you'll need the Iterable [`Server-side` API Key with `standard` permissions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys-).
|
||||
To set up the Iterable source connector, you need:
|
||||
|
||||
- An Iterable [`Server-side` API Key with `Standard` permissions](https://support.iterable.com/hc/en-us/articles/360043464871-API-Keys-). Mobile and Browser API keys are not supported.
|
||||
|
||||
## Set up the Iterable connector in Airbyte
|
||||
|
||||
@@ -72,9 +74,21 @@ The Iterable source connector supports the following [sync modes](https://docs.a
|
||||
- [CustomEvent](https://api.iterable.com/api/docs#export_exportDataJson) \(Incremental\)
|
||||
- [HostedUnsubscribeClick](https://api.iterable.com/api/docs#export_exportDataJson) \(Incremental\)
|
||||
|
||||
## Additional notes
|
||||
## Performance and data retrieval
|
||||
|
||||
[List Users](https://api.iterable.com/api/docs#lists_getLists_0) Stream when meeting `500 - Generic Error` will skip a broken slice and keep going with the next one. This is related to unexpected failures when trying to get users list for specific list ids. See #[24968](https://github.com/airbytehq/airbyte/issues/24968) issue for more details.
|
||||
### Incremental sync behavior
|
||||
|
||||
Streams that support incremental sync use the following approaches:
|
||||
|
||||
- **Users stream**: Splits data retrieval into 90-day intervals to manage large data volumes efficiently. The stream uses the `profileUpdatedAt` field as the cursor for incremental syncs.
|
||||
- **Event streams** (Email, Push, SMS, In-App, Web Push, Inbox events): Use adaptive date range slicing to handle varying data volumes. The connector automatically adjusts slice sizes based on API response times and handles connection timeouts gracefully.
|
||||
- **Templates stream**: Supports incremental sync using the `updatedAt` field as the cursor.
|
||||
|
||||
### Error handling
|
||||
|
||||
The List Users stream handles `500 - Generic Error` responses gracefully by skipping the affected list and continuing to process remaining lists. This behavior prevents individual list errors from blocking the entire sync and is related to intermittent API failures when retrieving users for specific list IDs.
|
||||
|
||||
The connector implements retry logic with exponential backoff for rate limiting (HTTP 429) and server errors (HTTP 500-599), with up to 10 retries and delays ranging from 20 to 400 seconds.
|
||||
|
||||
## Changelog
|
||||
|
||||
|
||||
Reference in New Issue
Block a user