jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-21 11:01:41 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

71 Commits

Author SHA1 Message Date
Aaron ("AJ") Steers
4cde75568c feat: Add external documentation URLs to Group I source connectors (source-outbrain-amplify through source-pypi) (#69736) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2025-11-18 18:10:01 -08:00
octavia-bot-hoard[bot]
749468bf8a 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-10-14] (#67754) 
# Update source-outbrain-amplify

This PR was autogenerated by running `airbyte-ci connectors
--name=source-outbrain-amplify up_to_date --pull`

We've set the `auto-merge` label on it, so it will be automatically
merged if the CI pipelines pass.
If you don't want to merge it automatically, please remove the
`auto-merge` label.
Please reach out to the Airbyte Connector Tooling team if you have any
questions or concerns.


## Operations

- Upgrade the base image to the latest version in metadata.yaml: Skipped

- Update versions of libraries in poetry: Successful

- PATCH bump source-outbrain-amplify version to 0.2.21: Successful

- Build source-outbrain-amplify docker image for platform(s)
linux/amd64, linux/arm64: Successful

- Get dependency updates: Successful

- Create or update pull request on Airbyte repository: Successful

- Add changelog entry: Successful




## Dependency updates

We use [`syft`](https://github.com/anchore/syft) to generate a SBOM for
the latest connector version and the one from the PR.
It allows us to spot the dependencies that have been updated at all
levels and for all types of dependencies (system, python, java etc.).
Here are the dependencies that have been updated compared to
`airbyte/source-outbrain-amplify:latest`.
Keep in mind that `:latest` does not always match the connector code on
the main branch.
It is the latest released connector image when the head commit of this
branch was created.

| Type | Name | State | Previous Version | New Version |
|------|------|-------|-------------|------------------|
| python | MarkupSafe | updated | 3.0.2 | **3.0.3** |
| python | PyYAML | updated | 6.0.2 | **6.0.3** |
| python | attrs | updated | 25.3.0 | **25.4.0** |
| python | cachetools | updated | 6.1.0 | **6.2.1** |
| python | cattrs | updated | 25.1.1 | **25.3.0** |
| python | certifi | updated | 2025.8.3 | **2025.7.9** |
| python | charset-normalizer | updated | 3.4.3 | **3.4.4** |
| python | idna | updated | 3.10 | **3.11** |
| python | platformdirs | updated | 4.3.8 | **4.4.0** |
| python | pydantic | updated | 1.10.22 | **1.10.24** |
| python | requests | updated | 2.32.4 | **2.32.5** |
| python | typing_extensions | updated | 4.14.1 | **4.15.0** |
| python | source-outbrain-amplify | removed | 0.2.15 | **not present**
|

> [!IMPORTANT]
> **Auto-merge enabled.**
> 
> _This PR is set to merge automatically when all requirements are met._

Co-authored-by: octavia-bot-hoard[bot] <230633153+octavia-bot-hoard[bot]@users.noreply.github.com>
 2025-10-14 16:31:33 -04:00
Anatoli Makarevich
eba7941311 fix: fixed broken Outbrain connector pagination (#67079) 
## What
<!--
* Describe what the change is solving. Link all GitHub issues related to
this change.
-->

I recently noticed that our Outbrain connector doesn't return all
campaigns and promoted links for some marketers (Outbrain ad accounts).

After quick debugging and building a script to download the data with a
fresh API token I figured that the problem is in Airbyte connector.

This PR fixes broken pagination for the Outbrain connector.

Issue: https://github.com/airbytehq/airbyte/issues/67080

## How
<!--
* Describe how code changes achieve the solution.
-->

By adjusting the limit/offset logic in the connector, it was clearly
broken.

## Review guide
<!--
1. `x.py`
2. `y.py`
-->

Ideally you want to download marketers/campaigns/promoted links and
promoted links performance to make sure.

I did this by running:

```bash
python main.py read --config secrets/config.json --catalog secrets/catalog.json
```

with an actual access token and the following `secrets/catalog.json`
file:

```json
{
  "streams": [
    {
      "stream": {
        "name": "marketers",
        "json_schema": {},
        "supported_sync_modes": ["full_refresh"]
      },
      "sync_mode": "full_refresh",
      "destination_sync_mode": "append_dedup"
    },
    {
      "stream": {
        "name": "campaigns",
        "json_schema": {
          "$schema": "http://json-schema.org/schema#",
          "type": "object",
          "additionalProperties": true,
          "properties": {
            "id": {
              "type": ["string", "null"]
            },
            "name": {
              "type": ["string", "null"]
            },
            "enabled": {
              "type": ["boolean", "null"]
            },
            "creationTime": {
              "type": ["string", "null"]
            },
            "lastModified": {
              "type": ["string", "null"]
            },
            "cpc": {
              "type": ["number", "null"]
            },
            "autoArchived": {
              "type": ["boolean", "null"]
            },
            "minimumCpc": {
              "type": ["number", "null"]
            },
            "currency": {
              "type": ["string", "null"]
            },
            "targeting": {
              "type": "object",
              "properties": {
                "platform": {
                  "type": "array",
                  "items": {
                    "type": ["string", "null"]
                  }
                },
                "language": {
                  "type": ["string", "null"]
                },
                "excludeAdBlockUsers": {
                  "type": ["boolean", "null"]
                },
                "nativePlacements": {
                  "type": "object",
                  "properties": {
                    "enabled": {
                      "type": ["boolean", "null"]
                    }
                  }
                },
                "includeCellularNetwork": {
                  "type": ["boolean", "null"]
                },
                "nativePlacementsEnabled": {
                  "type": ["boolean", "null"]
                },
                "locationsVersion": {
                  "type": ["string", "null"]
                }
              }
            },
            "marketerId": {
              "type": ["string", "null"]
            },
            "autoExpirationOfAds": {
              "type": ["integer", "null"]
            },
            "contentType": {
              "type": ["string", "null"]
            },
            "budget": {
              "type": "object",
              "properties": {
                "id": {
                  "type": ["string", "null"]
                },
                "name": {
                  "type": ["string", "null"]
                },
                "shared": {
                  "type": ["boolean", "null"]
                },
                "amount": {
                  "type": ["number", "null"]
                },
                "currency": {
                  "type": ["string", "null"]
                },
                "creationTime": {
                  "type": ["string", "null"]
                },
                "lastModified": {
                  "type": ["string", "null"]
                },
                "startDate": {
                  "type": ["string", "null"]
                },
                "endDate": {
                  "type": ["string", "null"]
                },
                "runForever": {
                  "type": ["boolean", "null"]
                },
                "type": {
                  "type": ["string", "null"]
                },
                "pacing": {
                  "type": ["string", "null"]
                }
              }
            },
            "prefixTrackingCode": {
              "type": "object",
              "properties": {
                "prefix": {
                  "type": ["string", "null"]
                },
                "encode": {
                  "type": ["boolean", "null"]
                }
              }
            },
            "liveStatus": {
              "type": "object",
              "properties": {
                "onAirReason": {
                  "type": ["string", "null"]
                },
                "campaignOnAir": {
                  "type": ["boolean", "null"]
                },
                "amountSpent": {
                  "type": ["number", "null"]
                },
                "onAirModificationTime": {
                  "type": ["string", "null"]
                }
              }
            },
            "readonly": {
              "type": ["boolean", "null"]
            },
            "startHour": {
              "type": ["string", "null"]
            },
            "trackingPixels": {
              "type": "object",
              "properties": {
                "enabled": {
                  "type": ["boolean", "null"]
                },
                "urls": {
                  "type": "array",
                  "items": {
                    "type": ["string", "null"]
                  }
                }
              }
            },
            "pixels": {
              "type": "object",
              "properties": {
                "impressionPixels": {
                  "type": "array",
                  "items": {
                    "type": ["string", "null"]
                  }
                },
                "trackingPixels": {
                  "type": "array",
                  "items": {
                    "type": ["string", "null"]
                  }
                }
              }
            },
            "onAirType": {
              "type": ["string", "null"]
            },
            "objective": {
              "type": ["string", "null"]
            },
            "creativeFormat": {
              "type": ["string", "null"]
            },
            "dynamicRetargeting": {
              "type": ["boolean", "null"]
            }
          }
        },
        "supported_sync_modes": ["full_refresh"]
      },
      "sync_mode": "full_refresh",
      "destination_sync_mode": "append_dedup"
    },
    {
      "stream": {
        "name": "promoted_links",
        "json_schema": {
          "$schema": "http://json-schema.org/schema#",
          "type": "object",
          "additionalProperties": true,
          "properties": {
            "id": {
              "type": ["string", "null"]
            },
            "text": {
              "type": ["string", "null"]
            },
            "creationTime": {
              "type": ["string", "null"]
            },
            "lastModified": {
              "type": ["string", "null"]
            },
            "url": {
              "type": ["string", "null"]
            },
            "siteName": {
              "type": ["string", "null"]
            },
            "sectionName": {
              "type": ["string", "null"]
            },
            "status": {
              "type": ["string", "null"]
            },
            "enabled": {
              "type": ["boolean", "null"]
            },
            "campaignId": {
              "type": ["string", "null"]
            },
            "archived": {
              "type": ["boolean", "null"]
            },
            "documentLanguage": {
              "type": ["string", "null"]
            },
            "onAirStatus": {
              "type": "object",
              "properties": {
                "onAir": {
                  "type": ["boolean", "null"]
                },
                "reason": {
                  "type": ["string", "null"]
                }
              }
            },
            "baseUrl": {
              "type": ["string", "null"]
            },
            "documentId": {
              "type": ["string", "null"]
            },
            "metaData": {
              "type": ["string", "null"]
            },
            "approvalStatus": {
              "type": "object",
              "properties": {
                "status": {
                  "type": ["string", "null"]
                },
                "isEditable": {
                  "type": ["boolean", "null"]
                }
              }
            },
            "description": {
              "type": ["string", "null"]
            },
            "callToAction": {
              "type": "object",
              "properties": {
                "type": {
                  "type": ["string", "null"]
                },
                "value": {
                  "type": ["string", "null"]
                }
              }
            },
            "imageType": {
              "type": ["string", "null"]
            },
            "language": {
              "type": ["string", "null"]
            }
          }
        },
        "supported_sync_modes": ["full_refresh"]
      },
      "sync_mode": "full_refresh",
      "destination_sync_mode": "append_dedup"
    },
    {
      "stream": {
        "name": "performance_promoted_links",
        "json_schema": {
          "$schema": "http://json-schema.org/schema#",
          "type": "object",
          "properties": {
            "metadata": {
              "type": "object",
              "properties": {
                "id": {
                  "type": ["string", "null"]
                },
                "fromDate": {
                  "type": ["string", "null"]
                },
                "toDate": {
                  "type": ["string", "null"]
                }
              }
            },
            "metrics": {
              "type": "object",
              "properties": {
                "impressions": {
                  "type": ["number", "null"]
                },
                "clicks": {
                  "type": ["number", "null"]
                },
                "totalConversions": {
                  "type": ["number", "null"]
                },
                "conversions": {
                  "type": ["number", "null"]
                },
                "viewConversions": {
                  "type": ["number", "null"]
                },
                "spend": {
                  "type": ["number", "null"]
                },
                "ecpc": {
                  "type": ["number", "null"]
                },
                "ctr": {
                  "type": ["number", "null"]
                },
                "conversionRate": {
                  "type": ["number", "null"]
                },
                "viewConversionRate": {
                  "type": ["number", "null"]
                },
                "cpa": {
                  "type": ["number", "null"]
                },
                "totalCpa": {
                  "type": ["number", "null"]
                },
                "totalSumValue": {
                  "type": ["number", "null"]
                },
                "sumValue": {
                  "type": ["number", "null"]
                },
                "viewSumValue": {
                  "type": ["number", "null"]
                },
                "totalAverageValue": {
                  "type": ["number", "null"]
                },
                "averageValue": {
                  "type": ["number", "null"]
                },
                "viewAverageValue": {
                  "type": ["number", "null"]
                },
                "totalRoas": {
                  "type": ["number", "null"]
                },
                "roas": {
                  "type": ["number", "null"]
                },
                "videoReachedFirstQ": {
                  "type": ["integer", "null"]
                },
                "videoReachedSecondQ": {
                  "type": ["integer", "null"]
                },
                "videoReachedThirdQ": {
                  "type": ["integer", "null"]
                },
                "videoReachedCompletion": {
                  "type": ["integer", "null"]
                },
                "videoViewDuration": {
                  "type": ["integer", "null"]
                },
                "videoAvgViewDuration": {
                  "type": ["integer", "null"]
                },
                "videoPlays": {
                  "type": ["integer", "null"]
                }
              }
            },
            "marketer_id": {
              "type": ["string", "null"]
            },
            "campaign_id": {
              "type": ["string", "null"]
            },
            "promoted_link_id": {
              "type": ["string", "null"]
            }
          }
        },
        "supported_sync_modes": ["full_refresh"]
      },
      "sync_mode": "full_refresh",
      "destination_sync_mode": "append_dedup"
    }
  ]
}
```

I see that we got all campaigns, promoted links and daily performance
records (we had 65 campaigns before, limited by the broken pagination):

<img width="2314" height="320" alt="Screenshot 2025-10-06 at 12 43 32"
src="https://github.com/user-attachments/assets/3b12234c-4fe4-40c0-99d9-bf71ad34be79"
/>

## User Impact
<!--
* What is the end result perceived by the user?
* If there are negative side effects, please list them. 
-->

💀 💣 Every user with at least 50 campaigns won't get the full data from
Airbyte connector.

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌

---------

Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
 2025-10-08 22:39:07 -04:00
octavia-bot-hoard[bot]
16be5bcae9 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-10-07] (#67354) 
Co-authored-by: octavia-bot-hoard[bot] <230633153+octavia-bot-hoard[bot]@users.noreply.github.com>
 2025-10-07 14:13:54 -04:00
octavia-bot-hoard[bot]
6590a09fc0 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-09-30] (#66383) 
Co-authored-by: octavia-bot-hoard[bot] <230633153+octavia-bot-hoard[bot]@users.noreply.github.com>
 2025-09-30 12:14:10 -04:00
Airbyte
43f41bbebe 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-09-09] (#65847) 2025-09-09 18:11:19 -04:00
Airbyte
13f34fed4d 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-08-23] (#65178) 2025-08-23 10:11:31 -04:00
Airbyte
73ebcd21e4 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-08-16] (#64979) 2025-08-16 20:06:26 +02:00
Airbyte
26907b201c 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-08-09] (#64678) 2025-08-09 20:32:30 +02:00
Airbyte
8ab16c513a 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-08-02] (#64286) 2025-08-02 18:07:19 +02:00
Charles
50a97db804 chore: migrate connectors from MIT to ELv2 (#63723) 2025-07-22 14:26:58 -07:00
Airbyte
73fbfcad82 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-07-19] (#63413) 2025-07-19 16:07:18 +02:00
Airbyte
6f722d43a6 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-07-12] (#63186) 2025-07-12 15:07:14 -07:00
Airbyte
e36dfdaa5e 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-07-05] (#62630) 2025-07-05 09:10:24 -07:00
Airbyte
87f686d4d5 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-06-28] (#62353) 2025-06-28 15:19:17 -07:00
Airbyte
da25f86eec 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-06-21] (#60489) 2025-06-21 15:12:16 -07:00
Airbyte
cf2bd7b340 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-05-10] (#60191) 2025-05-11 15:12:24 +03:00
Airbyte
d6e61be8c9 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-05-04] (#59511) 2025-05-04 09:12:58 +03:00
Airbyte
6583d22ef1 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-04-27] (#59100) 2025-04-27 15:11:56 +03:00
Airbyte
eccbbee47f 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-04-19] (#58502) 2025-04-20 09:12:33 +03:00
Airbyte
5cf0206c15 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-04-12] (#57901) 2025-04-13 06:08:36 +03:00
Airbyte
d75d6cf880 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-04-05] (#57318) 2025-04-06 05:40:48 +03:00
Airbyte
becf5d9278 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-03-29] (#56805) 2025-03-30 05:38:33 +03:00
FelixJetschiny
4017232035 ✨Source Outbrain Amplify: Add new optional parameter 'conversionsByClickDate' for performance reports (#55746) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
 2025-03-26 14:45:56 -03:00
Airbyte
24838f9883 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-03-22] (#56217) 2025-03-23 04:38:18 +02:00
Airbyte
1efa1b83c6 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-03-08] (#55067) 2025-03-09 02:30:44 +02:00
devin-ai-integration[bot]
74e610c8ea Update pytest to version 8.0+ across all packages (#54708) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: natik@airbyte.io <natik@airbyte.io>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
 2025-02-28 15:12:54 -08:00
Airbyte
c01002a5b7 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-02-23] (#54557) 2025-02-23 06:10:09 +02:00
Airbyte
671445e779 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-02-15] (#53963) 2025-02-18 07:51:16 +02:00
devin-ai-integration[bot]
ed19a158c4 chore: standardize Poe tasks configuration across all connectors (#53233) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Aaron <AJ> Steers <aj@airbyte.io>
 2025-02-07 15:26:48 -08:00
Airbyte
d5d7506ff5 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-02-01] (#53010) 2025-02-02 04:32:08 +02:00
Airbyte
1a3f28fb04 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-01-25] (#52461) 2025-01-26 04:30:18 +02:00
Airbyte
079e91cb65 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-01-18] (#51889) 2025-01-19 02:33:31 +02:00
Airbyte
cea094a213 🐙 source-outbrain-amplify: run up-to-date pipeline [2025-01-11] (#51319) 2025-01-12 02:35:42 +02:00
Airbyte
7e89b60b76 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-12-28] (#50746) 2024-12-29 04:35:07 +02:00
Airbyte
c844505c0d 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-12-21] (#50236) 2024-12-22 02:35:58 +02:00
Aaron ("AJ") Steers
83ecbe0fc3 CI: apply pre-commit format fix from #49806 (#49852) 2024-12-18 14:05:43 -08:00
Airbyte
1b21e3ba96 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-12-14] (#49715) 2024-12-15 04:40:20 +02:00
Airbyte
99cf9e157f 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-12-12] (#49065) 2024-12-13 02:35:00 +02:00
Airbyte
dca736a2a5 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-11-25] (#48634) 2024-11-25 14:14:30 +02:00
Airbyte
d27c72147a 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-10-29] (#47849) 2024-10-29 18:10:54 +02:00
Airbyte
580dae0b4f 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-10-28] (#47040) 2024-10-28 16:15:18 +02:00
Airbyte
0c92c8a9da 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-10-12] (#46775) 2024-10-13 03:21:21 +03:00
Airbyte
25094afb98 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-10-05] (#46403) 2024-10-06 03:22:06 +03:00
Airbyte
8a5f7aec7a 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-09-28] (#46195) 2024-09-29 03:19:00 +03:00
Airbyte
b85212a150 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-09-21] (#45832) 2024-09-22 03:18:07 +03:00
Airbyte
7eeddeeba4 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-09-14] (#45502) 2024-09-15 03:21:57 +03:00
Airbyte
e3e1b63312 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-09-07] (#45240) 2024-09-08 03:21:20 +03:00
Airbyte
bf39c39933 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-08-31] (#45024) 2024-09-01 03:20:13 +03:00
Airbyte
9e53abbae0 🐙 source-outbrain-amplify: run up-to-date pipeline [2024-08-24] (#44737) 2024-08-25 03:17:21 +03:00