jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-01-05 21:02:13 -05:00
Code Issues Packages Projects Releases Wiki Activity
11,968 Commits 5,162 Branches 518 Tags
9f6963ccfcddd7a573c4ca0ff94de41ce6175cc0
Commit Graph

85 Commits

Author SHA1 Message Date
Denys Davydov
ccac29af1b 🚨 🚨 Source Amazon Ads: fix schemas (#27868) 
* Connector health: source hubspot, gitlab, snapchat-marketing: fix builds

* #1955 source amazon ads: fix stream schemas

* source amazon ads: upd changelog

* source amazon ads: upd CAT config

* upd backwards incompatible version

* update metadata with breakingChanges entry

* fix CATs

* fix md
 2023-07-24 16:29:42 +03:00
Roman Yermilov [GL]
aea5a9ec8e Source Amazon Ads: fix Nonetype error when recordId is missing (#28155) 
* Source Amazon Ads: fix Nonetype error when recordId is missing

* Source Amazon Ads: bump version and update changelog

* Source Amazon Ads: unittest for recordId generation
 2023-07-13 15:10:41 +04:00
midavadim
d1802154ae 🎉 Amazon Ads - new streams for bids and keyword recommendations (#28002) 
* added new streams for amazon adgroup bids keyword recommendations

* updated docs and configured catalog

* updated version in metadata

* fix test

* updated docs

* fix doc after merge

* fix comments

* updated version

* fux typo
 2023-07-06 13:18:33 -04:00
Anatolii Yatsuk
df135afa26 ✨ Source Amazon-Ads: Add streams for portfolios and sponsored brands v3 (#27607) 
* Add stream for sponsored brands v3
* Add new stream Portfolios
 2023-07-05 15:56:04 +03:00
midavadim
c7b2f99af5 🎉 Source Amazon Ads add campaign negative keywords stream (#27482) 
* Add sponsored_product_campaign_negative_keywords stream

* revert changes

* 🤖 Auto format source-amazon-ads code [skip ci]

* Add sponsored_display_budget_rules stream

* 🤖 Auto format source-amazon-ads code [skip ci]

* ignore dailyBudget field
it can be updated, also it is sometimes integer, sometimes float"

* Fix sponsored_product_campaigns

* Fix sponsored_product_campaigns

* merged with the latest version

* updated source test

---------

Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
 2023-06-23 17:31:20 +03:00
Roman Yermilov [GL]
80e6a2666b 🐛Source Amazon Ads: fix portfolioId type (#25874) 
* Source Amazon Ads: fix portfolioId type

* Source Amazon Ads: update metadata

* Source Amazon Ads: disable backward compatibility tests for 1.1.0

* 🤖 Auto format source-amazon-ads code [skip ci]

---------

Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
 2023-06-20 11:22:43 -05:00
Augustin
d8820d6e80 Fix python connectors formatting errors with black / isort (#26951) 
* format source-tiktok-marketing

* format source-twilio

* format source-instgram

* format source-amazon-ads

* format source-google-search-console

* format source-stripe

* format source-google-analytics-data-api

* format source-facebook-marketing

* format source-zendesk-support

* format source-file
 2023-06-02 09:26:04 -05:00
Ben Church
1dabc6208e Metadata: add tags field (#26320) 
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
 2023-05-26 16:13:09 -07:00
Marcos Marx
215abfe4a8 bump amazon ads medatafile file (#26657) 
Co-authored-by: Augustin <augustin@airbyte.io>
 2023-05-26 14:57:37 +02:00
Shishir Rajmohan Verma
26c5cc0d6d 🐛 Source Amazon Ads: Add missing reporting metrics to the source connector (#25412) 
* Add missing reporting metrics to source connector for Amazon Ads

* update changelog

* correct version
 2023-05-25 15:57:21 -03:00
Ben Church
248bbf94c1 HACKDAY: Icon CDN (#26158) 
* Move icons to connector folder

* Delete old icons

* Update upload logic

* Add icon url to definitions

* Update registry model

* Populate cdn url

* DNC butcher the pipeline

* Low hanging fruit fixes

* Fix bucket name

* Merge old and new approaches

* Fix metadata upload step

* Format

* Fix test
 2023-05-24 17:25:41 -07:00
Roman Yermilov [GL]
94d0d61248 Source Amazon Ads: get rid of fail_on_extra_columns: false in SAT (#25913) 
* Source Amazon Ads: small schema fixes

* Source Amazon Ads: update changelog

* Source Amazon Ads: update unittest

* Source Amazon Ads: unittest additional property is boolean

* Source Amazon Ads: bump version

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-16 21:47:33 +04:00
Serhii Chvaliuk
1cbd9252ba Source: Amazon Ads - Improve error handling for attribution_report(s) streams (#25885) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-05-16 17:43:26 +03:00
Roman Yermilov [GL]
c577e44633 ✨ Source Amazon Ads: add availability strategy for basic streams (#25792) 
* Source Amazon Ads: add availability strategy for basic streams

* auto-bump connector version

* Source Amazon Ads: update metadata.yaml

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-05-08 11:06:17 +04:00
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254) 
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
 2023-04-18 22:15:11 +02:00
Serhii Lazebnyi
ab784c5d90 Source: Amazon Ads - Add pk validator for reports when expected pk is not returned (#25146) 
* Add pk validator for reports when expected pk is not returned

* Update PR number

* Fix formating

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-04-13 19:06:29 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
Serhii Lazebnyi
d9a6ac9db8 Source Amazon Ads: Migrate products_report stream to API v3 (#22355) 
* Migrate products_report stream to API v3

* Updated PR number

* fix

Signed-off-by: Your Name <you@example.com>

* fix tests

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* Fixed unttests

* Fix Metric response

* Fix typo

* Fix integrations test tdue migration

* auto-bump connector version

---------

Signed-off-by: Your Name <you@example.com>
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-03-31 02:01:38 +02:00
Ella Rohm-Ensing
9cfc647fa5 Ella/fail on extra fields (#23985) 
* Pass argument along, add test that should pass and test that should fail

* Add tests with additionalProperties

* Set additionalproperties=false when not set|

* Parametrize test cases

* Make the behavior 'optional'

* Fix parametrization for all combinations

* Improve CI credentials README and rename param

* Update naming to be clearer about columns only

* record_has_unexpected_field > record_has_unexpected_column

* Automated Change

* Add bypass for beta + ga connectors that failed

* Update docs and TODOs

* Update changelog and dockerfile

* Update TODO

* Update a few neglected connectors

* Remove uploaded file

* Update dockerfile after merge conflict

---------

Co-authored-by: marcosmarxm <marcosmarxm@users.noreply.github.com>
 2023-03-23 22:22:44 +00:00
Augustin
bad5bce8ce CDK: remove unexpected error swallowing on abstract source's check method (#24240) 2023-03-23 13:04:51 +00:00
Ganpat Agarwal
2b81405f2f 🎉 Source Amazon Ads : use optional config report_record_types (#18677) 
* use optional config report_record_types

* update spec

* update image version

* update

* correct indentation

* remove lint changes

* add tests for sponsored products report

* update spec

* add test for video report

* increment amazon-ads connector version

* add test_strictness_level as high

* gradle format update

* update

* update tests for new METRIC_RESPONSE

* align validation and transformation

* bump connector version

* auto-bump connector version

---------

Co-authored-by: Juozas V <monai@cure.lt>
Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Nataly Merezhuk <65251165+natalyjazzviolin@users.noreply.github.com>
Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
Co-authored-by: sh4sh <6833405+sh4sh@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-03-14 15:43:04 -03:00
Serhii Lazebnyi
3d8f81a3e4 Connector Health: Fixed SAT for Amazon Ads, Faker, Hubspot, Instagram, Pinterest, Sentry, Zendesk Talk (#23600) 
* Fixed SAT for some connecotrs

* Fix unittest pinterest

* add allowedHosts

* Updated unittest for instagram

* Fix expected records instagram
 2023-03-01 10:09:19 +01:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084) 
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2023-02-24 16:13:42 -05:00
Augustin
fbd6dbf091 python-connectors: remove explicit dependencies to connector-acceptance-test build (#23332) 2023-02-22 21:21:38 +01:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594) 
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
 2023-02-08 13:01:16 -08:00
Roman Yermilov [GL]
7d13ad5d5d Source Amazon Ads: fix reports stream records primary keys (#21677) 
* Source Amazon Ads: fix reports stream records primary keys

* Source Amazon Ads: update changelog

* #1332 source Amazon Ads: review fixes

* #1332 source amazon ads: remove unused imports

* #1332 source amazon ads: bump major version

* auto-bump connector version

---------

Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-02-06 18:53:18 +02:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846) 
Rename source acceptance test to connector acceptance test
 2023-02-02 11:38:19 +01:00
Ella Rohm-Ensing
fe2ae98767 Turn off default httpavailabilitystrategy for source-amazon-ads (GA) (#22038) 
* Turn off default for source-amazon-ads (GA)

* update import

* Update Dockerfile

* Update amazon-ads.md

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2023-01-30 13:02:19 -05:00
Daniele Dapuzzo
13eac98309 🎉 Source Amazon Ads: add option to customize look back window value (#19491) 
* add new configurable field

* update docker image version

* amazon-ads.md updated

* remove unnecessary constant

* rollback changes inside seed files

* fully revert changes

* pre load lbw default value

* fix value access

* update config fixture

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2023-01-27 06:03:00 -03:00
Arsen Losenko
d7cbc790b8 Source Amazon Ads: enable SAT high strictness (#21646) 
* Source Amazon Ads: enable SAT high strictness

* Update SAT config, remove config_report.json for now
 2023-01-24 11:38:48 +01:00
Serhii Chvaliuk
e51853e046 Source Amazon Ads: fix bug with handling: "Report date is too far in the past." (partial revert of #20662) (#21082) 
* Revert "Source Amazon Ads: fix bug with handling: "Report date is too far in the past." (#20662)"

This reverts commit ec995959f7.

* fix

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* revert master

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* revert to master

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* bump 0.1.27

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* fix

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* amazon-ads.md updated

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* amazon-ads.md updated

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* auto-bump connector version

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
 2023-01-06 09:18:50 -06:00
Serhii Chvaliuk
ec995959f7 Source Amazon Ads: fix bug with handling: "Report date is too far in the past." (#20662) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-01-04 20:09:00 +02:00
Evan Tahler
e39e1898c5 Expected Records to .jsonl format (#20850) 
* Expected Records to `.jsonl` format

* fix formatting template

* remove endline

* update templates

* Update docs/connector-development/testing-connectors/source-acceptance-tests-reference.md

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
 2023-01-03 15:55:36 -08:00
Serhii Chvaliuk
70635632ce Source Amazon Ads: remove "report_wait_timeout", "report_generation_max_retries" from config (#18985) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-11-09 00:10:59 +02:00
darynaishchenko
02095d20da fixed acceptance tests (#18699) 2022-11-02 14:03:36 +02:00
Brian Lai
3521802545 Update Python Source connectors to use the latest airbyte-cdk minor version (#18261) 
* update all connectors currently only adhering to patch versions to pull latest minor version

* also add greenhouse

* fix scaffolds
 2022-10-20 16:06:02 -04:00
Sage Watterworth
0d9b4e5dce 🎉 Source Amazon Ads: filters for state on brand, product and display campaigns (#17475) 
* amazon ad status filter

* update test report stream with parametrized campaigns

* remove config files related to previous acceptance test. revert to master acceptance test set up

* amazon ad status filter

* update test report stream with parametrized campaigns

* remove config files related to previous acceptance test. revert to master acceptance test set up

* oct 17 edits

* oct 17 edits: 2

* bump dockerfile to 1.24

* fix: match the cdk version to new one

* auto-bump connector version

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Harshith Mullapudi <harshithmullapudi@interviewkickstart.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-10-20 12:50:55 -04:00
Sajarin
3154cfa68e Source Amazon Ads - bump dockerfile to 1.23 (#17906) 
* bump dockerfile to 1.23

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-10-19 12:47:04 -04:00
Serhii Chvaliuk
220513db96 Source Amazon Ads : SAT recover expected_records.txt (#17938) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-10-13 15:05:45 +03:00
Greg Solovyev
034b3fb100 Remove unpulbished version of source-amazon-ads from seed file (#17904) 2022-10-12 13:24:53 -07:00
Ganpat Agarwal
ca198ecf36 update amazon-ads version to 0.1.23 (#17870) 2022-10-12 14:27:44 -04:00
Evan Tahler
49cb3360de Remove redundant title labels from connector specs (#17544) 
* Remove redundant title labels from connector specs

* Manually update specs

* add env variable

* Remove debugging log
 2022-10-05 12:58:38 -07:00
Ganpat Agarwal
454f84d3ed 🎉 Source Amazon Ads : Add attribution reports (#16342) 
* use data field for json response

* add attribution reports

* update changelog

* add atrribution report integration test

* clean up expected_records
add empty streams to acceptance test config

* handle exception for profile

* update tests

Co-authored-by: Yiyang Li <yiyangli2010@gmail.com>
Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
Co-authored-by: Sajarin <sajarindider@gmail.com>
 2022-10-03 10:24:33 -04:00
Augustin
ff4ea3961a Republish connectors using CDK 0.1.88 to 0.1.89 (#17304) 2022-09-28 18:18:59 +02:00
Serhii Chvaliuk
35230f189e 🎉 Source Amazon Ads - improve handling reporting errors (#17202) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-09-27 16:45:14 +03:00
Serhii Chvaliuk
0ab777f23e 🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes (#16453) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-09-10 13:48:00 +03:00
Serhii Chvaliuk
13a238c2d5 🎉 Source Amazon Ads: improve config.start_date validation (#16191) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-09-05 13:03:35 +03:00
Serhii Chvaliuk
f962b8fdec 🐛 Source Amazon Ads: Skip API error "Tactic T00020 is not supported for report API in marketplace A1C3SOZRARQ6R3." (#15951) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-08-25 15:16:07 +03:00
Serhii Chvaliuk
f952ace006 🎉 Source Amazon Ads: Skip API error "Report date is too far in the past." (#15921) 
* _check_report_date_error added

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* amazon-ads.md updated

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* improve docs

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* improve logging, improve type hints

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* remove ReportInitFatal

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* amazon-ads.md updated

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* auto-bump connector version [ci skip]

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-08-25 08:38:50 +03:00