1
0
mirror of synced 2026-01-06 06:04:16 -05:00
Commit Graph

1541 Commits

Author SHA1 Message Date
Denis Davydov
922bf46e8f Source Hubspot: fix for property_history stream (#13596)
* #259 on call: Source Hubspot - fix for the property_history stream which did not emit any records

* #259 Source Hubspot: upd changelog

* #259 oncall: hubspot review fixes

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-09 10:21:08 +03:00
Denis Davydov
60f5f4ce46 Source Google Ads: add missing dependency (#13618)
* Source Google Ads: add missing dependency

* Source Google Ads: upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-09 09:12:36 +03:00
Serhii Chvaliuk
9a4582ad7d 🐛 Source Github: enable caching only for parent streams (#13558)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-09 07:21:10 +03:00
LiRen Tu
bb46941867 🎉 Databricks destination: rename to Databricks delta lake (#13630)
* Rename databricks connector

* Rename connector in the seed

* Update changelog with pr id

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-08 18:22:55 -07:00
Serhii Chvaliuk
38fa0fa8b3 🐛 Source Github: improve docs for GA (#13593)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-08 20:19:11 +03:00
Serhii Chvaliuk
5444317513 🐛 Source Github: fix different result from get_starting_point when reading by pages (#13531)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-08 20:06:06 +03:00
Baz
ec6a58c8a2 🐛 Source LinkedIn Ads: fixed issue with normalization on redshift (#13495) 2022-06-08 17:53:40 +03:00
pcsoni89
b4e43c2ddd 🎉 Destination MSSQL: added custom JDBC parameters support. (#13054)
* 🎉 Destination MSSQL: added custom JDBC parameters support.

* 🎉  Destination MSSQL: added custom JDBC parameters support.

* 🎉  Destination MSSQL: added custom JDBC parameters support.

* airbyte-101718: Bumped version.

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Alexander Tsukanov <alexander.tsukanovvv@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-08 14:55:45 +03:00
LiRen Tu
fd2048da53 🎉 Postgres source: publish new version with JDBC fetch size v2 (#13576)
* Update postgres doc

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 19:01:50 -07:00
LiRen Tu
e996c2c14f 🐛 BigQuery destination: always run GCS destination check for GCS staging method (#13579)
* Run gcs destination check all the time

* Add missing permissions to the status message

* Add more error messages

* Bump bigquery version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 17:52:49 -07:00
LiRen Tu
545a7a3eb6 🎉 JDBC source: adjust fetch size based on max memory and max row size (#13435)
* Switch to measure max row byte size

* Reduce fetch size change logs

* Update unit tests

* Determine jdbc buffer size based on max memory

* Bump postgres version

* Bump postgres version

* Bump mysql version

* Bump mssql version

* Format java code

* Increase hikari connection timeout

* Update data source default parameters

* auto-bump connector version

* Mark postgres 0.4.21 as not published

* Revert "Bump mysql version"

This reverts commit ad9135258c.

* Fix unit test

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 14:29:34 -07:00
Alexandre Girard
f1d79b201f Source Hubspot: report which scopes are missing to user (#13566)
* Report missing scopes

* simplify

* bump

* emit trace message

* remove print

* more careful error handling

* Add typing

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 14:10:03 -07:00
Yurii Bidiuk
d6b50c1ce0 Destination BigQuery: refactored spec and doc (#13424)
* change spec

* refactored readme file for bigquery

* change spec order

* bump version
2022-06-07 18:36:40 +03:00
Ganpat Agarwal
1721827814 🎉 Source Amazon Seller Partner: add financial streams (#13364)
* add new schemas

* add catalog in integration tests

* add streams for ListFinancialEvents and ListFinancialEventGroups

* bump to version 0.2.21

* update docs

* add unit tests

* expand schema up to 2 levels for better normalisation

* fix unit test

* lint fix

* add missing attribute

* chore: bump version in source definitions

* chore: update seed file

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-06-07 18:26:51 +05:30
midavadim
0796b35e55 🎉 Source Stripe - to ga (#13449)
* Added incremental sync support (filtering by date - expired_at attr) for CheckourSessions stream and CheckoutSessionsLineItems
Added unittests

* updated docs

* updated connector version

* airbyte config

* auto-bump connector version

* updated connector version in config

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 01:55:09 +03:00
Yurii Bidiuk
4b266c1b7d 🐛 Source Postgres: write hstore as json (#13367)
* src-postgres: write hstore as json

* fix PR remarks

* bump version

* fix PR comments

* add text blocks
2022-06-06 22:25:29 +03:00
Serhii Chvaliuk
7800b52f4d 🐛 Source Hubspot: scope crm.objects.feedback_submissions.read added for feedback_submissions stream (#13475)
* crm.objects.feedback_submissions.read added

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* hubspot.md updated

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-06 14:11:01 +03:00
Marcos Marx
9aa0220820 revert redshift and json operations (#13465) 2022-06-03 19:44:40 -03:00
Alexandre Girard
1225f80bd8 Source hubspot: filter out streams we don't have access to (#13455)
* filter out streams we don't have access to

* cleanup

* fix associations

* fix for api key auth

* this doesn't need to be a property method

* Bump

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-03 12:06:51 -07:00
Adam
071a8e9dc8 Destination Redshift: fixed array contents verification for SUPER (#13069)
* add array handling to json flatten

* bump version, add changelog entry

* test: add flatten unit tests

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-03 15:46:16 -03:00
Baz
94dc19168c 🐛 Source Amplitude: Fixed JSON Validator date-time validation (#13373) 2022-06-03 17:41:53 +03:00
Luis Gomez
75e4114eb1 🎉 Source Freshdesk: Added new streams (#13332) 2022-06-03 14:55:35 +02:00
Alexandre Girard
19a20f0efc 🐛 Source Google Sheets: Retry on server errors (#13446)
* Retry on server errors

* retry on any 5XX

* bump version

* rename function for clarity

* reset

* delete unused code

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-02 18:28:45 -07:00
Serhii Chvaliuk
537c9bab0e 🐛 Source HubSpot: Transform contact_lists data to comply with schema (#13218)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-02 23:07:07 +03:00
Denis Davydov
a7b6487189 Source Google Ads: drop schema field that filters out the data from stream (#13423)
* #12999 drop schema field that filters out the data from stream

* #12999 up changelog

* #12999 source google ads: add note to the doc

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-02 21:42:17 +03:00
Marcos Marx
b5b43d3fa1 Source MSSQL: correct enum Standard method (#13419)
* correct enum MSSQL source

* bump strict version and update doc

* correct expect spec test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-02 14:16:47 -03:00
LiRen Tu
9648a5ceb6 Update postgres doc about cdc publication (#13433)
* Update postgres doc about cdc publication

* Update wording
2022-06-02 09:25:22 -07:00
Serhii Chvaliuk
d0d74ec730 🎉Source Hubspot: Fix connector incremental state handling (#13320)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-06-02 18:26:55 +03:00
Serhii Lazebnyi
bdf6ba8463 🎉Source Hubspot: Associated line items to deals (#13383)
* Associated line items to deals

* Bumped version

* Bumped version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-02 14:54:27 +03:00
Roman
d79b319819 🎉 Source Mixpanel: Beta preparation (#13372)
* Add extra mode to Source, to allow run acceptance tests
* move streams into distinct modules
* Add property name transformation for Export stream for avoiding collisions
* Update doc
* Add `date_window_size`
2022-06-02 13:31:23 +03:00
Alexandre Girard
1b6779d619 [OC-252] Hubspot Source: Request optional scopes (#13381)
* Set optional scopes

* update test

* required scopes

* fix

* Add a comment

* try empty scopes

* remove scopes to ensure it fails

* Revert "remove scopes to ensure it fails"

This reverts commit ba65642945.

* reorder

* set at least one required scope

* dont check for authorized scopes

* update docs

* Don't fail on missing scopes

* Revert "Don't fail on missing scopes"

This reverts commit 43ee93af9d.

* bump docker tag

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-01 19:35:06 -07:00
Denis Davydov
6442ad6f33 🐛 Source Zendesk Support: use CDK v.0.1.60 to handle ConnectionResetError, increase test coverage (#13261) 2022-06-01 13:50:50 +03:00
VitaliiMaltsev
0a56b1d43f Postgres Source: add timezone awareness and handle BC dates (#13166)
* Postgres Source: add timezone awareness and handle BC dates

* fixed checkstyle

* add tests

* updated changelog

* removed star import

* fixed tests

* refactoring

* removed star import

* fixed bytea type

* created final static constants

* bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-01 10:46:39 +03:00
VitaliiMaltsev
748291bb0c Snowflake Destination apply buffering strategy on GCS staging (#12952)
* Destination Snowflake apply buffering strategy for GCS

* Snowflake Destination apply buffering strategy on GCS staging

* updated CHANGELOG

* fixed checkstyle

* fixed compile error

* refactoring

* bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-01 10:28:02 +03:00
Serhii Lazebnyi
f9348b2251 🐛 Source Amazon S3: solve possible case of files being missed during incremental syncs (#12568)
* Added history to state

* Deleted unused import

* Rollback abnormal state file

* Rollback abnormal state file

* Fixed type error issue

* Fix state issue

* Updated after review

* Bumped version
2022-05-31 21:39:10 +03:00
Topher Lubaway
0c5cdc7e66 Adds redirects to the sidebars (#13334)
fixes two HTML tag like errors docusaurs isn't into
2022-05-31 09:51:32 -05:00
Vladimir remar
76032e66dd Source Facebook Marketing: Add lookback window to insights streams (#12402) 2022-05-31 09:37:37 +02:00
Ahmed Buksh
c55f185109 🚀 Source Klaviyo: New Stream addition along with update to existing (#11685)
* 🚀 flow stream added along with flow, campaign and flow message addition to event

*  liniting fix

* 🔨 annotations updated along with update method for klaviyo

* 💥 docker version updated and log added

* 🔨 fixed acceptance test for klaviyo flow stream

* 🔨 unused import removed

* fix: flows stream has no records thus tests are failing

* chore: update seed file

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-05-31 12:12:46 +05:30
Mainara Cavalcanti
1d1476e493 Jira Source: Added resolutiondate to fields on issues stream (#13202)
* Added resolutiondate to fields on issues stream (jira source)

* updated jira source changelog

* updated dockerfile version

* correct jira ci

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-05-30 17:49:20 -03:00
Michael Nguyen
ab81e4d1a0 Source Mixpanel: Added "end_date" field in specs (#12801)
* Added end_date for source mixpanel

* Update airbyte-integrations/connectors/source-mixpanel/source_mixpanel/spec.json

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* add end_date

* auto-bump connector version

Co-authored-by: Cheryl0402 <yi.cheryl.chen@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-30 16:40:27 -03:00
Misko Lee
721b5b69c7 Downgrade ElasticSearch batch size from 256mib to 32mib (#11752)
* Update ElasticsearchAirbyteMessageConsumerFactory.java

* add changleog and bump connector version seed file

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-05-30 14:37:32 -03:00
Ganpat Agarwal
2293f897b8 🎉 Source Amazon Seller Partner: add replication end date to config (#13059) 2022-05-30 19:23:08 +02:00
Anna Lvova
1dbd7ab99c Source Amazon Seller Partner: OAuth support (#13119)
* Update docs

* add oauth

* add spec.py

* test

* fix

* fix

* add lwa_app_id

* test

* test

* fix

* add beta version

* fix

* test

* test

* test

* merge

* fix

* fix

* fix

* test

* test

* test

* test

* sort AWSRegion

* up

* add import

* up

* up

* up

* fix

* fix tests

* bump version

* upd

* auto-bump connector version

* format

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-30 16:24:10 +03:00
Zawar Khan
d6d52c5d99 🎉 New Source: RKI (Robert Koch-Institut) Covid Public API (#11732)
* Added source for RKI-covid-germany, updated spec.json, implemented source with check and discover method added germany.json.

* implemented incremental method for germany history cases with date as parameters, updated streams, added cursor field for incremental streams.

* main file added.

* added Incidence source with date as parameter. Incremental stream. spec, source, schemas updated, added class GermanHistoryIncidence added.

* Added a full-refresh stream for germany/age-group .Added incremental streams for deaths/:days, recovered/:days, frozen-incidence/:days, hospitalization/:days. Updated source.py methods. Updated sepc.json properties key. Updated configured_catalogue.json with required streams. Updated config.json

* writting test unit test cases for incremental streams: german history cases and german history incidence.

* Incremental streams for germanhistorydeaths, germanhistoryfrozenIncidence, germanhistoryhospitalization, germanhistoryrecovered. Fixing other test cases.

* Added test stream for age group and germany.

* changes in Readme and source.py.

* IncrementalMixin added to class GermanyHistoryCases. AFter review IncrementalMixin will be implemented to all incremental classes.

* Implemented Incremental mixin or GermanHistory Cases connector.

* corrected changes.

* adding integration test

* comment acceptence test.

* On path with master

* updated the changes for PR request.

* changed file source-rki-covid/integration_tests/configured_catalog.json

* corrected flake and blackformatting. Build gradel.

* Worked on the suggestions.

* source_rki_covid/schemas/germany_age_groups.json

* uodated abnormal_state.json

* updated the schemas for german age groups and history hospitalization.

* correct dockerfile and update airbyte_cdk version

* run format

* update python version

* correct dockerfile build

* add source in seed

* update uuid for rki-covid source

* change docker

* add bash

* auto-bump connector version

* run seed file

* correct doc

* auto-bump connector version

Co-authored-by: Marcos Marx Millnitz <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-30 10:02:02 -03:00
Serhii Chvaliuk
ad1fd2b742 🐛 Source Hubspot: use RFC3339 datetime (#13159)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-05-30 15:54:13 +03:00
Luis Gomez
9d1cd42ff9 🎉 Source Freshdesk: Migrated to latest CDK (#12334) 2022-05-30 12:28:50 +02:00
Serhii Lazebnyi
3bddf875a6 Refactored docs (#13306) 2022-05-30 13:13:55 +03:00
Yevhen Sukhomud
da88158c2f 🎉 New SFTP source connector (#13120)
* 11797 New SFTP source
* 11797 Bump sftp source version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-30 14:28:13 +07:00
tirth7777777
83b9b4fca8 Source Kafka: added option for giving polling time when consuming records using consumer (#12903)
* added option for giving polling time using spec

* add doc and seed file

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-05-29 16:22:02 -03:00
Yurii Bidiuk
16de42fde3 🎉Source Postgres (CDC) - add support of tsquery type (#13083)
* cdc - add support of tsquery type

* rebase on master

* Add changes made by the build

* Update version in Dockerfile

* auto-bump connector version

Co-authored-by: grishick <greg@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-28 22:03:59 -07:00