1
0
mirror of synced 2026-01-02 12:02:47 -05:00
Commit Graph

1973 Commits

Author SHA1 Message Date
Eugene
7ce39ed8a1 🐛Destination-snowflake: added connection string identifier (#17115)
* [16767] Destination-snowflake: added connection string identifier
2022-09-26 22:09:40 +03:00
Subodh Kant Chaturvedi
99746082a4 mysql source : implement support for snapshot of new tables in cdc mode (#16954)
* mysql source : implement support for snapshot of new tables in cdc mode

* undo unwanted changes

* add more assertions

* format

* fix build

* fix build

* revert acceptance test changes

* bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-27 00:14:07 +05:30
Björn Schuberg
05bd34f477 🎉 New Source: Adjust (#16051)
* Initial version of the adjust source connector

* source-adjust: add bootstrap.md

* adjust-source: add setup guide

* adjust-source: update integration READMEs

* source-adjust: better stream name

* source-adjust: fix sample conf

* source-adjust: add spec order metadata

* source-adjust: improve spec dimension doc

* source-adjust: warn on custom metric cast failure

* source adjust: Update source_definitions.yaml

* source adjust: Update documentation url
2022-09-26 14:25:51 -04:00
Eugene
4e381abbd2 🐛Source-snowflake: added connection string identifier (#17116)
* [16767] Source-snowflake: added connection string identifier
2022-09-26 21:19:56 +03:00
Yurii Bidiuk
e07bd6bee3 Destination BigQuery: add user-agent header (#16890)
* Destiantion BigQuery: add user-agent header

* fixed comments

* fix tests

* refactor code

* refactoring

* bump version

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 20:31:27 +03:00
Krisjan O
3939ff34cd Source: Amazon Seller Partner - Updated Report API version + New reports (#16629)
* Updated Report API version + New reports

* Fixed typo and added ability to set fetch period to a fixed range

* Removed old code + updated connection test doc

* minor fix for python format

* Bump version to 0.2.26 and updated docs

* Docs update, fix typo in PR number

Co-authored-by: Tuan Nguyen <anhtuan.nguyen@me.com>
Co-authored-by: Sajarin <sajarindider@gmail.com>
2022-09-26 13:19:06 -04:00
Subodh Kant Chaturvedi
cb7076bffe allow nullable columns to be used as cursor (#17131)
* allow nullable columns to be used as cursor

* add test

* fix test

* bump version

* Revert "bump version"

This reverts commit a3cf9aff84.

* bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 21:20:53 +05:30
Serhii Chvaliuk
e3553fea22 🎉 Source mixpanel: Service Accounts added (#16915)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-26 16:34:45 +03:00
darynaishchenko
b8a647270a Chargebee: Added "custom fields" to the relevant Chargebee source data streams. (#17056)
* Chargebee: added custom fields

* Chargebee: added value types json schema

* Chargebee: added docs, updated docker label, refactored utils

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 11:16:16 +03:00
Liren Tu
d14b4423ea Publish R2 destination (#17122)
* Format code

* Update doc

* Add r2 to build doc

* Update readme

* Add r2 to destination definitions

* Move test resource to base-java-s3

* Remove redundant dependency

* Add missing dependency

* Format json

* Remove s3-destination-base-integration-test setting.gradle root

* Add missing lzo dependency

* Fix import in destination jdbc
2022-09-25 21:45:00 -07:00
Nugusbayev Kanagat
3494405078 🎉 New Destination: R2 Cloudflare (#15296)
* added destination-r2 spec

added base-java-s3 module

updated common s3 lib

local

changed dependency of s3 in all realted modules

minor style fixes

added common module for s3 integration tests

* fix configuration for r2 integration tests

* fix configuration for r2 integration tests

* added docs for destination-r2

* minor import fixes

* mark test as disabled

* fixed imports in destination-snowflake

* restored styling task

* added tests for S3DEstinationConfig

* added upload threads count configuration for r2(due to limitations)

* deleted parquet format type

* Fix import in gcs

* Fix import in redshift

* Fix import in snowflake

* Fix one more import in gcs

* Fix one more import in redshift

* Fix import in databricks

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
2022-09-25 19:20:47 -07:00
Denys Davydov
68cae56610 Source Facebook Marketing: limit time range to 37 months (#17027)
* #481 oncall source fb marketing: limit time range to 37 months

* source fb marketing: upd changelog

* #391 source facebook-marketing: retry on getting error 960

* source facebook marketing: format code

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-23 21:57:27 +03:00
Denys Davydov
d62f88196f Source Salesforce: perform call to objects list in connection check (#17094)
* #629 oncall - source salesforce: perform call to objects list in connection check

* source salesforce: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-23 21:57:11 +03:00
Baz
45503de7d9 Source Netsuite: mention API method used to fetch the data for connector (#17082) 2022-09-23 13:51:53 +03:00
Baz
5928b48c41 🐛 Source Recharge: fix Charges.total_weight field data-type (#17080) 2022-09-23 13:26:23 +03:00
andriikorotkov
dbe25f6588 updated DB2 source documentation (#17078) 2022-09-23 12:10:31 +03:00
Edward Gao
23c8efa2b2 Destination Bigquery / Bigquery-denormalized: Only override dataset ID if stream namespace is null/empty (#17054)
* stop overriding namespace?

* set namespace if needed

* also check for empty namespace

* version bump + changelog

* auto-bump connector version [ci skip]

* sanitize dataset id

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 12:41:47 -07:00
Oleksandr Sheheda
6465aae0dc [16617] Source BigQuery: added user agent header (#16902)
* [16617] Source BigQuery: added user agent header

* [16617] Source BigQuery: added user agent header

* [16617] Source BigQuery: added user agent header
updated image tag, added changelog

* [16617] Source BigQuery: added user agent header
fixed NPE

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 21:45:06 +03:00
Cole Snodgrass
e662cf0d3a rm airbyte-scheduler; migrate existing client to airbyte-server (#17051)
* wip; move scheduler client to server package

* remove additional references to airbyte-scheduler

* removed more references to airbyte-scheduler
2022-09-22 10:25:52 -07:00
Yashkumar Makwana
abed1ce8b6 🎉New Source: Zenefits (#14809)
* New Source: Zenefits

* updated Readme.md

* Update spec.json

* Update request headers

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update source.py

* removed sample_files

* 1. updated source_definition
2. created a new file zenefits.md

* Delete TODO.md

* fix: remove .class files

* fix: added back source_definition

* fix: rebase with origin master

* fix: updated specs

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-09-22 22:25:21 +05:30
Yurii Bidiuk
bb39b36016 Destiantion Kinesis: add required fields (#16952)
* Destiantion Kinesis: add required fields

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 18:03:26 +03:00
Baz
550dfb9827 Source Netsuite: fix public documentation (#17037) 2022-09-22 15:27:34 +03:00
Baz
ea3d57b6f3 🐛 Source Recharge: data types should be transformed based on schema types (#16959) 2022-09-22 15:21:21 +03:00
Denys Davydov
596a4367af Source salesforce: handle japanese characters (#17001)
* #454 oncall source salesforce: handle japanese characters

* source salesforce: upd changelog

* source salesforce: flake fix

* #454 source salesforce: adjust public interface to CDK, do not take into account state when choosing the API type

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 14:27:28 +03:00
Akash Kulkarni
c83a5953bd 🎉 Destination Snowflake : Update JDBC driver for Snowflake to 3.13.19 (#16839)
Destination Snowflake : Update JDBC driver for Snowflake to 3.13.19 (#16839)

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:27:51 -07:00
Brian Lai
d0386249de bump notion, pipedrive, and pinterest to the latest CDK including per-stream changes (#16799) 2022-09-21 18:17:42 -05:00
Akash Kulkarni
44ef1c4ad6 🎉 Source Snowflake : Update JDBC driver for Snowflake to 3.13.22 (#16766)
Update JDBC driver for Snowflake

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:07:34 -07:00
Denys Davydov
57c269d9c6 Source github: improve error logging (#16947)
* #410 oncall source github: improve error logging

* source github: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 23:16:40 +03:00
Andrzej Dackiewicz
59a699be7b 🎉 New source: Primetric (#15880)
* Added source-primetric connector

* Fixed folder name

* Made Primetric client_id an Airbyte secret

* Updated schemas, added source connector documentation

* Added money object schema, updated docs

* Updated builds.md

* Fixed typos in configured catalog

* Fixed backoff errors in integration tests

* Removed comments

* Fixing Python formatting

* Fixing placeholder

* Fixed unit tests

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:12:05 -04:00
drrest
db86fcb84e 🎉 New Source: NetSuite (#16093) 2022-09-21 21:43:49 +03:00
Sophia Wiley
1e399c7c99 Deleted Cloud information from Harvest Doc (#16988)
Harvest is not yet available for Cloud, so I deleted the Cloud instructions of this doc.
2022-09-21 09:35:37 -07:00
Mainara Cavalcanti
97b9c74718 Source Github: add workflow_jobs stream (#16534)
* feat(workflow-jobs)!: new class 'WorkflowJobs' to get job informations from Github;

* feat(workflow-jobs)!: unit test to new class 'WorkflowJobs';

* feat(test_stream/test_stream_workflow_jobs_read_incremental)!: support to method incremetal in unit test to class 'WorkflowJobs';

* feat(test_workflow): unit test incremental and full refresh to class 'WorkflowJobs';

* docs: updated github docs

* docs: updated github docs

* docs: updated github.md

* docs: updated github.md

* fix: add workflow_jobs in abnormal tests, use 'start date' in source and unit test;

* fix: add workflow_jobs in 'abnormal_state.json' and 'sample_state.json' for integration tests;

Co-authored-by: Paulo Roberto <paulo.rb.beserra@gmail.com>
Co-authored-by: Paulo Roberto <71179739+py-paulo@users.noreply.github.com>
2022-09-21 10:19:35 -04:00
Serhii Lazebnyi
64a1c2e04e Source Stripe: Updated customer schema field type (#16537)
* Updated customer schema field type

* Updated PR number

* Updated setup file

* Updated acceptance test config

* Updated acceptance test config

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-20 21:57:39 +02:00
Denys Davydov
9054468c21 Source s3: upgrade pyarrow (#16921)
* #423 oncall source s3: upgrade pyarrow

* source s3: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-20 19:24:07 +03:00
Juan
dc328ca14e Adds newly created IP addresses to docs (#16901)
* Adds newly created IP addresses to docs

* Update docs/operator-guides/security.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/integrations/sources/postgres.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/integrations/sources/alloydb.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/cloud/getting-started-with-airbyte-cloud.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>
2022-09-19 14:22:39 -04:00
ernestasga
973fa68001 🎉 New Source: Yandex metrica (#15061)
* Initial commit

* Add option to skip connectivyty check

* Update unit tests

* Simplify check_connection method

* Small updates

* Some changes

* Resolve PR commented parts

* Fix flakeCheck

* Drop field prefixes

* Formatting

* Fix unit tests

* Fix integration tests and remove dateTimeUTC field

* Fix acceptance tests

* auto-bump connector version [ci skip]

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:07:17 -04:00
Yurii Bidiuk
dacf3813b9 Source AlloyDB: add strict-encrypt version (#16776)
* Source AlloyDB: add strict-encrypt version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 18:08:17 +03:00
Roman Yermilov [GL]
8d61711c13 Source Google Search Console: fix 403 error (#16819)
* Source Google Search Console: fix 403 error

* Source Google Search Console: bump version and update changelog

* Source Google Search Console: fix test abnormal state data

* Updated release stage

* auto-bump connector version [ci skip]

Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 17:16:55 +04:00
jablonskijakub
1c0d237cd5 Source Zenloop: add new stream properties (#15843)
* add new stream properties

* add PR number

* change additionalProperties to true

* revert bumoed version of an image, add json schema in the tests files, remove additionalProperties: true

* gradle build completed

* improve test acceptance quality, adjust the order of objects for the properties json schema

* delete reduntant configured_catalog, add json schema for all streams

* auto-bump connector version [ci skip]

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Marie Amelie Sandrock <marieamelie94@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 05:39:11 -04:00
Denys Davydov
d780141f89 Source File: fix schema generation for json files containing an array (#16772)
* #547 oncall Source File: fix schema generation for json files containing arrays

* source file: upda changelog

* #547 oncall: source file - upgrade source-file-secure

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:18:05 +03:00
Denys Davydov
4dc394cb9a Source S3: fix reading jsonl files with nested data (#16607)
* #531 source s3: fix reading nested jsonl files

* #531 source s3: upd changelog

* oncall #531 source s3: fix sample file

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:09:40 +03:00
Denys Davydov
73ba7b63d5 Source S3: choose between data types when merging master schema (#16631)
* #422 source s3: choose broadest data type when there is a mismatch during merging json schemas

* #422 source s3: upd changelog
2022-09-19 10:50:18 +03:00
Serhii Chvaliuk
a670d45afd 🎉 Source mixpanel: add streaming for export stream :) (#16843)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-18 16:30:10 +03:00
Roman Yermilov [GL]
18552b49ab 🎉Source SurveyMonkey: handle 404 (#16706)
* Source Survey Monkey: handle nonexistant survey

* Deleted airbyte cdk version from setup

* Updated release stage to GA

* Updated log in doc

* auto-bump connector version [ci skip]

Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-16 19:10:42 +02:00
Sophia Wiley
4ed7e2291f deleted spaces (#16787) 2022-09-16 10:10:28 -04:00
Greg Solovyev
0d1f0e3c2e Add instructions for upgrading mysql source from 0.6.8 to 0.6.9 and MS SQL source from 0.4.17 to 0.4.18 (#16411)
Add instructions for upgrading
2022-09-15 17:04:21 -07:00
Amruta Ranade
e9f669304d Fixed nits 2022-09-15 18:10:11 -04:00
Sophia Wiley
3e80a074da Edit Bing Ads doc (#16755)
* Edited content

* formatting changes
2022-09-15 17:39:38 -04:00
Amruta Ranade
a6af9611a6 added SSL Tunnel method rec (#16782) 2022-09-15 17:36:32 -04:00
Földi Tamás
a61ac9f05b 🎉 New Source: Wrike (#15638)
* Connector skeleton

* Implement the first two endpoints with incomplete schema

* Add more fields to download from tasks

* Add more fields to download from tasks

* Add new entities plus refector a few stuff

* Complete all schema jsons

* Add slice management for comments

* complete schema definition for folders

* remove incremental tests

* Removing as no incremental support yet

* Format as pep8

* sample config files

* add newer version of the catalog

* remove unused import

* use wrike cataglog

* remove additional properties

* remove incremental tests

* reformat everything with Prettifier

* Add developer notes

* Add source-wrike badge

* Add Wrike source as alpha

* Document Wrike connector usage

* Add title fields for input parameters

* fix typo

* handle empty start date string

* add unit tests for wrike source

* remove unnecessary comments

* duplicated file

* document how to get permanent tokens

* Add link to API documentation

* Apply gradlew format

* Add workflow endpoint

* Change youtube back to Beta

* Remove unnecesary TODO comment

* Add null constraints

* Code cleanup in check_connection and streams

* Improved null constraints in schema

* Convert arrays with one element to string

* chore: update source_definitions.yaml

* chore: generate/update source_specs.yaml

Co-authored-by: sajarin <sajarindider@gmail.com>
2022-09-15 13:57:46 -04:00