1
0
mirror of synced 2026-01-02 12:02:47 -05:00
Commit Graph

3340 Commits

Author SHA1 Message Date
Sophia Wiley
821c98e66c Added Google OAuth login info to docs (#17070)
* added google login info

* edited doc
2022-09-26 15:43:03 -07:00
Eugene
7ce39ed8a1 🐛Destination-snowflake: added connection string identifier (#17115)
* [16767] Destination-snowflake: added connection string identifier
2022-09-26 22:09:40 +03:00
Subodh Kant Chaturvedi
99746082a4 mysql source : implement support for snapshot of new tables in cdc mode (#16954)
* mysql source : implement support for snapshot of new tables in cdc mode

* undo unwanted changes

* add more assertions

* format

* fix build

* fix build

* revert acceptance test changes

* bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-27 00:14:07 +05:30
Björn Schuberg
05bd34f477 🎉 New Source: Adjust (#16051)
* Initial version of the adjust source connector

* source-adjust: add bootstrap.md

* adjust-source: add setup guide

* adjust-source: update integration READMEs

* source-adjust: better stream name

* source-adjust: fix sample conf

* source-adjust: add spec order metadata

* source-adjust: improve spec dimension doc

* source-adjust: warn on custom metric cast failure

* source adjust: Update source_definitions.yaml

* source adjust: Update documentation url
2022-09-26 14:25:51 -04:00
Eugene
4e381abbd2 🐛Source-snowflake: added connection string identifier (#17116)
* [16767] Source-snowflake: added connection string identifier
2022-09-26 21:19:56 +03:00
Davin Chia
517466eb66 Remove unused webbackend search code. (#17008)
* Remove unused webbackend search code.

* Remove unused request body.

* Remove unused request body.

* Remove failing test.

* Remove failing test.

* Remove failing test.

Co-authored-by: Parker Mossman <parker@airbyte.io>
2022-09-26 10:46:01 -07:00
Yurii Bidiuk
e07bd6bee3 Destination BigQuery: add user-agent header (#16890)
* Destiantion BigQuery: add user-agent header

* fixed comments

* fix tests

* refactor code

* refactoring

* bump version

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 20:31:27 +03:00
Krisjan O
3939ff34cd Source: Amazon Seller Partner - Updated Report API version + New reports (#16629)
* Updated Report API version + New reports

* Fixed typo and added ability to set fetch period to a fixed range

* Removed old code + updated connection test doc

* minor fix for python format

* Bump version to 0.2.26 and updated docs

* Docs update, fix typo in PR number

Co-authored-by: Tuan Nguyen <anhtuan.nguyen@me.com>
Co-authored-by: Sajarin <sajarindider@gmail.com>
2022-09-26 13:19:06 -04:00
Subodh Kant Chaturvedi
cb7076bffe allow nullable columns to be used as cursor (#17131)
* allow nullable columns to be used as cursor

* add test

* fix test

* bump version

* Revert "bump version"

This reverts commit a3cf9aff84.

* bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 21:20:53 +05:30
Serhii Chvaliuk
e3553fea22 🎉 Source mixpanel: Service Accounts added (#16915)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-26 16:34:45 +03:00
Michael Siega
32a984123e convert the workspace update api to patch style (#16739)
* convert the workspace api to patch style

* check for empty email in workspace update
2022-09-26 11:29:51 +02:00
darynaishchenko
b8a647270a Chargebee: Added "custom fields" to the relevant Chargebee source data streams. (#17056)
* Chargebee: added custom fields

* Chargebee: added value types json schema

* Chargebee: added docs, updated docker label, refactored utils

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-26 11:16:16 +03:00
Liren Tu
d14b4423ea Publish R2 destination (#17122)
* Format code

* Update doc

* Add r2 to build doc

* Update readme

* Add r2 to destination definitions

* Move test resource to base-java-s3

* Remove redundant dependency

* Add missing dependency

* Format json

* Remove s3-destination-base-integration-test setting.gradle root

* Add missing lzo dependency

* Fix import in destination jdbc
2022-09-25 21:45:00 -07:00
Nugusbayev Kanagat
3494405078 🎉 New Destination: R2 Cloudflare (#15296)
* added destination-r2 spec

added base-java-s3 module

updated common s3 lib

local

changed dependency of s3 in all realted modules

minor style fixes

added common module for s3 integration tests

* fix configuration for r2 integration tests

* fix configuration for r2 integration tests

* added docs for destination-r2

* minor import fixes

* mark test as disabled

* fixed imports in destination-snowflake

* restored styling task

* added tests for S3DEstinationConfig

* added upload threads count configuration for r2(due to limitations)

* deleted parquet format type

* Fix import in gcs

* Fix import in redshift

* Fix import in snowflake

* Fix one more import in gcs

* Fix one more import in redshift

* Fix import in databricks

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
2022-09-25 19:20:47 -07:00
Denys Davydov
68cae56610 Source Facebook Marketing: limit time range to 37 months (#17027)
* #481 oncall source fb marketing: limit time range to 37 months

* source fb marketing: upd changelog

* #391 source facebook-marketing: retry on getting error 960

* source facebook marketing: format code

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-23 21:57:27 +03:00
Denys Davydov
d62f88196f Source Salesforce: perform call to objects list in connection check (#17094)
* #629 oncall - source salesforce: perform call to objects list in connection check

* source salesforce: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-23 21:57:11 +03:00
Octavia Squidington III
8dfb8b0c92 Bump Airbyte version from 0.40.8 to 0.40.9 (#17093)
Co-authored-by: terencecho <terencecho@users.noreply.github.com>
2022-09-23 09:46:09 -05:00
Baz
45503de7d9 Source Netsuite: mention API method used to fetch the data for connector (#17082) 2022-09-23 13:51:53 +03:00
Baz
5928b48c41 🐛 Source Recharge: fix Charges.total_weight field data-type (#17080) 2022-09-23 13:26:23 +03:00
andriikorotkov
dbe25f6588 updated DB2 source documentation (#17078) 2022-09-23 12:10:31 +03:00
Edward Gao
23c8efa2b2 Destination Bigquery / Bigquery-denormalized: Only override dataset ID if stream namespace is null/empty (#17054)
* stop overriding namespace?

* set namespace if needed

* also check for empty namespace

* version bump + changelog

* auto-bump connector version [ci skip]

* sanitize dataset id

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 12:41:47 -07:00
Oleksandr Sheheda
6465aae0dc [16617] Source BigQuery: added user agent header (#16902)
* [16617] Source BigQuery: added user agent header

* [16617] Source BigQuery: added user agent header

* [16617] Source BigQuery: added user agent header
updated image tag, added changelog

* [16617] Source BigQuery: added user agent header
fixed NPE

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 21:45:06 +03:00
Cole Snodgrass
e662cf0d3a rm airbyte-scheduler; migrate existing client to airbyte-server (#17051)
* wip; move scheduler client to server package

* remove additional references to airbyte-scheduler

* removed more references to airbyte-scheduler
2022-09-22 10:25:52 -07:00
Yashkumar Makwana
abed1ce8b6 🎉New Source: Zenefits (#14809)
* New Source: Zenefits

* updated Readme.md

* Update spec.json

* Update request headers

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update source.py

* removed sample_files

* 1. updated source_definition
2. created a new file zenefits.md

* Delete TODO.md

* fix: remove .class files

* fix: added back source_definition

* fix: rebase with origin master

* fix: updated specs

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-09-22 22:25:21 +05:30
Yurii Bidiuk
bb39b36016 Destiantion Kinesis: add required fields (#16952)
* Destiantion Kinesis: add required fields

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 18:03:26 +03:00
Robert
c13d03b06c Docs: Fix some typos in connection-checking.yml (#17028)
Fixes some formatting issues and hopefully makes the guide slightly easier to follow.
2022-09-22 11:59:37 -03:00
Baz
550dfb9827 Source Netsuite: fix public documentation (#17037) 2022-09-22 15:27:34 +03:00
Baz
ea3d57b6f3 🐛 Source Recharge: data types should be transformed based on schema types (#16959) 2022-09-22 15:21:21 +03:00
Denys Davydov
596a4367af Source salesforce: handle japanese characters (#17001)
* #454 oncall source salesforce: handle japanese characters

* source salesforce: upd changelog

* source salesforce: flake fix

* #454 source salesforce: adjust public interface to CDK, do not take into account state when choosing the API type

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 14:27:28 +03:00
Akash Kulkarni
c83a5953bd 🎉 Destination Snowflake : Update JDBC driver for Snowflake to 3.13.19 (#16839)
Destination Snowflake : Update JDBC driver for Snowflake to 3.13.19 (#16839)

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:27:51 -07:00
Brian Lai
d0386249de bump notion, pipedrive, and pinterest to the latest CDK including per-stream changes (#16799) 2022-09-21 18:17:42 -05:00
Akash Kulkarni
44ef1c4ad6 🎉 Source Snowflake : Update JDBC driver for Snowflake to 3.13.22 (#16766)
Update JDBC driver for Snowflake

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:07:34 -07:00
Octavia Squidington III
387b96a924 Bump Airbyte version from 0.40.7 to 0.40.8 (#17009)
Co-authored-by: gosusnp <gosusnp@users.noreply.github.com>
2022-09-21 15:57:29 -07:00
Liren Tu
9356d6adf5 📝 Update json avro conversion doc about logical type union (#17010)
* Update json avro doc to include logical type union

* Add a note about the potential problem

* Add issue link
2022-09-21 15:43:44 -07:00
Davin Chia
03ae8bc4a4 Remove the webbackend list all route. (#17006)
Web backend routes are used by the Frontend.

This is not used today, and we don't imagine using it in the near future.
2022-09-21 16:21:09 -05:00
Denys Davydov
57c269d9c6 Source github: improve error logging (#16947)
* #410 oncall source github: improve error logging

* source github: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 23:16:40 +03:00
Andrzej Dackiewicz
59a699be7b 🎉 New source: Primetric (#15880)
* Added source-primetric connector

* Fixed folder name

* Made Primetric client_id an Airbyte secret

* Updated schemas, added source connector documentation

* Added money object schema, updated docs

* Updated builds.md

* Fixed typos in configured catalog

* Fixed backoff errors in integration tests

* Removed comments

* Fixing Python formatting

* Fixing placeholder

* Fixed unit tests

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:12:05 -04:00
drrest
db86fcb84e 🎉 New Source: NetSuite (#16093) 2022-09-21 21:43:49 +03:00
Sophia Wiley
1e399c7c99 Deleted Cloud information from Harvest Doc (#16988)
Harvest is not yet available for Cloud, so I deleted the Cloud instructions of this doc.
2022-09-21 09:35:37 -07:00
Mainara Cavalcanti
97b9c74718 Source Github: add workflow_jobs stream (#16534)
* feat(workflow-jobs)!: new class 'WorkflowJobs' to get job informations from Github;

* feat(workflow-jobs)!: unit test to new class 'WorkflowJobs';

* feat(test_stream/test_stream_workflow_jobs_read_incremental)!: support to method incremetal in unit test to class 'WorkflowJobs';

* feat(test_workflow): unit test incremental and full refresh to class 'WorkflowJobs';

* docs: updated github docs

* docs: updated github docs

* docs: updated github.md

* docs: updated github.md

* fix: add workflow_jobs in abnormal tests, use 'start date' in source and unit test;

* fix: add workflow_jobs in 'abnormal_state.json' and 'sample_state.json' for integration tests;

Co-authored-by: Paulo Roberto <paulo.rb.beserra@gmail.com>
Co-authored-by: Paulo Roberto <71179739+py-paulo@users.noreply.github.com>
2022-09-21 10:19:35 -04:00
Sophia Wiley
1d7e2163f4 Added info about cron scheduling (#16835)
* added info about cron scheduling

* added additional info about cron scheduling
2022-09-20 13:33:04 -07:00
Serhii Lazebnyi
64a1c2e04e Source Stripe: Updated customer schema field type (#16537)
* Updated customer schema field type

* Updated PR number

* Updated setup file

* Updated acceptance test config

* Updated acceptance test config

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-20 21:57:39 +02:00
Denys Davydov
9054468c21 Source s3: upgrade pyarrow (#16921)
* #423 oncall source s3: upgrade pyarrow

* source s3: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-20 19:24:07 +03:00
Parker Mossman
1a0ea82c34 Change ConnectionUpdate to PATCH behavior (#16157)
* remove operationIds from WebBackendConnectionUpdate, just use operations

* refactor connection updates to patch-style update, where null fields remain unchanged

* better comment and arg name

* format

* make sure we are still 'dual-writing' to the old schedule column, even when the patch doesn't specify anything for it

* update acceptance test to update with new schedule syntax

* add catalog sorting to preserve stream order during patch, and more tests

* format

* add description, throw runtime exception for impossible branches, move streamReset to private helper

* PR suggestions

* add nested test classes and write a test for the catalog sorting method

* format

* add comment clarifying that the catalog sort is for UX, and isn't critical

* format

* format

* update acceptance tests to send proper catalog patches instead of whole new catalog

* format

* format

* simplify catalog patching - now, if a catalog is present on the request, replace the entire catalog with it.
Otherwise, if catalog on the request is null, leave the catalog unchanged

* format

* format

* Revert "update acceptance tests to send proper catalog patches instead of whole new catalog"

This reverts commit 71922648b4e070f46ff6c468813b7ab8dd9d6651.

* adjust description
2022-09-19 18:36:34 -05:00
Juan
dc328ca14e Adds newly created IP addresses to docs (#16901)
* Adds newly created IP addresses to docs

* Update docs/operator-guides/security.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/integrations/sources/postgres.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/integrations/sources/alloydb.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/cloud/getting-started-with-airbyte-cloud.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>
2022-09-19 14:22:39 -04:00
ernestasga
973fa68001 🎉 New Source: Yandex metrica (#15061)
* Initial commit

* Add option to skip connectivyty check

* Update unit tests

* Simplify check_connection method

* Small updates

* Some changes

* Resolve PR commented parts

* Fix flakeCheck

* Drop field prefixes

* Formatting

* Fix unit tests

* Fix integration tests and remove dateTimeUTC field

* Fix acceptance tests

* auto-bump connector version [ci skip]

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:07:17 -04:00
Yurii Bidiuk
dacf3813b9 Source AlloyDB: add strict-encrypt version (#16776)
* Source AlloyDB: add strict-encrypt version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 18:08:17 +03:00
Roman Yermilov [GL]
8d61711c13 Source Google Search Console: fix 403 error (#16819)
* Source Google Search Console: fix 403 error

* Source Google Search Console: bump version and update changelog

* Source Google Search Console: fix test abnormal state data

* Updated release stage

* auto-bump connector version [ci skip]

Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 17:16:55 +04:00
jablonskijakub
1c0d237cd5 Source Zenloop: add new stream properties (#15843)
* add new stream properties

* add PR number

* change additionalProperties to true

* revert bumoed version of an image, add json schema in the tests files, remove additionalProperties: true

* gradle build completed

* improve test acceptance quality, adjust the order of objects for the properties json schema

* delete reduntant configured_catalog, add json schema for all streams

* auto-bump connector version [ci skip]

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Marie Amelie Sandrock <marieamelie94@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 05:39:11 -04:00
Denys Davydov
d780141f89 Source File: fix schema generation for json files containing an array (#16772)
* #547 oncall Source File: fix schema generation for json files containing arrays

* source file: upda changelog

* #547 oncall: source file - upgrade source-file-secure

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:18:05 +03:00