1
0
mirror of synced 2026-01-07 09:05:45 -05:00
Commit Graph

1953 Commits

Author SHA1 Message Date
Yurii Bidiuk
bb39b36016 Destiantion Kinesis: add required fields (#16952)
* Destiantion Kinesis: add required fields

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 18:03:26 +03:00
Baz
550dfb9827 Source Netsuite: fix public documentation (#17037) 2022-09-22 15:27:34 +03:00
Baz
ea3d57b6f3 🐛 Source Recharge: data types should be transformed based on schema types (#16959) 2022-09-22 15:21:21 +03:00
Denys Davydov
596a4367af Source salesforce: handle japanese characters (#17001)
* #454 oncall source salesforce: handle japanese characters

* source salesforce: upd changelog

* source salesforce: flake fix

* #454 source salesforce: adjust public interface to CDK, do not take into account state when choosing the API type

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-22 14:27:28 +03:00
Akash Kulkarni
c83a5953bd 🎉 Destination Snowflake : Update JDBC driver for Snowflake to 3.13.19 (#16839)
Destination Snowflake : Update JDBC driver for Snowflake to 3.13.19 (#16839)

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:27:51 -07:00
Brian Lai
d0386249de bump notion, pipedrive, and pinterest to the latest CDK including per-stream changes (#16799) 2022-09-21 18:17:42 -05:00
Akash Kulkarni
44ef1c4ad6 🎉 Source Snowflake : Update JDBC driver for Snowflake to 3.13.22 (#16766)
Update JDBC driver for Snowflake

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:07:34 -07:00
Denys Davydov
57c269d9c6 Source github: improve error logging (#16947)
* #410 oncall source github: improve error logging

* source github: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 23:16:40 +03:00
Andrzej Dackiewicz
59a699be7b 🎉 New source: Primetric (#15880)
* Added source-primetric connector

* Fixed folder name

* Made Primetric client_id an Airbyte secret

* Updated schemas, added source connector documentation

* Added money object schema, updated docs

* Updated builds.md

* Fixed typos in configured catalog

* Fixed backoff errors in integration tests

* Removed comments

* Fixing Python formatting

* Fixing placeholder

* Fixed unit tests

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:12:05 -04:00
drrest
db86fcb84e 🎉 New Source: NetSuite (#16093) 2022-09-21 21:43:49 +03:00
Sophia Wiley
1e399c7c99 Deleted Cloud information from Harvest Doc (#16988)
Harvest is not yet available for Cloud, so I deleted the Cloud instructions of this doc.
2022-09-21 09:35:37 -07:00
Mainara Cavalcanti
97b9c74718 Source Github: add workflow_jobs stream (#16534)
* feat(workflow-jobs)!: new class 'WorkflowJobs' to get job informations from Github;

* feat(workflow-jobs)!: unit test to new class 'WorkflowJobs';

* feat(test_stream/test_stream_workflow_jobs_read_incremental)!: support to method incremetal in unit test to class 'WorkflowJobs';

* feat(test_workflow): unit test incremental and full refresh to class 'WorkflowJobs';

* docs: updated github docs

* docs: updated github docs

* docs: updated github.md

* docs: updated github.md

* fix: add workflow_jobs in abnormal tests, use 'start date' in source and unit test;

* fix: add workflow_jobs in 'abnormal_state.json' and 'sample_state.json' for integration tests;

Co-authored-by: Paulo Roberto <paulo.rb.beserra@gmail.com>
Co-authored-by: Paulo Roberto <71179739+py-paulo@users.noreply.github.com>
2022-09-21 10:19:35 -04:00
Serhii Lazebnyi
64a1c2e04e Source Stripe: Updated customer schema field type (#16537)
* Updated customer schema field type

* Updated PR number

* Updated setup file

* Updated acceptance test config

* Updated acceptance test config

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-20 21:57:39 +02:00
Denys Davydov
9054468c21 Source s3: upgrade pyarrow (#16921)
* #423 oncall source s3: upgrade pyarrow

* source s3: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-20 19:24:07 +03:00
Juan
dc328ca14e Adds newly created IP addresses to docs (#16901)
* Adds newly created IP addresses to docs

* Update docs/operator-guides/security.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/integrations/sources/postgres.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/integrations/sources/alloydb.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

* Update docs/cloud/getting-started-with-airbyte-cloud.md

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>

Co-authored-by: Sophia Wiley <106352739+sophia-wiley@users.noreply.github.com>
2022-09-19 14:22:39 -04:00
ernestasga
973fa68001 🎉 New Source: Yandex metrica (#15061)
* Initial commit

* Add option to skip connectivyty check

* Update unit tests

* Simplify check_connection method

* Small updates

* Some changes

* Resolve PR commented parts

* Fix flakeCheck

* Drop field prefixes

* Formatting

* Fix unit tests

* Fix integration tests and remove dateTimeUTC field

* Fix acceptance tests

* auto-bump connector version [ci skip]

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:07:17 -04:00
Yurii Bidiuk
dacf3813b9 Source AlloyDB: add strict-encrypt version (#16776)
* Source AlloyDB: add strict-encrypt version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 18:08:17 +03:00
Roman Yermilov [GL]
8d61711c13 Source Google Search Console: fix 403 error (#16819)
* Source Google Search Console: fix 403 error

* Source Google Search Console: bump version and update changelog

* Source Google Search Console: fix test abnormal state data

* Updated release stage

* auto-bump connector version [ci skip]

Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 17:16:55 +04:00
jablonskijakub
1c0d237cd5 Source Zenloop: add new stream properties (#15843)
* add new stream properties

* add PR number

* change additionalProperties to true

* revert bumoed version of an image, add json schema in the tests files, remove additionalProperties: true

* gradle build completed

* improve test acceptance quality, adjust the order of objects for the properties json schema

* delete reduntant configured_catalog, add json schema for all streams

* auto-bump connector version [ci skip]

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Marie Amelie Sandrock <marieamelie94@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 05:39:11 -04:00
Denys Davydov
d780141f89 Source File: fix schema generation for json files containing an array (#16772)
* #547 oncall Source File: fix schema generation for json files containing arrays

* source file: upda changelog

* #547 oncall: source file - upgrade source-file-secure

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:18:05 +03:00
Denys Davydov
4dc394cb9a Source S3: fix reading jsonl files with nested data (#16607)
* #531 source s3: fix reading nested jsonl files

* #531 source s3: upd changelog

* oncall #531 source s3: fix sample file

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 12:09:40 +03:00
Denys Davydov
73ba7b63d5 Source S3: choose between data types when merging master schema (#16631)
* #422 source s3: choose broadest data type when there is a mismatch during merging json schemas

* #422 source s3: upd changelog
2022-09-19 10:50:18 +03:00
Serhii Chvaliuk
a670d45afd 🎉 Source mixpanel: add streaming for export stream :) (#16843)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-18 16:30:10 +03:00
Roman Yermilov [GL]
18552b49ab 🎉Source SurveyMonkey: handle 404 (#16706)
* Source Survey Monkey: handle nonexistant survey

* Deleted airbyte cdk version from setup

* Updated release stage to GA

* Updated log in doc

* auto-bump connector version [ci skip]

Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-16 19:10:42 +02:00
Sophia Wiley
4ed7e2291f deleted spaces (#16787) 2022-09-16 10:10:28 -04:00
Greg Solovyev
0d1f0e3c2e Add instructions for upgrading mysql source from 0.6.8 to 0.6.9 and MS SQL source from 0.4.17 to 0.4.18 (#16411)
Add instructions for upgrading
2022-09-15 17:04:21 -07:00
Amruta Ranade
e9f669304d Fixed nits 2022-09-15 18:10:11 -04:00
Sophia Wiley
3e80a074da Edit Bing Ads doc (#16755)
* Edited content

* formatting changes
2022-09-15 17:39:38 -04:00
Amruta Ranade
a6af9611a6 added SSL Tunnel method rec (#16782) 2022-09-15 17:36:32 -04:00
Földi Tamás
a61ac9f05b 🎉 New Source: Wrike (#15638)
* Connector skeleton

* Implement the first two endpoints with incomplete schema

* Add more fields to download from tasks

* Add more fields to download from tasks

* Add new entities plus refector a few stuff

* Complete all schema jsons

* Add slice management for comments

* complete schema definition for folders

* remove incremental tests

* Removing as no incremental support yet

* Format as pep8

* sample config files

* add newer version of the catalog

* remove unused import

* use wrike cataglog

* remove additional properties

* remove incremental tests

* reformat everything with Prettifier

* Add developer notes

* Add source-wrike badge

* Add Wrike source as alpha

* Document Wrike connector usage

* Add title fields for input parameters

* fix typo

* handle empty start date string

* add unit tests for wrike source

* remove unnecessary comments

* duplicated file

* document how to get permanent tokens

* Add link to API documentation

* Apply gradlew format

* Add workflow endpoint

* Change youtube back to Beta

* Remove unnecesary TODO comment

* Add null constraints

* Code cleanup in check_connection and streams

* Improved null constraints in schema

* Convert arrays with one element to string

* chore: update source_definitions.yaml

* chore: generate/update source_specs.yaml

Co-authored-by: sajarin <sajarindider@gmail.com>
2022-09-15 13:57:46 -04:00
Serhii Chvaliuk
e03f47ef6a 🎉 Source mixpanel: Use "Retry-After" header for backoff (#16770)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-15 15:40:29 +03:00
Liren Tu
2e17e3f13a 🐛 Source google ads: mark custom query fields required (#15858)
* Mark query and table_name as required

* Bump version

* bypass backward compat tests

* bypass backward compat tests

* revert change on source-google-sheets/acceptance-test-config

* oncall 467 - source google ads: fix sats

* oncall #467 source google-ads: one more SATs fix

* auto-bump connector version [ci skip]

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-15 14:25:37 +03:00
Amalia Advincula-Roye
d4e18c6313 Remove Cloud from Kafka docs (#16753)
Kafka is currently not available on Cloud
2022-09-14 18:09:00 -04:00
Yurii Bidiuk
10e4aae891 Use LOG4J2 to wrap connectors logs to JSON format (#15668)
* Use LOG4J2 to wrap connectors logs to JSON format

* log connector's stack trace directly as a message

* add stack_trace field to json template

* bump versions

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* auto-bump connector version [ci skip]

* revert versions for destinations: postgres, mssql, mysql

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-14 23:27:13 +03:00
Amruta Ranade
425ab81b37 Update connector catalog (#16749)
* updated product stages

* updated cloud status
2022-09-14 16:16:21 -04:00
Roman Yermilov [GL]
55f875bce6 Source Google Search Console: add custom analytics stream (#16433)
* Source Google Search Console: add custom analytics stream

* Source Google Search Console: fix flake warnings

* Source Google Search Console: validate custom report string

* Source Google Search Console: update catalog to include custom report to acceptance tests

* Source Google Search Console: update abnormal state for custom report

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-14 14:44:38 +04:00
Ben Strawson
8b67f03d19 🐛 Source Tempo: Avoid infinite loop for non-paginated APIs (#16361)
* 🐛 Source Tempo: Avoid infinite loop for APIs that do not support pagination and support more than DEFAULT_ITEMS_PER_PAGE results.

* Source Tempo: Update changelog for version 0.2.6

* fix: update additionalProperties to true

* fix: remove additionalProperties field in schema files

Co-authored-by: Sajarin <sajarindider@gmail.com>
2022-09-13 20:11:04 -04:00
Edward Gao
39b816eb02 CDC sources: avoid busy wait when pushing to queue (#16657) 2022-09-13 13:53:23 -07:00
Arne Zeising
1888c27da9 🐛 Source Notion: Update spec description to include working timestamp example (#16272)
* Update description to include working timestamp example

* Update Docker version

* Update docs for Notion source connector

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-13 16:18:39 -03:00
VitaliiMaltsev
022b8db345 Destination: BigQuery (denormalized typed struct) - Could not evaluate union, field is expected to be one of these: NULL, STRING. (#16401)
* Fixed bucket naming for S3

* removed redundant configs

* BigQuery Denormalized: Could not evaluate union on Mongo formatted type

* add test

* fixed BigQuery tests

* updated changelog

* replaced star import

* bump version

* auto-bump connector version [ci skip]

* fixed compilation error

* bump mongodb-strict-encrypt

* auto-bump connector version [ci skip]

* removed unused method

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-13 19:10:09 +03:00
sivankumar86
ce64e96d71 Source:Kafka - Avro format message support (#15827)
* new format is added

* Avro support

Avro support

* new format is added

* schema namespace updated

* multi topic schema name is added

* max_records_process param is added

* review 1

* Schema registry details are added

* note update

* auto-bump connector version [ci skip]

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-13 11:28:25 -04:00
Denys Davydov
7934eed363 Source Hubspot: remove FeedbackSubmissions stream (#16632)
* #586 source hubspot: rm FeedbackSubmissions stream

* #586 source hubspot: upd changelog

* #586 source hubspot: fix unit tests

* #586 source hubspot: fix SATs

* #586 source hubspot: review fixes

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-13 18:08:51 +03:00
Serhii Chvaliuk
5edc575333 🎉 Source mixpanel: Improved connector's input configuration validation (#16570)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-13 12:12:14 +03:00
Ryan Fu
50a8d03c4c Cleans and Rebase Error Message Factory PR (#16202)
* Cleaned error messages factory PR

* Bumped MySQL and Postgres version

* Fixed messages and typos in test

* Fixes the changelog conflict with per-stream state

* Added note for flaky test

* Bumps mysql version to match changelog

* Added exception objects to all LOGGER.error for more visibility

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 16:08:11 -07:00
Evan Tahler
dcfcb75d0f AirbyteLogMessage.stack_trace for logging messages with related (non-fatal) errors (#16479)
* Test log message from faker

* AirbyteLogMessage gains stack_trace

* Fixup spacing

* bump python protocol

* fixup additionalProperties in faker spec

* bump faker version

* update docs

* use lineSeparator vs \r\n

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 16:01:57 -07:00
Benoit Moriceau
a0b4044bef Fix intercom (#16540)
* Fix intercom

* PR comments

* test

* Restore bmusko version

* Bump version and update changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 09:19:54 -07:00
Arsen Losenko
0f26426183 Source Recharge: Use TypeTransformer to convert integer values in Orders stream (#16592)
* Use TypeTransformer to convert integer values in Orders stream

* Update changelog

* Leave just the declaration of transformer
2022-09-12 17:19:41 +03:00
Denys Davydov
60888c2707 Source Sendgrid: change start time param type to datetime string (#16400)
* #4842 Source Sendgrid: change start time param type to datetime string

* #4842 source sendgrid: upd changelog

* #4842 source sendgrid: fix SAT

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 13:21:38 +03:00
Yurii Bidiuk
73653b3b02 Destiantion Mariadb: add custom JDBC params (#16391)
* Destiantion Mariadb: add custom JDBC params

* bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 12:07:00 +03:00
Serhii Chvaliuk
0ab777f23e 🎉 Source Amazon Ads: increase report_wait_timeout 30 -> 60 minutes (#16453)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-10 13:48:00 +03:00