1
0
mirror of synced 2026-01-06 06:04:16 -05:00
Commit Graph

10441 Commits

Author SHA1 Message Date
Maxime Carbonneau-Leclerc
98719cf3f3 [ISSUE #23794] CDK's read command handler supports Connector Builder … (#24204)
* [ISSUE #23794] CDK's read command handler supports Connector Builder list_streams requests

* [ISSUE #23794] code review
2023-03-21 09:01:33 -04:00
Dick Tang
7696b30f1c 📝 Source Google Search Console: update documentation (#22295)
* update docs for Google Search Console integration

* update connector version

* bump connector version to 0.1.22

* auto-bump connector version

---------

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-21 09:44:48 -03:00
Baz
775f203e0c 🎉 Source YoutubeAnalyticsBusiness Demo: remove the source-youtube-analytics-business from OSS part 2/3 (#24148) 2023-03-21 13:10:59 +02:00
Joe Reuter
6f8eec73f3 Adjust CDK publish command (#23935)
* adjust publish command

* fix

* Automated Change

* reset connectors

---------

Co-authored-by: flash1293 <flash1293@users.noreply.github.com>
2023-03-21 11:02:06 +01:00
Denys Davydov
6c434c885e Source zendesk support: validate datetime on check (#24252)
* #1683 source zendesk support: validate datetime on check

* #1683 source zendesk support: upd changelog

* Automated Change

* rm extra files

* auto-bump connector version

---------

Co-authored-by: davydov-d <davydov-d@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-21 10:48:07 +02:00
Augustin
2262cef74a re-set ci pipeline start timestamp (#24268) 2023-03-21 02:45:17 -05:00
Yevhen Sukhomud
7ae1de24af Destination Databricks: Support managed tables (#23965)
* 23036 Destination Databricks: support managed tables

* 23036 Destination Databricks: Update Databricks version to 1.0.0

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-21 10:51:08 +07:00
Conor
9d0aae671f Eks runner branch (#23943)
* run on eks runner

* retrigger build

* remote cache

* new dagger cli config
2023-03-20 22:13:26 -05:00
Praty Sharma
0cf36ed5d3 Source Slack: Increase page_size/limit param for API calls (#24126)
* Update page size for slack requests to 999

Slack APIs are heavily rate limited, and this minor change improves the performance of this connector by 10x with no impact to quality.

* update page_size limit to 1000

* bump connector version

* auto-bump connector version

---------

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 21:09:04 -03:00
Catherine Noll
fbe4ecc5be Fix: only build docker images with the local CDK for connectors (#24259) 2023-03-20 19:41:18 -04:00
Catherine Noll
deadeca0cf Connector Builder: assert catalog exists on test_read requests (#24199) 2023-03-20 19:35:11 -04:00
Artem Inzhyyants
6f98a68e28 Source Yandex Metrica: certify to beta (#24188)
* Source Yandex: http -> https

* Source Yandex: fix schema

* Source Yandex: bump to latest version

* Source Yandex: Ref Clean Job

* Source Yandex: Ref

* Source Yandex: update schemas; ref

* Source Yandex: refactor

* Source Yandex: refactor; add tests

* Source Yandex: bump version; update setup; update docs

* Source Yandex Metrica: fix tests

* Source Yandex Metrica: add allowed hosts; change releaseStage

* Source Yandex Metrica: fix docs

* Source Yandex Metrica: fix docs

* Automated Change

* Source Yandex Metrica: update docs

* Source Yandex Metrica: update docs

* Source Yandex Metrica: add expected_records for views stream

* auto-bump connector version

* Automated Commit - Formatting Changes

* Source Yandex Metrica: empty commit

---------

Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 21:26:21 +01:00
Serhii Chvaliuk
b588a52bbd Source Xero: Certify to Beta (#24217)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-03-20 21:28:43 +02:00
Ben Church
2d6f5ee88a Dagster PoC for Metadata service (#23989)
* Add airbyte-ci folders

* Add poetry

* Add first dagster job

* Get sensors properly registering

* Trigger job on new files

* Add etag cursor

* Wire up resources and ops

* Parse destinations dataframe

* Add multiple dataframes

* Compute markdown

* Write html and markdown

* Move columns to variable

* move to a folder centered file structure

* Move to sensor factory

* Add resource def to sensor

* Use appropriate credentials

* Use GCSFileManager

* use catalog_directory_manager

* Generalize the gcp catalog resources

* Move bucket to env var

* Clean up and add comments

* Update readme

* Move dependencies into orchestrator

* Add gcs section to readme

* Clean up debug

* Add merge catalog tests

* Run code formatter

* Apply flake8 fixes

* Remove suffix

* Move tests up one level

* Folder rename

* Update readme and rename env

* Add jinja templates

* Rename connectors_ci to connectors for lib
2023-03-20 12:28:10 -07:00
Benoit Moriceau
511d2b00a8 Update default threshold value for Jira (#24245)
* Update default threshold value for Jira

* Automated Change

---------

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2023-03-20 17:49:35 +00:00
Serhii Lazebnyi
e94e62fc85 Fix expected records for chargebee (#24239) 2023-03-20 18:25:11 +01:00
Brandon Beidel
db388a5cea Docs: update twitter.md (#24241)
fix spelling errors; documentation
2023-03-20 14:19:08 -03:00
Daryna Ishchenko
edf5b19ef1 Source Instagram: decrease give up rate (#23671)
* updated spec and added more specific error message

* added change log

* refactored error handling

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 17:19:36 +02:00
Koji Matsumoto
73eb77f2a5 🎉 New Source: Firebase Realtime Database (#18029)
* implement check for source-firebase-realtime-database

* implement discover

* support only one stream

* implement read and tests

* add docs

* remove unnecessary file

* add pr number

* fix sat

* add bootstrap.md

* add badge in builds.md

* fix acceptance.py

* add supported_sync_modes in configured_catalog

* add database_name description and example

* add info about required role for service account

* remove abnormal_state.json

* fill catalog.json with values for testing

* loosen requirements

* Update Dockerfile

* auto-bump connector version

* readd firebase to seed file

* fix build gradle for firebase

---------

Co-authored-by: Tuan Nguyen <anhtuan.nguyen@me.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2023-03-20 11:42:02 -03:00
VitaliiMaltsev
e5f96a3678 MySQL Source : support different table structure during "DESCRIBE" query (#24147)
* MySQL Source : support different table structure during "DESCRIBE" query

* bump version

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 14:28:29 +00:00
Arsen Losenko
508de0fb9c Source Github: Mark datefields in spec as dates (#22993)
* Source Github: Mark datefields in spec as dates

* Update changelog

* Update Dockerfile

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 16:21:23 +02:00
Arsen Losenko
550fe10591 Source Google Search Console: Mark datefields in spec as dates (#22984)
* Source Google Search Console: Mark datefields in spec as dates

* Update changelog

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 16:20:24 +02:00
Artem Inzhyyants
4eb896bf0c Source Airtable: make stream name unique; add id to names (#22704)
* Source Airtable: make stream name unique; add id to names

* Source Airtable: update docs

* Source Airtable: fix test

* Source Airtable: fix test

* Source Airtable: update version

* Source Airtable: update version

* Source Airtable: disable backward_compatibility_tests

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 14:08:26 +01:00
Arsen Losenko
27de42eff6 Source Google Ads: Mark datefields in spec as dates (#22985)
* Source Google Ads: Mark datefields in spec as dates

* Update changelog

* Update Dockerfile

* Update Dockerfile

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 15:05:19 +02:00
Joe Reuter
0d93fff47f Fix spec orders for yugabyte, sftp-bulk, gnews (#24180)
* fix spec orders

* Update yugabytedb.md

* Update gnews.md

* Update yugabytedb.md

* Update gnews.md

* Update sftp-bulk.md

* auto-bump connector version

* Update seed file

* fix gnews

* rollback gnews changes

* remove accidental commit

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-20 10:35:51 +01:00
Denys Davydov
096e76f3b6 Source Salesloft to beta (#24202)
* #19217 source salesloft to beta

* Automated Commit - Formatting Changes

* Automated Change

---------

Co-authored-by: davydov-d <davydov-d@users.noreply.github.com>
2023-03-20 09:32:08 +02:00
midavadim
f2ba0873f3 🎉 Source ZohoCRM - enable in cloud (#23906)
* Fixed authentication to support latest CDK, fixed SAT, updated expected record, enabled high strictness

* updated docs and connector version

* updated format of abnormal state

* updated version is seed

* merged

* updated connector version

* fixed SAT

* fixed SAT

* added empty streams

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-19 23:24:50 +02:00
Lake Mossman
2d3c48da8d Revert "Simplify postgres and GitHub forms (#24127)" (#24219)
This reverts commit fcc80cb5be.
2023-03-18 19:22:38 -07:00
Eugene
c4fbc8c9a5 S3-Parquet: added handler to process null values in arrays (#23788)
* [22807] Parquet - added handler to null values in arrays
2023-03-18 08:52:37 +00:00
Benoit Moriceau
5cc7055cb5 Update sentry heartbeat timeout treshold (#24208)
* Update sentry heartbeat timeout treshold

* Automated Commit - Formatting Changes

---------

Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
2023-03-17 17:38:35 -07:00
Lake Mossman
fcc80cb5be Simplify postgres and GitHub forms (#24127)
* add grouping and collapsing fields to postgres source

* add auth group to github source connector

* revert postgres field order changes and adjust group of schemas field

* inject group into ssh tunnel spec for postgres only, through overloaded methods

* Automated Change

* bump Dockerfile versions and update changelogs

* bump strict encrypt version as well

* fix postgres acceptance test

* fix acceptance test again

---------

Co-authored-by: lmossman <lmossman@users.noreply.github.com>
2023-03-17 23:02:03 +00:00
Baz
c3247d3f7c 🐛 Source Freshsales: update CAT before Enable in Cloud (#24155) 2023-03-17 23:23:36 +02:00
Serhii Chvaliuk
f06be037a5 Source Trello: Certify to Beta (#24141)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-03-17 20:47:50 +02:00
Ben Church
1ed9adda05 Add metadata validator (#24198)
* Add begining of lib folder

* Generate models from poetry command

* Run the validation script

* Add the catalog overrides type

* Add test for valid metadata files

* Add error state tests

* Expand valid and invalid test cases

* Update readme

* Run formatter

* Delete remaining catalogs
2023-03-17 11:33:54 -07:00
Denys Davydov
6a88625cca Source s3: fix datetime conversion (#24178)
* #1669 source s3: fix datetime conversion

* #1669 source s3: review fixes

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-17 20:14:08 +02:00
Baz
c86199c417 updated CAT config (#24154) 2023-03-17 20:08:23 +02:00
Nikolay Shebanov
3b0a710d25 Update the warning on stripe full-refresh-overwrite (#21007)
This commit fixes the warning message in the stripe source doc. The previous message wasn't specific enough and could be interpreted as if all the data older than 30 days will be deleted.

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2023-03-17 15:04:27 -03:00
Cyril ALFARO
85ba3683df Fix: Make the test commands consistent with the first line (#21904)
Small fixed just to be clear

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2023-03-17 15:02:23 -03:00
sandip-baidya
d4d9f569f6 Updated Braintree connector docs (#21447)
* Updated Braintree connector docs

Hi,
Followed Amruta's youTube video's reference to edit Braintree's connector docs. This is my first open source contribution!
1. Added prerequisites section
2. consistent header sizes
3. Added set up guide (similar to freshdesk and greenhouse)

Following are some of the areas I did not touch because there was ambiguity there and I did not want to change anything around because of my limited knowledge of both Airbyte and Braintree:-
1. The sentence "creating read-only key specifically for Airbyte access" could do well with specifying what key are we talking about exactly because there are two keys; public and private and I couldn't really find that out on my own either from google. (So, perhaps someone can edit that in the future)
2. Sync Overview (I was not sure about whether all 4 kinds of syncs are being supported for braintree from what was originally written in the docs, so I let the original sentence be as is; did not modify it one bit)
3. Data type mapping (Again, I was not sure what data mapping truly meant here, so I let it be, as is)

Thanks.

* Incorporated the suggested changes

---------

Co-authored-by: sandip <sandip@FC-IN-FVFJ3M5FQ6LR.local>
Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
2023-03-17 14:55:48 -03:00
Julian
3e81c9b306 Docs: Information to setup source GA4 #21670 (#22416)
* Added: Information to setup source #21670

* Changed: Added PR number to changelog entry

* Removed: Version bump as per change request

* Update google-analytics-v4.md

---------

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2023-03-17 14:54:17 -03:00
Sophia Wiley
69378d8d7e Docs: added info about default data residency (#23990)
* added info about default data residency

* small edit

* edited wording and added info about workflows

* more edits

* added link to data residency in getting started doc

* added info to getting started guide
2023-03-17 18:43:04 +01:00
Leo Schick
325599bd2a fix trustpilot changelog date #24009 (#24189) 2023-03-17 14:41:37 -03:00
Catherine Noll
e890d01d55 Connector builder: handle empty catalog (#24184) 2023-03-17 12:51:10 -04:00
Denys Davydov
29587dc623 Source stripe: safe get customer's attributes (#24179)
* #1663 source stripe: safe get customer's attributes

* #1663 source stripe: upd changelog

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-17 16:21:08 +02:00
Carlo Nuccio
f9f57ac421 🎉 Source Facebook Marketing: Added level into custom ads insights (#19141)
* Source Facebook Marketing - Added Level to Custom Ads Insights

* Added Row to changelog

* added level to integration tests

* edited spec.json

* edited to array

* added TODO in spec.py

* converted to type=string with pattern

* removed Literal from import

* Level with type string and enum list

* delete unused LEVEL_PATTERN

* deleted todo

* deleted level variable

* bump connector version

* solve conflict in doc

* auto-bump connector version

---------

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-17 10:24:57 -03:00
Serhii Lazebnyi
c1a4a6f3d3 Fix expected records for jira and zenloop (#24182) 2023-03-17 12:55:11 +01:00
Ella Rohm-Ensing
52db686e6e ella/even more tiny docs changes! (#24171)
* Fix heirarchy

* Move info on CI workflows to code contributing section

* Remove build badge, fix leftover bolded headers
2023-03-16 17:17:19 -05:00
Akash Kulkarni
c772bc2472 Some more cleanup in abstract classes in preparation of splitting (#24164) 2023-03-16 14:56:32 -07:00
Evan Tahler
7538437452 Fixup dead links (#24167) 2023-03-16 22:42:30 +01:00
Denys Davydov
db45f05814 Source S3: fix discovery issues (#24157)
* #1652 #1664 Source S3: fix discovery issues

* #1652 #1664 source s3: upd changelog

* #1652 #1664 source s3: review comments

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-16 22:39:29 +02:00