1
0
mirror of synced 2025-12-20 18:39:31 -05:00
Commit Graph

204 Commits

Author SHA1 Message Date
Dhroov Makwana
94dfe73e70 🎉 New Source: Gutendex API [low-code CDK] (#18075)
* Initial commit for source-gutendex

* Clean up unnecessary code

* solve conflict

* Fix schema, Rename results_stream to books_stream

* Add parameters in gutendex.md, change doc url to official Airbyte url

* Add pagination, read each and every record instead of just 32

* Add bootstrap.md, add badge in builds.md

* add source definition to airbyte-config

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-20 19:53:55 -03:00
Henri Blancke
f861a7580c 🎉 New Source: Insightly [python cdk] (#18164)
* [ADD] logic for source insightly

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] schema cleanup

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [ADD] documentation

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] configured catalog

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* [RMV] catalog and logs

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] source tests

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* format files

* fix pk

* add seed config

* auto-bump connector version

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-20 16:17:28 -03:00
Gabriel Samson
5c5e7a180e 🎉 New Source: Courier [low-code cdk] (#16847)
* Add Courier connector

* Add docs

* Rename api key input + Cleanup fixtureS

* Fix bad BearerAuthenticator param

* Improve schemas

* Fix schemas

* Fix integrat testsg

* solve conflict

* redone some code

* correct source

* add seed config

* auto-bump connector version

* run format

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-19 14:49:32 -03:00
Amruta Ranade
8560f19022 Updated connector catalog page (#18076) 2022-10-17 15:41:50 -04:00
Michael Siega
d8e9f512d6 Fix multiple breakages in the platform build (#18011)
* Revert "🎉 New Destination: bit.io (#15821)"

This reverts commit c05135b336.

* Fix broken acceptance tests
2022-10-14 22:23:43 +02:00
Adam Fletcher
c05135b336 🎉 New Destination: bit.io (#15821)
* tests pass

* solve conflict

* Add readme

* add bootstrap.md

* solve doc conflict

* update spec

* fix spec test

* add bitdotio icon

* change dockerfile version

* add releaseStage for tidb

* fix icon name

* correct conflict

* remove dest def for bit.io

* run config process seed

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-10-14 14:14:28 -03:00
Amruta Ranade
4c7002ec05 Updated GA4 status 2022-10-05 15:52:10 -04:00
Neil Macneale V
65e616877a 🎉 New source: Fauna (#15274)
* Add fauna source

* Update changelog to include the correct PR

* Improve docs (#1)

* Applied suggestions to improve docs (#2)

* Applied suggestions to improve docs

* Cleaned up the docs

* Apply suggestions from code review

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

* Update airbyte-integrations/connectors/source-fauna/source_fauna/spec.yaml

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>

* Flake Checker (#3)

* Run ./gradlew :airbyte-integrations:connectors:source-fauna:flakeCheck

* Fix all the warnings

* Set additionalProperties to true to adhere to acceptance tests

* Remove custom fields (#4)

* Remove custom fields from source.py

* Remove custom fields from spec.yaml

* Collections that support incremental sync are found correctly

* Run formatter

* Index values and termins are verified

* Stripped additional_columns from collection config and check()

* We now search for an index at the start of each sync

* Add default for missing data in collection

* Add a log message about the index chosen to sync an incremental stream

* Add an example for a configured incremental catalog

* Check test now validates the simplified check function

* Remove collection name from spec.yaml and CollectionConfig

* Update test_util.py to ahere to the new config

* Update the first discover test to validate that we can find indexes correctly

* Remove other discover tests, as they no longer apply

* Full refresh test now works with simplified expanded columns

* Remove unused imports

* Incremental test now adheres to the find_index_for_stream system

* Database test passes, so now all unit tests pass again

* Remove extra fields from required section

* ttl is nullable

* Data defaults to an empty object

* Update tests to reflect ttl and data select changes

* Fix expected records. All unit tests and acceptance tests pass

* Cleanup docs for find_index_for_stream

* Update setup guide to reflect multiple collections

* Add docs to install the fauna shell

* Update examples and README to conform to the removal of additional columns

Co-authored-by: Ewan Edwards <46354154+faunaee@users.noreply.github.com>
2022-09-29 10:37:03 -03:00
Amruta Ranade
f854222187 archived duplicate docs + updated connector catalog (#17180) 2022-09-26 18:49:04 -04:00
Björn Schuberg
05bd34f477 🎉 New Source: Adjust (#16051)
* Initial version of the adjust source connector

* source-adjust: add bootstrap.md

* adjust-source: add setup guide

* adjust-source: update integration READMEs

* source-adjust: better stream name

* source-adjust: fix sample conf

* source-adjust: add spec order metadata

* source-adjust: improve spec dimension doc

* source-adjust: warn on custom metric cast failure

* source adjust: Update source_definitions.yaml

* source adjust: Update documentation url
2022-09-26 14:25:51 -04:00
Liren Tu
d14b4423ea Publish R2 destination (#17122)
* Format code

* Update doc

* Add r2 to build doc

* Update readme

* Add r2 to destination definitions

* Move test resource to base-java-s3

* Remove redundant dependency

* Add missing dependency

* Format json

* Remove s3-destination-base-integration-test setting.gradle root

* Add missing lzo dependency

* Fix import in destination jdbc
2022-09-25 21:45:00 -07:00
Yurii Bidiuk
dacf3813b9 Source AlloyDB: add strict-encrypt version (#16776)
* Source AlloyDB: add strict-encrypt version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-19 18:08:17 +03:00
Földi Tamás
a61ac9f05b 🎉 New Source: Wrike (#15638)
* Connector skeleton

* Implement the first two endpoints with incomplete schema

* Add more fields to download from tasks

* Add more fields to download from tasks

* Add new entities plus refector a few stuff

* Complete all schema jsons

* Add slice management for comments

* complete schema definition for folders

* remove incremental tests

* Removing as no incremental support yet

* Format as pep8

* sample config files

* add newer version of the catalog

* remove unused import

* use wrike cataglog

* remove additional properties

* remove incremental tests

* reformat everything with Prettifier

* Add developer notes

* Add source-wrike badge

* Add Wrike source as alpha

* Document Wrike connector usage

* Add title fields for input parameters

* fix typo

* handle empty start date string

* add unit tests for wrike source

* remove unnecessary comments

* duplicated file

* document how to get permanent tokens

* Add link to API documentation

* Apply gradlew format

* Add workflow endpoint

* Change youtube back to Beta

* Remove unnecesary TODO comment

* Add null constraints

* Code cleanup in check_connection and streams

* Improved null constraints in schema

* Convert arrays with one element to string

* chore: update source_definitions.yaml

* chore: generate/update source_specs.yaml

Co-authored-by: sajarin <sajarindider@gmail.com>
2022-09-15 13:57:46 -04:00
Amruta Ranade
425ab81b37 Update connector catalog (#16749)
* updated product stages

* updated cloud status
2022-09-14 16:16:21 -04:00
Eugene
63deca2e0a 🎉Source-AlloyDB for Postgres: added new connector (#16323)
* [16174] Source-AlloyDB for Postgres: added new connector
2022-09-07 11:41:37 +03:00
Daemonxiao
d4524032ae 🎉 New Destination: TiDB (#15592)
* Add new destination-tidb

* support sync

* Add normalization-tidb

* fix failed tests

* Add unnest marco

* fmt

* Add new destination-tidb

* support sync

* Add normalization-tidb

* fix failed tests

* Add unnest marco

* fmt

* fmt

* fix integration test

* Update docs/integrations/destinations/tidb.md

Co-authored-by: Xiang Zhang <angwerzx@126.com>

* Update doc

* Update doc

* Update doc

* bump normalization version

* update normalization changelog

* run format

* add dest def

* generat spec

Co-authored-by: Xiang Zhang <angwerzx@126.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-08-31 16:50:27 -03:00
Amruta Ranade
8998bad8c7 Updated connector catalog page (#16181) 2022-08-31 14:32:02 -04:00
Amruta Ranade
6f098603e6 Update README.md 2022-08-24 17:36:44 -04:00
Jordan Scott
c6ed31078f 🎉 New Source: LinkedIn Pages (#13098)
* added linkedin pages source connector

* fixed last source acceptance test issue and passed all

* requested changes completed

* fixed last source acceptance test issue and passed all

* requested changes completed

* added nulls to share_statistics schema

* added linkedin pages source connector

* solve conflict in readme

* requested changes completed

* fixed last source acceptance test issue and passed all

* requested changes completed

* added nulls to share_statistics schema

* remove utg posts, page stats and other stream

* remove gh file

* make org_id secret

* change org_id to string

* rollback org_id

* return org_id to integer

* rollback to integrr after correct ci

* correct docs

* auto-bump connector version [ci skip]

Co-authored-by: Jordan Scott <jscottpolevault@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-08-11 16:42:41 -03:00
Yuhui Shi
474e8c40f4 🎉 New Destination: SQLite (#15018)
* sqlite destination skeleton

* Add doc

* Add destination to  general docs

* clean unused files

* add seed and icon

* sqlite destination skeleton

* Add doc

* Add destination to  general docs

* add seed and icon

* correct config file

* remove unused import

* run seed spec file

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-08-09 14:00:38 -03:00
Amruta Ranade
b9b064fa44 Updated postgres doc and fixed other minor nits (#15297)
* Updated postgres doc and other minor nits

* fixed formatting

* added note about connectTimeout

* added limits to the initial wait time values
2022-08-05 14:24:31 -04:00
Amruta Ranade
d57a6a62f2 Update README.md 2022-08-03 17:49:12 -04:00
Amruta Ranade
b57ec0f7c4 Update README.md 2022-08-03 11:28:02 -04:00
Amruta Ranade
1282193244 Updated connector status page (#15240) 2022-08-03 10:56:28 -04:00
Rodrigo Parra
a5eb87220e 🎉 New source connector: Glassfrog (#13868)
* Add Glassfrog native connector

* fix: tests and formatting

* chore: added connector to definitions

* Add Glassfrog native connector

* fix: tests and formatting

* chore: added connector to definitions

* fix: tests and formatting

* auto-bump connector version [ci skip]

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-27 09:23:45 +05:30
Harshith Mullapudi
a8745db31a chore: add elasticsearch to documentation (#14948) 2022-07-22 19:32:15 -04:00
wjwatkinson
19df797ac0 New Source: Kyriba (#12748)
* remove redundant error status retry logic

* added kyriba source

* implemented account incremental sync

* remove unnecessary scheams and rename accounts

* added cash flows

* allow for different kyriba domains

* added account balances

* added bank balances and intraday streams

* refactored with account sub stream class

* airbyte cannot accomodate nested incremental date fields

* fix nested date fields in schema

* fix cash flow schema and logic

* specify max retries

* change 401 status code, so it gets retried

* moved to requests native auth

* compare tokens

* update request auth on 401

* set request auth header on 401

* increase cash flow page size to 1000

* add retries to login

* only sync cash balances yesterday and earlier

* corrected account schema

* increased max retries

* create uuid for eod bank balances

* add standard cash balance sync

* add standard bank balance sync

* fixed cash flow schema

* full refresh cash balances to enable proper ELT

* full refresh bank balances to enable proper ELT

* add end date

* fix formatting None end date

* don't require end_date in regex

* fixed cash balance range increment

* added unit tests

* added tests updated update date time parameter formatting

* added unit tests

* removed modification

* formatted

* fix integration tests

* fix test connection integration test

* added lastKnownBalanceDate

* produce STATE with abnormal future state

* formatted

* added source kyriba

* added kyriba build

* removed unnecessary files

* address PR comments to improve code

* make api version name consistent

* decrease size of cash flow slices

* improve cash flow sync performance with 1 day stream slices

* reduce number of retries as slices checkpoint state

* currently only one version of the API

* fixed pr link

* fixed static code check issues

* added source kyriba

* correct doc

* remove redundant error status retry logic

* added kyriba source

* implemented account incremental sync

* remove unnecessary scheams and rename accounts

* added cash flows

* added account balances

* added bank balances and intraday streams

* airbyte cannot accomodate nested incremental date fields

* fix cash flow schema and logic

* specify max retries

* change 401 status code, so it gets retried

* compare tokens

* set request auth header on 401

* add retries to login

* increased max retries

* create uuid for eod bank balances

* add standard cash balance sync

* add standard bank balance sync

* fixed cash flow schema

* full refresh cash balances to enable proper ELT

* full refresh bank balances to enable proper ELT

* added unit tests

* added tests updated update date time parameter formatting

* removed modification

* formatted

* fix integration tests

* fix test connection integration test

* added source kyriba

* removed unnecessary files

* address PR comments to improve code

* currently only one version of the API

* fixed static code check issues

* correct mypy checks

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-13 11:11:07 -03:00
Topher Lubaway
9c6c092a22 Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596)
This reverts commit a2c194a11f.
2022-07-11 15:27:14 -05:00
Mykyta Serbynevskiy
a2c194a11f Improving docusaurus sidebar generation (#1927) (#14369)
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
2022-07-08 14:18:27 -05:00
Amruta Ranade
ae1734c7dd Fixed Postgres status 2022-07-06 17:27:21 -04:00
Amruta Ranade
a31ef3bd5c Updated connector status page (#14466)
* Updated connector status page

* fixed google analytics filenames

* Fixed links
2022-07-06 17:04:46 -04:00
Harshith Mullapudi
9c289516ee Harshith/test pr 13118 (#14192)
* Firebolt destination

* feat: Write method dropdown

* feat: Use future-proof Auth in SDK

* refactor: Move writer instantiation

* fix: tests are failing

* fix: tests are failing

* fix: tests are failing

* chore: added connector to definitions

* fix: formatting and spec

* fix: formatting for orbit

Co-authored-by: ptiurin <petro.tiurin@firebolt.io>
2022-06-28 02:16:30 +05:30
Mohamed Magdy
a6bb6e0ead 🎉 Introduce Google Analytics Data API source (#12701)
* Introduce Google Analytics Data API source

https://developers.google.com/analytics/devguides/reporting/data/v1

* Add Google Analytics Data API source PR link

* Add `client` class for Google Analytics Data API

* Move dimensions and metrics extraction to the `client` class

In the Google Analytics Data API

* Change the copyright date to 2022 in Google Analytics Data API

* fix: removing incremental syncs

* fix: change project_id to string

* fix: flake check is failing

* chore: added it to source definitions

* chore: update seed file

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-06-27 23:25:53 +05:30
Christophe Duong
1498ce9483 🎉New source connector: source-metabase (#13752)
* Add docs

* Close metabase session when sync finishes

* Close session in check_connection

* Add source definition to seed

* Add icon
2022-06-23 09:46:50 +02:00
George Claireaux
30c5d2d9a7 🎉 New Source Dockerhub (#13931)
* init

* implement working source + tests

* add docs

* add docs

* fix bad comments

* Update airbyte-integrations/connectors/source-dockerhub/acceptance-test-config.yml

* Update airbyte-integrations/connectors/source-dockerhub/Dockerfile

* Update airbyte-integrations/connectors/source-dockerhub/.dockerignore

* Apply suggestions from code review

* Update docs/integrations/sources/dockerhub.md

* Update airbyte-integrations/connectors/source-dockerhub/integration_tests/acceptance.py

Co-authored-by: George Claireaux <george@airbyte.io>

* address @Phlair's feedback

* address @Phlair's feedback

* each record is now a Docker image rather than response page

* format

* fix unit tests

* fix acceptance tests

* add icon, definition and generate seed spec

* add requests to requirements

Co-authored-by: sw-yx <shawnthe1@gmail.com>
2022-06-20 15:13:29 +01:00
Harshith Mullapudi
074adefcdc Harshith/test pr 12536 (#13874)
* 🎉 New Source: Firebolt (#1)

* fix: Boolean type cast

* test: Improve format testing and doc

* refactor: Move some db functionality

* docs: Adding types doc link in utils

* feat: Use future-proof Auth in SDK

* fix: integration tests are failing

* chore: update seed file

Co-authored-by: Petro Tiurin <93913847+ptiurin@users.noreply.github.com>
Co-authored-by: ptiurin <petro.tiurin@firebolt.io>
2022-06-17 17:47:08 +05:30
Amruta Ranade
96a4d7515f Updated connector catalog (#13862) 2022-06-16 17:32:01 -04:00
Amruta Ranade
1393b4a43d Updated connector catalog 2022-06-09 10:45:22 -04:00
Yevhen Sukhomud
da88158c2f 🎉 New SFTP source connector (#13120)
* 11797 New SFTP source
* 11797 Bump sftp source version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-30 14:28:13 +07:00
Amruta Ranade
5aeae3b2e1 update Amazon Ads cloud availability (#13249) 2022-05-26 21:22:17 -04:00
Topher Lubaway
3a64c25073 Fixes all broken links and enforces no broken (#13152)
* Fixes all broken links and enforces no broken

* Addresses peter's comments

removes unused summary.md: an artifact of gitbook
2022-05-25 15:59:29 -05:00
Amruta Ranade
4a9d9296d2 updated connector catalog based on the May 25 changelog (#13190) 2022-05-25 15:06:22 -04:00
Erica Struthers
e542a4bd73 Update README.md (#12891) 2022-05-19 11:43:06 -04:00
Baz
feb0d2f378 🎉 New Destination: Implement Destination Google Sheets using CDK (#12135) 2022-05-12 01:10:28 +03:00
Amruta Ranade
2617f833bb Updated connector statuses (#12765) 2022-05-11 17:45:04 +05:30
Amruta Ranade
2a8df3342e Fixed broken things + updated connector status (#12373) 2022-04-26 15:53:08 -04:00
Amruta Ranade
e0a13405e2 Add cloud availability status on connector catalog page (#12081) 2022-04-18 12:56:14 -04:00
Evan Tahler
8293ce3fbe 🎉 New Source: Faker (#11738)
* Faker WIP

* Update catalog to handle dates better

* Adding unit tests for faker source

* WIP  - tests mostly passing

* add docs

* bump python version and fix unit tests

* test array types

* remove comment

* better python map

* update `moduleDirectory`

* simplify intiilization\

* use `ConfiguredAirbyteCatalog` in test rather than custom dict class

* Tests passing by using deterministic time

* Bump birthdays

* Update airbyte-integrations/connectors/source-faker/integration_tests/acceptance.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* remove bootstrap and stronger types

* better incremental support

* fixup un-used imports

* bump to test codecov

* Add connector to medatata files

Co-authored-by: George Claireaux <george@claireaux.co.uk>
2022-04-14 15:21:30 -07:00
Jared Rhizor
14dfc1d417 fix parentheses and info/warning/danger formatting in docs (#11981)
* fix parens

* fix formatting for hints

* remove hint from non-docusaurus or gitbook doc
2022-04-13 12:08:34 -07:00
Topher Lubaway
03ed894917 Updated GA connector docs and connector (#11640)
pipeline trigger because gitbook is sad

This reverts commit 230134068f.
2022-03-31 15:29:34 -05:00