1
0
mirror of synced 2025-12-20 18:39:31 -05:00
Commit Graph

204 Commits

Author SHA1 Message Date
Topher Lubaway
230134068f Revert "Updated GA connector docs and connector " (#11639)
gitbook pipelines failed.  reverting to trigger gitbook pipeline again
This reverts commit 6aa7e4ce79.
2022-03-31 15:24:17 -05:00
Amruta Ranade
6aa7e4ce79 Updated GA connector docs and connector stages (#11628)
* updated GA connector docs and README

* minor edit
2022-03-31 10:07:03 -04:00
Juan
e661d8aeff Update integrations README.md (#10851)
Updated verbiage from grades to stages
Updated connector stages to match cloud stage tags
Added connectors missing on README.md that appear on cloud drop down
2022-03-04 08:25:15 -07:00
Harshith Mullapudi
382e2b7476 Publish all faros ai sources (#10526)
* Add more sources sponsored by Faros AI

* Fix doc link

* Add icons

* Fix phabricator links

* Remove references to phabricator source because we dont have stable source acceptance test for it yet

* fix: rebased with master

* fix: account name in docker image

* Delete config.json

* chore: added seed spec

* chore: fix formatting

Co-authored-by: Chris Wu <chris@faros.ai>
2022-02-22 11:34:47 +05:30
Serhii Chvaliuk
3eb3367cc3 🐛 Source Quickbooks: update label Quickbooks -> QuickBooks (#10346)
* Quickbooks -> QuickBooks

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-02-17 16:52:20 +02:00
Harshith Mullapudi
f4365c59cf Add Jenkins source from Faros AI to connector catalog (#9929)
* Add Jenkins source from Faros AI to connector catalog

* todos

* Add setup instruction

* Update doc

* Feedback and add to dropdown

Co-authored-by: Chris Wu <chris@faros.ai>
2022-02-02 16:09:28 +05:30
Harshith Mullapudi
51af69b3b8 Add streamr destination catalog (#9928)
* Add streamr document

* add missing end line

* fix name: streamr > Streamr

Co-authored-by: Thanh Le <mihthanh27@gmail.com>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
2022-02-02 15:54:30 +05:30
Harshith Mullapudi
df332c7baa Revert "Add Streamr destination document (#9155)" (#9918)
This reverts commit 94d74bb4c6.
2022-01-31 16:33:21 +05:30
Harshith Mullapudi
a8135dd5be Revert "Add Jenkins source from Faros AI to connector catalog (#7837)" (#9917)
This reverts commit 5fd3292e2d.
2022-01-31 16:32:50 +05:30
Chris Wu
5fd3292e2d Add Jenkins source from Faros AI to connector catalog (#7837)
* Add Jenkins source from Faros AI to connector catalog

* todos

* Add setup instruction

* Update doc

* Feedback and add to dropdown
2022-01-31 15:55:58 +05:30
Thanh Le
94d74bb4c6 Add Streamr destination document (#9155)
* Add streamr document

* add missing end line

* fix name: streamr > Streamr

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
2022-01-31 15:55:47 +05:30
LiRen Tu
b269b9fb1e 🎉 Source e2e test: support custom catalog (#9720)
* Add continuous feed mode to source e2e-test

* Update connector catalog doc

* Fix sonar qube issues

* Add cloud variant

* Format code

* Add testing source connector to seed
2022-01-23 21:09:03 -08:00
Zaimwa9
1a74de6156 🎉 New Source: PersistIq (#9515) 2022-01-21 19:40:17 +01:00
Juozas V
9afbbff454 🎉 New Source: Flexport (#8777) 2022-01-20 15:03:12 +01:00
Titas Skrebe
081510017a 🎉 New Source: Chartmogul (#9381)
* New source: Chartmogul

* update changelog

* bootstrap.md

* reorder schema

* make acceptance test pass

* add chartmogul to builds.md

* Update airbyte-integrations/connectors/source-chartmogul/source_chartmogul/source.py

Co-authored-by: Augustin <augustin@airbyte.io>

* Update airbyte-integrations/connectors/source-chartmogul/source_chartmogul/source.py

Co-authored-by: Augustin <augustin@airbyte.io>

* address comments

* fix unused imports

* update source_specs

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
2022-01-18 19:45:59 +01:00
Juozas V
22dcb0f674 🎉 New Source: 3PL Central (#7322)
* Add source-tplcentral

* Add source definition

* Add missing file

* Update catalog

* Update sample config

* Update formatting

* Update invalid config

* Fix primary key identifier

* Update naming

* Rename variable

* Parametrize page size

* Implement Items stream

* Update schemas

* Update response normalization

* Add arrow

* Implement items stream cursor

* Fix types

* Cleanup

* Add customers stream

* Fix items sort

* Add stock_details stream

* Add inventory stream

* Add orders stream

* Refactor streams to a separate file

* Update cursor handling

* Refactor request_params

* Simplify

* Update params generation

* Update response parse

* Refactor stream+slices

* Refactor parse_response

* Cleanup

* New order

* Fix style

* Refactor parse_reponse

* Fix cursor

* Don't fetch next page after the last

* Fix schema errors

* Flatten shared schemas

* Inline refs

* Fix handling None stream_state

* Update schemas

* Implement primary_key and cursor fields

* Fix style

* Fix style

* Fix schemas

* Fix deep_map

* Fix items schema

* Fix cursor formatting

* Update integration tests assets

* Update README.md

* Remove TODO.md

* Update spec.json

* Cleanup

* Add bootstrap.md

* Increase page sizes

* Increase state checkpoint interval

* Add documentation

* Update links to the documentation

* Add build status badge

* Implement test_source.py

* Improve code testability

* Add stream tests

* Implement test_incremental_streams

* Add requests-mock dependency

* Fix formatting

* Update author

* Run ./gradlew format

* Cleanup failed merge artifacts

* Update sample_state.json

* Add stream documentation urls

* Cleanup

* Preserve upstream naming

* Fix primary key

* Fix configured catalog

* Update schemas

* Update catalog.json

* Fix tests

* Add schema source files

* Split configured catalogs

* Cleanup

* Update documentationUrl

* Add new properties

* Run gradlew format

* Remove additionalProperties: false from the schemas

* Revert "Remove additionalProperties: false from the schemas"

This reverts commit d4e8fea2fd.
2021-12-28 13:35:36 +05:30
Serhii Lazebnyi
6d3a80aafe 🎉 New Source: SearchMetrics (#6992)
* Added streams

* SearchMetrics implementation

* Updated unit tests

* Update to review and tests

* Add SEARCH_METRICS_TESTS_CREDS to ci creds

* Add CountDomainKeyword stream and updated to review

* Updated to new master

* Updated to linter

* Deleted windows in days from config

* Updated version in seed
2021-12-22 22:31:21 +02:00
Daniel Diamond
105bb58ac6 🎉 New source: Qualaroo (#8623)
* init

* Add additional docs per connector checklist

* De-nest answered_questions

* gradlew format

* fix integration tests and remove incremental stream

* Remove bin files
2021-12-21 13:17:52 +05:30
Augustin
369571c400 🎉 New Source: OpenWeather (#7434) 2021-12-15 22:17:36 +01:00
sbaldwin-rs
3748c66daa 🎉 New Destination: Rockset (#8006) 2021-12-15 15:53:58 +01:00
Bo Lu
bbcd461bc5 🎉 New Destination: ClickHouse (#7620)
* add ClickHouse destination

* update docs

* format code

* code improvement as per code review

* add ssh tunneling and ssl/tls support and code enhancement

* merge from master

* disable testCustomDbtTransformationsFailure test

* fix string format bug

* fix reserved keywords bug and disable dbt

* disable dbt in expect result

* add type hints

* bump connector version

Co-authored-by: Alexander Tsukanov <alexander.tsukanovvv@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-12-13 19:39:19 -03:00
Luis Gomez
cba7285c99 🎉 New destination: RabbitMQ (#7560)
* Added RabbitMQ destination

* Added unit tests

* Added exchange to config

* Updated to support different connection parameters

* Added integration tests

* Added RabbitMQ destination documentation

* small change

* Addressed PR comments

* add creds

* format files

* bump connector config files version

* run format

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-12-13 19:11:32 -03:00
Marcos Marx
47e03a3d2d Docs: Update connector list (#8683) 2021-12-09 20:10:41 -03:00
Alasdair Brown
84b3bf55ac 🎉 Destination Amazon SQS: New connector (#7503)
* initial commit, working sending single messages

* Adds Unit Tests, removes un-implemented features from Spec, yields sent messages, fixed fifo suffix match, adds bootstrap

* Remove yield from RECORD message

* Complete readme for sqs

* Formatting

* Minor tidying of unit tests

* Add references in docs pages

* formatting

* Add docs links

* formatting
2021-12-10 03:51:51 +05:30
Madison Swain-Bowden
b28f20bb53 🎉 New Destination: SFTP JSON (#4924) 2021-11-24 15:23:21 +01:00
Koji Matsumoto
694cbed455 🎉 New Destination: MariaDB ColumnStore (#7961) 2021-11-23 19:31:27 +01:00
Augustin
dfa3714895 Consistent WooCommerce naming (#8133) 2021-11-23 18:04:53 +01:00
Luis Gomez
4b59fe4382 🎉 New Source: Outreach (#7507)
* Generated connector source code

* Added streams: prospect, sequence, sequenceState

* Updated spec and schemas

* Removed print statement

* Added unit tests

* Updated acceptance test files

* Updated spec

* Ran gradlew format

* Added comments

* Added Outreach source documentation
2021-11-23 10:33:25 +05:30
AlexanderBatoulis
fd90c95e18 🎉 New Source: Zenloop (#7380) 2021-11-22 17:14:43 +01:00
Ivica Taseski
0eba52e002 🎉 New destination: Kinesis (#7799)
* add kinesis destination connector

* format code

* add javadoc and general documentation

* fix buffer bug and refactor code

* rename vars to common airbyte usage

* run format

* run format + seed file

* add eof

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-21 23:17:38 -03:00
Adam Dobrawy
519ea2a58a 🎉 New Destination: Google Firestore (#7231)
* destination-firestore: add connector

* destination-firestore: Add connector to indexes in docs

* destination-firestore: fix sync mode spec

* destination-firestore: fix typo after connector rename

* destination-firestore: fix required field in spec

* destination firestore: fix code formatting

* destionation-firestore: add python adnotation for writer

* destination firestore: add all used libraries into the setup file

* destionation-firestore: fix python formatting
2021-11-22 02:20:45 +05:30
Mario Molina
5b0b5840d6 🎉 New Destination: MQTT (#7718)
* MQTT destination

* Reformat code

* Updating readme

* run seed

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-21 14:20:42 -03:00
Ivica Taseski
8a957caf14 🎉 New destination: Redis (#7653)
* add redis destination connector

* add redis destination connector

* refactor & docs

* format

* refactor code

* debug checkpoint

* refactor and code cleanup

* format code

* add redis docs

* bump version, docs and run seed:

* correct uuid

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-20 11:26:37 -03:00
Tuan Nguyen
74c13b5193 🎉 New Source: Airtable (#7639) 2021-11-19 19:00:55 +01:00
Marcos Marx
678b731a33 change cart naming to cart.com (#8115) 2021-11-19 14:57:31 -03:00
Serhii Chvaliuk
530773fc32 🎉 New Source: Youtube analytics (#7407)
* init source-youtube-analytics

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-11-19 12:17:19 +02:00
Bo Lu
595ed6b80a 🎉 New Source: Notion (#7092)
* add source notion

* update PR number in change log

* bug fix and code improvement as code review suggestions

* code improvement as review advices

* new connector notion

* format

* correct creds file

* run seed

* bump connector version

* format

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-18 18:38:36 -03:00
Sherif Nada
894672b50c remove merge conflict from connector catalog docs page 2021-11-16 15:04:52 -08:00
Ivica Taseski
fcba5263e1 🎉 New destination: ScyllaDB (#7468)
* add scylla destination connector

* minor fixes

Co-authored-by: itaseski <ivica.taseski@seavus.com>
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-16 13:28:46 -03:00
Juozas V
ea738f1680 🎉 New Source: Linnworks (#7588) 2021-11-15 18:58:29 +01:00
Serhii Chvaliuk
1b38fc7b79 bugfix 'Hubspot' -> 'HubSpot' (#7683)
* bugfix 'Hubspot' -> 'HubSpot'

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-11-09 20:43:17 +02:00
Serhii Lazebnyi
2d2965b879 🎉 New Source: Pinterest (#7493)
* Add Pinterest implementation

* Updated to review

* Added rate limits and PR number to doc

* Fixed unit tests and data slicing bug

* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/connectors/source-pinterest/build.gradle

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Updated to review

* Updated window in days

* Fixed unit test

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-11-09 12:54:12 +02:00
Mario Molina
16d9d15f5a 🎉 New Destination: Pulsar (#7315)
* Adding Pulsar destination

* Add enable_chunking property

* Add dependency for DNS resolve for Mac

* Updating build.gradle

* Adding definition

* Refactor and fix tests

* Adding new config properties

* Reformat code

* Close resources when checking connection

* Updating doc

* Adding more test to build the producer map

* Rename brokers property

* Avoid emitting null states to the output collector

* fix config file

* run seed source/destination

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-05 20:03:27 -03:00
itaseskii
f53fd5e66b 🎉 New destination: Cassandra (#7186)
* add cassandra destination connector

* refactor and docs.

* delete test dockerfile

* revert Dockerfile rm change

* refactor & fix acceptance tests & format

* revert stream peek

* remove get pip

* add address example

* improved copy and code refactor

* add docker-compose and improved docs

Co-authored-by: itaseski <ivica.taseski@seavus.com>
2021-11-05 19:02:01 -03:00
Koji Matsumoto
c689594069 🎉 New Source: Sentry (#6975)
* add Events, Issues, ProjectDetail stream

* add P/R number

* add SUMMARY entry

* add docs/integrations/README.md entry

* add source_definitions.yaml entry

* add connector JSON definition

* add builds.md entry

* SentryStream keeps primary_key

* add Projects stream

* change stream for connection checking

* handling errors with try-catch in next_page_token function

* remove required key field from schemas

* remove DEFAULT_HOST

* raise error if link header don't have mandatory field

* fix unit test for streams

* update cursor for pagination

* add docs to each endpoint

* add hostname property to invalid_config

* fix schema

* add hostname to sample_config
2021-11-02 19:41:27 +05:30
Jeremy
fb313a6d26 🎉 New Destination: Elasticsearch (#7005)
* feat: adds destination-elasticsearch

* feat: adds destination-elasticsearch es server container

* refactor: header configuration

* update: only call createIndex when preparing the writes

* update: reuse container

* fix: make index names valid and use namespace

* refactor: use bulk process and buffered consumer

* refactor: fix bulk process and buffered consumer

* chore: update documentation

* update: remove ssl reference

* fix: bulk indexing

adds test logging config to inspect http wire
begins work for overrwriting existing records

* docs: update for authentication

* refactor: simplify config

* refactor: cleanup indices, implement auth

* update: cleanup equals/toString in Elasticsearch ConnectionConfiguration

* chore: use conventions and remove unused code

* update: close underlying rest connection

* update: enable `supportsNormalization`

* refactor: better encapsulate index naming

* update: allow upserting

* update: use oneOf for auth method

* refactor: use encapsulated auth object

* chore: pretty

* update: simplify auth header creation

* chore: remove unused class

* update: use boolean as field type

* adds: elasticsearch example server

* fix: api secret test
2021-11-01 22:52:01 -03:00
folusoogunlana
0e70491e0f 🎉 New Source: Paystack 🌟 (#7214)
* feat(67): add support for 'spec' using Python HTTP API source template and stripe as an example

* chore(67): add sample state and config

* feat(67): add check functionality for paystack source by fetching first customer

* feat(67): add support for discover and read customer stream

* feat(67): add paystack source connector to UI

* feat(67): update source definitions to use 0.1.0

* Hacktoberfest 67 paystack source (#1)

* feat(67): add support for 'spec' using Python HTTP API source template and stripe as an example

* chore(67): add sample state and config

* feat(67): add check functionality for paystack source by fetching first customer

* feat(67): add support for discover and read customer stream

* feat(67): add paystack source connector to UI

* feat(67): update source definitions to use 0.1.0

Co-authored-by: Foluso Ogunlana <foluso_ogunlana@stearsng.com>

* feat(67): update stream state cursor field to be integer and to match API record field name

* chore(67): add unit tests for source and streams

* chore(67): store formatted date time in state to match type of catalog

* chore(67): add configuration for acceptance integration tests

* docs(67): update docs and summary with paystack

* chore(67): add essential schemas to be catalogued for new streams

* feat(67): add support for critical streams - transactions subscriptions transfers refunds settlements

* docs(67): update image and bootstrap

* chore(67): update builds.md to include paystack badge

* docs(67): add changelog and source definition JSON file

* docs(67): add paystack to integrations readme

* chore(67): update check_connection to airbyte standard

* refactor to simplify streams and remove constants file

* fix(67): correct "null, null" values in schemas

* chore(67): update file formatting with gradle format

Co-authored-by: Foluso <5675998+foogunlana@users.noreply.github.com>
2021-11-01 23:37:00 +05:30
terencecho
1b1b52d6c9 🎉 New Source: Strava (#7151)
* Initial commit for new source: Strava

* Update readme and invalid_config

* Update PR number in docs

* Update docs/integrations/README

* Annotae spec with airbyte_secret

* Change starting_after input to start_date and add try/catch connection check

* Add connector to connector index

* Address PR commends and separate streams

* Set start_date as a required field

* Remove nexted objects from schema

* Remove schema TODO.md

* Fix schema

* Edit activites schema to allow for null values

* Fix formatting

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Fix formatting

* Update source_definitions.yaml

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2021-10-26 17:06:12 -03:00
Igli
19f2541a03 🎉 New Source: Lemlist (#7062)
* add source lemlist

* update change log

* apply default formating

* add comment, delete unnecessary files

* remove unnecessary things

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2021-10-26 14:16:49 -03:00
Alasdair Brown
58b569da18 🎉 Source Amazon SQS: New connector (#6937)
* Initial commit, working source with static Creds

* Typo in example queue url

* Adds auto delete of messages after read

* Adds visibility timeout

* remove insecure comments from AWS IAM Key spec

* explicitly set supported sync modes

* explicit sync mode should be lower case

* Adds unit tests for check, discover, read

* remove incremental acceptance test block

* remove incremental from conf catalog sample

* remove test requirement moto from main req

* align int catalog sample with sample_files

* fixing catalog configs

* acceptance testing config

* adds expected records txt

* automated formatting changes

* remove expected records block from acpt test

* Adds Docs page

* Ammends formatting on readme

* Adds doc link to summary

* Improve error handling & debug logging

* Adds bootstrap.md

* Add a todo suggestion for batch output

* Adds SQS to integrations readme list

* lower case properties

* removed unused line

* uses enum for aws region

* updates sample configs to use lowercase

* required props to lower case

* add missed property to lowercase

* gradle formatting

* Fixing issues from acceptance tests

* annotate secrets in spec.json with airbyte_secret

* Adds explicit warnings about data less when using Delete Message option
2021-10-26 13:23:10 -03:00