1
0
mirror of synced 2025-12-21 11:01:41 -05:00
Commit Graph

851 Commits

Author SHA1 Message Date
Baz
c790a7f15e 🎉 Source Drift: add OAuth 2.0 authentication option (#7337) 2021-11-22 17:28:49 +02:00
Harshith Mullapudi
1f7a42b018 added firestore credentials to CI and fix grpc installation (#8158) 2021-11-22 02:24:46 +05:30
Charles
005e4f21ba remove todo to handle bug in old version of docker compose; not an issue in new versions (#8122) 2021-11-19 13:19:40 -08:00
Serhii Chvaliuk
778b0f2e53 🎉 Source Harvest: add OAuth 2.0 (#7952)
* add OAuth 2.0 for source-harvest

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-11-19 20:23:19 +02:00
Tuan Nguyen
74c13b5193 🎉 New Source: Airtable (#7639) 2021-11-19 19:00:55 +01:00
Anna Lvova
3226b22dea 🎉 Source Snapchat Marketing: support oauth (#7811)
* add python oauth part

* bump version

* remove old creds

* bump version

* change spec

* bump version

* bump version

* bump version
2021-11-19 12:20:20 +02:00
Bo Lu
595ed6b80a 🎉 New Source: Notion (#7092)
* add source notion

* update PR number in change log

* bug fix and code improvement as code review suggestions

* code improvement as review advices

* new connector notion

* format

* correct creds file

* run seed

* bump connector version

* format

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-18 18:38:36 -03:00
Maksym Pavlenok
82c4c8223b 🎉 Linkedin Ads: support of oAuth2 (#7839)
* fix 404 responses for the ticket_comments stream

* add unit test

* add unit test

* add oauth2 access token

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* update spec file

* update CI secrets logic

* remove debug data

* add a debug message

* fix json convertation

* support one secret by several connectors

* Update tools/bin/ci_credentials.sh

Co-authored-by: LiRen Tu <tuliren@gmail.com>

* update function names

* update docs

* reset failed changes

* update json set value

* update spec file

* add oauth2 logic

* update doc and version

* add tests

* correction of spec

* update tests

* update spec file

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: LiRen Tu <tuliren@gmail.com>
2021-11-18 16:36:16 +02:00
Maksym Pavlenok
9a3a32711c 🐛 Source salesforce: failed async bulk jobs (#8009)
* update dockerfile

* update version

* update changelogs

* remove test config

* fix after flake8

* bump versio to 0.1.6

* remove secrets from config

* update source specs
2021-11-18 01:45:35 +02:00
Juozas V
ea738f1680 🎉 New Source: Linnworks (#7588) 2021-11-15 18:58:29 +01:00
Eugene
3a0eaafb5e 🎉 Source Mysql - added basic MySql performance tests (#7820)
* [ticket 7489, PR 7820] Source Mysql - added skeleton for MySql performance tests
2021-11-15 12:16:36 +02:00
Harshith Mullapudi
7c28f25959 🚀 Publish new source: Azure table storage (#7953)
* add azure table storage source connector

* fix imports

* add connector docs

* Add azure table connector to the API/UI

* fix imports

* add azure table to builds.md

* Delete launch.json

* fix import paths

* remove azure prefic from spec params and cleanup

* gradlew format

* Merge branch 'patch-1' of https://github.com/geekwhocodes/airbyte into patch-1

* rename specs key names and cleanup

* rename specs key names and cleanup (#1)

* add comments

* fix source definitions

* added azure table credentials to ci

* chore: added axure-table to seed spec

Co-authored-by: geekwhocodes <ganeshraskar@outlook.com>
2021-11-15 12:06:39 +05:30
Benoit Moriceau
9d05b1c477 Faux Major Version Bump (#7876) 2021-11-11 13:40:09 -08:00
Benoit Moriceau
d3ba217bf9 Update README.md (#7833) 2021-11-10 15:54:29 -08:00
Benoit Moriceau
73a9131526 Revert "Custom auto-setup temporal docker image (#7681)" (#7835)
This reverts commit e20d98fe0a.
2021-11-10 15:09:24 -08:00
Benoit Moriceau
e20d98fe0a Custom auto-setup temporal docker image (#7681)
This is a custom auto-setup script for the temporal environment. Unfortunately there is no other way properly update the DB without copy pasting parts of the temporal auto-setup script. Ideally temporal would provide a dedicated container for it DB but it is not the case right now.
2021-11-10 14:32:36 -08:00
Maksym Pavlenok
c0cc288f97 🎉 CI secrets: migration to GSM (#7528)
* fix 404 responses for the ticket_comments stream

* add unit test

* add unit test

* add oauth2 access token

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* update spec file

* update CI secrets logic

* update CI secrets logic

* remove debug data

* add a debug message

* fix json convertation

* fix json convertation

* support one secret by several connectors

* Update tools/bin/ci_credentials.sh

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update tools/bin/ci_credentials.sh

Co-authored-by: LiRen Tu <tuliren@gmail.com>

* update function names

* update docs

* reset failed changes

* update json set value

* update secrets json key

* Update tools/bin/ci_credentials.sh


* add JSON validation

* Update docs/connector-development/README.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* update doc

* Update README.md

* rename the function write_standard_creds

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: LiRen Tu <tuliren@gmail.com>
2021-11-10 10:49:44 +02:00
midavadim
c3c3e318c6 🎉 intercom added oauth support (#7060)
* Added oauth support

* fixed description of access_token field in Oauth authentication

* test selenium

* add java implementation

* remove commented dependencies| remove star import

* remove System.out.println

* added support for old format of spec.json files.

* added support for old format of spec.json files.

* added support for old format of spec.json files

* added credential with new spec format

* bumped connector version

* Reverted format of spec.json file

* Override extractRefreshToken method

* java oauth test fixes, minor updates, cloud spec version update

* Fixed docs strings

* Added HttpClient argument

* revert changes in seed folder to follow new dev flow

* Updated image version in seed

* Fixed import of IntercomOAuthFlow

* updated IntercomOAuthFlow to match base class changes

* Change image version to 0.1.8

Co-authored-by: vmaltsev <vitalii.maltsev@globallogic.com>
2021-11-09 22:13:46 +02:00
Serhii Lazebnyi
2d2965b879 🎉 New Source: Pinterest (#7493)
* Add Pinterest implementation

* Updated to review

* Added rate limits and PR number to doc

* Fixed unit tests and data slicing bug

* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/connectors/source-pinterest/build.gradle

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/connectors/source-pinterest/source_pinterest/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Updated to review

* Updated window in days

* Fixed unit test

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-11-09 12:54:12 +02:00
James Sewa Wilson
46fa44ab80 🎉 New Source: Commerce Tools (#5957)
* adding new source connector Commercetools

* adding new source connector Commercetools

* quick fix

* quick config

* changes

* format files

* add ci credentials run

* correct bin creds

* add publish cmd

* add spec host enum

* correct itnegration tests

* run seed file

Co-authored-by: jsewawilson <jwilson@pebblepost.com>
Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
2021-11-08 23:59:29 -03:00
Benoit Moriceau
42b378fe37 Revert "Revert "Bmoric/remove docker compose for build (#7500)" (#7698)" (#7746)
This reverts commit 797d11a8d7.

Restore the removal of the docker compose file
2021-11-08 14:16:21 -08:00
Tuan Nguyen
0b9b8ba472 🎉 New Source: Monday (#7168)
* add items stream

* add boards stream

* add remaining streams

* check point

* lint

* fix unit test & linting

* Delete launch.json

* fix schema

* add retry logic

* Update airbyte-integrations/connectors/source-monday/source_monday/source.py

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* small fix + add creds

* bump version and config file

Co-authored-by: Marcos Marx <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2021-11-06 17:32:13 -03:00
Benoit Moriceau
797d11a8d7 Revert "Bmoric/remove docker compose for build (#7500)" (#7698)
This reverts commit 4e17fa21a5.
2021-11-05 18:55:15 -07:00
vitaliizazmic
1f295f24e6 🎉 Source Google Directory: support oauth
* Source Google Directory #6265 - add oauth support

* Source Google Directory #6265 - update credentials

* Source Google Directory #6265 - fixing according to PR

* Source Google directory #6265 - update docs

* Source Google directory #5190 - update doc

* Source Google Directory #6265 - resolve merge conflict

* Source Google Directory #6265 - SAT for oauth

* Source Google Directory #6265 - bump version and update changelog

* Source Google Directory #6265 - bump version and update changelog (update publish)
2021-11-05 21:40:56 +02:00
Benoit Moriceau
4e17fa21a5 Bmoric/remove docker compose for build (#7500)
This making the build using a gradle plugin instead of using docker-compose build.
It aims to make the build to be more incremental as described in #7306

Building the docker image don't rely on docker-compose anymore.
The docker build step is isolated into a dedicated folder (in order to make sure that gradle plugin don't recompute the build of the docker container)
Gradle is responsible for copying the files that docker needs to build its image.
That removes the need of having a dockerignore file.
This might not be effective until #7539 is solved.
2021-11-05 11:58:23 -07:00
Harshith Mullapudi
af1c62f0c6 Publish new source Confluence #7241 (#7666)
* initial commit

* add remaining streams

* fix warning and unit test

* clean up

* revert unrelated files

* remove state files

* lint

* add documentation strings

* added confluence credentials to ci

* fix: removed audit from configured catalog as it cannot be pulled in free account

* fix: format

Co-authored-by: Tuan Nguyen <anhtuan.nguyen@me.com>
2021-11-05 15:17:26 +05:30
Jared Rhizor
8981228fc4 stop trying to copy normalization image for kube tests (#7580) 2021-11-03 09:18:08 -07:00
Jared Rhizor
00356f9910 add temporal and db logs for kube test output (#7583) 2021-11-03 09:17:54 -07:00
Marcos Marx
0e09ac3032 Publish PR 6963: new source freshsales (#7585)
* initial commit

* finish implementing full_refresh

* add other tables

* cleaning up

* add docs and use requests_native_auth

* fix function return different number of values

* change package author

* fix schema

* fix schema bug

* linting

* fix unit test

* clean up

* add null for schemas

* remove fc_widget_collaboration col

* change accpt test

* add creds

* run format

* add config files

* run format

Co-authored-by: Tuan Nguyen <anhtuan.nguyen@me.com>
2021-11-03 01:06:38 -03:00
Harshith Mullapudi
8ddf686e9e added credentials to CI for new source sentry
* add credentials to CI for new source sentry

* add ci cred
2021-11-02 19:55:58 +05:30
Dmytro
85381bde3c Surveymonkey: oAuth backend and connector's spec change. (#7524) 2021-11-02 10:40:39 +02:00
Marcos Marx
c846188772 Publish PR 6966: new source retently (#7552)
* retently

* PR comments

* PR comments

* mask api keys

* README

* keep gradle happy

* add github creds

* run format

Co-authored-by: Subhash <subhash@kloud.io>
2021-11-01 23:58:16 -03:00
Marcos Marx
fe118e676e Publish PR 4551: new source delighted correct format (#7540)
* Add Delighted source with people, unsubscribes and bounces streams

* Add survey responses stream

* Add documentation for Delighted source

* publish delighted source

* correct acceptance test

Co-authored-by: Rodrigo Parra <rodpar07@gmail.com>
2021-11-01 22:09:13 -03:00
Harshith Mullapudi
bdaec1cf3f add credentials in for New Source Paystack: #7214 (#7529)
* added credentials to ci for new source paystack

* format
2021-11-01 23:49:53 +05:30
Yevhenii
269298c854 🎉 Source File: Migrate File source to CDK structure (#7387)
* Migrate File source to CDK structure

* fix .dockerignore file

* remove SAT requirements

* update Dockerfile

* change Dockerfile to base images python:3.7-slim

* add SAT tests

* update tests

* add secret/config.json for source-file

* update changelogs
2021-10-29 13:15:13 +03:00
Marcos Marx
2c8151fef2 Add creds for strava, lemlist, amazon sqs and freshservice (#7388)
* add creds for strava, lemlist, amazon sqs and freshservice

* correct source name in ci_creds
2021-10-26 11:42:36 -03:00
Yevhenii
928aa6280c 🐛 Source Hubspot: Fix issue with getting 414 HTTP error for streams (#6954)
* Source Hubspot: Fix issue with getting 414 HTTP error for streams

* update code and schemas

* bump version
2021-10-26 15:37:07 +03:00
Eugene
9a0b8dac6c 🎉Source Clickhouse: added ssl support and "strict-encrypt" connector (#7127)
* [Ticket 6435] Source-ClickHouse: added ssl support

* [Ticket 6716] Source-ClickHouse: added secure only connector
2021-10-25 13:33:25 +03:00
Marcos Marx
c83cafc5c1 add salesloft credentials github wf and ci script (#7275) 2021-10-22 01:28:46 -03:00
Marcos Marx
4efadb228c add onesignal credentials + gradlew format (#7270) 2021-10-21 23:55:06 -03:00
irynakruk
265986b5bc Create a secure-only MongoDb destination (#6945)
* Added mongodb destination strict encrypt
2021-10-20 09:04:30 -04:00
Maksym Pavlenok
059645fad4 🎉 Source File: separate secure fork (#6768)
* create a fork for the source-file connector

* update docs and tests

* update docs and tests

* add secrets

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* refactoring after reviews

* correction of doc

* update spec file



Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
2021-10-20 12:28:26 +03:00
Harsha Teja Kanna
3e7f95c25a 🎉 Support build on MacOS M1 (Apple Silicon) (#7104)
- See this doc for details: https://github.com/airbytehq/airbyte/blob/master/docs/contributing-to-airbyte/developing-locally.md
- Unit test does not work yet.
2021-10-19 11:20:21 -07:00
Jared Rhizor
0f93224769 kube build fixes (#7139)
* upgrade ec2-github-runner

* remaining build fixes

* fix build
2021-10-18 14:46:17 -07:00
Charles
f194f354c7 move basic normalization to connectors_base build (#7068) 2021-10-15 17:26:08 -07:00
Maksym Pavlenok
592f7fe2b2 🐛 Source ZenDesk Support: fix 404 responses for the ticket_comments stream (#6513)
* fix 404 responses for the ticket_comments stream

* add unit test

* add unit test

* add oauth2 access token

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* Update docs/integrations/sources/zendesk-support.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* add log message for 404 errors

* Update docs/integrations/sources/zendesk-support.md

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* remove oauth logic from this PR

* remove oauth logic from this PR

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Davin Chia <davinchia@gmail.com>
2021-10-16 01:20:41 +03:00
Maksym Pavlenok
713fe61efd 🎉 New Source: TikTok Marketing (#5887)
* create a new connector folder

* add base classes

* add schemas

* add ads/ad_groups streams

* update tests and docs

* add a bootstrap file

* update the base domain name for API

* update the base domain name for API

* update Dockerfile libs

* remove unused comments

* Update airbyte-integrations/connectors/source-tiktok-marketing/bootstrap.md

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* Update airbyte-integrations/connectors/source-tiktok-marketing/bootstrap.md

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* fix UI error with input parameters

* fix problem with updated state

* Update airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/spec.py

Co-authored-by: Davin Chia <davinchia@gmail.com>

* add an unit test

* update README.md

* bump version

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Davin Chia <davinchia@gmail.com>
2021-10-15 19:24:14 +03:00
Baz
cfae84cbb0 🎉 Source Shopify: support oauth flow (#6951) 2021-10-14 22:37:30 +03:00
Christophe Duong
c4620559d7 🎉 Refactor Normalization docker images and upgrade to use dbt 0.21.0 (#6959)
* Split normalization docker images for some connectors with specifics dependencies

* Regenerate (#7003)
2021-10-14 20:29:16 +02:00
Arthur Galuza
bfe865aebf 🎉 Source Google Sheets: Support connecting via oAuth webflow (#6354)
* Add service account support

* Upd oauth support

* Upd auth creds selector
2021-10-13 23:02:22 +03:00