jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-01-03 15:04:01 -05:00
Code Issues Packages Projects Releases Wiki Activity
3,680 Commits 5,149 Branches 518 Tags
b94ee00fd8e5220e3191073cd711a57e052d7fda
Commit Graph

145 Commits

Author SHA1 Message Date
Marcos Marx
0e09ac3032 Publish PR 6963: new source freshsales (#7585) 
* initial commit

* finish implementing full_refresh

* add other tables

* cleaning up

* add docs and use requests_native_auth

* fix function return different number of values

* change package author

* fix schema

* fix schema bug

* linting

* fix unit test

* clean up

* add null for schemas

* remove fc_widget_collaboration col

* change accpt test

* add creds

* run format

* add config files

* run format

Co-authored-by: Tuan Nguyen <anhtuan.nguyen@me.com>
 2021-11-03 01:06:38 -03:00
Harshith Mullapudi
8ddf686e9e added credentials to CI for new source sentry 
* add credentials to CI for new source sentry

* add ci cred
 2021-11-02 19:55:58 +05:30
Marcos Marx
c846188772 Publish PR 6966: new source retently (#7552) 
* retently

* PR comments

* PR comments

* mask api keys

* README

* keep gradle happy

* add github creds

* run format

Co-authored-by: Subhash <subhash@kloud.io>
 2021-11-01 23:58:16 -03:00
Marcos Marx
fe118e676e Publish PR 4551: new source delighted correct format (#7540) 
* Add Delighted source with people, unsubscribes and bounces streams

* Add survey responses stream

* Add documentation for Delighted source

* publish delighted source

* correct acceptance test

Co-authored-by: Rodrigo Parra <rodpar07@gmail.com>
 2021-11-01 22:09:13 -03:00
Harshith Mullapudi
bdaec1cf3f add credentials in for New Source Paystack: #7214 (#7529) 
* added credentials to ci for new source paystack

* format
 2021-11-01 23:49:53 +05:30
Marcos Marx
2c8151fef2 Add creds for strava, lemlist, amazon sqs and freshservice (#7388) 
* add creds for strava, lemlist, amazon sqs and freshservice

* correct source name in ci_creds
 2021-10-26 11:42:36 -03:00
Marcos Marx
c83cafc5c1 add salesloft credentials github wf and ci script (#7275) 2021-10-22 01:28:46 -03:00
Marcos Marx
4efadb228c add onesignal credentials + gradlew format (#7270) 2021-10-21 23:55:06 -03:00
Maksym Pavlenok
059645fad4 🎉 Source File: separate secure fork (#6768) 
* create a fork for the source-file connector

* update docs and tests

* update docs and tests

* add secrets

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* refactoring after reviews

* correction of doc

* update spec file



Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
 2021-10-20 12:28:26 +03:00
George Claireaux
1d3a17a8fb 🎉 Source S3 - memory & performance optimisations + advanced CSV options (#6615) 
* memory & performance optimisations

* address comments

* version bump

* added advanced_options for reading csv without header, and more custom pyarrow ReadOptions

* updated to use the latest airbyte-cdk

* updated docs

* bump source-s3 to 0.1.6

* remove unneeded lines

* Use the all dep ami for python builds.

* ec2-instance-id should be ec2-image-id

* ec2-instance-id should be ec2-image-id

Co-authored-by: Jingkun Zhuang <Jingkun.Zhuang@icims.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2021-10-19 16:50:51 +01:00
Davin Chia
214b158bde Add labels to EC2 build machines. Refactor actions to reduce duplicate start ec2 runner actions. (#7119) 
As the title.

I also took the chance to compose the actions, instead of injecting it everywhere.
 2021-10-19 18:02:18 +08:00
Maksym Pavlenok
592f7fe2b2 🐛 Source ZenDesk Support: fix 404 responses for the ticket_comments stream (#6513) 
* fix 404 responses for the ticket_comments stream

* add unit test

* add unit test

* add oauth2 access token

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* switching among auth methods

* Update docs/integrations/sources/zendesk-support.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* add log message for 404 errors

* Update docs/integrations/sources/zendesk-support.md

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* Update airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/streams.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* remove oauth logic from this PR

* remove oauth logic from this PR

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2021-10-16 01:20:41 +03:00
Maksym Pavlenok
713fe61efd 🎉 New Source: TikTok Marketing (#5887) 
* create a new connector folder

* add base classes

* add schemas

* add ads/ad_groups streams

* update tests and docs

* add a bootstrap file

* update the base domain name for API

* update the base domain name for API

* update Dockerfile libs

* remove unused comments

* Update airbyte-integrations/connectors/source-tiktok-marketing/bootstrap.md

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* Update airbyte-integrations/connectors/source-tiktok-marketing/bootstrap.md

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* fix UI error with input parameters

* fix problem with updated state

* Update airbyte-integrations/connectors/source-tiktok-marketing/source_tiktok_marketing/spec.py

Co-authored-by: Davin Chia <davinchia@gmail.com>

* add an unit test

* update README.md

* bump version

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2021-10-15 19:24:14 +03:00
Baz
cfae84cbb0 🎉 Source Shopify: support oauth flow (#6951) 2021-10-14 22:37:30 +03:00
Arthur Galuza
bfe865aebf 🎉 Source Google Sheets: Support connecting via oAuth webflow (#6354) 
* Add service account support

* Upd oauth support

* Upd auth creds selector
 2021-10-13 23:02:22 +03:00
Arthur Galuza
99b1e91c65 🎉 Source Pipedrive: add oAuth support (#6821) 
* Add oauth support

* Upd changelog

* Update airbyte-integrations/connectors/source-pipedrive/source_pipedrive/spec.json

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-10-12 09:45:46 +03:00
Arthur Galuza
bf3189cd12 🎉 Source Google Analytics v4: Declare oauth parameters in google sources (#6414) 
* Upd auth: oauth support

* Rename authorization/credentials, upd spec, refactor

* Add backward compatibility

* Upd CI

* Bump version

* Upd changelog
 2021-10-08 12:38:40 +03:00
Yevhenii
00d56e7711 🎉 Source Slack: Implement OAuth support with OAuth authenticator (#6570) 
* Source Slack: Implement OAuth support with OAuth authenticator
 2021-10-07 14:01:15 +03:00
Sherif A. Nada
b470489983 🧹delete marketo singer source (#6800) 2021-10-06 22:53:21 -07:00
Vadym
351e998394 🎉 Source Hubspot: Support OAuth (#6465) 
* Add required scopes to hubspot.md docs

* Update spec with oauth creds.
Update acceptance test config with oauth tests.
Add oauth test creds configs.

* Fix spec.js `oneOf` usage

* Add CDK authenticators usage.
Update source acceptance tests.
Update spec.json.

* Fix source unit tests

* Update hubspot.md scopes docs

* Add spec field titles.
Update comments.

* Update spec.json oauthFlowOutputParameters.

* Update spec.json to pass SAT.
 2021-10-04 17:11:34 +03:00
oneshcheret
11561290be 🎉 Source MSSQL: Implement SSH bastion for acceptance/integration tests (#6608) 
* 🎉 Source MSSQL: Implement SSH bastion for acceptance/integration tests

* 🎉 Source MSSQL: remove ci creds for mssql source
 2021-10-04 16:08:41 +03:00
VitaliiMaltsev
c2561141fc 🎉 Destination Oracle - Added support for connection via ssh tunnel (#6370) 
* Oracle destination with ssh tunneling

* add ssh key integration testing for Oracle destination

* fix checkstyle

* add container approach to Oracle Destination Integration Tests

* add container approach to Oracle Destination Integration Tests

* add container approach to Oracle Source Integration Tests/ delete secrets for Oracle db

* Oracle Destination - bump version to 0.1.8

* remove oracle ssh secrets from ci_credentials.sh

Co-authored-by: vmaltsev <vitalii.maltsev@globallogic.com>
 2021-09-28 14:25:33 +03:00
Baz
2f835e4e39 🎉 New Source: LinkedIn Ads (#5285) 
Resolving: https://github.com/airbytehq/airbyte/issues/2589

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
 2021-09-25 01:58:58 +03:00
Eugene
6658ecd5aa 🎉 Source Oracle: Added support for connection via SSH tunnels (#6356) 
* [5968] Source Oracle - Added option to connect to Oracle DB via SSH (aka bastion server)
 2021-09-24 20:52:47 +03:00
irynakruk
5551698002 Source MongoDb: added support via TLS/SSL (#6364) 
* updated ssl configs;
fixed replica connection;
fixed incremental read;

* added MongoDbSourceAtlasAcceptanceTest with ssl enabled

* updated docs, moved TLS option for standalone instance type, enabled it by default for other types

* Update mongodb-v2.md

* updated README.md

* updated spec.json

* Code review changes

* updated ci_credentials.sh
 2021-09-24 09:22:54 -04:00
Serhii Lazebnyi
26c9546a1c 🎉 New Source: Marketo (#5863) 
* Add Marketo implementation

* Updated PR in docs

* Add bootstrap file

* Update to review

* Add gcc to docker
 2021-09-23 18:51:47 +03:00
Yevhenii
63425923d0 Remove deprecated connectors: source-google-adwords-singer, source-google-search-console-singer, source-googleanalytics-singer (#6378) 2021-09-23 15:24:49 +03:00
Yevhenii
73162062db 🎉 New Source: Lever Hiring (#6141) 
* Lever Hiring - New Connector
 2021-09-22 12:51:00 +03:00
Oleksandr Sheheda
59cd5ca308 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966) (#6318) 
* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)

* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
updated image version

* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
correction of remarks

* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
bumb image version
 2021-09-21 17:25:04 +03:00
midavadim
12cb3e3f9f 🎉 Source greenhouse: added identification of accessible streams for API keys with limited permissions (#6238) 
* Added identification of accessible streams for API keys with limited permissions

* added credentials for API key with limited permissions (for user stream only)

* update docs and image version

* updated error message

* fixed formatting
 2021-09-21 15:02:12 +03:00
Eugene
11f64882a9 🎉Source MySQl - Added support for connection via ssh (aka bastion server) (#6146) 
* Source MySQL - Added ssh support
 2021-09-21 10:57:31 +03:00
Dmytro
7a6da86baa add coverage report (#6045) 
Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
 2021-09-15 10:50:07 +03:00
LiRen Tu
e837048418 🎉 New Destination: Databricks (#5998) 
Implement new destination connector for databricks delta lake.
Resolves #2075.

Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-09-14 16:55:53 -07:00
Anna Lvova
03e1e08047 🎉 New Source: Close.com (#5366) 
* added close com source

* added docs

* added activities streams

* added tasks streams

* updated doc

* fixed indent

* added docs

* add PR to doc

* fix doc

* fix doc

* update after review

* update after review

* remove streams from empty_streams

* move path from streams to Mixin classes

* update README

* add streams

* update doc

* fix opportunities cursor_field

* remove Roles stream from tests

* add Roles stream to SAT after fix in 5738 PR

* add bootstrap.md

* add bootstrap.md

* update after review

* update after review

* remove state_checkpoint_interval
 2021-09-13 17:44:31 +03:00
Arthur Galuza
c0d4652852 🎉 New Source: Google Search Console (#5350) 
* Generate Google Search Console connector

* Add schema

* Upd schema

* Upd authenticator

* Add creds retrieving script

* Remove legacy dep

* Upd dockerfile base image

* Add sample config

* Upd source definitions, add ci_credentials injection

* Upd schema

* Upd GSC creds injection

* Cleanup

* Add tzdata

* Upd tzdata installing

* Change base docker image

* Upd streams

* Fix typo

* Upd supported_sync_modes list

* Add multiple site, service account, search type support

* Fix typo

* Upd streams, pagination, multi site support

* Add service account secrets

* Remove source-google-search-console-singer from source definitions

* Upd creds retrieving base image

* Upd schema

* Upd docs

* Add badge

* Upd schema

* Upd docs

* Move the cursor field to the top

* Upd docs
 2021-09-10 11:47:02 +03:00
vovavovavovavova
aa9786df4e 🎉 Google Ads improvement: Support user-specified queries (#5302) 
*Add google ads custom queries stream

*Display link to gradle scan on PR comment if test build failed
 2021-09-10 09:17:16 +03:00
Eugene
bb043b7f50 🐛 Destination BigQuery Denormalized: Fixed compilation error (#5917) 
* Fixed destination bigquery denormalized compilation error (caused by https://github.com/airbytehq/airbyte/pull/5614)
 2021-09-09 12:54:46 +03:00
Yevhenii
fdb4c852a8 🎉 Salesforce Native Connector: init version (#5619) 
* Native Salesforce Connector

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
 2021-09-08 14:00:08 +03:00
Charles
8ad43afb07 SSH for Postgres Destination (#5743) 
Co-authored-by: George Claireaux <phlair@users.noreply.github.com>
 2021-09-07 17:06:25 -07:00
Maksym Pavlenok
e5c44e64b1 🎉 Source S3: support of Parquet format (#5305) 
* add parquet parser

* add integration tests for partquet formats

* add unit tests for parquet

* update docs and secrets

* fix incorrect import for tests

* add lib pandas for unit tests

* revert changes of foreign connectors

* update secret settings

* fix config values

* Update airbyte-integrations/connectors/source-s3/source_s3/source_files_abstract/formats/parquet_spec.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* Update airbyte-integrations/connectors/source-s3/source_s3/source_files_abstract/formats/parquet_spec.py

Co-authored-by: George Claireaux <george@claireaux.co.uk>

* remove some unused default options

* update tests

* update docs

* bump its version

* fix expected test

Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
Co-authored-by: George Claireaux <george@claireaux.co.uk>
 2021-09-05 02:40:49 +03:00
Charles
7bf531a967 SSH for Postgres Source (#5742) 2021-09-02 11:32:04 -07:00
Arthur Galuza
736126330a 🎉 New Source: Facebook Pages (#5158) 
* [WIP] Add basic facebook pages support

* Upd schema

* Hardcode metrics to reduce API call amount

* Refactoring

* Upd test configs

* Add secrets to CI

* Fix typo

* Add a source definition

* fix secrets in CI

* Cleanup with black

* Upd metrics

* Cleanup

* Rm start_date, add test

* Upd schema

* Cleanup

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-09-01 19:11:35 +03:00
Eugene
61842ed7c3 🎉 Destination Azure blob storage: introduced new connector with jsonl and csv formats (#5332) 
* [3447] Added destination Azure blob storage connector (CSV and JSONL formats)
 2021-08-29 12:23:59 +03:00
LiRen Tu
073a1d53d1 Publish source bigcommerce (#5722) 
* Fix integration test

* Format code

* Setup ci credential

* Add doc links
 2021-08-28 15:22:22 -07:00
Marcos Marx
6633eb4f60 add dynamo credentials (#5604) 2021-08-27 14:04:26 -03:00
LiRen Tu
61a18af5a8 Publish source bamboo hr (#5698) 
* Add secret variable for source bamboo hr

* Add bamboo hr to connector catalog
 2021-08-27 04:28:49 -07:00
Dmytro
3630cdeded 🎉 New Source: Amazon Ads (initial version) (#5023) 
* 3772: Amazon Ads source, initial version


Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-08-26 11:15:38 +03:00
Serhii Lazebnyi
2ee462a9a1 🎉 New Source-connector for Google Analytics v4 (#5290) 
* Add Google Analytics v4 implementation

* Add docs and connector index

* Fix a broken link to Airbyte CDK

* Fix a broken link to source acceptance tests docs

* Add condition for a full refresh or incremental stream

* Add unit tests

* Fix formatting to flake8

* Updated to review

* Added logger for custom reports validation

* Add comments to code

* Updated to format
 2021-08-25 17:55:08 +03:00
Serhii Lazebnyi
ce5d8d6758 🎉 New Source: Trello (#5501) 
* Add Trello implementation

* Add doc string to TrelloAuthenticator

* Add spaces to Docker file

* Add rate limit balancer

* Add loger to load balancer
 2021-08-25 14:29:50 +03:00
Marcos Marx
088a138c0e Add oracle dest creds to get from secrets (#5578) 
* add oracle dest creds to get from secrets

* add oracle in publish and test workflow
 2021-08-23 16:11:18 -03:00