jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-01-04 09:04:47 -05:00
Code Issues Packages Projects Releases Wiki Activity
2,364 Commits 5,149 Branches 518 Tags
fd5c5be352bfa2ca744abe0ece76dc7aed1b2b8a
Commit Graph

233 Commits

Author SHA1 Message Date
Abhi Vaidyanatha
d985fe73f4 Add note about rate limits with the HubSpot API (#4491) 
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-07-01 23:04:35 -07:00
Sherif A. Nada
9708e4c8de 🐛 bugfix Source Okta: Fix endless loop while syncing data from logs stream (#4464) 
Co-authored-by: sabifranjo <sabifranjo@gmail.com>
Co-authored-by: Sabolc Franjo <sabolc.franjo@ev-box.com>
 2021-07-01 17:39:51 -07:00
Eugene
4399656ec9 🎉 Source Cockroachdb: Added new source for Cockroachdb (#4382) 
* [1705] Added cockroachdb source connector
 2021-07-01 22:04:21 +03:00
Sherif A. Nada
71c7cecd74 Source postgres: document that replicating views is supported (#4008) 2021-06-28 00:11:37 -05:00
Kirill Chernov
40aed2249d 🎉 fix Google Adwords CI & publish new version (#4205) 
* added SAT tests, deleted incremental test sfrom pipeline

* pr changes and tests uncommented

* deleted facebook file from pr

* deleted facebook file from pr

* removed tap adwords from install_requires

* Update airbyte-integrations/connectors/source-google-adwords-singer/acceptance-test-config.yml

* added changelog info

* version bump

* version bump

Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
 2021-06-26 01:45:22 +03:00
LiRen Tu
8954a99290 Bump source looker connector version (#4350) 
* Bump looker connector version

* Update changelog

* Format code
 2021-06-25 11:23:16 -07:00
Oleksandr
0e0d0a102e 🐛 Bugfix for pagination in Shopify Source Connector #4284 (#4290) 
#4284 - Bugfix for pagination in Shopify Source Connector

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
 2021-06-25 19:45:45 +03:00
Michael Irvine
842f10bd83 🎉 Source Looker: add run-look endpoint (#3911) 
* looker run_looks method works with normalization

* update catalogs

* validate that look(s) entered actually exist at config time

* field type lookup works, though it is a bit messy

* don't re-run explore query if we have already run it

* add more type mappings

* add some docstrings and comments

* add title for ease of use

* Revert "update catalogs"

This reverts commit 7ecaaa4d0d.

* fix configuration file formatting

* fix some issues from code review and reorder methods

* update docs to include run_look endpoint
 2021-06-25 08:31:56 -07:00
Yaroslav Dudar
2ccb1c2de0 🎉 New source: AWS CloudTrail (#4122) 
AWS CloudTrail connector initial release
 2021-06-25 11:10:07 +03:00
Marcos Marx
77f7096d14 🐛 Source Mailchimp: use datacenter parameter from apikey info (#4285) 
* replace datacenter parameter using apikey info

* change to use dynamic property

* add dynamic property

* bump version

* change config versino
 2021-06-24 21:55:13 -03:00
Sherif A. Nada
c130465048 Release Shopify source 0.1.6 (#4332) 2021-06-24 16:39:35 -07:00
jacqueskpoty
e01576d461 🎉 Source Shopify: add pages, price rules and discount codes streams (#4009) 
Co-authored-by: jsewawilson <jwilson@pebblepost.com>
 2021-06-24 16:18:15 -07:00
Harshith Mullapudi
f4ddf4042d 🐛 Bugfix incorrectly failing check connection operations in Google Ads and Amazon AP API connectors (#4288) 2021-06-24 15:58:37 -07:00
irynakruk
0a4712cbe7 🎉 New Source: IBM DB2 (#4197) 
* Db2 source: implementation and acceptance tests

* Db2 source: added documentation, build info, Db2SourceComprehensiveTest

* Db2 source: improved Db2SourceComprehensiveTest

* Db2 source: improved Db2SourceComprehensiveTest

* Db2 source: updated docs

* Db2 source: added test for the special values in Decfloat

* Formatter changes

Co-authored-by: Iryna Kruk <iryna.o.kruk@globallogic.com>
 2021-06-23 18:06:31 +03:00
Eugene Kulak
2f7c15a540 Source Facebook: migrate to CDK (#3743) 
* move to CDK implementation
* make insights streams configurable
* improve Async implementation using slices
* minimize set of streams in configured_catalog.json
* apply suggestions from @sherifnada
* improve async jobs handling even more

Co-authored-by: Ka-Ping Yee <zestyping@gmail.com>
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-06-22 18:48:17 -04:00
Vadym
4c2fc3ba00 Bump harvest version (#4277) 
* Bump source Harvest docker version

* Update harvest.md docs
 2021-06-22 20:22:02 +03:00
Marcos Marx
156629ca95 Bump Harvest Source connector 0.1.2 (#4223) 
* fixes #4221. Fix broken spec file property and point to correct docs page

* bump harvest connector

* add changelog for harvest

* bump dockerfile version

Co-authored-by: Tyler Schroeder <tyler@tylerschroeder.com>
 2021-06-21 15:22:48 -03:00
Marcos Marx
88d7953a18 Add CHANGELOG for Sources (#4215) 
* add changelog for postres source

* add changelog for mysql source

* add changelog for mssql source

* add changelog for file source

* add changelog for google sheets

* add full changelog for file source

* add full changelog for google sheet source

* add full changelog for mssql source

* add changelog for http requests

* add changelog for salesforce source

* add changelog for facebook marketing source

* add changelog for google analytics source

* add changelog for shopify native source connector
 2021-06-18 17:45:09 -03:00
LiRen Tu
14b605e064 Bump version and fix format (#4167) 
* Bump version

* Move changelog to connector doc

* Fix flakeCheck

* Fix format
 2021-06-16 13:20:31 -07:00
Harshith Mullapudi
78d17bc1a4 Source Amazon Seller Partner: Error for max wait time reached when create report status is CANCELLED or FATAL (#4108) 
* 1. Use pendulum - user friendly
2. Fix: Error for max wait time reached when create report status is CANCELLED or FATAL

* Write test cases and changelog
 2021-06-16 11:59:40 -07:00
LiRen Tu
4e449e98a5 Bump jira version (#4126) 2021-06-15 03:34:47 -07:00
Harshith Mullapudi
5a37359389 🎉 Added getting started for google ads connector (#4125) 2021-06-15 17:52:19 +08:00
irynakruk
446ec0407f 🎉 New Source: Snowflake DB (#3844) 2021-06-10 12:52:47 -07:00
vovavovavovavova
07b09a9a12 🎉 Posthog: New Source (#3768) 
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-06-08 16:55:05 -07:00
Oleksandr
01dc7bbde9 🎉 Add Airbyte-native shopify connector (replaces Singer-based) (#3787) 
* first commit

* Delete source copy.py

File doesn't need

* IAdded Incremental Refresh + stream_slice for Refunds, Transactions

* small changes to source.py

* fixed Incremental Refresh

* Fixed acceptence-tests

* Edited no-refunds-catalog for acceptence-tests

* changed the state

* Commit for Review

* removed spaces

* added parameter to orders stream to read all the records by default

* added parameter to abandoned_checkouts stream to read all the records by default

* minor fix

* minor fix

* changed authentication method from basic http > header based auth, removed usage of api_key

* reformated source.py

* improved source.py, removed dependencies block from build.gradle

* removed unneeded dependencies + formated source.py

* changed Shopify in source_definitions.yaml from singer-based to cdk version, changed years in the documents

* added source_definition for source-shopify cdk json file

* Delete airbyte-config/init/bin/main/config/STANDARD_SOURCE_DEFINITION directory

* Add files via upload

* added secret credentials for source-shopify

* Delete 9da77001-af33-4bcd-be46-6252bf9342b9.json

* re-added source_definition file

* Delete airbyte-config/init/bin/main/config/STANDARD_SOURCE_DEFINITION directory

* re-added source definition

* Delete airbyte-config/init/bin/main/config/STANDARD_SOURCE_DEFINITION directory

* edit source definitions

* source definition change: removed icon block

* ...

* changed source_definition.json

* changed source_definition.json

* Added ability to specify the start_date of data replica.

* changed docker version to 0.1.1 - bumped version on image, aedited request_params for metafields

* changed docker version to 0.1.1 - bumped version on image, aedited request_params for metafields

* changed docker version to 0.1.1 - bumped version on image, aedited request_params for metafields

* changed collects to full_refresh by default

* changed collects to full_refresh by default

* Delete airbyte-config/.settings directory

* Delete .classpath

* Delete .project

* Delete .project

* Delete .classpath

* Delete airbyte-config/init/.settings directory

* Delete airbyte-config/init/bin directory

* Delete airbyte-config/models/.settings directory

* Delete airbyte-config/models/bin directory

* Delete .classpath

* Delete .project

* Delete airbyte-config/persistence/.settings directory

* Delete airbyte-config/persistence/bin directory

* Delete .classpath

* Delete .project

* changed python to python3 in README.md

* small changes to code source.py, bump the version, edited .gitignore

* edited spec.json with pattern for start_date

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
 2021-06-08 12:46:35 +03:00
Yevhenii
a90e5f0eca 🎉 New Source: Amplitude (#3664) 
* New Amplitude Source: Full Refresh and Incremental Sync

* update api.py, add CHANGELOG.md file

* bump version

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
 2021-06-08 10:36:31 +03:00
Vadym
e699f39740 🎉 New Source: Harvest (#3709) 
* Add harvest source.
Add Clients, Contacts, Company, Invoices, InvoiceMessages, InvoicePayments streams full_refresh support.

* Add all Harvest source streams.
Add config and schema files for all the streams.

* Complete streams implementation.
Add config and schemas.

* Fix expenses stream calls.
Fix ExpensesBase request_params method.
Add source docs and definitions.

* Update harvest.md docs.

* Comment source_definitions.yaml harvest definition.

* Delete Harvest standard source definitions.

* Add Harvest creds to commands.

* Update Harvest build.gradle.

* Add extra report streams schemas.
Update config files.
Update incremental streams mixin.

* Fix HarvestStreamWithPaginationSliced stream_slices method for incremental stream support

* Remove source-acceptance-test test requirement

* * Fix schema.
* Update streams
* Update docs

* Update billable_rates and cost_rates to full_refresh only

* Fix HarvestStreamIncrementalMixin request_params method

* Fix BillableRates and CostRates streams initialization

* Refactor streams.py code common usages.
Update HarvestTokenAuthenticator class definition.

* Add streams docstring.
Fix sub streams slicing.
Update spec and schemas.
Update md docs.

* Update report streams to be support incremental sync.
Update report schemas.

* Removed Uninvoiced and ProjectBudget incremental sync support.
Fix configs and configured catalog.

* Fix streams cursor value.
Update IncrementalReportsBase request_params.
Update schemas and config.

* Add harvest source definitions.
 2021-06-07 15:11:30 +03:00
Abhi Vaidyanatha
a262f46e42 Add changelog for Stripe connector (#3832) 
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-06-03 15:40:55 -06:00
vitaliizazmic
2072b77e97 🎉 Source Jira: Support all available entities in Jira Cloud 
* Jira source #1389  - adding schemas for streams

* Jira source #1389  - supporting streams

* Jira source #1389  - creating_project script

* Jira source #1389  - updating docs

* Jira source #1389  - fixing check method

* Jira source #1389  - uploading missing schemes

* Jira source #1389  - disabling JQL and Server info streams

* Jira source #1389 - fixing according to PR comments

* Jira source #1389 - fixing filter_sharing and screen_tab_fields streams

* Update airbyte-integrations/connectors/source-jira/source_jira/client.py

* Google search console source #2257 - improving configured catalog(adding sync_mode and destination_sync_mode to streams)

* Jira Source - incremental sync

* Jira source #1390 - issues incremental sync

* Jira source #1390 - issue worklogs incremental sync

* Source Jira #1390 - incremental sync improving

* Source Jira #1390 - migrating to airbyte-cdk, creating CHANGELOG.md

* Source Jira #1389 - reformat

* Jira Source HTTP CDK

* Source Jira #3453 - cleaning branch

* Source Jira #3453 - cleaning branch (fix)

* Source Jira #3453 - abstractmethod get_updated_state

* Jira dummy data #2100 #2101

* Jira source #2100 - data generator

* Jira source #2100 - issue related streams populating

* Jira source #2100 - project related streams populating

* Jira source #2101 - populating data for non issue or project related streams

* Source Jira #2100 - improving according to comments

* Source Jira #2100 - format

* Source Jira #1389 - bump version

* Source Jira #1389 - enabling base_read acceptance test divided by stream groups

* Source Jira #1389 - bump version

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-06-02 22:13:36 +03:00
Gabriel Rebessi
eade88efa7 New Source: Okta (#3563) 
* Add source-okta

* Add PR suggestions

* Remove typo

* Remove unused file

* hard code page size

* add missing fields to catalog.json

* Add missing fields to catalog

* Add docs to Okta

* Add credential for okta

* Update docs/integrations/README.md

* Fix linting

* fix dependency conflict

* Fix integration tests

* update ci_credentials

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-06-02 14:03:00 -03:00
oleh.zorenko
4b3e345866 Source Amazon SP: Add missing files (#3782) 
* Add missing files

Add files for publishing the connector.
Update typing in few files.
Add `amazon-seller-partner.md` file.
Add `Amazon Seller Partner` to `builds.md` and to `README.md` files.

* Remove release files

* Comment out tests in `acceptance-test-config.yml`

* Update few files

Remove `!Dockerfile.test` from `.dockerignore` file.
Add `dependencies` to `build.gradle` file.

* Update `amazon-seller-partner.md` file

* Add stream to docs

Add `GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL` stream.

* Add release info

Add connector to `source_definitions.yaml` file.
Add connector to `STANDARD_SOURCE_DEFINITION` folder.
 2021-06-01 22:55:38 +03:00
Phlair
9360ed0da9 🎉 File source: Add support for Azure blob storage (#3660) 
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2021-06-01 10:42:13 +08:00
Marcos Marx
769d8b1216 removing <details> gitbook doenst support it (#3749) 2021-05-31 19:14:40 -03:00
Marcos Marx
2435012a46 Update exchange source doc and add changelog exp (#3682) 
* update exchange source doc and add changelog exp

* add break line and 0.1.0 commit
 2021-05-30 20:06:59 -03:00
John Lafleur
a0a6d02824 Update asana.md 2021-05-27 16:38:23 +11:00
oleh.zorenko
5aa03643cf Update source-exchange-rates connector (#3566) 2021-05-26 21:59:42 +03:00
oleh.zorenko
61cb866faa Fix typo in docs/integrations/sources/asana.md (#3589) 2021-05-25 20:10:30 +03:00
oleh.zorenko
c8711abbd6 🎉 New Source: Asana (#3510) 
* Asana source

* Fix creds for CI.

* Update connection status in acceptance test config

Change status from `exception` to `failed`.

* Implement change request.

Remove few files from /integration_tests folder.
Use `stream_slices` and/or `request_params` functions instear of `read_records` function

* Update sample_config.json file.

* Update airbyte-integrations/connectors/source-asana/CHANGELOG.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update `stream_slices` using.

Create reneric `read_stream` function in AsanaStream class and move there logic from `stream_slices` function.

* Rename functions.

rename `read_stream` to `read_slices_from_records`.

* Changes about publishing.

Add asana source to `source_definitions.yaml`.
Add `asana.svg`.
Create connector related file in `STANDARD_SOURCE_DEFINITION` folder.

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-05-25 18:55:46 +03:00
Subodh Kant Chaturvedi
7ccc4fafe8 🎉 source: implementation for mysql cdc (#3505) 
* source: implementation for mysql cdc

* add target file and position

* dont want to add file in this PR

* refine tests + add comments

* fix typo

* address review comments

* fix formatting error

* resolve conflicts

* update docs + bump docker minor version

* remove un-necessary new lines + add multiple checks for cdc

* address review comments from Davin

* increase the version in source_definitions.yaml

* rebuild seed
 2021-05-25 00:31:57 +05:30
Marcos Marx
df3d37650d correct smarthsheet markdown (#3568) 2021-05-24 15:19:33 -03:00
John Lafleur
e563936c2c Updated description of Clickhouse source connector (#3556) 
* Updated description of Clickhouse source connector

The first sentence of the overview seemed to indicate it only supported Full Refresh, while it supports both full refresh and incremental

* Update clickhouse.md

* Update clickhouse.md
 2021-05-24 20:26:52 +11:00
Yevhenii
3647c3cf1d 🎉 New Source: Recharge (#3466) 
* Recharge Connector: Full Refresh add Incremental syncs, Acceptance tests, docs

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
 2021-05-20 14:45:52 +03:00
Vadym
f9feb2e903 🎉 Source Slack bugfix: don't overwrite thread_ts in threads stream (#3483) 
* setup.py install_requirements updated

* setup.py formatting fix.
slack.md docks update.

* Bump source-slack-singer docker version
 2021-05-20 12:38:05 +03:00
Eugene Kulak
d2eb5c0564 🎉 New Source: Klaviyo (#3463) 
* Klaviyo source

* add test creds

* format and docs

* remove accidentally committed files

* fix build

* format

* restore page size

* add creds to CI

* update configured_catalog.json

* register in default sources

* fix spec

* Update airbyte-integrations/connectors/source-klaviyo/source_klaviyo/source.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* address comments from @sherifnada

* dump spec and enable spec test

* fix acceptance-test-config.yml

* fix configured_catalog.json

* fix configured_catalog.json

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-05-19 18:36:09 -04:00
Marcos Marx
a18b0e80d0 GitBook: [master] 161 pages and 75 assets modified 2021-05-16 22:51:02 +00:00
Davin Chia
fbbd25069c Correct GA readme error. (#3407) 2021-05-14 09:58:18 +08:00
Yevhenii
d44fc16e65 Source Stripe: Add Acceptance Tests to Stripe Connector (#3367) 
* Add Acceptance Tests to Stripe Connector

* move configured_catalog.json to sample_files

* bump version

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
 2021-05-13 18:40:24 +03:00
Subodh Kant Chaturvedi
69125cfbeb source: clickhouse setup docs (#3375) 
* clickhouse source setup docs

* fix typo

* address review comment
 2021-05-13 13:35:10 +05:30
vitaliizazmic
735b0783dc 🎉 New Source: google search console 
* Google search console source #2257 - new source

* Google search console source #2257 - reformat

* Google search console source #2257 - adding gcc to docker container

* Google search console source #2257 - remove unused files, update acceptance test configs, change tap repo to airbyte

* Google search console source #2257 - updating acceptance tests configs

* Google search console source #2257 - updating acceptance cursor_paths

* Google search console source #2257 - temporary disable tests

* Google search console source #2257 - disable performance_report_date stream

* Google search console source #2257 - disable performance_report_date stream (update docs)

* Google search console source #2257 - disable performance_report_date stream for tests

* Google search console source #2257 - updating singer tap fork
 2021-05-10 09:59:44 +03:00
Sherif A. Nada
32773cb7f8 update FB marketing rate limiting link (#3294) 2021-05-07 17:32:36 -07:00