jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-22 03:21:25 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

31012 Commits

Author SHA1 Message Date
Christophe Duong
257b14ef13 Fix duplicated paragraph in docs and broken link (#3377) 2021-05-12 19:35:25 +02:00
Christophe Duong
80125fe8a3 Add support for Operations to Sync/Connections API (#3244) 
* Add Operations (for normalization and custom DBT) to API

* Add Operations to StandardSync backend configuration objects (#3325)
 2021-05-12 10:49:53 +02:00
John Lafleur
e4715e3019 Update connectors.md 2021-05-12 15:30:51 +11:00
John Lafleur
f7829f4c45 Update platform.md 2021-05-12 15:21:20 +11:00
Davin Chia
e9731eb698 Bump version: 0.22.2-alpha → 0.22.3-alpha (#3374) 2021-05-12 10:18:23 +08:00
Abhi Vaidyanatha
40e664fcec Add guide on how to upgrade a connector (#3369) 
* Add guide on how to upgrade a connector.

* Update intrusctions on navigating to DockerHub

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-05-11 16:03:04 -07:00
vitaliizazmic
735b0783dc 🎉 New Source: google search console 
* Google search console source #2257 - new source

* Google search console source #2257 - reformat

* Google search console source #2257 - adding gcc to docker container

* Google search console source #2257 - remove unused files, update acceptance test configs, change tap repo to airbyte

* Google search console source #2257 - updating acceptance tests configs

* Google search console source #2257 - updating acceptance cursor_paths

* Google search console source #2257 - temporary disable tests

* Google search console source #2257 - disable performance_report_date stream

* Google search console source #2257 - disable performance_report_date stream (update docs)

* Google search console source #2257 - disable performance_report_date stream for tests

* Google search console source #2257 - updating singer tap fork
 2021-05-10 09:59:44 +03:00
Abhi Vaidyanatha
4a397d2524 Rewrite and break out CDK Python overview to be more concise and digestible (#3291) 
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-05-07 20:30:21 -07:00
Sherif A. Nada
32773cb7f8 update FB marketing rate limiting link (#3294) 2021-05-07 17:32:36 -07:00
Abhi Vaidyanatha
05bdb368a0 Reorganize CDK documentation directory structure to be less nested (#3270) 
* Move CDK docs down a level, consolidate directories.

* Change name of tutorial

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-05-07 12:50:33 -07:00
Subodh Kant Chaturvedi
a2ec9ee1e5 mysql destination docs (#3275) 
* setup doc for mysql destination

* update the documentation in right places
 2021-05-07 13:38:07 +05:30
Abhi Vaidyanatha
0c3cca2f24 Document usage of ./gradlew format and workaround for files without import statements (#3269) 
* Make Spotless license check look for variables.

* Test and confirm.

* Document real workaround.

* Address review comments.

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-05-06 18:33:01 -07:00
Jared Rhizor
beeee785f2 clarify publication and replication identity handling in cdc docs (#3258) 
* clarify publication and replication identity handling in cdc docs

* remove character
 2021-05-06 17:34:42 -07:00
Abhi Vaidyanatha
e2d5bfa3ea Add primary_key noop to docs and PokeAPI connector. (#3245) 
* Add primary_key noop to docs and PokeAPI connector.

* Small grammar fix in FAQ.

* Change empty string to None for noop.

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-05-06 10:54:42 -07:00
John Lafleur
2dd604c1eb Update README.md (#3231) 2021-05-06 04:35:03 +11:00
Abhi Vaidyanatha
b215733e22 Add source connector for the PokéAPI (#3149) 
* Add toy source for the PokeAPI

* Address review comments.

* Run gradlew format

* Move Pokemon list constant to its own file

* Fix gradle formatting issues

* Prepare connector for publishing.

* Fix some naming bugs

* Update config file path

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-05-04 19:18:35 -07:00
Ganesh Raskar
36784c60d4 🎉 Added tutorial to deploy Airbyte on Azure VM #3170 (#3171) 
* Added tutorial to deploy Airbye on Azure VM #3170

* Updated summary.md to have link to deploy on azure vm

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* emended article and added step to download ssh key

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* Update docs/deploying-airbyte/on-azure-vm-cloud-shell.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* grouped bash commands and made few corrections

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2021-05-04 23:09:39 -03:00
John Lafleur
f942f7f6ff Update connectors.md 2021-05-05 11:39:53 +11:00
John Lafleur
67c156bbba Update platform.md 2021-05-05 11:31:44 +11:00
John Lafleur
6032026424 Update platform.md 2021-05-05 11:30:26 +11:00
Abhi Vaidyanatha
4710e5836f Adding concepts docs, structure for CDK documentation (#3220) 
* Adding concepts docs, structure for CDK documentation

* Add more context to the CDK README

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-05-04 17:09:29 -07:00
Jared Rhizor
32ee10e76d snowflake gcs staging (#3183) 
* spec

* snowflake gcs

* spec comment

* fmt

* remove sout

* add docs

* format

* respond to comments

* fix

* bump to 0.3.3
 2021-05-04 09:59:32 -07:00
Davin Chia
a1ca5d5427 Bump version: 0.22.1-alpha → 0.22.2-alpha (#3205) 2021-05-04 12:36:44 +08:00
Charles
bdbd48a245 Bump version: 0.22.0-alpha → 0.22.1-alpha (#3194) 2021-05-03 15:38:44 -07:00
John Lafleur
fa882621ca Quick access to the 2 different changelogs when displaying main changelog (#3164) 2021-05-04 09:01:22 +11:00
Yevhenii
6bc63264e7 🎉 New Source: Zendesk Chat (#3088) 
* Zendesk Chat connector: implement full_refresh sync

* format

* Source Zendesk Chat: Implement Incremental sync (#3157)

* Zendesk Chat: Implement Incremental sync

* Roll back the stream Chats to the previous endpoint

* update integration-base-python to v0.1.6

* update limit to 100

* add acceptence tests

* update test and setup configs

* remove unknown file

* update naming of classes

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>

* register Zendesk Chat as source

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
 2021-05-03 16:02:00 -04:00
Marcos Marx
8742bce8bd GitBook: [master] one page modified 2021-05-03 12:15:09 +00:00
Marcos Marx
7c70eb02cd GitBook: [master] 84 pages and 72 assets modified 2021-05-03 12:14:16 +00:00
John Lafleur
9a45f7ca04 Update connectors.md 2021-05-03 18:49:18 +11:00
John Lafleur
fe77828896 Update connectors.md 2021-05-03 18:48:27 +11:00
John Lafleur
35fa7bee63 Update SUMMARY.md 2021-05-01 17:12:21 +11:00
Marcos Marx
237b99e774 correct typo in mysql source docs (#3143) 
* correct mysql source docs

* change bigint order
 2021-04-30 18:45:40 -03:00
Yevhenii
672ebb49a3 Source MongoDB: add supporting TLS and Replica Sets (#3111) 
* MongoDB Source: add config for TLS and Replica Sets

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
 2021-04-30 22:57:34 +03:00
Christophe Duong
77ffd74b32 Ignore records that are too big in Redshift destinations (instead of failing) (#2988) 
* Abort sync if one of the part fails to copy to temp table

* Check for record size when copying data from s3 to redshift

* Handle big record in RedshiftInsertDestination too
 2021-04-30 21:04:03 +02:00
Abhi Vaidyanatha
d940c78307 Add Python CDK Alpha Tutorial to Docs (#3147) 
* Fix Transformations article link

* Add Python CDK Alpha documentation.

* Make requested changes, add to SUMMARY.md

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-04-30 11:10:04 -07:00
Eugene Kulak
7f66569436 Standard Tests: allow specifying exact record matches #2186 (#2960) 
* records matching

+ extra_fields
+ exact_order
+ extra_records

* fix serialization before comparision

* fix pydantic validation

* fix reporting and enable for hubspot

* format

* update docs

* update examples

* fix template

* Update docs/contributing-to-airbyte/building-new-connector/source-acceptance-tests.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update docs/contributing-to-airbyte/building-new-connector/source-acceptance-tests.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-04-30 04:21:35 -04:00
John Lafleur
7d0a794bb4 Update stitchdata-vs-airbyte.md 2021-04-30 16:25:15 +11:00
John Lafleur
1084bb5758 Update fivetran-vs-airbyte.md 2021-04-30 16:11:49 +11:00
John Lafleur
ea5b1a931b Update platform.md 2021-04-30 10:25:25 +11:00
John Lafleur
761f0f77d7 Update connectors.md 2021-04-30 10:14:01 +11:00
Abhi Vaidyanatha
e378d40236 Documentation Reorganization (#3124) 
* First reorganization pass.

* Stop auto-generating api docs html file.

* Update spelling

* Final cleanup.

* Final changes_REAL_actual_2_thisone

* fix path for generating api docs

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: jrhizor <me@jaredrhizor.com>
 2021-04-29 15:18:52 -07:00
Eugene Kulak
34433d972c Remove refences to ./acceptance-test-docker.sh (#3138) 2021-04-29 17:30:24 -04:00
Marcos Marx
e7007268f1 add some connectors to connector health page (#3099) 
* add oracle to connector health page

* add appstore and appsflyer
 2021-04-28 22:03:04 -03:00
Eugene Kulak
88b77aa9f3 Standard tests: update template (#2937) 
* update template for python sources
* update templates and their docs
* update docs, rename folders, add acceptance script
* adding hooks for custom setup and teardown
+ improve plugin behaviour, enable it explicitly
+ add acceptance.py to run acceptance tests together with integration tests
+ remove helper acceptance-test-python.sh
+ fix gradle command for acceptance tests
+ update docs

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-04-28 18:11:48 -04:00
Abhi Vaidyanatha
0904f9dda2 Command line fixes for toy connector tutorial. (#3101) 
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-04-27 20:53:31 -07:00
Davin Chia
4f03fc4603 Release 0.22.0-alpha. (#3097) 2021-04-28 07:40:13 +08:00
Abhi Vaidyanatha
cc876499da Fix Postgres replication tutorial command (#3095) 
* Update docker exec command to pull replicated rows.

* Grammar fix on Getting Started guide

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-04-27 15:28:58 -07:00
Sherif A. Nada
4e9f4bdd06 CDK: tutorial for implementing an HTTP source (#3079) 2021-04-27 00:28:10 -07:00
Davin Chia
e63ab8490a Make Redshift part size configurable. (#3053) 
The Redshift Copy strategy currently has it's part size set to 10 MB. Since S3 allows a file to be broken into max of 10k parts, this results in a 100GB table limit. A user is trying to sync a table of 115GB and running into this issue.

This makes the part size configurable so users can increase this size if needed.
 2021-04-27 13:35:35 +08:00
Christophe Duong
4a795f6d7a Update upgrade docs (#2991) 
* Update upgrade docs

* Apply changes from review
 2021-04-26 18:58:02 +02:00