jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-22 03:21:25 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

31012 Commits

Author SHA1 Message Date
Marcos Marx
b414f694eb Add troubleshooting section and remove FAQ file (#3902) 
* add troubleshooting section and remove FAQ file

* add windows new tutorial to summary

* remove change on normalization

* add eof

* transfer user handbook to docs

* review comments and made changes
 2021-06-08 20:31:22 -03:00
etsybaev
eec3e03ddb 🎉 Added JDBC source connector bootstrap template (#3819) 
* [3560] Added JDBC source connector bootstrap template
 2021-06-08 21:04:10 +03:00
Oleksandr
01dc7bbde9 🎉 Add Airbyte-native shopify connector (replaces Singer-based) (#3787) 
* first commit

* Delete source copy.py

File doesn't need

* IAdded Incremental Refresh + stream_slice for Refunds, Transactions

* small changes to source.py

* fixed Incremental Refresh

* Fixed acceptence-tests

* Edited no-refunds-catalog for acceptence-tests

* changed the state

* Commit for Review

* removed spaces

* added parameter to orders stream to read all the records by default

* added parameter to abandoned_checkouts stream to read all the records by default

* minor fix

* minor fix

* changed authentication method from basic http > header based auth, removed usage of api_key

* reformated source.py

* improved source.py, removed dependencies block from build.gradle

* removed unneeded dependencies + formated source.py

* changed Shopify in source_definitions.yaml from singer-based to cdk version, changed years in the documents

* added source_definition for source-shopify cdk json file

* Delete airbyte-config/init/bin/main/config/STANDARD_SOURCE_DEFINITION directory

* Add files via upload

* added secret credentials for source-shopify

* Delete 9da77001-af33-4bcd-be46-6252bf9342b9.json

* re-added source_definition file

* Delete airbyte-config/init/bin/main/config/STANDARD_SOURCE_DEFINITION directory

* re-added source definition

* Delete airbyte-config/init/bin/main/config/STANDARD_SOURCE_DEFINITION directory

* edit source definitions

* source definition change: removed icon block

* ...

* changed source_definition.json

* changed source_definition.json

* Added ability to specify the start_date of data replica.

* changed docker version to 0.1.1 - bumped version on image, aedited request_params for metafields

* changed docker version to 0.1.1 - bumped version on image, aedited request_params for metafields

* changed docker version to 0.1.1 - bumped version on image, aedited request_params for metafields

* changed collects to full_refresh by default

* changed collects to full_refresh by default

* Delete airbyte-config/.settings directory

* Delete .classpath

* Delete .project

* Delete .project

* Delete .classpath

* Delete airbyte-config/init/.settings directory

* Delete airbyte-config/init/bin directory

* Delete airbyte-config/models/.settings directory

* Delete airbyte-config/models/bin directory

* Delete .classpath

* Delete .project

* Delete airbyte-config/persistence/.settings directory

* Delete airbyte-config/persistence/bin directory

* Delete .classpath

* Delete .project

* changed python to python3 in README.md

* small changes to code source.py, bump the version, edited .gitignore

* edited spec.json with pattern for start_date

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
 2021-06-08 12:46:35 +03:00
Dmytro
66d96967df 🎉 Containerize connector code generator (#3763) 2021-06-08 12:25:27 +03:00
Yevhenii
a90e5f0eca 🎉 New Source: Amplitude (#3664) 
* New Amplitude Source: Full Refresh and Incremental Sync

* update api.py, add CHANGELOG.md file

* bump version

Co-authored-by: ykurochkin <y.kurochkin@zazmic.com>
 2021-06-08 10:36:31 +03:00
jacqueskpoty
8c9bf60bd8 updating integrations destinations local files doccs (#3910) 
* updating integrations destinations local files doccs

* updating integrations destinations local files doccs

* Update docs/integrations/destinations/local-json.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* updating integrations destinations local files doccs

* updating integrations destinations local files doccs

* Update docs/integrations/destinations/local-csv.md

Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>

* Update docs/integrations/destinations/local-csv.md

Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>

* Update docs/integrations/destinations/local-csv.md

Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>

* Update docs/integrations/destinations/local-csv.md

Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>

* updating integrations destinations local files doccs

Co-authored-by: jsewawilson <jwilson@pebblepost.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>
 2021-06-07 21:34:35 -03:00
Sherif A. Nada
73c8dfa7e2 Add Harvest to summary.md 2021-06-07 09:16:11 -07:00
Christophe Duong
913c55df6b 🎉 Add dataset location to BigQuery connector (issue #3277) (#3889) 
Authored-by: sabifranjo <sabifranjo@gmail.com>
Authored-by: Sabolc Franjo <sabolc.franjo@ev-box.com>
 2021-06-07 15:33:47 +02:00
Vadym
e699f39740 🎉 New Source: Harvest (#3709) 
* Add harvest source.
Add Clients, Contacts, Company, Invoices, InvoiceMessages, InvoicePayments streams full_refresh support.

* Add all Harvest source streams.
Add config and schema files for all the streams.

* Complete streams implementation.
Add config and schemas.

* Fix expenses stream calls.
Fix ExpensesBase request_params method.
Add source docs and definitions.

* Update harvest.md docs.

* Comment source_definitions.yaml harvest definition.

* Delete Harvest standard source definitions.

* Add Harvest creds to commands.

* Update Harvest build.gradle.

* Add extra report streams schemas.
Update config files.
Update incremental streams mixin.

* Fix HarvestStreamWithPaginationSliced stream_slices method for incremental stream support

* Remove source-acceptance-test test requirement

* * Fix schema.
* Update streams
* Update docs

* Update billable_rates and cost_rates to full_refresh only

* Fix HarvestStreamIncrementalMixin request_params method

* Fix BillableRates and CostRates streams initialization

* Refactor streams.py code common usages.
Update HarvestTokenAuthenticator class definition.

* Add streams docstring.
Fix sub streams slicing.
Update spec and schemas.
Update md docs.

* Update report streams to be support incremental sync.
Update report schemas.

* Removed Uninvoiced and ProjectBudget incremental sync support.
Fix configs and configured catalog.

* Fix streams cursor value.
Update IncrementalReportsBase request_params.
Update schemas and config.

* Add harvest source definitions.
 2021-06-07 15:11:30 +03:00
John Lafleur
a7f8f76951 Update SUMMARY.md 2021-06-05 14:06:07 +11:00
Davin Chia
75960e25fd Bump version: 0.24.6-alpha → 0.24.7-alpha (#3907) 2021-06-05 09:47:29 +08:00
Haziq Roslan
1ad283b058 🎉 Added document for locating files in local destination on windows (#3892) 
* add coument for locating files in local destination on windows

* add document for locating files in local destination on windows

* Update docs/tutorials/locating-files-local-destination.md

Co-authored-by: Christophe Duong <christophe.duong@gmail.com>

* Update docs/tutorials/locating-files-local-destination.md

Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>

* Update docs/tutorials/locating-files-local-destination.md

Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>

* Update docs/tutorials/locating-files-local-destination.md

Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>

Co-authored-by: Haziq Roslan <haziq.roslan@digitas.com>
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
Co-authored-by: Abhi Vaidyanatha <abhi@airbyte.io>
 2021-06-04 17:15:22 -03:00
John Lafleur
c4b943326b Updated job desc with investors (#3901) 
* Updated people-operations-manager

* Update founding-dev-evangelist

* Update senior-software-engineer
 2021-06-05 06:38:14 +11:00
Abhi Vaidyanatha
46e62f1c4d Add documentation around what the reset button does. (#3878) 
* Add documentation around what the reset button does.

* real fix

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-06-04 11:07:20 -06:00
Sherif A. Nada
887c7cce18 Add more detailed instructions for how to publish new versions of connectors without using gradle (#3859) 2021-06-04 08:00:08 -07:00
Christophe Duong
3c763d9bfd Add namespaceDefinitionType to standardSync (#3813) 
* Add NamespaceDefinition

* Add namespaceFormat to standardSync (#3815)
 2021-06-04 10:59:30 +02:00
Sherif A. Nada
7ec2649fbf 📚 add simple gradle cheatsheet (#3861) 2021-06-03 22:40:35 -07:00
Sherif A. Nada
36e2b2b6e0 📚 Update documentation best practices (#3870) 2021-06-03 22:38:55 -07:00
Sherif A. Nada
73cdae370f Add S3 to SUMMARY.md 2021-06-03 22:26:09 -07:00
Davin Chia
2621e03dd4 Bump version: 0.24.5-alpha → 0.24.6-alpha (#3876) 2021-06-04 08:32:35 +08:00
Marcos Marx
de5d62352b Add kubernetes danger-warning for deploy (#3871) 
* add kubernetes danger-warning for deploy

* change note
 2021-06-03 20:31:58 -03:00
masonwheeler
8dadd1cebd Oracle destination implementation (#3498) 
Working implementation of Oracle destination

Co-authored-by: cgardens <giardina.charles@gmail.com>
 2021-06-03 16:27:09 -06:00
Abhi Vaidyanatha
a262f46e42 Add changelog for Stripe connector (#3832) 
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-06-03 15:40:55 -06:00
LiRen Tu
038b84832b 🎉 Add Java destination generator (#3820) 
* Add java destination generator

* Fix typos
 2021-06-03 09:42:29 -07:00
LiRen Tu
c13b9883e8 🎉 New destination: S3 (#3672) 
* Update README icon links

* Update airbyte-specification doc

* Extend base connector

* Remove redundant region

* Separate warning from info

* Implement s3 destination

* Run format

* Clarify logging message

* Rename variables and functions

* Update documentation

* Rename and annotate interface

* Inject formatter factory

* Remove part size

* Fix spec field names and add unit tests

* Add unit tests for csv output formatter

* Format code

* Complete acceptance test and fix bugs

* Fix uuid

* Remove generator template files

They belong to another PR.

* Add unhappy test case

* Checkin airbyte state message

* Adjust stream transfer manager parameters

* Use underscore in filename

* Create csv sheet generator to handle data processing

* Format code

* Add partition id to filename

* Rename date format variable
 2021-06-03 09:40:51 -07:00
Davin Chia
131f824201 Bump version: 0.24.4-alpha → 0.24.5-alpha (#3841) 2021-06-03 13:55:51 +08:00
Vivek Ayer
2580a64c3c docs: link to 'Transformations with SQL' page (#3835) 
- basic-normalization markdown page was missing a link.
 2021-06-02 22:04:12 -03:00
Abhi Vaidyanatha
9c11940acc 0.24.3 Release Notes (#3827) 
* Add Release notes for 0.24.1 -> 0.24.3

* gradle format

* Add amazon seller partner to SUMMARY.md

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-06-02 14:37:31 -06:00
vitaliizazmic
2072b77e97 🎉 Source Jira: Support all available entities in Jira Cloud 
* Jira source #1389  - adding schemas for streams

* Jira source #1389  - supporting streams

* Jira source #1389  - creating_project script

* Jira source #1389  - updating docs

* Jira source #1389  - fixing check method

* Jira source #1389  - uploading missing schemes

* Jira source #1389  - disabling JQL and Server info streams

* Jira source #1389 - fixing according to PR comments

* Jira source #1389 - fixing filter_sharing and screen_tab_fields streams

* Update airbyte-integrations/connectors/source-jira/source_jira/client.py

* Google search console source #2257 - improving configured catalog(adding sync_mode and destination_sync_mode to streams)

* Jira Source - incremental sync

* Jira source #1390 - issues incremental sync

* Jira source #1390 - issue worklogs incremental sync

* Source Jira #1390 - incremental sync improving

* Source Jira #1390 - migrating to airbyte-cdk, creating CHANGELOG.md

* Source Jira #1389 - reformat

* Jira Source HTTP CDK

* Source Jira #3453 - cleaning branch

* Source Jira #3453 - cleaning branch (fix)

* Source Jira #3453 - abstractmethod get_updated_state

* Jira dummy data #2100 #2101

* Jira source #2100 - data generator

* Jira source #2100 - issue related streams populating

* Jira source #2100 - project related streams populating

* Jira source #2101 - populating data for non issue or project related streams

* Source Jira #2100 - improving according to comments

* Source Jira #2100 - format

* Source Jira #1389 - bump version

* Source Jira #1389 - enabling base_read acceptance test divided by stream groups

* Source Jira #1389 - bump version

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-06-02 22:13:36 +03:00
Gabriel Rebessi
eade88efa7 New Source: Okta (#3563) 
* Add source-okta

* Add PR suggestions

* Remove typo

* Remove unused file

* hard code page size

* add missing fields to catalog.json

* Add missing fields to catalog

* Add docs to Okta

* Add credential for okta

* Update docs/integrations/README.md

* Fix linting

* fix dependency conflict

* Fix integration tests

* update ci_credentials

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2021-06-02 14:03:00 -03:00
Christophe Duong
368a7ce4ef Webackend Connection API routes should return Webbackend objects (#3806) 2021-06-02 12:33:50 +02:00
Dmytro
a1035e007b 📚 Update all Python connectors READMEs with minimum required version (#3760) 2021-06-02 12:35:29 +03:00
Abhi Vaidyanatha
bc44cae39b Add more references to the CDK and clarify destination support (#3791) 
* Add more references to the CDK and clarify destination support

* gradle format

Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
 2021-06-02 00:25:24 -06:00
John Lafleur
25acff5974 Update SUMMARY.md 2021-06-02 16:16:31 +11:00
John Lafleur
cde7dda6a5 Rename operations-manager.md to people-operations-manager.md 2021-06-02 16:15:38 +11:00
John Lafleur
f3820237ba Update operations-manager.md 2021-06-02 16:15:10 +11:00
oleh.zorenko
4b3e345866 Source Amazon SP: Add missing files (#3782) 
* Add missing files

Add files for publishing the connector.
Update typing in few files.
Add `amazon-seller-partner.md` file.
Add `Amazon Seller Partner` to `builds.md` and to `README.md` files.

* Remove release files

* Comment out tests in `acceptance-test-config.yml`

* Update few files

Remove `!Dockerfile.test` from `.dockerignore` file.
Add `dependencies` to `build.gradle` file.

* Update `amazon-seller-partner.md` file

* Add stream to docs

Add `GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL` stream.

* Add release info

Add connector to `source_definitions.yaml` file.
Add connector to `STANDARD_SOURCE_DEFINITION` folder.
 2021-06-01 22:55:38 +03:00
Sherif A. Nada
691f8b17cb 📚 Document CI secrets workflow for connector development (#3733) 2021-06-01 08:29:36 -07:00
Christophe Duong
bb4dcb1987 🎉 Remove hash when it is not necessary from normalization outputs (#3704) 
* Refactor `generate_new_table_name` using a table name registry class instead

* update normalization docs

* Enable MyPy

* Regenerate output files

* Closes https://github.com/airbytehq/airbyte/issues/2389

* Bumpversion normalization
 2021-06-01 17:07:22 +02:00
etsybaev
1fed4b0e2c 🐛 Destination MSSQL: fix issue with unicode symbols handling (ex.\u2028 showed as ?) (#3671) 
* [3552] Fixed MSSQL _airbyte_data field's type to support unicode
 2021-06-01 09:38:13 +03:00
Phlair
9360ed0da9 🎉 File source: Add support for Azure blob storage (#3660) 
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2021-06-01 10:42:13 +08:00
Marcos Marx
769d8b1216 removing <details> gitbook doenst support it (#3749) 2021-05-31 19:14:40 -03:00
John Lafleur
68b39e52e1 Removing the empty Connector Health page from the docs (#3747) 
* Removing the empty Connector Health page from the docs

* Update .gitbook.yaml
 2021-05-31 10:47:42 +11:00
Marcos Marx
2435012a46 Update exchange source doc and add changelog exp (#3682) 
* update exchange source doc and add changelog exp

* add break line and 0.1.0 commit
 2021-05-30 20:06:59 -03:00
Sherif A. Nada
248c285c6b remove connector health page and display all connectors on the root connectors page (#3478) 2021-05-29 09:09:58 -07:00
Charles
d16245fd0b Bump version: 0.24.3-alpha → 0.24.4-alpha (#3724) 2021-05-28 12:36:04 -07:00
etsybaev
07d7420731 📚 📝 Build a connector-The hard way: added missed import 
Co-authored-by: ievgeniit <ievgeniit>
 2021-05-28 08:46:22 -07:00
Subodh Kant Chaturvedi
7993c3f99b Bump version: 0.24.2-alpha → 0.24.3-alpha (#3703) 2021-05-28 15:36:30 +05:30
Charles
7b4b804e9b move the upgrading airbyte doc to the top level (#3674) 2021-05-27 21:16:32 -07:00
Sherif A. Nada
51f7a64e74 fix founding developer job posting markdown 2021-05-27 17:47:07 -07:00