jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-01-27 16:02:00 -05:00
Code Issues Packages Projects Releases Wiki Activity
8,873 Commits 5,399 Branches 518 Tags
3b0512fe87d3dac412e869b81dfbbc829fb01ac4
Commit Graph

3866 Commits

Author SHA1 Message Date
Xiaohan Song
2a1628529d remove un-used update/delete custom connector api (#20034) 
* remove un-used update/delete custom connector api

* also remove create path

* more tests to fix
 2022-12-05 10:11:59 -08:00
sarafonseca
c471be2913 🐛 Source RD Station Marketing: Fix stream analytics conversions (#18826) 
* Fix stream analytics conversions on RD Station Marketing

* update connector's version

* Fix PR number on changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-05 11:48:38 -05:00
Mahdi Dibaiee
34e85e3cf1 source-mailchimp: set primary_key for EmailActivity (#19023) 
* source-mailchimp: set primary_key for EmailActivity

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-05 11:47:41 -05:00
jerome DOUCET
5149108020 🐛 Source ElasticSearch: avoid too_long_frame_exception (#18134) 
* Fix: (elasticsearch source) avoid too_long_frame_exception

batch the queries on mapping with a arbitrary (but reasonable) chunk size
to avoid reaching the 4096 bytes limits url size.

* bump connector

* auto-bump connector version

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-05 08:33:23 -03:00
Philip Corr
b452d53e97 🎉 Source Shopify: Add a graphql products stream (#19789) 
* add graphql stream for shopify products

* Update docs

* add json schema back in

* Fix graphql source tests

* fix: format python files with flake8

* auto-bump connector version

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-04 18:00:56 -05:00
Serhii Chvaliuk
f73dc44858 Source: Facebook Marketing - action_breakdowns can be empty list - bugfix for #20016 (#20043) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-12-04 16:56:59 +02:00
Serhii Chvaliuk
e1700fdb21 Source: Facebook Marketing - action_breakdowns can be empty list (#20016) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-12-03 21:18:28 +02:00
Kris Sikora
b7d26812a8 Destination:PubSub Add Batching and Ordering configuration (#15705) 
* Add Batching and Ordering configuration

* Add changelog

* fix config for tests

* format files

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-02 20:13:15 -03:00
Vincent Koc
acc7b324a8 🐛Destination-snowflake: Resolved typo in documentation (#18326) 
* chore: update typo in source docs

* chore: bump pr number for change

* Update Dockerfile

* Update snowflake.md

Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2022-12-02 19:46:56 -03:00
resoliwan
88c9844265 Source file: fix pandas read_csv header none issue. (#18587) 
* Source file: fix pandas read_csv header none issue.

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2022-12-02 19:43:03 -03:00
Mohamed Magdy
a7f32c00e8 🎉 New source: Babelforce (#12700) 
* solve deleted file

* Set Babelforce version to `0.1.0`

* Update Babelforce source documentaiton with the PR link

* Remove `.python-version` file in Babelforce source from Git

* Fix typo in Babelfoce `spec` description field

* Remove redaundant `title` field from JSON schema in Babelforce source

* Add examples for the `date_created_from` and `date_created_to` Babelforce source spec

* Remove redaundant `static` `authenticator` method in Babelforce source

* Check if `date_created_from` less than `date_created_to`

In Babelforce source

* Add parsing details to `docstring`

For Babelforce source

* Fix fetching next page token

In Babelforce source

* Remove parent class `docstring`

In Babelforce source

* Use `pendulum` to parse date/time

In Babelforce source

* add babelforce to source index

* remove icon

* format files

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2022-12-02 19:29:04 -03:00
Serhii Chvaliuk
5fc2a6b354 Source smartsheets: OAuth2.0 support - disabled (#20017) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-12-02 23:26:40 +02:00
JJ Nilbodee
1e96e54e6e Source Zoom: upgrade CDK version (#19939) 
* upgrade cdk

* update changelog

* update changelog

* bump version & add readme

* auto-bump connector version

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-02 17:22:17 -03:00
Denys Davydov
bd96b6144e Source gitlab - fix duplicate records issue (#20023) 
* #20022 source gitlab - fix duplicate records issue

* #20022 source gitlab - upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-02 21:19:16 +02:00
oneshcheret
1fd4a03fda Source postgres: fix schema permission issue (#19024) 
* Source postgres: fix schema permission issue

* Source postgres: add test for schema permission issue

* Source postgres: format code

* Source postgres: added unit test

* Source postgres: bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
 2022-12-02 10:04:45 -08:00
Serhii Chvaliuk
764496e930 Source Slack - remove OAuth2.0 refresh_token (#19970) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-12-02 14:37:53 +02:00
Serhii Chvaliuk
3c6f9792e5 Source iterable: fix pendulum.parse memory leak (#19913) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-12-02 12:08:58 +02:00
Evan Tahler
1e160b2858 Faker V0.3 - CDK, consistent naming, determinism, display name, and SAT (#19490) 
* better comment to show sync types

* WIP - basics in place

* test cleanup

* emit `AirbyteEstimateTraceMessage`

* assert estimates emitted

* unit tests passing

* format

* flake

* check tests

* add newline

* lint

* check in with new file name

* bump acceptance test yml

* acceptance test progress

* deterministic purchases

* SAT making progress

* better disable

* adjust yaml

* change abnormal_state.json format

* per-stream state

* products for abnormal state

* products have state + incremental

* purchases state = total_user_records

* update docs and name

* disable proper version

* typo

* try disable 0.2.1

* kwargs

* v1.0.0

* lint

* backward_compatibility_tests_config

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-01 14:53:38 -08:00
Denys Davydov
5da3a7c2a8 Source Gitlab: fix stream schema (#19986) 
* #12901 source gitlab: fix schema

* #12901 source gitlab: upd changelog

* #12901 fix SAT

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-12-01 20:51:20 +02:00
Octavia Squidington III
77112b02bf Bump Airbyte version from 0.40.22 to 0.40.23 (#19941) 
Co-authored-by: davinchia <davinchia@users.noreply.github.com>
 2022-11-30 20:02:24 +00:00
Evan Tahler
e21500ee73 Evan/fix docs (#19942) 
* fix docs build

* remove errant <br>
 2022-11-30 11:50:43 -08:00
Siddhant Singh
4c0ffb6c40 🎉 New Source: SurveyCTO (#19371) 
* survey cto connector

* datetime fix

* removed some unused tests

* intergration tests

* acceptance test is passing

* airbyte_secret

* removed unused files

* removed key

* removed some unused items

* added the error handling

* catalog fix to add form_id

* config improvement

* full refresh test added

* incremental sync test passed

* docs added surveycto.md

* added the docs improvement

* improve docs and spec

* fix: add source definition for surveycto

* flake improvements

* refactor

* refactor

* python format fix

* auto-bump connector version

* docs update

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-30 13:26:12 -05:00
Evan Tahler
ea58882fa3 Fix affected connectors (#19865) 
* source-appsflyer

* source-cart

* source-chargebee

* source-lemlist

* source-linnworks

* source-mailgun

* source-pardot

* source-plaid

* source-prestashop

* source-quickbooks-singer

* source-recharge

* source-tplcentral

* remove icons that aren't real

* Remove TODOs

* add missing specs

* Update docs/integrations/sources/primetric.md

Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>

* bump readmes

* revert ci_check_deps logic

* New name

Co-authored-by: Ella Rohm-Ensing <erohmensing@gmail.com>
 2022-11-30 09:51:21 -08:00
Denys Davydov
fa8ea96e70 Source freshdesk - fix schema (#19496) 
* #1038 source freshdesk - fix schema

* #1038 source freshdesk: upd changelog

* #1038 source freshdesk - remove temp file

* #1038 source freshdesk: fix tests

* #1038 source freshdesk: update expected records again

* #1038 source freshdesk: rm unstable expected record

* #1038 source freshdesk: edit another expected record

* #1038 source freshdesk: edit another expected record

* #1038 source freshdesk: fix expected records

* #1038 source freshdesk: upd expected_records

* #1038 source freshdesk: resolve dependency conflict

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-30 15:19:43 +02:00
Jimmy Ma
5a2642af69 Airbyte Protocol v1 (#19846) 
* Fix versioning of protocol

* Update protocol changelog

* Update protocol $id

* Fix PR reference in the changelog
 2022-11-29 16:54:13 -08:00
Denys Davydov
1ab02de43b Source Zendesk Support: do not use nonpublic API (#19432) 
* #1010 oncall Source Zendesk Support: do not use nonpublic API

* #1010 source zendesk-support: upd changelog

* #1010 source zendesk-support: log an error during discover

* #1010 source zendesk support: fix unit test

* #1010 source zendesk support: fix expected records

* #1010 source zendesk support: upd expected records

* #1010 source zendesk support: another records update

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-30 01:25:21 +02:00
brad-meru
329f654e04 Update on-gcp-compute-engine.md (#18643) 
All caps SSH throws an error from gcloud now. Changed to lowercase ssh.
 2022-11-29 17:53:16 -03:00
Mark Roberts
87d69a9a7e DocS: Fix typo in BigQuery source documentation (#19543) 
Data type mapping table referred to "CockroachDb Type" instead of "BigQuery Type".
 2022-11-29 17:52:47 -03:00
btkcodedev
db836b3b5a 🎉 New Source: Pexels API [low-code cdk] (#18854) 
* New Source: Pexels API

* Resolved all comments

* feat: Added paginator, refactor: moved params to request options

* refactor: configured params, moved request option to referenced

* Corrected comments

* correct test and add pexels api to source def

* auto-bump connector version

Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-29 17:43:17 -03:00
Luke Hsiao
84ab12f93d Docs: Update building-a-python-destination.md (#19516) 
If a user used `append_dedupe`, they get the error:

```
{
  "type": "LOG",
  "log": {
    "level": "FATAL",
    "message": "1 validation error for ConnectorSpecification\nsupported_destination_sync_modes -> 2\n  value is not a valid enumeration member; permitted: 'append', 'overwrite', 'append_dedup' (type=type_error.enum; enum_values=[<DestinationSyncMode.append: 'append'>, <DestinationSyncMode.overwrite: 'overwrite'>, <DestinationSyncMode.append_dedup: 'append_dedup'>])..."
  }
}
```
 2022-11-29 17:42:36 -03:00
Anne
f07cef808d Disable broken connections when feature flag is on (#19730) 
When auto-detect schema changes feature flag is on, disable connections that have breaking schema changes and connections that have any schema changes where the user has set their preference to disable.
 2022-11-29 12:20:26 -08:00
Akash Kulkarni
865b580776 DB sources data type support - adding new data types (#19679) 
* Introduce JsonSchemaTypeUtil + refactor code

* Update JsonSchemaPrimitiveUtil.java

Comment

* Update dynamodb.md

* Create acceptance-test-config.yml

* fix merge break

* formatting

* Fix formatting

* Code cleanup

* Update DataTypeEnumTest.java
 2022-11-29 09:58:42 -08:00
Denys Davydov
34c0e953fb Source twilio: fix generating date ranges (#19479) 
* #1046 source twilio: fix generating date ranges

* #1046 source twilio - upd changelog

* #1046 source twilio: add freezegun to requirements

* #1046 source twilio: fix incremental tests

* #1046 source twilio: fix expected records

* #1046 source twilio - fix expected records

* #1046 source twilio: cut expected records

* #1046 source twilio: cut expected records

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-29 19:29:01 +02:00
oneshcheret
494349f966 Revert "Source postgres: mark termination connection PSQLException as config error (#19623)" (#19881) 
This reverts commit 92282ded3b.
 2022-11-29 18:10:09 +02:00
oneshcheret
92282ded3b Source postgres: mark termination connection PSQLException as config error (#19623) 
* Source postgres: catch termination connection PSQLException

* Source postgres: move common code to util method

* Source postgres: clean code

* Source postgres: review fixes and added unit tests

* Source postgres: clean code

* Source postgres: bump version

* Source postgres: format

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-29 16:47:09 +02:00
Augustin
b78fdadaf2 SAT: optionally disable discovered catalog caching (#19806) 2022-11-29 13:41:45 +01:00
Augustin
f262bef9d2 source-surveymonkey: fix getBaseURLByOrigin (#19868) 2022-11-29 11:51:56 +01:00
Denys Davydov
8ac5d50293 Source Salesforce: make AccountHistory support BULK API (#19869) 
* #17503 source salesforce: make AccountHistory support BULK API

* #17503 source salesforce: upd changelog

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-29 12:11:56 +02:00
Anne
8287362247 Add schema change attrs to connection create (#19771) 
* Add schema change attrs to connection create
 2022-11-28 16:53:19 -08:00
Evan Tahler
e73a5337ca Increase Database Source SELECT Batch Size (#19514) 
* Increase Database Source SELECT Batch Size

* 60% is a little safer

* comment

* format...

* Update byte-size tests

* fixup tests

* test fixes

* format

* readmes and connector versions

* match strict-encrypt versions

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-28 14:19:47 -08:00
Ben Church
61bf812105 Fix markdown typo in airbyte-protocol-docker.md (#19842) 
## Overview
I noticed the `I/O` heading was not displaying properly due to a missing space. 

This fixes that.
 2022-11-28 11:42:45 -08:00
Jimmy Ma
43e6a52413 Add versioning documentation (#19738) 
* On Deploying versioning error from the bootloader

* Add ProtocolVersion description to airbyte protocol

* Add general protocol versioning doc
 2022-11-28 11:32:09 -08:00
jerri-airbyte
10b701d8c9 Update slack-code-of-conduct.md 2022-11-28 11:04:48 -08:00
Andy
0b6999993d docs: Include alpha call out in lowcode builder UI docs (#19841) 
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
 2022-11-28 10:51:48 -08:00
Serhii Chvaliuk
e38bd5d510 Source: Facebook Marketing - New default for action_breakdowns, improve "check" speed (#19803) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-11-28 20:25:06 +02:00
Sophia Wiley
d155c49fd6 deleted onboarding info (#19576) 2022-11-28 12:59:40 -05:00
Simon Späti
b85efb1ac4 📝 Add reference to newly created Airbyte Monitoring Tutorial (#19598) 
* add reference from metric collection

* adding more context

* remove empty line

Co-authored-by: swyx <shawnthe1@gmail.com>
 2022-11-28 12:55:55 -05:00
Eugene
be530bbf2e [18713] Destination-bigquery: updated Check method to test for non-billable project (#19489) 
* [18713] Destination-bigquery: updated Check method to test for non-billable project

* fixed typo in comments

* Bumped version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-28 09:36:03 -08:00
Serhii Chvaliuk
ccda38bfbc Source zendesk support: bugfix transform "-" to null (#19792) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2022-11-28 19:03:02 +02:00
VitaliiMaltsev
d04a01a8ad Source Postgres: Handle Arrays data types (#16990) 
* Source Postgres: Handle Arrays data types

* bump version

* updated items mapping

* updated mapping of common types

* add datatype tests

* removed redundant variables

* Fixed getSpec schema test

* removed redundant variables

* refactoring

* updated json schema mappings

* updated debezium converters

* removed unused logging

* refactoring

* refactoring and added comments

* fixed checkstyle

* refactoring

* bump version

* auto-bump connector version

Co-authored-by: ievgeniit <etsybaev@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2022-11-28 17:53:26 +02:00