1
0
mirror of synced 2026-01-06 06:04:16 -05:00
Commit Graph

1541 Commits

Author SHA1 Message Date
Denis Davydov
e49faedaed Source Google Ads to GA: update spec and docs (#12807)
* #12650: source google ads to GA: mock input config for unit tests

* #12650 source google ads to GA: doc updates

* #12650 source google ads to GA: upd changelog

* #12650 source GAds to GA: doc review fixes

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-13 20:31:18 +03:00
LiRen Tu
7d16fdafa1 🐞 Postgres source: fix azure database catalog retrieval (#12834)
* Remove host in azure username

* Bump version

* Fix test

* Replace host with jdbc_url

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-13 01:35:00 -07:00
Marcos Marx
e0f821f6f3 Docs: remove deprecated variable for mysql binlog config (#12824) 2022-05-12 18:50:18 -03:00
Andy
a3658ba13a Update spec to replace markdown (#12797)
* Update spec to replace markdown

* Update version

* S3 Destination: Update change log

* auto-bump connector version

Co-authored-by: oneshcheret <sashkaliife@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-12 19:16:00 +03:00
Oleksandr Sheheda
259b2f585f Follow up on #2655 Clean up kafka source connector docs (#12343)
* 2655 Clean up kafka source connector docs

* 2655 Clean up kafka source connector docs

* 2655 Clean up kafka source connector docs
2022-05-12 17:07:32 +03:00
George Claireaux
440f3db8bb Destinations BigQuery & Snowflake: use latest base-java to emit AirbyteTraceMessage on errros (#12805)
* bumping BQ and Snowflake dests to get AirbyteTraceMessage from base-java

* add PR link to changelogs

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-12 14:23:29 +01:00
Serhii Chvaliuk
1c8b18890d Source Hubspot: ensure all oauth2.0 scopes in "check" command (#12711)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-05-12 12:50:43 +03:00
VitaliiMaltsev
4b5bd2b5a4 Redshift Destination Apply buffering strategy (#12601)
* Redshift Destination: Apply buffering strategy

* add manifest uploading

* refactoring

* fixed checkstyle

* updated CHANGELOG

* removed redundant static

* airbyte-12265: Added  stagingOperations.onDestinationCloseOperations() to StagingConsumerFactory.java.

* airbyte-12265: Created operations and copiers java packages.

* safe delete of RedshiftCopyS3Destination.java

* rename tests

* bump version

* auto-bump connector version

Co-authored-by: Oleksandr Tsukanov <alexander.tsukanovvv@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-12 12:30:54 +03:00
Baz
feb0d2f378 🎉 New Destination: Implement Destination Google Sheets using CDK (#12135) 2022-05-12 01:10:28 +03:00
Serhii Lazebnyi
225aecd37c 🐛Source Amazon S3: Fixed empty options issue (#12730)
* Fixed empty oprions issue

* Update airbyte-integrations/connectors/source-s3/source_s3/utils.py

Co-authored-by: Denis Davydov <denys.i.davydov@globallogic.com>

* Bumped version

* Fix typo

* Bumped seed version

* Fix changelog

* Bumped version in docker file

* auto-bump connector version

Co-authored-by: Denis Davydov <denys.i.davydov@globallogic.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-11 21:21:54 +03:00
Melker Öhrman
f9188590cc Add avro parser to s3 source (#12602)
* added MVP avro parser running fine locally

* added unit tests for avro

* added wip state of avro integration test setup

* deleted unused files

* added avro specific config path

* fixed comments. Added nested record support, simplify code and minor fixes

* bumped version + docs update

* Added working acceptance tests + format

* auto-bump connector version

Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-11 16:59:58 +01:00
Amruta Ranade
2617f833bb Updated connector statuses (#12765) 2022-05-11 17:45:04 +05:30
Davin Chia
f8a35eaa80 Add Java Catalog documentation. (#12751)
Clean up and add better guidelines on how to use the Java catalogs we recently added.

Took the chance to move existing documentation to improve reading flow.
2022-05-11 13:02:07 +08:00
Baz
0b2eb7a8fc Source Amplitude: fix the doc to match the template (#12698) 2022-05-10 10:56:28 +03:00
Denis Davydov
d4f8b25b8e Source Google Ads: Improve unit and integration tests (#12651)
* #12650 source Googel ads: tests

* #12650 source google ads: add changelog item

* #12650 source google ads: add comments to tests

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-09 18:34:28 +03:00
Serhii Lazebnyi
27e6ce2ca8 Source Amazon S3: Refactored docs (#12534)
* Refactored spec and docs

* Updated spec.json

* Rollback spec fromating

* Rollback spec fromating

* Rollback spec fromating
2022-05-09 14:56:52 +03:00
LiRen Tu
3e6151a35a 🎉 Source postgres: publish fix of missing streams (#12689)
* Add logging

* Bump connector version

* Mute fb errors

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-08 21:06:06 -07:00
Serhii Chvaliuk
886da80174 🎉 Source Amazon Ads: improve docs for beta (#12541)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-05-08 09:46:55 +03:00
Pedro S. Lopez
816fece247 bump GA connectors to latest version of python cdk (#12685)
* bump GA connectors to latest version of python cdk

* update changelogs

* fix changelogs

* more changelog fixes

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-06 20:52:59 -04:00
oneshcheret
cc6e18320b Postgres source: skipping columns with nullable values (#11760)
* Postgres source: skipping nullable columns during

* Postgres source: handling null value

* Postgres source: bump versions

* Postgres source: revert bump versions in definition yaml

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-06 16:49:06 -07:00
midavadim
22ecd282d2 🎉 Tiktok marketing - remove odd auth method (#12545)
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: ievgeniit <etsybaev@gmail.com>
2022-05-06 16:48:51 -07:00
Stéphane Kattoor
66d760955d Fix typo in spec.json for destination AWS Datalake & fix python license plugin setup (#11811) 2022-05-06 01:28:00 -07:00
Mahdi Dibaiee
06016055ab 🎉 source-facebook-marketing: allow configuration of page_size (#12171)
* source-facebook-marketing: allow configuration of page_size

* source-facebook-marketing: fix integration test for page_size conf

* fix: bin are added

* chore: update seed file

Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
2022-05-05 16:40:36 +05:30
Yurii Bidiuk
b3194b2200 🎉🐛: Source mongoDB: implement building JsonSchema with 'properties' for fields with type 'object' (#12428)
* mongodb: build JsonSchema with 'properties'

* add tests

* bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-05 12:56:07 +03:00
Dany Chepenko
4b9ff01d46 Source Hubspot: Added deals associations for quotes (#12198) 2022-05-05 09:00:25 +02:00
Sherif A. Nada
b8e147538c Update various connector input configs & docs copy (#12500) 2022-05-04 23:37:10 -07:00
Andy
5aed8e2147 Update connector specs (#12482) 2022-05-04 22:44:44 -07:00
LiRen Tu
bd59726668 Publish new connectors to log offending json fields in json to avro conversion for debugging (#12578)
* Publish new connectors to log debugging info in json to avro conversion

* Add pull request id

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-04 11:47:07 -07:00
Alexandre Girard
22b67d8528 source salesforce: use utf8 by default and iso as fallback (#12576)
* use utf8 by default and iso as fallback

* test both

* add comment

* Bump version
2022-05-04 11:04:18 -07:00
Pedro Calleja
4a13b60560 source amazon-seller-partner: allow to use IAM user arn or IAM role arn (#12523) 2022-05-04 17:26:27 +02:00
Edward Gao
43470a2a87 🎉 Destination Snowflake: Add option to stage encrypted files via S3; fix purge_staging_data (#12452) 2022-05-04 07:31:53 -07:00
Augustin
8ca229bb8f Source Jira: change wording for description of array fields (#10835) 2022-05-04 14:30:51 +02:00
Baz
b61f790682 🎉 Source Amplitude: increase unit_tests coverage up to 90% (#12479) 2022-05-04 13:12:20 +03:00
Alexandre Girard
90162b6e76 Encoding is ISO-8859-1 (#12552)
* Encoding is ISO-8859-1

* rename test

* bump

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-03 19:23:41 -07:00
Edward Gao
3d416129c7 🐛 Prevent sources from hanging if they have orphaned threads (#12544) 2022-05-03 18:48:43 -07:00
Denis Davydov
e6ee556f0c Source Google Analytics: always sync data from two days ago (#12426)
* #12013 source GA to Beta: always sync data from two days ago

* #12013 GA to Beta: fix changelog

* #12013 source GA to Beta: rm odd file

* #12013 Source GA to Beta: comment out integration tests

* #12013 expose isDataGolden field, assume missing field equals False

* #12013 expose isDataGOlden flag: reword docs

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-03 17:24:33 +03:00
George Claireaux
9ffd5bb0e8 Source Facebook Marketing: Attempt to retry failing jobs that are already split to minimum size (#12390)
* restart jobs that are already split to smallest size

* manager now fails on nested jobs hitting max attempts

* version bump

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-03 12:32:20 +01:00
Alexandre Girard
ce1936e491 Source Hubspot - Log instructions to update scopes when hitting 403 HTTP error (#12515)
* read workflows

* try except

* Revert "read workflows"

This reverts commit 86db8ff850.

* Check campaigns

* Check by reading data

* Check all streams

* requests_mock.ANY

* mock all http methods

* Try with get and post

* reset to master

* log name

* bump version
2022-05-02 17:45:50 -07:00
Alexandre Girard
1021428998 Destination Bigquery: Update Dataset location field description (#12528)
* Update field description

* reset

* Bump

* reset

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-02 17:45:36 -07:00
Ivica Taseski
aab4f34ca4 🎉 Source Mixpanel: Enhance streams to incremental (#11501)
* enhance streams to incremental

* revert file structure

* 11501

* add engage

* doc and bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-05-02 19:58:06 -03:00
jordan-glitch
98e5953d8d 🐛 Source Gitlab: allow container_expiration_policy to be nullable + fix null projects list (#11907)
* allow nullable container_expiration_policy

* Update Dockerfile

* Update source_definitions.yaml

* Update source_specs.yaml

* fix unspecified projects

* add doc update

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-05-02 18:44:21 -03:00
midavadim
915573e898 🎉 Source Tiktok Marketing - updated specs description (#12435)
* updated seed files

* updated change history

* updated docs

* updated connector version

* auto-bump connector version

* updated source specs

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-02 19:37:05 +03:00
LiRen Tu
35f2aa9aed 🎉 Jdbc sources: publish new version with adaptive fetch size (#12480)
* Default scaffold to use adaptive streaming config

* Switch more connectors to use adaptive streaming config

* Bump version for cockroach db

* Bump version for db2

* Bump mssql version

* Bump mysql version

* Bump oracle version

* Bump postgres version

* Bump redshift version

* Bump snowflake version

* Bump tidb version

* auto-bump connector version

* Fix db2 findbug issue

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* Fix more findbug issues

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* Fix findbug issue for mysql-strict-encrypt

* Fix findbugs issue for oracle source

* auto-bump connector version

* Remove suppress warnings annotation

* Fix oracle encrypt tests

* Fix oracle encrypt acceptance test

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-29 23:14:58 -07:00
Alexandre Girard
553024780f Destination Bigquery: Dataset location is a required field (#12477)
* Dataset location is required

* Update doc

* Bump versions

* reset

* Bump

* downgrade

* Revert "downgrade"

This reverts commit b482c54d6f.

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-29 15:34:43 -07:00
oneshcheret
fd7f21e071 S3 destination: updating docs regarding certification (#11966)
* S3 destination: updating docs regarding certification

* S3 destination: updating docs by new template

* Apply suggestions from code review

Co-authored-by: Andy <andy@airbyte.io>

* S3 destination: updating docs by new template

Co-authored-by: Andy <andy@airbyte.io>
2022-04-29 18:17:13 +03:00
Luis Gomez
1642630461 🐛 Source Intercom: Fixed filtering of conversation_parts (#12374) 2022-04-29 15:55:26 +02:00
Serhii Lazebnyi
28c04efa99 Source Instagram: refactor instagram docs (#12469)
* Updated doc to new template

* Fix typo
2022-04-29 14:26:10 +03:00
Eugene
6565315a04 Destination-Postgres: updated postgres.md documentation as per BETA template (#12279)
* Updated postgres.md documentation as per BETA template
2022-04-29 09:55:11 +03:00
Brian Lai
bb2da42fde source freshdesk, hubspot, mixpanel, paypal-transaction, salesforce, zendesk-support: adding fixtures to mock time.sleep for connectors that explicitly sleep (#12335)
* adding fixtures to mock time.sleep for connectors that explicitly sleep

* bump connector versions

* update changelog doc for each connector

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* remove version bump for freshdesk because connector tests are in a bad state

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-28 12:21:45 -07:00
Marcos Marx
e0902e6e30 🐛 Source Hubspot: correct createAt and updateAd data type (#12424)
* correct createAt and updateAd data type

* bump connectorversion

* bump connector version again
2022-04-28 13:00:28 -03:00