1
0
mirror of synced 2026-01-01 09:02:59 -05:00
Commit Graph

38 Commits

Author SHA1 Message Date
Evan Tahler
e6b06a88ac AirbyteEstimateTraceMessage (#18875)
* `AirbyteEstimateTraceMessage`

* Add PR number

* fix method name

* Lint

* Lint

* fix merge

* Update docs/understanding-airbyte/airbyte-protocol.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* `EstimateType` sub type in python

* lint

Co-authored-by: Davin Chia <davinchia@gmail.com>
2022-11-07 12:45:39 -08:00
Evan Tahler
02459e8354 Protocol Change: AirbyteControlMessage.ConnectorConfig (#17907)
* Protocol Change: AirbyteConfigMessage

* update PR link in docs

* Lint

* Update python files

* Update docs/understanding-airbyte/airbyte-protocol.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update docs/understanding-airbyte/airbyte-protocol.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* `AirbyteConfigMessage` -> `AirbyteConnectorConfigMessage`

* AirbyteOrchestratorMessage

* Update docs

* `AirbyteControlConnectorConfigMessage`

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2022-10-28 11:11:31 -07:00
Evan Tahler
9126dc21e3 Add missing generated python files (#18580) 2022-10-27 18:01:04 -07:00
Augustin
b564f3eb78 Protocol: make supported_sync_modes a required not empty list on AirbyteStream (#15591) 2022-10-19 15:22:25 +02:00
Alexandre Girard
18da341b91 Revert: #17047 Airbyte CDK: Improve error for returning non-iterable from connectors parse_response (#17707)
* Bump cdk version

* Revert "#17047 Airbyte CDK: Improve error for returning non-iterable from connectors parse_response (#17626)"

This reverts commit d9ad2725b1.

* Bump
2022-10-06 18:43:01 -07:00
Robert
d9ad2725b1 #17047 Airbyte CDK: Improve error for returning non-iterable from connectors parse_response (#17626)
* Improve airbyte cdk invalid message data type error message

* Test cdk invalid message data type custom error is raised

* Fix test to pass stream as a string

* Add valid record message data input type test

* Add object type and value to AirbyteRecordMessage validator message

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
2022-10-06 16:46:33 -07:00
Evan Tahler
dcfcb75d0f AirbyteLogMessage.stack_trace for logging messages with related (non-fatal) errors (#16479)
* Test log message from faker

* AirbyteLogMessage gains stack_trace

* Fixup spacing

* bump python protocol

* fixup additionalProperties in faker spec

* bump faker version

* update docs

* use lineSeparator vs \r\n

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-12 16:01:57 -07:00
Jimmy Ma
4fbe03bebe Add explicit tracking of Airbyte Protocol Version in ConnectorSpecification (#15340)
* Add Version to AirbyteMessage

* Move protocol version to ConnectorSpecification

* Add cdk generated protocol model

* Add protocol_version to the sample ConnectorSpec in the docs

* Update airbyte-protocol/protocol-models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* update doc

* Update CDK changelog

* Update CDK protocol model

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2022-08-29 14:46:56 -07:00
Evan Tahler
d449fb6067 All objects in the Airbyte Proticol have additionalProperties: true (#15081)
* All objects in the Airbyte Proticol have `additionalProperties: true`

* order of keys

* rebuild airbyte proticol for python CDK
2022-07-27 18:03:13 -07:00
Lake Mossman
c6d83b3239 Fix per stream state protocol backward compatibility (#14032)
* rename state type field to fix backwards compatibility issue

* replace usages of stateType with type
2022-06-22 17:42:02 -07:00
Charles
22b727c0ea Update Airbyte Protocol Docs (#13709) 2022-06-14 15:04:38 -07:00
Benoit Moriceau
704dd8b534 Update schema (#13573)
* Update schema

* generate python

* Stream as an object

* PR comments

* generate python

* rm unused required

* Description the state with no type

* Fix connector build

* Format

* format

Co-authored-by: cgardens <charles@airbyte.io>
2022-06-10 16:35:59 -07:00
Benoit Moriceau
5a7b1aaacd Bmoric/add namespace to protocol (#13356)
* Add namespace

* Matching python update

* Pr comments

* regenerate and format python cdk

* format

Co-authored-by: Pedro Lopez <pedroslopez@me.com>
2022-06-06 15:41:56 -07:00
Alexandre Girard
731c8bf178 Add airbyte protocol file (#13234) 2022-05-26 11:22:46 -07:00
Alexandre Girard
3894134d11 Bump year in license short to 2022 (#13191)
* Bump to 2022

* format
2022-05-25 17:56:49 -07:00
Pedro S. Lopez
b8703ea6a1 update CDK airbyte protocol models to fix master build (#12829) 2022-05-12 22:38:30 -04:00
Pedro S. Lopez
3c1eab3e27 generate AirbyeTraceMessage.type enum with descriptive class name (#12581)
* generate AirbyeTraceMessage `type` enum with descriptive class name

* add comment on `title` usage

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* apply changes to bases/airbyte-protocol

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2022-05-04 18:48:34 -04:00
LiRen Tu
a294f40383 Use jsongenerator 1.0.1 (#12554)
* Use jsongenerator 1.0.1

* Fix connectors base
2022-05-03 21:31:12 -07:00
LiRen Tu
81417e6728 Add connector metadata as sentry tags (#10475)
* Pass worker metadata to connector

* Fix compilation

* Pass in job id and image from worker

* Remove application version

* Add default job environment variables

* Add back removed comment

* Rename env map to job metadata

* Fix env configs

* Read connector from application

* Use empty string

* Remove println

* Fix unit test

* Fix compilation error

* Introduce constants for worker env

* Add worker env to ENV_VARS_TO_TRANSFER

* Pass into getWorkerMetadata map to all constructions

* Format code

* Format octavia cli

* Fix test compilation

* Fix typos
2022-03-09 07:36:03 -08:00
Eugene
44cb30aca5 🐛Destination-gcs\destination-bigquery(gcs) - updated check() method to handle that user has both storage.objects.create and storage.multipartUploads.create roles (#9121)
* [9044] Destination-gcs\destination-bigquery(gcs) - updated check() method to handle that user has both storage.objects.create and storage.multipartUploads.create roles
2022-01-10 22:00:16 +02:00
LiRen Tu
0ee4d09eed Revert "Fix connector base test (#9348)" (#9373)
This commit reverts #9348 (9bb28939ee) because it does not work. The `test_docker_runner[standard]` and `test_docker_runner[waiting]` test cases still fail transiently.
2022-01-09 22:17:24 -08:00
LiRen Tu
9bb28939ee Fix connector base test (#9348)
* Format code

* Add sleep back

* Revert wait time

* Decrease waiting time

* Adjust waiting time

* Delay container command
2022-01-09 20:06:13 -08:00
Christophe Duong
fa040da084 Implement protocol change for OAuth outputs (#7917)
* Change OAuth API

* Change protocol for new OAuthConfigSpecification

* Refactor OAuth classes and tests

* Remove webbackend source/destination creation

* Change from webback to normal API

* Implement new protocol change with OAuth specs

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* format

* format

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-11-15 14:36:50 +01:00
Christophe Duong
cdb476ecad Add auth config to oauth api (#7798)
* Change OAuth API

* Change protocol for new OAuth Spec (#7827)

* Add examples

* Add protocol object to api too

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-11-15 12:56:48 +01:00
vitaliizazmic
0c7f06018b 🐛 Airbyte CDK: transforming Python log levels to Airbyte protocol log levels
* Airbyte CDK native logger #1279 - transforming Python log levels to Airbyte protocol log levels

* Airbyte CDK native logger #1279 - add test for level types transforming

* CDK standard logger #1279 - test for transform critical level to fatal
2021-10-22 17:14:01 +03:00
George Claireaux
d660661b64 add google sheets oauth flow to server + fix auth rootObject type in protocol (#7131)
* adding google sheets oauth flow to server

* fix oauth type in protocol yaml

* bump sheets version in definitions

* added GDrive scope

* update sheets to master changes

* update protocol incl. cdk

* protocol typing for oauth rootobject

* format
2021-10-21 18:28:50 +01:00
vitaliizazmic
d5c04991bc 🎉 CDK: use standard logger with custom handler
* CDK - using native logger with custom formatter

* CDK - using native logger(fix exception and add trace)

* CDK - using native logger in AirbyteEntrypoint

* CDK - CHANGELOG.md

* CDK - remove unnecessary imports

* CDK - fixing according to PR review

* CDK native logger #1279 - annotations

* CDK native logger #1279 - fixing according to PR review

* CDK standard logger #1279 - tests

* CDK standard logger #1279 - reformat

* Airbyte CDK #1279 - improve docstrings

* Airbyte CDK #1279 - improve log levels

* Airbyte CDK #1279 - fix init get source name

* Airbyte CDK #1279 - update test licence

* Airbyte CDK #1279 - bump version
2021-10-12 20:56:56 +03:00
Christophe Duong
960214a48a Change Airbyte Protocol for OAuth and Make protocol more permissive (#6542) 2021-09-29 16:27:10 +02:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
Sherif A. Nada
89dccf3897 Modify oauthSpecification to allow working with oneOfs (#6456) 2021-09-27 18:00:36 +02:00
Baz
9c05308d1f 🐛 Source Shopify: Fixed line_items/discount allocations & duties parts of orders schema (#5801)
* edited orders schema
* added changelog
* formated

Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
2021-09-02 13:58:47 +03:00
Sherif A. Nada
75a6a75b43 [protocol] add oauth params to connector specification (#5776) 2021-09-01 11:27:11 -07:00
Christophe Duong
7c26305865 Add supportsDBT and supportsNormalization to API objects (#4031)
* Add supportsDBT and supportsNormalization to API objects
2021-06-11 10:11:42 +02:00
Subodh Kant Chaturvedi
6adad7d98e destination-specification: add supportsNormalization and supportsDBT attributes (#3779)
* destination-specification: add supportsNormalization and supportsDBT attributes

* address review comment

* missed this one

* output after gradle format
2021-06-01 17:40:28 +05:30
Charles
0df53170c9 Stop formatting python with spotless (#3388) 2021-05-13 17:46:34 -07:00
Davin Chia
fa33c9c126 Initial CDK Test set up and one test file. (#3207)
Co-authored-by: Sherif Nada <snadalive@gmail.com>
2021-05-04 14:27:58 -07:00
Christophe Duong
8dff1586ab Fix spotlessApply conflict with black (#3152)
* Fix spotlessApply conflict with black
2021-04-30 21:09:42 +02:00
Sherif A. Nada
e9287e7964 Naively merge all Python modules into one. (#3148)
* naively merge all python base modules into one

* Formatting changes.

Co-authored-by: Davin Chia <davinchia@gmail.com>
2021-04-30 15:17:55 +08:00