1
0
mirror of synced 2026-01-01 09:02:59 -05:00
Commit Graph

23 Commits

Author SHA1 Message Date
Alexandre Girard
e029353b9f SimpleRetriever yield request and response as log messages (#18644)
* method yielding airbytemessage

* move to Stream

* update abstract source

* reset

* missing file

* Yield request and response as log messages

* only emit request and responses if the debug flag is on

* add test docker image

* script to run acceptance tests with local cdk

* Update conftest to use a different image

* extract to method

* dont use a different image tag

* Always install local cdk

* break the cdk

* get path from current working directory

* or

* ignore unit test

* debug log

* Revert "AMI change: ami-0f23be2f917510c26 -> ami-005924fb76f7477ce (#18689)"

This reverts commit bf06decf73.

* build from the top

* Update source-acceptance-test

* fix

* copy setup

* some work on the gradle plugin

* reset to master

* delete unused file

* delete unused file

* reset to master

* optional argument

* delete dead code

* use latest cdk with sendgrid

* fix sendgrid dockerfile

* break the cdk

* use local file

* Revert "break the cdk"

This reverts commit 600c195541.

* dont raise an exception

* reset to master

* unit tests

* missing test

* more unit tests

* remove deprecated comment

* newline

* reset to master

* remove files

* reset

* Update abstract source

* remove method from stream

* convert to airbytemessage

* unittests

* Update

* unit test

* remove debug logs

* Revert "remove debug logs"

This reverts commit a1a139ef37.

* Revert "Revert "remove debug logs""

This reverts commit b1d62cdb60.

* Revert "reset to master"

This reverts commit 3fa6a004c1.

* fix

* slightly better test

* typing

* extract method

* Revert "Revert "reset to master""

This reverts commit 5dac7c2804.

* reset to master

* reset to master

* Revert "reset to master"

This reverts commit 3fa6a004c1.

* Comment

* operate on the message

* Revert "Revert "reset to master""

This reverts commit 5833c84d0a.

* comment

* test

* Revert "test"

This reverts commit 2f91b803b0.

* test

* Revert "test"

This reverts commit 62d95ebbb5.

* test

* Revert "test"

This reverts commit 27150ba341.

* format

* format

* symlink

* Update setup

* update path

* reset to master

* update

* Add local files

* greenhouse

* format

* symlink

* try reordering

* better error message

* better log message

* reset to master

* Revert "merge for qa"

This reverts commit ad7128f2c5, reversing
changes made to 7196c22a73.

* reset to master

* reset to master

* reset to master

* format

* gradlew format

* right type hints

* reset to master

* reset to master

* gradlew format

* a bunch of small fixes

* Update output format

* fixes from feedback

* fixme comment

* streams cannot return AirbyteRecordMessage

* fix

* format

* only return logs when running on debug mode

* move branching

* update typing

* remove dead code

* fix simpleretriever name

* i think this is better

* log response.text

* debug flag

* comment

* pass config

* comments

* run SATs

* fix most of the unit tests

* fix unit test

* reset to master

* runFromPath

* Revert "runFromPath"

This reverts commit 85979a801a.

* Revert "run SATs"

This reverts commit a8a8a2da95.

* no need to convert to dict

* fix test
2022-11-09 13:26:50 -08:00
Alexandre Girard
1a608f846a Stream returns AirbyteMessages (#18572)
* method yielding airbytemessage

* move to Stream

* update abstract source

* reset

* missing file

* add test docker image

* script to run acceptance tests with local cdk

* Update conftest to use a different image

* extract to method

* dont use a different image tag

* Always install local cdk

* break the cdk

* get path from current working directory

* or

* ignore unit test

* debug log

* Revert "AMI change: ami-0f23be2f917510c26 -> ami-005924fb76f7477ce (#18689)"

This reverts commit bf06decf73.

* build from the top

* Update source-acceptance-test

* fix

* copy setup

* some work on the gradle plugin

* reset to master

* delete unused file

* delete unused file

* reset to master

* optional argument

* delete dead code

* use latest cdk with sendgrid

* fix sendgrid dockerfile

* break the cdk

* use local file

* Revert "break the cdk"

This reverts commit 600c195541.

* dont raise an exception

* reset to master

* unit tests

* missing test

* more unit tests

* newline

* reset to master

* remove files

* reset

* Update abstract source

* remove method from stream

* convert to airbytemessage

* unittests

* Update

* unit test

* remove debug logs

* Revert "remove debug logs"

This reverts commit a1a139ef37.

* Revert "Revert "remove debug logs""

This reverts commit b1d62cdb60.

* Revert "reset to master"

This reverts commit 3fa6a004c1.

* Revert "Revert "reset to master""

This reverts commit 5dac7c2804.

* reset to master

* reset to master

* test

* Revert "test"

This reverts commit 2f91b803b0.

* test

* Revert "test"

This reverts commit 62d95ebbb5.

* test

* Revert "test"

This reverts commit 27150ba341.

* format
2022-11-07 21:23:11 -08:00
Pedro S. Lopez
ad29513a43 CDK: Handle config validation errors as config_error and failed check status (#18214)
* schema helpers raises a config_error

* traced exceptions can be turned into connection status messages

* add tests for schema helpers

* return a failed status message rather than throwing in check command

* remove unused imports

* add comment

* bump version / update changelog
2022-10-20 13:23:06 -04:00
Pedro S. Lopez
58f5128adb 🐞 CDK: fix error when logging warning for types that could not be transformed (#17367)
* fix error when logging warning for types that could not be transformed

* changelog and bump
2022-09-29 18:48:53 -04:00
Serhii Chvaliuk
74e936bf7d CDK: improve TypeTransformer to convert simple types to array of simple types (#16636)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-15 11:34:17 +03:00
Serhii Chvaliuk
0d55a895cb CDK: TypeTransformer - warning message more informative (#16695)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-15 10:24:37 +03:00
Serhii Chvaliuk
5a3b6d82b6 CDK: make TypeTransformer more robust to incorrect incoming records (#16544)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-13 10:30:16 +03:00
Sherif A. Nada
7acbcfbfb7 Fix CDK obfuscation of nested secrets (#14035) 2022-06-23 09:23:27 -07:00
George Claireaux
4d279f8238 Remove legacy sentry code from cdk (#14016)
* rip sentry out of cdk

* remove sentry dsn from gsc
2022-06-23 12:14:09 +01:00
Alexandre Girard
3894134d11 Bump year in license short to 2022 (#13191)
* Bump to 2022

* format
2022-05-25 17:56:49 -07:00
Pedro S. Lopez
73c7fad7fc CDK: emit AirbyteTraceMessage with exception trace information (#12593) 2022-05-06 17:16:57 -04:00
Dmytro
fe954c1a16 Integrate Sentry for performance and errors tracking. (#8248)
* Integrate Sentry for performance and errors tracking.

* Add sentry sensitive data scrubbing.

* updated cdk version and changelog

* Integrate Sentry for performance and errors tracking

Add `SENTRY_DSN` environment variable

* Integrate Sentry for performance and errors tracking

Add `sentry_sdk` to install requirements

* format cdk

* enable Sentry for google-search-console

* updated connector version

* update spec and source yamls

Co-authored-by: auganbay <auganenu@gmail.com>
Co-authored-by: Sergei Solonitcyn <sergei.solonitcyn@zazmic.com>
2021-12-21 20:02:02 +06:00
Eugene Kulak
d50ae47b49 🐛 CDK: Fix the logging of unhandled exceptions to show stacktrace (#8704)
* print stacktrace for unhandled exceptions

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
2021-12-14 00:04:29 +02:00
Eugene Kulak
aa67604f09 CDK: Add base pydantic model for connector config and schemas (#8485)
* add base spec model

* fix usage of state_checkpoint_interval in case it is dynamic

* add schema base models, fix spelling, signatures and polishing

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
2021-12-08 01:14:59 +02:00
Dmytro
e56e86dd28 Fix schema transformation false warnings. (#7863) 2021-11-12 18:20:53 +02:00
Dmytro
16b626cc91 Resolve discover catalog ref fields (#7734) 2021-11-09 12:42:15 +02:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
Dmytro
1c5ac5b5ea 🏗️ Python CDK: add schema transformer class (#6139)
* Python CDK: add schema transformer class
2021-09-27 13:07:17 +03:00
Dmytro
474b8f1065 CDK: support nested refs resolving (#6044)
Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
2021-09-16 18:38:08 +03:00
Marcos Eliziario Santos
1314a6a04c #5796 silence printing full config when config validation fails (#5879)
* - #5796 silence printing full config when config validation fails

* fix unit tests after config validation check changes

Co-authored-by: Marcos Eliziario Santos <marcos@coremarcos.com>
2021-09-14 11:00:47 -03:00
Charles
0df53170c9 Stop formatting python with spotless (#3388) 2021-05-13 17:46:34 -07:00
Davin Chia
1d8e6da511 Various CDK build fixes/clean up. (#3312)
* Better MyPy command to not type check singer. Export UserDefinedException as that is used in tests.
2021-05-10 10:02:04 +08:00
Sherif A. Nada
184dab77eb CDK: overhaul directory structure (#3295) 2021-05-09 15:27:38 -07:00