1
0
mirror of synced 2026-01-06 06:04:16 -05:00
Commit Graph

11 Commits

Author SHA1 Message Date
Maxime Carbonneau-Leclerc
3cc67a6d9e [ISSUE #23382] ignore backoff configuration on test reads (#25429) 2023-04-26 08:36:59 -04:00
Maxime Carbonneau-Leclerc
4d65fa1b98 [ISSUE #23994] make MessageGrouper use AirbyteEntrypoint (#25402)
* [ISSUE #23994] make MessageGrouper use AirbyteEntrypoint

* [ISSUE #23994] code review
2023-04-24 11:24:15 -04:00
Alexandre Girard
f3799280f2 connector builder: Emit message at start of slice (#25180)
* Move condition for yielding the slice message to an overwritable method

* Automated Commit - Formatting Changes

* yield the slice log messages

* same for incremental

* refactor

* Revert "refactor"

This reverts commit c594365bd8.

* move flag from factory to source

* set the flag

* remove debug print

* halfmock

* clean up

* Add a test for a single page

* Add another test

* Pass the flag

* rename

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
2023-04-14 10:23:59 -07:00
Alexandre Girard
71fc3dd517 Connector builder: set pages and slices limits (#25121)
* Set limits

* refactor and add unit tests

* Update as per comments
2023-04-12 14:46:43 -07:00
Alexandre Girard
edfc59533d Connector builder: Port "send stacktrace when error on read" to CDK connector builder module (#24173)
* wip

* fix unit test

* fix other unit test

* format

* reset

* format

* missing unit test

* yield a LogMessage on error

* format

* format

* fix unit tests

* yield a trace message instead of a log message

* format

* fix bad merge
2023-03-21 17:22:08 -07:00
Catherine Noll
f4fd4d98a2 Connector Builder: Make connector_builder part of the CDK package (#24280) 2023-03-21 13:31:16 -04:00
Maxime Carbonneau-Leclerc
98719cf3f3 [ISSUE #23794] CDK's read command handler supports Connector Builder … (#24204)
* [ISSUE #23794] CDK's read command handler supports Connector Builder list_streams requests

* [ISSUE #23794] code review
2023-03-21 09:01:33 -04:00
Catherine Noll
e890d01d55 Connector builder: handle empty catalog (#24184) 2023-03-17 12:51:10 -04:00
Brian Lai
903d34e5f1 [Low-Code CDK] Enforce manifest against the airbyte-cdk version and the Beta version 0.29.0 (#23796)
* enforce manifest version correctness against the CDK package being used

* parse versions into parts for better comparisons and error checking

* fix pr feedback and derp forgot to actually add the commit with the low-code manifests updated to the beta version

* pr feedback and fix new tests since last rebase
2023-03-16 00:50:30 -04:00
Alexandre Girard
bb5741a0c0 Connector builder: support for test read with message grouping per slices (#23925)
* New connector_builder module for handling requests from the Connector Builder.

Also implements `resolve_manifest` handler

* Automated Commit - Formatting Changes

* Rename ConnectorBuilderSource to ConnectorBuilderHandler

* Update source_declarative_manifest README

* Reorganize

* read records

* paste unit tests from connector builder server

* compiles but tests fail

* first test passes

* Second test passes

* 3rd test passes

* one more test

* another test

* one more test

* test

* return StreamRead

* test

* test

* rename

* test

* test

* test

* main seems to work

* Update

* Update

* Update

* Update

* update

* error message

* rename

* update

* Update

* CR improvements

* fix test_source_declarative_manifest

* fix tests

* Update

* Update

* Update

* Update

* rename

* rename

* rename

* format

* Give connector_builder its own main.py

* Update

* reset

* delete dead code

* remove debug print

* update test

* Update

* set right stream

* Add --catalog argument

* Remove unneeded preparse

* Update README

* handle error

* tests pass

* more explicit test

* reset

* format

* fix merge

* raise exception

* fix

* black format

* raise with config

* update

* fix flake

* __test_read_config is optional

* fix

* Automated Commit - Formatting Changes

* fix

* exclude_unset

---------

Co-authored-by: Catherine Noll <noll.catherine@gmail.com>
Co-authored-by: clnoll <clnoll@users.noreply.github.com>
Co-authored-by: girarda <girarda@users.noreply.github.com>
2023-03-15 17:12:37 -07:00
Catherine Noll
8ee32b1132 New connector_builder module for handling requests from the Connector Builder (#23888)
Also implements `resolve_manifest` handler
2023-03-14 13:51:27 -04:00