1
0
mirror of synced 2025-12-30 21:02:43 -05:00
Commit Graph

18 Commits

Author SHA1 Message Date
Alexandre Girard
df01616951 [Issue #23497] Deduplicate query parameters for declarative connectors (#28550)
* remove duplicate param

* remove duplicate params

* fix some of the typing issues

* fix typing issues

* newline

* format

* Enable by default

* Add missing file

* refactor and remove flag

* none check

* move line of code

* fix typing in rate_limiting

* comment

* use typedef

* else branch

* format

* gate the feature

* rename test

* fix the test

* only dedupe if the values are the same

* Add some tests

* convert values to strings

* Document the change

* implement in requester too
2023-07-25 14:22:25 -07:00
Alexandre Girard
3ae73fb0ff connector builder: Set test_read_limit_reached to true if we hit the max records limit (#28293)
* set test_read_limit_reached to true if we hit the max records limit

* rename slice to _slice to avoid shadowing a builtin keyword

* newline

* fix some of the typing issues

* fix some more typing issues

* another fix

* fix last typing issue

* format

* Automated Commit - Formatting Changes

* reset type

* fix the type

* Update for clarity

* Update types

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
2023-07-18 15:53:53 -07:00
Maxime Carbonneau-Leclerc
df2a6e50bb Issue 21014/oauth requests (#27973)
* [ISSUE #27494] fix type issue caused by connector builder logging

* [ISSUE #21014] log request/response for oauth as 'global_requests'

* formatcdk

* [ISSUE #21014] support DeclarativeOauth2Authenticator as well

* [ISSUE #21014] improving message grouper tests

* formatcdk

* Test solution with logic in MessageRepository (#27990)

* Test solution with logic in MessageRepository

* Solution without creating a new ModelToComponentFactory

* [ISSUE #21014] adding tests

* [ISSUE #21014] add title and description to global requests

* Revert "Solution without creating a new ModelToComponentFactory"

This reverts commit f17799ecff.

* Automated Commit - Formatting Changes

* [ISSUE #21014] code review

* [ISSUE #21014] do not break on log appender conflict

* Automated Commit - Formatting Changes

* [ISSUE #21014] code review

* formatcdk

* [ISSUE #21014] moving is_global to is_auxiliary
2023-07-11 13:37:38 -04:00
Joe Reuter
8aba48810c Low-code CDK: Serialize request body as string for connector builder module (#27657)
* serialize request body as string

* fix some bugs
2023-06-27 08:27:16 +02:00
Maxime Carbonneau-Leclerc
f48849fdb4 [ISSUE #26909] adding message repository (#27158)
* [ISSUE #26909] adding message repository

* Automated Commit - Formatting Changes

* [ISSUE #26909] improve entrypoint error handling

* format CDK

* [ISSUE #26909] adding an integration test
2023-06-13 08:40:55 -04:00
Joe Reuter
d6512dea2c CDK: Datetime format inferrer (#27071)
* datetime inferrer class

* format

* pass inferred date formats along

* review comments
2023-06-09 10:33:54 +02:00
Maxime Carbonneau-Leclerc
4625cef571 [ISSUE #26909] add latest connector config control message to connect… (#26922)
* [ISSUE #26909] add latest connector config control message to connector builder API

* [ISSUE #26909] flake

* Automated Commit - Formatting Changes

* [ISSUE #26909] fallback on in-memory dict if no config control message

* [ISSUE #26909] update and add tests
2023-06-07 08:31:45 -04:00
Maxime Carbonneau-Leclerc
d54a68640f Improving error messages to have better messaging in datadog and the … (#26860)
* Improving error messages to have better messaging in datadog and the frontend

* fixing tests
2023-05-31 15:36:27 -04:00
Maxime Carbonneau-Leclerc
0efc18a114 [ISSUE #24720] connector builder set slice descriptor (#25677) 2023-05-01 12:18:22 -04:00
Joe Reuter
5ec81cc8d1 Connector builder: Restructure documentation (#25353)
* authentication documentation

* some fixes

* fix broken link

* remove session token auth

* review comments

* first version of tutorial

* Revert "first version of tutorial"

This reverts commit bbdeb997f2.

* first version of tutorial

* review comments

* improvements

* partitioning documentation

* incremental sync

* review comments

* improvements

* record processing doc

* improvements

* error handler connector builder ui documentation

* add missing link

* overview page

* mention current limitation

---------

Co-authored-by: brianjlai <brian.lai@airbyte.io>
2023-04-27 12:32:26 +02:00
Maxime Carbonneau-Leclerc
3cc67a6d9e [ISSUE #23382] ignore backoff configuration on test reads (#25429) 2023-04-26 08:36:59 -04:00
Maxime Carbonneau-Leclerc
4d65fa1b98 [ISSUE #23994] make MessageGrouper use AirbyteEntrypoint (#25402)
* [ISSUE #23994] make MessageGrouper use AirbyteEntrypoint

* [ISSUE #23994] code review
2023-04-24 11:24:15 -04:00
Alexandre Girard
f3799280f2 connector builder: Emit message at start of slice (#25180)
* Move condition for yielding the slice message to an overwritable method

* Automated Commit - Formatting Changes

* yield the slice log messages

* same for incremental

* refactor

* Revert "refactor"

This reverts commit c594365bd8.

* move flag from factory to source

* set the flag

* remove debug print

* halfmock

* clean up

* Add a test for a single page

* Add another test

* Pass the flag

* rename

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
2023-04-14 10:23:59 -07:00
Alexandre Girard
71fc3dd517 Connector builder: set pages and slices limits (#25121)
* Set limits

* refactor and add unit tests

* Update as per comments
2023-04-12 14:46:43 -07:00
Brian Lai
3ba15b5fcb Decouple flags for debug messages from connector builder log messages (#24881)
* decouple debug message flags from connector builder grouping messages

* Automated Commit - Formatting Changes

* pr feedback simplifying configs a bit

---------

Co-authored-by: brianjlai <brianjlai@users.noreply.github.com>
2023-04-06 12:16:05 -04:00
Alexandre Girard
aaf00deb38 Add missing type hints (#24345) 2023-03-22 07:15:54 -07:00
Alexandre Girard
edfc59533d Connector builder: Port "send stacktrace when error on read" to CDK connector builder module (#24173)
* wip

* fix unit test

* fix other unit test

* format

* reset

* format

* missing unit test

* yield a LogMessage on error

* format

* format

* fix unit tests

* yield a trace message instead of a log message

* format

* fix bad merge
2023-03-21 17:22:08 -07:00
Catherine Noll
f4fd4d98a2 Connector Builder: Make connector_builder part of the CDK package (#24280) 2023-03-21 13:31:16 -04:00