jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-30 21:02:43 -05:00
Code Issues Packages Projects Releases Wiki Activity
11,968 Commits 5,135 Branches 518 Tags
9f6963ccfcddd7a573c4ca0ff94de41ce6175cc0
Commit Graph

538 Commits

Author SHA1 Message Date
brianjlai
4b44a51eb1 🤖 Bump patch version of Airbyte CDK 2023-05-09 02:09:58 +00:00
Alexandre Girard
7443970de3 low-code: Use Jinja sandbox environment and prevent use of range method (#25589) 
* secure the jinja environment

* format

* Update comment

* remove extra test

* remove lambda

* Update

* Raise an error on undefined variables

* remove unused import

* add unit tests to missing context vars and adjust error message

---------

Co-authored-by: brianjlai <brian.lai@airbyte.io>
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
 2023-05-08 19:34:53 -04:00
girarda
93d89a85ee 🤖 Bump patch version of Airbyte CDK 2023-05-04 22:53:38 +00:00
Alexandre Girard
0f0e8c8f96 pin the version of requests lib (#25820) 2023-05-04 15:46:23 -07:00
maxi297
bb5efe4191 🤖 Bump patch version of Airbyte CDK 2023-05-01 16:26:03 +00:00
Maxime Carbonneau-Leclerc
0efc18a114 [ISSUE #24720] connector builder set slice descriptor (#25677) 2023-05-01 12:18:22 -04:00
Maxime Carbonneau-Leclerc
b26c897a8b [ISSUE #25646] support parsing of non utc dates (#25665) 
* [ISSUE #25646] support parsing of non utc dates

* [ISSUE #25646] improve parsing

* [ISSUE #25646] removing timezone for DatetimeParser interface

* [ISSUE #25646] fix tests
 2023-05-01 12:16:44 -04:00
Alexandre Girard
a4b518fb30 low-code: Update docs on incremental syncs (#25696) 
* Update docs

* Automated Commit - Formatting Changes

* update

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
 2023-04-28 18:55:07 -07:00
Joe Reuter
5ec81cc8d1 Connector builder: Restructure documentation (#25353) 
* authentication documentation

* some fixes

* fix broken link

* remove session token auth

* review comments

* first version of tutorial

* Revert "first version of tutorial"

This reverts commit bbdeb997f2.

* first version of tutorial

* review comments

* improvements

* partitioning documentation

* incremental sync

* review comments

* improvements

* record processing doc

* improvements

* error handler connector builder ui documentation

* add missing link

* overview page

* mention current limitation

---------

Co-authored-by: brianjlai <brian.lai@airbyte.io>
 2023-04-27 12:32:26 +02:00
girarda
263a132a31 🤖 Bump patch version of Airbyte CDK 2023-04-27 00:22:45 +00:00
Alexandre Girard
e41060c02c low-code: Fix type check in DeclarativeStream (#25533) 
* Set right type

* Update the comment

* Update

* format

* Update comment
 2023-04-26 15:49:10 -07:00
jdpgrailsdev
31baedaa41 🤖 Bump minor version of Airbyte CDK 2023-04-26 14:36:04 +00:00
Jonathan Pearlin
2ebfa459cf Publish stream status messages in CDK (#24994) 
* Publish stream status messages in CDK

* Automated Commit - Formatting Changes

* Convert to StreamDescriptor

* Automated Commit - Formatting Changes

* Bump to latest protocol model

* Automated Commit - Formatting Changes

* Bump protocol version

* Add tests for stream status message creation

* Formatting

* Formatting

* Fix failing test

* Actually emit state message

* Automated Commit - Formatting Changes

* Bump airbyte-protocol

* PR feedback

* Fix parameter input

* Correctly yield status message

* PR feedback

* Formatting

* Fix failing tests

* Automated Commit - Formatting Changes

* Revert accidental change

* Automated Change

* Replace STOPPED with COMPLETE/INCOMPLETE

* Update source-facebook-marketing changelog

* Revert "Update source-facebook-marketing changelog"

This reverts commit 709edb800c.

---------

Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
 2023-04-26 10:30:36 -04:00
Maxime Carbonneau-Leclerc
3cc67a6d9e [ISSUE #23382] ignore backoff configuration on test reads (#25429) 2023-04-26 08:36:59 -04:00
girarda
e7aff71ba7 🤖 Bump patch version of Airbyte CDK 2023-04-26 03:46:29 +00:00
Alexandre Girard
1c13a0af6a Update jinja variable description (#25440) 
* Update description

* Add examples

* format

* update

* update

* Update declarative_component_schema.yaml

* remove superfluous -

---------

Co-authored-by: Joe Reuter <joe@airbyte.io>
 2023-04-25 20:44:05 -07:00
Alexandre Girard
250c3b1c87 low-code: Delete now_local macro (#25404) 
* Delete now_local macro

* Remove from reference docs

* remove example
 2023-04-25 20:40:46 -07:00
girarda
7aad27b8a8 🤖 Bump patch version of Airbyte CDK 2023-04-25 01:37:44 +00:00
Alexandre Girard
645763588c low-code: Alias stream_interval and stream_partition to stream_slice in interpolation context (#25373) 
* add aliases

* Raise error if the alias is found in the context

* format

* Comment

* Automated Commit - Formatting Changes

* rename to stream partition in greenhouse manifest

* Revert "rename to stream partition in greenhouse manifest"

This reverts commit d513ef418f.

* Clean up test

* Other test

* last test

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
 2023-04-24 18:25:54 -07:00
maxi297
cff896f4f2 🤖 Bump patch version of Airbyte CDK 2023-04-24 15:42:05 +00:00
Maxime Carbonneau-Leclerc
4d65fa1b98 [ISSUE #23994] make MessageGrouper use AirbyteEntrypoint (#25402) 
* [ISSUE #23994] make MessageGrouper use AirbyteEntrypoint

* [ISSUE #23994] code review
 2023-04-24 11:24:15 -04:00
girarda
ca344afbb2 🤖 Bump patch version of Airbyte CDK 2023-04-21 18:12:55 +00:00
Alexandre Girard
15f90e3a2f Fix and document macros and interpolation variables (#25305) 
* Fix and document macros

* cleanup

* dots

* Add tests and refactor

* Update

* Add an example

* Document variables

* Mention now_local is not recommended
 2023-04-21 10:58:53 -07:00
Alexandre Girard
0a91a98370 Add descriptions and examples to component schema (#25117) 
* Reference docs for backoff strategies

* Docs for most authentication mechansims

* Docs for CheckStream

* MinMaxDatetime

* DeclarativeStream

* DefaultErrorHandler

* CompositeErrorHandler

* update

* Update token expiry date description

* DPath extractor

* Add interpolation_context

* HttpResponseFilter

* RecordFilter

* RecordSelector

* DefaultPaginator

* CursorPagination

* OffsetIncrement

* Page Increment

* PrimaryKey

* HttpRequester

* request option and request path

* Schemas

* Spec

* Add field

* remove fields

* no auth and no pagination

* Delete deprecated comment

* Missing description

* Json Decoder

* OAuthConfigSpecification

* reorder

* add titles, examples, and descriptions for partition routers and datetime based cursor

* updates

* Update

* fix indentation

* Automated Commit - Formatting Changes

* Update as per feedback

* html tag

* generate models

* Update name

* do not use title as class name

* Update

* Add stream_interval and stream_partition to interpolation_context

* Automated Commit - Formatting Changes

* fix path

* format

---------

Co-authored-by: brianjlai <brian.lai@airbyte.io>
Co-authored-by: girarda <girarda@users.noreply.github.com>
 2023-04-21 10:58:23 -07:00
Alexandre Girard
1e8cf8f5d5 low-code: Do not apply transforms on AirbyteLogMessages and AirbyteTraceMessages (#25290) 
* Check the input type before applying transformations

* format

* remove debug prints
 2023-04-20 14:12:22 -07:00
girarda
dd68bb17b0 🤖 Bump minor version of Airbyte CDK 2023-04-17 21:49:21 +00:00
Alexandre Girard
fc3655c12a low-code: Clean up SessionTokenAuthenticator interface (#25086) 
* Username and session token are optional fields

* update

* Add titles, descriptions, and examples

* Automated Commit - Formatting Changes

* fix a small typo

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
 2023-04-17 14:42:49 -07:00
girarda
6fa2f21b3d 🤖 Bump patch version of Airbyte CDK 2023-04-14 21:11:36 +00:00
Alexandre Girard
3841141913 Fix manifest_declarative_source + add unit tests (#25217) 
* Fix + unit test

* Add a test with pagination

* Add a test with partition router

* Make sure _fetch_next_page is called with the right arguments

* Automated Commit - Formatting Changes

* pagination with partitions

* refactor

* clean up

* format

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
 2023-04-14 14:05:22 -07:00
girarda
0d9392e4ae 🤖 Bump patch version of Airbyte CDK 2023-04-14 17:30:23 +00:00
Alexandre Girard
f3799280f2 connector builder: Emit message at start of slice (#25180) 
* Move condition for yielding the slice message to an overwritable method

* Automated Commit - Formatting Changes

* yield the slice log messages

* same for incremental

* refactor

* Revert "refactor"

This reverts commit c594365bd8.

* move flag from factory to source

* set the flag

* remove debug print

* halfmock

* clean up

* Add a test for a single page

* Add another test

* Pass the flag

* rename

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
 2023-04-14 10:23:59 -07:00
girarda
08067008ca 🤖 Bump patch version of Airbyte CDK 2023-04-12 21:54:31 +00:00
Alexandre Girard
71fc3dd517 Connector builder: set pages and slices limits (#25121) 
* Set limits

* refactor and add unit tests

* Update as per comments
 2023-04-12 14:46:43 -07:00
Alexandre Girard
f969ebd63b low-code: Add some unit tests for CompositeErrorHandler (#24930) 
* Add some unit tests

* update

* fix indent

* fix warnings
 2023-04-10 12:12:36 -07:00
Brian Lai
3ba15b5fcb Decouple flags for debug messages from connector builder log messages (#24881) 
* decouple debug message flags from connector builder grouping messages

* Automated Commit - Formatting Changes

* pr feedback simplifying configs a bit

---------

Co-authored-by: brianjlai <brianjlai@users.noreply.github.com>
 2023-04-06 12:16:05 -04:00
davydov-d
2b09f4faa0 🤖 Bump minor version of Airbyte CDK 2023-04-06 09:21:37 +00:00
Denys Davydov
30e2447ceb Lowcode CDK: implement single use refresh token oauth authenticator (#24891) 
* #24658 Lowcode CDK: implement single use refresh token oauth authenticator

* #24658 lowcode cdk: review fixes
 2023-04-06 12:15:36 +03:00
girarda
e1fab6c98c 🤖 Bump patch version of Airbyte CDK 2023-04-05 00:06:47 +00:00
Alexandre Girard
4b324c3084 Low-code: fix duplicate stream_slicer update (#24827) 
* first tentative fix

* cleaner fix

* refactor test

* format

* format

* move to utils file

* use simpler implementation
 2023-04-04 15:40:33 -07:00
grubberr
28c49d4048 🤖 Bump patch version of Airbyte CDK 2023-04-03 10:36:32 +00:00
Serhii Chvaliuk
032f9b8045 Low-Code CDK: make RecordFilter.filter_records as generator (#24772) 
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
 2023-04-03 13:29:24 +03:00
girarda
cf94c14ae5 🤖 Bump minor version of Airbyte CDK 2023-03-29 21:24:30 +00:00
Alexandre Girard
c3b017c7b5 Add auth flow to declarative manifest schema (#24441) 
* Add auth flow to declarative manifest schema

* Rename

* fix rename

* set advanced_auth

* Automated Commit - Formatting Changes

* update unit test

* format

* Add examples

* example -> examples

* add missing examples

* Automated Commit - Formatting Changes

---------

Co-authored-by: girarda <girarda@users.noreply.github.com>
 2023-03-29 16:06:56 -05:00
alafanechere
69736d9f40 🤖 Bump minor version of Airbyte CDK 2023-03-23 14:09:17 +00:00
Augustin
bad5bce8ce CDK: remove unexpected error swallowing on abstract source's check method (#24240) 2023-03-23 13:04:51 +00:00
Alexandre Girard
aaf00deb38 Add missing type hints (#24345) 2023-03-22 07:15:54 -07:00
girarda
62caa92f22 🤖 Bump patch version of Airbyte CDK 2023-03-22 00:44:23 +00:00
Alexandre Girard
edfc59533d Connector builder: Port "send stacktrace when error on read" to CDK connector builder module (#24173) 
* wip

* fix unit test

* fix other unit test

* format

* reset

* format

* missing unit test

* yield a LogMessage on error

* format

* format

* fix unit tests

* yield a trace message instead of a log message

* format

* fix bad merge
 2023-03-21 17:22:08 -07:00
clnoll
99873c3a6c 🤖 Bump minor version of Airbyte CDK 2023-03-21 17:39:39 +00:00
Catherine Noll
f4fd4d98a2 Connector Builder: Make connector_builder part of the CDK package (#24280) 2023-03-21 13:31:16 -04:00