19 Commits

Author	SHA1	Message	Date
Brian Lai	374e265fcb	[Low Code CDK] configurable oauth request payload (#13993) ... * configurable oauth request payload * support interpolation for dictionaries that are not new subcomponents * rewrite a declarative oauth authenticator that performs interpolation at runtime * formatting * whatever i don't know why factory gets flagged w/ the newline change * we java now * remove duplicate oauth * add some comments * parse time properly from string interpolation * move declarative oauth to its own package in declarative module * add changelog info	2022-07-08 16:49:16 -04:00
Alexandre Girard	01fc7a48c7	move jinja macros to their own file (#14515) ... * move jinja macros to their own file * format * export as dict * format	2022-07-08 07:56:06 -07:00
Brian Lai	35ca7c8d10	[low code cdk] Add MinMaxDate component and allow for it to be specified for DatetimeStreamSlicer (#14288) ... * implement MinMaxDate and incorporate it into the DatetimeStreamSlicer * rename to datetime and restrict start/end to minmaxdatetime object * get rid of eval_date which isn't needed anymore and allow for datetimestreamslicer to set datetime format * add MinMaxDatetime to class types registry	2022-07-07 01:05:13 -04:00
Alexandre Girard	37acb9289f	[low-code cdk] Enable configurable state checkpointing (#14317) ... * checkout files from test branch * read_incremental works * reset to master * remove dead code * comment * fix * Add test * comments * utc * format * small fix * Add test with rfc3339 * remove unused param * fix test * configurable state checkpointing * update test	2022-07-05 19:20:23 -07:00
Alexandre Girard	e23789bbd6	Alex/declarative stream incremental fix (#14268) ... * checkout files from test branch * read_incremental works * reset to master * remove dead code * comment * fix * Add test * comments * utc * format * small fix * Add test with rfc3339 * remove unused param * fix test	2022-07-01 08:39:19 -07:00
Alexandre Girard	85449c975c	[low-code connectors] default types and default values (#14004) ... * default types and default values * cleanup * fixes so read works * remove prints and trycatch * comment * remove unused param * split file * extract method * extract methods * comment * optional * fix test * cleanup * delete interpolated request header provider * simplify next page url paginator interface * comment * format	2022-06-27 20:43:03 -07:00
Brian Lai	a61224887e	support semi incremental by adding extractor record filter (#13520) ... * support semi incremental by adding extractor record filter * refactor extractor into a record_selector that supports extraction and filtering of response records	2022-06-23 00:09:44 -04:00
Brian Lai	be01b476ce	Add new InterpolatedRequestOptionsProvider that encapsulates all variations of request arguments (#13472) ... * write out new request options provider and refactor components and parts of the YAML config * fix formatting * pr feedback to consolidate body_data_provider to simplify the code * pr feedback get rid of extraneous optional	2022-06-21 16:01:05 -04:00
Alexandre Girard	34e2dc36c6	[low-code connectors] Cartesian product stream slicer (#13740) ... * list slicer * Add comment * product stream slicer * comment * rename * format * Update comment * split on 2 lines for readability * Revert "rename" This reverts commit e801f2d1f8.	2022-06-16 08:48:32 -07:00
Alexandre Girard	2c8fef8b7d	[low-code connectors] Support for list-based stream slicers (#13668) ... * list slicer * Add comment * test * ast eval * refactor * fix * remove print * rename * return a generator * smaller fix and reset to master * union type hint * assert * comment * format	2022-06-14 17:12:53 -07:00
Alexandre Girard	15fe51b788	[low-code connectors] add a few unit tests (#13666) ... * add a couple of unit tests * refactor tests	2022-06-13 16:51:36 -07:00
Alexandre Girard	57dd7a9d4b	[low-code connectors] use jello instead of jq (#13570) ... * use jello instead of jq * reset * rename * remove print	2022-06-07 14:10:26 -07:00
Alexandre Girard	7fbe6dab5c	[Low code CDK] add support for iterating over parent stream slices (#13287) ... * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * Add missing tests * Add missing files * Add missing tests * add missing file * missing file * missing file * rename * doc * doc * remove broken test * rename * jinja dependency * Add comment * comment * comment * pyjq dependency * rename file * delete unused file * Revert "delete unused file" This reverts commit 758e939367. * fix * rename * abstract property * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * isort * update state * fix imports * update dependency * format * rename file * decoder * Use decoder * Update comment * dict_state is actually backed by a dict * Add a comment * update state takes kwargs * move state out of offset paginator * update jq parameter order * update * remove incremental mixin * delete comment * update comments * update comments * remove no_state * rename package * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * remove print * move test * delete duplicates * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * pass parameters through kwargs * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Allow passing a string or an enum * Define StateType enum * convert state_type if not of type type * convert state_type if not of type type * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * Delete dead code * rename methods * rename to declarative * rename the classes too * Try to install packages to build jq * isort * only automake * Revert "only automake" This reverts commit c8fe154ffc. * remove git * format * substream slicer * specify everything in the slice_definition * Add jq dependency * support multiple parent streams * support getting the parent stream name * cleanup * Add comment * format * delete duplicate file	2022-06-02 08:53:04 -07:00
Brian Lai	ccbd7c6508	conditional paginator + interpolated boolean components (#13389) ... * conditional paginator + interpolated boolean components * fix bug where empty arrays and dicts evaluated to true * add more falsey values * adjust falsey values and tweak based on review comments	2022-06-01 20:24:32 -04:00
Alexandre Girard	aa92518721	Low-code connectors: configurable source from yaml (#13038) ... * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * Add missing tests * Add missing files * Add missing tests * add missing file * missing file * missing file * sengrid low code connector * rename * doc * doc * remove broken test * rename * jinja dependency * Add comment * comment * comment * pyjq dependency * update import * rename file * delete unused file * Revert "delete unused file" This reverts commit 758e939367. * fix * rename * abstract property * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * isort * update state * fix imports * update * update dependency * remove dead code * remove dead code * format * rename file * decoder * Use decoder * Update comment * dict_state is actually backed by a dict * Add a comment * update state takes kwargs * move state out of offset paginator * fix * update jq parameter order * fix * pass config * update * update * remove incremental mixin * delete comment * start workin on yaml parser * fix test * progress * refer and overwrite partials * factory tests pass * fix * reset * Assert http_method is an enum value * fix auth * read lists works * fix test * comment * implement all streams * build connection checker * update comments * update comments * remove no_state * rename package * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * remove print * move test * delete duplicates * update * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * pass parameters through kwargs * fix test * update interface * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Allow passing a string or an enum * Define StateType enum * unit tests pass * update dict state * update * can read * fix test * fix from yaml update * elif * convert state_type if not of type type * convert state_type if not of type type * Add a test * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * remove sendgrid specific code * update * update * delete comment * remove sendgrid specific file * remove unused file * Delete dead code * rename methods * rename to declarative * rename the classes too * select streams to check * nit * rename method * rename class * {} is faster than dict() * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/parsers/yaml_parser.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * more precise exception * rename class * add comment * Try to install packages to build jq * isort * only automake * Revert "only automake" This reverts commit c8fe154ffc. * remove git * rename file * create components in kwargs * Use tuple of strings * parser doesn't need to be stored * move file and delete duplicates * Revert "Use tuple of strings" This reverts commit ab5a7afd08. * raise error if streams to check are not in the catalog * Revert "Revert "Use tuple of strings"" This reverts commit 7c9fb8eb33. * traverse tree * rename to options * move docstring * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/checks/check_stream.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * fix tests and format * format * update * better error message * Add jq dependency * Use request header provider * rename * rename field * remove get_context method * rename * add a comment * format Co-authored-by: Sherif A. Nada <snadalive@gmail.com>	2022-06-01 07:31:52 -07:00
Alexandre Girard	d9fa24ffff	Low code connectors: implement components for sendgrid (#12853) ... * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * Add missing tests * Add missing files * Add missing tests * add missing file * missing file * missing file * rename * doc * doc * remove broken test * rename * jinja dependency * Add comment * comment * comment * pyjq dependency * rename file * delete unused file * Revert "delete unused file" This reverts commit 758e939367. * fix * rename * abstract property * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * isort * update state * fix imports * update dependency * format * rename file * decoder * Use decoder * Update comment * dict_state is actually backed by a dict * Add a comment * update state takes kwargs * move state out of offset paginator * update jq parameter order * update * remove incremental mixin * delete comment * update comments * update comments * remove no_state * rename package * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * remove print * move test * delete duplicates * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * pass parameters through kwargs * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Allow passing a string or an enum * Define StateType enum * convert state_type if not of type type * convert state_type if not of type type * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * Delete dead code * rename methods * rename to declarative * rename the classes too * Try to install packages to build jq * isort * only automake * Revert "only automake" This reverts commit c8fe154ffc. * remove git * format * Add jq dependency * Use request header provider * rename * rename field * remove get_context method * rename	2022-06-01 07:02:35 -07:00
Marcos Marx	dca2256a7c	Bump 2022 license version (#13233) ... * Bump year in license short to 2022 * remove protocol from cdk	2022-05-26 15:00:42 -03:00
Brian Lai	c4ef7db5eb	[Low Code CDK] add support for string interpolation for request headers (#13215) ... * add support for string interpolation for request headers + small refactor * fix formatting * move parameter provider to folder for consistency	2022-05-25 22:15:19 -04:00
Alexandre Girard	150ab593f8	Low code connectors: core structure (#12850) ... * checkout from alex/cac * doc * doc * remove broken test * rename * rename file * delete unused file * rename * abstract property * isort * update state * Update comment * remove incremental mixin * delete comment * update comments * update comments * remove no_state * rename package * pass parameters through kwargs * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * rename to declarative * rename the classes too	2022-05-25 17:27:54 -07:00