10 Commits

Author	SHA1	Message	Date
Brian Lai	de24ddced5	make sure stream_slices is an iterator (#18092) ... * ensure output from stream_slices() is always an iterator * bump cdk version * bump version after merging latest cdk * bump cdk version post rebase	2022-10-19 15:37:00 -04:00
Alexandre Girard	df72bbdf2c	Low-code: Pass stream_slice to read_records when reading from CheckStream (#17804) ... * Implement a test * Implement fix * rename * extract method * bump	2022-10-17 05:47:56 -07:00
Brian Lai	bd31100774	initial first pass converting every component to dataclasses (#15189) ... * initial first pass converting every component to dataclasses * replace the hackier options pass through * get rid of the hackier way * fix issues w/ type hints by making options required and lots of fixes to the language to fix compatability for dataclasses * add dataclasses-jsonschema to setup * fix oauth authenticator to avoid dataclass name collisions * fix spacing for CI tests * remove property from oauth and fix a interpolation bug * pr feedback and cleaning up the code a bit, attempt at avoiding renaming * fix templates and bugs surfaced during greenhouse spec testing * fix tests * fix missing options in some declarative components * fix tests related to pulling latest master * fix issue w/ passing state, slice, and token to subcomponents * switch name back to get_access_token() since no name collision anymore	2022-08-05 17:39:27 -04:00
Alexandre Girard	a3ff80c179	[low-code-connectors] Disable parse-time interpolation in favor of runtime-only (#14923) ... * abstract auth token * basichttp * remove prints * docstrings * get rid of parse-time interpolation * always pass options through * delete print * delete misleading comment * delete note * reset * pass down options * delete duplicate file * missing test * refactor test * rename to '$options' * rename to '' * interpolatedauth * fix tests * fix * docstrings * update docstring * docstring * update docstring * remove extra field * undo * rename to runtime_parameters * docstring * update * / -> * * update template * rename to options * Add examples * update docstring * Update test * newlines * rename kwargs to options * options init param * delete duplicate line * type hints * update docstring * Revert "delete duplicate line" This reverts commit 4255d5b346. * delete duplicate code from bad merge * rename file * bump cdk version	2022-07-28 08:57:17 -07:00
Alexandre Girard	01fc7a48c7	move jinja macros to their own file (#14515) ... * move jinja macros to their own file * format * export as dict * format	2022-07-08 07:56:06 -07:00
Alexandre Girard	e23789bbd6	Alex/declarative stream incremental fix (#14268) ... * checkout files from test branch * read_incremental works * reset to master * remove dead code * comment * fix * Add test * comments * utc * format * small fix * Add test with rfc3339 * remove unused param * fix test	2022-07-01 08:39:19 -07:00
Alexandre Girard	7fbe6dab5c	[Low code CDK] add support for iterating over parent stream slices (#13287) ... * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * Add missing tests * Add missing files * Add missing tests * add missing file * missing file * missing file * rename * doc * doc * remove broken test * rename * jinja dependency * Add comment * comment * comment * pyjq dependency * rename file * delete unused file * Revert "delete unused file" This reverts commit 758e939367. * fix * rename * abstract property * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * isort * update state * fix imports * update dependency * format * rename file * decoder * Use decoder * Update comment * dict_state is actually backed by a dict * Add a comment * update state takes kwargs * move state out of offset paginator * update jq parameter order * update * remove incremental mixin * delete comment * update comments * update comments * remove no_state * rename package * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * remove print * move test * delete duplicates * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * pass parameters through kwargs * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Allow passing a string or an enum * Define StateType enum * convert state_type if not of type type * convert state_type if not of type type * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * Delete dead code * rename methods * rename to declarative * rename the classes too * Try to install packages to build jq * isort * only automake * Revert "only automake" This reverts commit c8fe154ffc. * remove git * format * substream slicer * specify everything in the slice_definition * Add jq dependency * support multiple parent streams * support getting the parent stream name * cleanup * Add comment * format * delete duplicate file	2022-06-02 08:53:04 -07:00
Alexandre Girard	aa92518721	Low-code connectors: configurable source from yaml (#13038) ... * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * Add missing tests * Add missing files * Add missing tests * add missing file * missing file * missing file * sengrid low code connector * rename * doc * doc * remove broken test * rename * jinja dependency * Add comment * comment * comment * pyjq dependency * update import * rename file * delete unused file * Revert "delete unused file" This reverts commit 758e939367. * fix * rename * abstract property * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * isort * update state * fix imports * update * update dependency * remove dead code * remove dead code * format * rename file * decoder * Use decoder * Update comment * dict_state is actually backed by a dict * Add a comment * update state takes kwargs * move state out of offset paginator * fix * update jq parameter order * fix * pass config * update * update * remove incremental mixin * delete comment * start workin on yaml parser * fix test * progress * refer and overwrite partials * factory tests pass * fix * reset * Assert http_method is an enum value * fix auth * read lists works * fix test * comment * implement all streams * build connection checker * update comments * update comments * remove no_state * rename package * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * remove print * move test * delete duplicates * update * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * pass parameters through kwargs * fix test * update interface * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Allow passing a string or an enum * Define StateType enum * unit tests pass * update dict state * update * can read * fix test * fix from yaml update * elif * convert state_type if not of type type * convert state_type if not of type type * Add a test * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * remove sendgrid specific code * update * update * delete comment * remove sendgrid specific file * remove unused file * Delete dead code * rename methods * rename to declarative * rename the classes too * select streams to check * nit * rename method * rename class * {} is faster than dict() * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/parsers/yaml_parser.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * more precise exception * rename class * add comment * Try to install packages to build jq * isort * only automake * Revert "only automake" This reverts commit c8fe154ffc. * remove git * rename file * create components in kwargs * Use tuple of strings * parser doesn't need to be stored * move file and delete duplicates * Revert "Use tuple of strings" This reverts commit ab5a7afd08. * raise error if streams to check are not in the catalog * Revert "Revert "Use tuple of strings"" This reverts commit 7c9fb8eb33. * traverse tree * rename to options * move docstring * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/checks/check_stream.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * fix tests and format * format * update * better error message * Add jq dependency * Use request header provider * rename * rename field * remove get_context method * rename * add a comment * format Co-authored-by: Sherif A. Nada <snadalive@gmail.com>	2022-06-01 07:31:52 -07:00
Alexandre Girard	d9fa24ffff	Low code connectors: implement components for sendgrid (#12853) ... * checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * Add missing tests * Add missing files * Add missing tests * add missing file * missing file * missing file * rename * doc * doc * remove broken test * rename * jinja dependency * Add comment * comment * comment * pyjq dependency * rename file * delete unused file * Revert "delete unused file" This reverts commit 758e939367. * fix * rename * abstract property * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * isort * update state * fix imports * update dependency * format * rename file * decoder * Use decoder * Update comment * dict_state is actually backed by a dict * Add a comment * update state takes kwargs * move state out of offset paginator * update jq parameter order * update * remove incremental mixin * delete comment * update comments * update comments * remove no_state * rename package * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * remove print * move test * delete duplicates * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * pass parameters through kwargs * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Allow passing a string or an enum * Define StateType enum * convert state_type if not of type type * convert state_type if not of type type * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * Delete dead code * rename methods * rename to declarative * rename the classes too * Try to install packages to build jq * isort * only automake * Revert "only automake" This reverts commit c8fe154ffc. * remove git * format * Add jq dependency * Use request header provider * rename * rename field * remove get_context method * rename	2022-06-01 07:02:35 -07:00
Alexandre Girard	150ab593f8	Low code connectors: core structure (#12850) ... * checkout from alex/cac * doc * doc * remove broken test * rename * rename file * delete unused file * rename * abstract property * isort * update state * Update comment * remove incremental mixin * delete comment * update comments * update comments * remove no_state * rename package * pass parameters through kwargs * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit 758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit 4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * rename to declarative * rename the classes too	2022-05-25 17:27:54 -07:00