3 Commits

Author	SHA1	Message	Date
Brian Lai	bd31100774	initial first pass converting every component to dataclasses (#15189) ... * initial first pass converting every component to dataclasses * replace the hackier options pass through * get rid of the hackier way * fix issues w/ type hints by making options required and lots of fixes to the language to fix compatability for dataclasses * add dataclasses-jsonschema to setup * fix oauth authenticator to avoid dataclass name collisions * fix spacing for CI tests * remove property from oauth and fix a interpolation bug * pr feedback and cleaning up the code a bit, attempt at avoiding renaming * fix templates and bugs surfaced during greenhouse spec testing * fix tests * fix missing options in some declarative components * fix tests related to pulling latest master * fix issue w/ passing state, slice, and token to subcomponents * switch name back to get_access_token() since no name collision anymore	2022-08-05 17:39:27 -04:00
Alexandre Girard	08239abafd	Alex/lowcode referencedocs (#14973) ... * Add docstrings for auth package * docstrings for the check package * docstrings for the datetime package * docstrings for the decoder package * docstrings for extractors package and fix tests * interpolation docstrings * ref -> and parser docstrings * docstrings for parsers package * error handler docstrings * requester docstrings * more docstrings * docstrings * docstrings * docstrings * Use defined type annotations * update * update docstrings * Update docstrings * update docstrings * update docstrings * update template * Revert "update template" This reverts commit eb4a11858b. * update template * update * move to interpolated_string * update docstring * update * fix tests * format * return type can also be an array * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/interpolated_boolean.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/interpolation.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/jinja.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/interpolated_boolean.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/requesters/error_handlers/backoff_strategy.py * Update as per comments Co-authored-by: Sherif A. Nada <snadalive@gmail.com>	2022-07-25 18:10:32 -07:00
Alexandre Girard	04a44b3d8d	[low-code connectors] Refactor paginator component so it owns the request options to set (#14433) ... * checkout files from test branch * read_incremental works * reset to master * remove dead code * comment * fix * Add test * comments * utc * format * small fix * Add test with rfc3339 * remove unused param * fix test * configurable state checkpointing * update test * start working on retrier * retry predicate * return response status * look in error message * cleanup test * constant backoff strategy * chain backoff strategy * chain retrier * Add to class types registry * extract backoff time from header * wait until * update * split file * parse_records * classmethod * delete dead code * comment * comment * comments * fix * test for instantiating chain retrier * fix parsing * cleanup * fix * reset * never raise on http error * remove print * comment * comment * comment * comment * remove prints * add declarative stream to registry * start working on limit paginator * support for offset pagination * tests * move limit value * extract request option * boilerplate * page increment * delete offset paginator * update conditional paginator * refactor and fix test * fix test * small fix * Delete dead code * Add docstrings * quick fix * exponential backoff * fix test * fix * delete unused properties * fix * missing unit tests * uppercase * docstrings * rename to success * compare full request instead of just url * renmae module * rename test file * rename interface * rename default retrier * rename to compositeerrorhandler * fix missing renames * move action to filter * str -> minmaxdatetime * small fixes * plural * add example * handle header variations * also fix wait time from * allow using a regex to extract the value * group() * docstring * add docs * update comment * docstrings * fix tests * rename param * cleanup stop_condition * cleanup * Add examples * interpolated pagination strategy * dont need duplicate class * docstrings * more docstrings * docstrings * update comment * Update airbyte-cdk/python/airbyte_cdk/sources/declarative/requesters/http_requester.py Co-authored-by: Sherif A. Nada <snadalive@gmail.com> * version: Update Parquet library to latest release (#14502) The upstream Parquet library that is currently pinned for use in the S3 destination plugin is over a year old. The current version is generating invalid schemas for date-time with time-zone fields which appears to be addressed in the `1.12.3` release of the library in commit c72862b613 * merge * 🎉 Source Github: improve schema for stream `pull_request_commits` added "null" (#14613) Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com> * Docs: Fixed broken links (#14622) * fixing broken links * more broken links * source-hubspot: change mentioning of Mailchimp into HubSpot doc (#14620) * Helm Chart: Add external temporal option (#14597) * conflict env configmap and chart lock * reverting lock * add eof lines and documentation on values yaml * conflict json file * rollback json * solve conflict * correct minio with new version Co-authored-by: Guy Feldman <gfeldman@86labs.com> * 🎉 Add YAML format to source-file reader (#14588) * Add yaml reader * Update docs * Bumpversion of connector * bump docs * Update pyarrow dependency * Upgrade pandas dependency * auto-bump connector version Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * 🎉 Source Okta: add GroupMembers stream (#14380) * add Group_Members stream to okta source - Group_Members return a list of users, the same schema of Users stream. - Create a shared schema users, and both group_members and users sechema use it as a reference. - Add Group_Members stream to source connector * add tests and fix logs schema - fix the test error: None is not one of enums though the enum type includes both string and null, it comes from json schema validator ddb87afad8/jsonschema/_validators.py (L279-L285) - change grouop_members to use id as the cursor field since `filter` is not supported in the query string - fix the abnormal state test on logs stream, when since is abnormally large, until has to defined, an equal or a larger value - remove logs stream from full sync test, because 2 full sync always has a gap -- at least a new log about users or groups api. * last polish before submit the PR - bump docker version - update changelog - add the right abnormal value for logs stream - correct the sample catalog * address comments:: - improve comments for until parameter under the logs stream - add use_cache on groupMembers * add use_cache to Group_Members * change configured_catalog to test * auto-bump connector version Co-authored-by: marcosmarxm <marcosmarxm@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> * split test files * renames * missing unit test * add missing unit tests * rename * assert isinstance * start extracting to their own files * use final instead of classmethod * assert we retry 429 errors * Add log * replace asserts with valueexceptions * delete superfluous print statement * fix factory so we don't need to union everything with strings * get class_name from type * remove from class types registry * process error handlers one at a time * sort * delete print statement * comment * comment * format * delete unused file * comment * interpolatedboolean * comment * not optional * not optional * unit tests * fix request body data * add test * move file to right module * update * reset to master * format * rename to pass_by * rename to page size * fix * add test * fix body data * delete extra newlines * move to subpackage * fix imports * handle str body data * simplify * fix typing * always return a map * rename to inject_into * only accept enum * delete conditional paginator * only return body data * missing test * update docstrings * update docstrings * update comment * rename Co-authored-by: Sherif A. Nada <snadalive@gmail.com> Co-authored-by: Tobias Macey <tmacey@boundlessnotions.com> Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com> Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com> Co-authored-by: Bas Beelen <bjgbeelen@gmail.com> Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com> Co-authored-by: Guy Feldman <gfeldman@86labs.com> Co-authored-by: Christophe Duong <christophe.duong@gmail.com> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com> Co-authored-by: Yiyang Li <yiyangli2010@gmail.com> Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>	2022-07-20 22:46:51 -07:00