* draft: first pass at complete schema language generation and factory validator
* actually a working validator and fixes to the schema that went uncaught
* remove extra spike file
* fix formatting file
* pr feedback and a little bit of refactoring
* fix some types that were erroneously marked as invalid schema
* some comments
* add jsonschemamixin to interfaces
* update changelog
* bump version
* initial first pass converting every component to dataclasses
* replace the hackier options pass through
* get rid of the hackier way
* fix issues w/ type hints by making options required and lots of fixes to the language to fix compatability for dataclasses
* add dataclasses-jsonschema to setup
* fix oauth authenticator to avoid dataclass name collisions
* fix spacing for CI tests
* remove property from oauth and fix a interpolation bug
* pr feedback and cleaning up the code a bit, attempt at avoiding renaming
* fix templates and bugs surfaced during greenhouse spec testing
* fix tests
* fix missing options in some declarative components
* fix tests related to pulling latest master
* fix issue w/ passing state, slice, and token to subcomponents
* switch name back to get_access_token() since no name collision anymore
* checkout files from test branch
* read_incremental works
* reset to master
* remove dead code
* comment
* fix
* Add test
* comments
* utc
* format
* small fix
* Add test with rfc3339
* remove unused param
* fix test
* configurable state checkpointing
* update test
* start working on retrier
* retry predicate
* return response status
* look in error message
* cleanup test
* constant backoff strategy
* chain backoff strategy
* chain retrier
* Add to class types registry
* extract backoff time from header
* wait until
* update
* split file
* parse_records
* classmethod
* delete dead code
* comment
* comment
* comments
* fix
* test for instantiating chain retrier
* fix parsing
* cleanup
* fix
* reset
* never raise on http error
* remove print
* comment
* comment
* comment
* comment
* remove prints
* add declarative stream to registry
* Delete dead code
* Add docstrings
* quick fix
* exponential backoff
* fix test
* fix
* delete unused properties
* fix
* missing unit tests
* uppercase
* docstrings
* rename to success
* compare full request instead of just url
* renmae module
* rename test file
* rename interface
* rename default retrier
* rename to compositeerrorhandler
* fix missing renames
* move action to filter
* str -> minmaxdatetime
* small fixes
* plural
* add example
* handle header variations
* also fix wait time from
* allow using a regex to extract the value
* group()
* docstring
* add docs
* update comment
* docstrings
* update comment
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/requesters/http_requester.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* version: Update Parquet library to latest release (#14502)
The upstream Parquet library that is currently pinned for use in the S3 destination plugin is over a year old. The current version is generating invalid schemas for date-time with time-zone fields which appears to be addressed in the `1.12.3` release of the library in commit c72862b613
* merge
* 🎉 Source Github: improve schema for stream `pull_request_commits` added "null" (#14613)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
* Docs: Fixed broken links (#14622)
* fixing broken links
* more broken links
* source-hubspot: change mentioning of Mailchimp into HubSpot doc (#14620)
* Helm Chart: Add external temporal option (#14597)
* conflict env configmap and chart lock
* reverting lock
* add eof lines and documentation on values yaml
* conflict json file
* rollback json
* solve conflict
* correct minio with new version
Co-authored-by: Guy Feldman <gfeldman@86labs.com>
* 🎉 Add YAML format to source-file reader (#14588)
* Add yaml reader
* Update docs
* Bumpversion of connector
* bump docs
* Update pyarrow dependency
* Upgrade pandas dependency
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* 🎉 Source Okta: add GroupMembers stream (#14380)
* add Group_Members stream to okta source
- Group_Members return a list of users, the same schema of Users stream.
- Create a shared schema users, and both group_members and users sechema use it as a reference.
- Add Group_Members stream to source connector
* add tests and fix logs schema
- fix the test error: None is not one of enums though the enum type includes both string and null, it comes from json schema validator
ddb87afad8/jsonschema/_validators.py (L279-L285)
- change grouop_members to use id as the cursor field since `filter` is not supported in the query string
- fix the abnormal state test on logs stream, when since is abnormally large, until has to defined, an equal or a larger value
- remove logs stream from full sync test, because 2 full sync always has a gap -- at least a new log about users or groups api.
* last polish before submit the PR
- bump docker version
- update changelog
- add the right abnormal value for logs stream
- correct the sample catalog
* address comments::
- improve comments for until parameter under the logs stream
- add use_cache on groupMembers
* add use_cache to Group_Members
* change configured_catalog to test
* auto-bump connector version
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* split test files
* renames
* missing unit test
* add missing unit tests
* rename
* assert isinstance
* start extracting to their own files
* use final instead of classmethod
* assert we retry 429 errors
* Add log
* replace asserts with valueexceptions
* delete superfluous print statement
* fix factory so we don't need to union everything with strings
* get class_name from type
* remove from class types registry
* process error handlers one at a time
* sort
* delete print statement
* comment
* comment
* format
* delete unused file
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Tobias Macey <tmacey@boundlessnotions.com>
Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
Co-authored-by: Bas Beelen <bjgbeelen@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Guy Feldman <gfeldman@86labs.com>
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Yiyang Li <yiyangli2010@gmail.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
* generate pdoc
* set python version to 3.8
* mimic doc
* set the os
* Revert "generate pdoc"
This reverts commit 82bdeabe8f.
* add missing docs
* add declarative to index
* missing init files
* generate rst files
* support semi incremental by adding extractor record filter
* refactor extractor into a record_selector that supports extraction and filtering of response records
