Brian Lai
ca6513625d
[low code connectors] read configs from package_data ( #15810 )
...
* read configs from package_data
* update changelog and setup
* commenting out failing tests in the short term
2022-08-19 21:16:20 -04:00
Brian Lai
7e158ef9af
[low code connectors] generate complete json schema from classes ( #15647 )
...
* draft: first pass at complete schema language generation and factory validator
* actually a working validator and fixes to the schema that went uncaught
* remove extra spike file
* fix formatting file
* Add method to generate the complete JSON schema of the low code declarative language
* add testing of a few components during schema gen
* pr feedback and a little bit of refactoring
* test for schema version
* fix some types that were erroneously marked as invalid schema
* some comments
* add jsonschemamixin to interfaces
* update tests now that interfaces are jsonschemamixin
* accidentally removed a mixin
* remove unneeded test
* make comment a little more clear
* update changelog
* bump version
* generic enum not enum class
* Add method to generate the complete JSON schema of the low code declarative language
* add testing of a few components during schema gen
* test for schema version
* update tests now that interfaces are jsonschemamixin
* accidentally removed a mixin
* remove unneeded test
* make comment a little more clear
* generic enum not enum class
* add generated json file and update docs to reference it
* verbage
2022-08-18 18:53:42 -04:00
Brian Lai
ca80d3782a
[low code connectors] perform schema validation of the input config against the declarative language schema ( #15543 )
...
* draft: first pass at complete schema language generation and factory validator
* actually a working validator and fixes to the schema that went uncaught
* remove extra spike file
* fix formatting file
* pr feedback and a little bit of refactoring
* fix some types that were erroneously marked as invalid schema
* some comments
* add jsonschemamixin to interfaces
* update changelog
* bump version
2022-08-18 15:29:26 -04:00
Alexandre Girard
f540499f43
[low-code connectors]: Assert there are no custom top-level fields ( #15489 )
...
* move components to definitions field
* Also update the references
* validate the top level fields and add version
* raise exception on unknown fields
* newline
* unit tests
* set version to 0.1.0
* newline
2022-08-10 11:37:07 -07:00
Brian Lai
ef712f18aa
[low-code connectors] fix so we don't display yaml when debug flag is turned off ( #15383 )
...
* fix so we don't display yaml when debug is turned off
* forgot to remove old debug level
2022-08-08 01:11:31 -04:00
Alexandre Girard
08239abafd
Alex/lowcode referencedocs ( #14973 )
...
* Add docstrings for auth package
* docstrings for the check package
* docstrings for the datetime package
* docstrings for the decoder package
* docstrings for extractors package and fix tests
* interpolation docstrings
* ref -> and parser docstrings
* docstrings for parsers package
* error handler docstrings
* requester docstrings
* more docstrings
* docstrings
* docstrings
* docstrings
* Use defined type annotations
* update
* update docstrings
* Update docstrings
* update docstrings
* update docstrings
* update template
* Revert "update template"
This reverts commit eb4a11858b .
* update template
* update
* move to interpolated_string
* update docstring
* update
* fix tests
* format
* return type can also be an array
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/interpolated_boolean.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/interpolation.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/jinja.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/interpolation/interpolated_boolean.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/requesters/error_handlers/backoff_strategy.py
* Update as per comments
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
2022-07-25 18:10:32 -07:00
Brian Lai
7bff12aea5
[ #3078 ] [CDK] Add support for enabling debug from command line and some basic general debug logs ( #14521 )
...
* allow for command line debug option and basic debug statements + declarative
* feedback from pr comments
* fix some tests w/ req/res mixed up and fixing logging tests
* formatting
* pr feedback: cleaning up traces in logger.py and update docs with debug configuration
* remove unneeded trace logger test
* remove extra print statement
2022-07-13 18:01:07 -04:00
Sherif A. Nada
743e6c2d6c
[Low-code CDK] Add ability to remove fields ( #14402 )
2022-07-11 19:04:57 -07:00
Alexandre Girard
85449c975c
[low-code connectors] default types and default values ( #14004 )
...
* default types and default values
* cleanup
* fixes so read works
* remove prints and trycatch
* comment
* remove unused param
* split file
* extract method
* extract methods
* comment
* optional
* fix test
* cleanup
* delete interpolated request header provider
* simplify next page url paginator interface
* comment
* format
2022-06-27 20:43:03 -07:00
Alexandre Girard
aa92518721
Low-code connectors: configurable source from yaml ( #13038 )
...
* checkout from alex/cac
* checkout from alex/cac
* checkout from alex/cac
* checkout from alex/cac
* Add missing tests
* Add missing files
* Add missing tests
* add missing file
* missing file
* missing file
* sengrid low code connector
* rename
* doc
* doc
* remove broken test
* rename
* jinja dependency
* Add comment
* comment
* comment
* pyjq dependency
* update import
* rename file
* delete unused file
* Revert "delete unused file"
This reverts commit 758e939367 .
* fix
* rename
* abstract property
* delete unused field
* delete unused field
* rename
* pass kwargs directly
* isort
* Revert "isort"
This reverts commit 4a79223944 .
* isort
* update state
* fix imports
* update
* update dependency
* remove dead code
* remove dead code
* format
* rename file
* decoder
* Use decoder
* Update comment
* dict_state is actually backed by a dict
* Add a comment
* update state takes kwargs
* move state out of offset paginator
* fix
* update jq parameter order
* fix
* pass config
* update
* update
* remove incremental mixin
* delete comment
* start workin on yaml parser
* fix test
* progress
* refer and overwrite partials
* factory tests pass
* fix
* reset
* Assert http_method is an enum value
* fix auth
* read lists works
* fix test
* comment
* implement all streams
* build connection checker
* update comments
* update comments
* remove no_state
* rename package
* checkout from alex/cac
* Add missing tests
* Add missing files
* missing file
* rename
* jinja dependency
* Add comment
* comment
* comment
* Revert "delete unused file"
This reverts commit 758e939367 .
* delete unused field
* delete unused field
* rename
* pass kwargs directly
* isort
* Revert "isort"
This reverts commit 4a79223944 .
* format
* decoder
* better error handling
* remove nostate
* isort
* remove print
* move test
* delete duplicates
* update
* delete dead code
* Update mapping type to [str, Any]
* add comment
* Add comment
* pass parameters through kwargs
* pass parameters through kwargs
* fix test
* update interface
* update interface to pass source in interface
* update interface to pass source in interface
* rename to stream_slicer
* Allow passing a string or an enum
* Define StateType enum
* unit tests pass
* update dict state
* update
* can read
* fix test
* fix from yaml update
* elif
* convert state_type if not of type type
* convert state_type if not of type type
* Add a test
* Low code connectors: string interpolation with jinja (#12852 )
* checkout from alex/cac
* Add missing tests
* Add missing files
* missing file
* rename
* jinja dependency
* Add comment
* comment
* comment
* Revert "delete unused file"
This reverts commit 758e939367 .
* delete unused field
* delete unused field
* rename
* pass kwargs directly
* isort
* Revert "isort"
This reverts commit 4a79223944 .
* format
* decoder
* better error handling
* remove nostate
* isort
* delete dead code
* Update mapping type to [str, Any]
* add comment
* Add comment
* pass parameters through kwargs
* move test to right module
* Add missing test
* Use authbase instead of deprecated class
* leverage generator
* remove sendgrid specific code
* update
* update
* delete comment
* remove sendgrid specific file
* remove unused file
* Delete dead code
* rename methods
* rename to declarative
* rename the classes too
* select streams to check
* nit
* rename method
* rename class
* {} is faster than dict()
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/parsers/yaml_parser.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
* more precise exception
* rename class
* add comment
* Try to install packages to build jq
* isort
* only automake
* Revert "only automake"
This reverts commit c8fe154ffc .
* remove git
* rename file
* create components in kwargs
* Use tuple of strings
* parser doesn't need to be stored
* move file and delete duplicates
* Revert "Use tuple of strings"
This reverts commit ab5a7afd08 .
* raise error if streams to check are not in the catalog
* Revert "Revert "Use tuple of strings""
This reverts commit 7c9fb8eb33 .
* traverse tree
* rename to options
* move docstring
* Update airbyte-cdk/python/airbyte_cdk/sources/declarative/checks/check_stream.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
* fix tests and format
* format
* update
* better error message
* Add jq dependency
* Use request header provider
* rename
* rename field
* remove get_context method
* rename
* add a comment
* format
Co-authored-by: Sherif A. Nada <snadalive@gmail.com >
2022-06-01 07:31:52 -07:00