* Update lowcode docs
* Update
* fix links
* more links
* Schemas
* indent
* Update docs/connector-development/config-based/advanced-topics.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/advanced-topics.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/low-code-cdk-overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/understanding-the-yaml-file/request-options.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/understanding-the-yaml-file/requester.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/understanding-the-yaml-file/stream-slicers.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/understanding-the-yaml-file/requester.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/understanding-the-yaml-file/error-handling.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* capitalization
* fix schema
* fix schema
* small changes so greenhouse validates
* fix brackets
* update code samples
* Add missing types
* fix
* reset
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Improve airbyte cdk invalid message data type error message
* Test cdk invalid message data type custom error is raised
* Fix test to pass stream as a string
* Add valid record message data input type test
* Add object type and value to AirbyteRecordMessage validator message
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
* input can only be a string
* remove debug print
* cleanup tests
* fix in the factory
* Add a unit test
* fix for lists
* Update changelog
* Update changelog
* update abstract source and connector state manager to update shared and stream state
* clean up some extra comments and extra lines
* a few changes based on PR feedback
* remove separate legacy map from state manager and simplify mapping to descriptor -> state blob
* rename a few paramets and add testing for state update via stream.state override
* replace shared_state processing with an explicit error and fix a few comments and pr feedback
* add some polish and additional test cases
* Add create_state_message to manager and emit per-stream state for connectors
* pr feedback and restructuring parts of the connector state manager initialization
* fix unfinished comment
* Update airbyte-cdk/python/unit_tests/sources/test_abstract_source.py
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* use pytest params to annotate tests better
* change to fix changed class name
* format test params a little better
* Update airbyte-cdk/python/airbyte_cdk/sources/connector_state_manager.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* a few bits of pr feedback
* do not emit namespace in the outbound message if it is None
* fix some issues w/ prior merge from master and pr feedback
* add new tests to ensure we never send None namespace to the platform
* bump CDK version number
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* update abstract source and connector state manager to update shared and stream state
* clean up some extra comments and extra lines
* a few changes based on PR feedback
* remove separate legacy map from state manager and simplify mapping to descriptor -> state blob
* rename a few paramets and add testing for state update via stream.state override
* replace shared_state processing with an explicit error and fix a few comments and pr feedback
* add some polish and additional test cases
* pr feedback and restructuring parts of the connector state manager initialization
* fix unfinished comment
* Update airbyte-cdk/python/unit_tests/sources/test_abstract_source.py
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* use pytest params to annotate tests better
* change to fix changed class name
* Update airbyte-cdk/python/airbyte_cdk/sources/connector_state_manager.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* a few bits of pr feedback
* pr feedback and cleaning up some comments and variable renames
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* interpret legacy and new per-stream format into AirbyteStateMessages
* add ConnectorStateManager stubs for future work
* remove frozen for the time being until we need to hash descriptors
* add validation that AirbyteStateMessage has at least one of stream, global, or data fields
* pr feedback and clean up of the code
* remove changes to airbyte_protocol and perform validation in read_state()
* fix import formatting
* Add Version to AirbyteMessage
* Move protocol version to ConnectorSpecification
* Add cdk generated protocol model
* Add protocol_version to the sample ConnectorSpec in the docs
* Update airbyte-protocol/protocol-models/src/main/resources/airbyte_protocol/airbyte_protocol.yaml
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* update doc
* Update CDK changelog
* Update CDK protocol model
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* replace file retrieval with pkgutil to fix getting schema files
* slightly better error handling on missing files
* filter our schema gen warnings for some classes that cannot generate schemas
* add comment for todo
* add changelog and setup before publish
* draft: first pass at complete schema language generation and factory validator
* actually a working validator and fixes to the schema that went uncaught
* remove extra spike file
* fix formatting file
* Add method to generate the complete JSON schema of the low code declarative language
* add testing of a few components during schema gen
* pr feedback and a little bit of refactoring
* test for schema version
* fix some types that were erroneously marked as invalid schema
* some comments
* add jsonschemamixin to interfaces
* update tests now that interfaces are jsonschemamixin
* accidentally removed a mixin
* remove unneeded test
* make comment a little more clear
* update changelog
* bump version
* generic enum not enum class
* Add method to generate the complete JSON schema of the low code declarative language
* add testing of a few components during schema gen
* test for schema version
* update tests now that interfaces are jsonschemamixin
* accidentally removed a mixin
* remove unneeded test
* make comment a little more clear
* generic enum not enum class
* add generated json file and update docs to reference it
* verbage
* draft: first pass at complete schema language generation and factory validator
* actually a working validator and fixes to the schema that went uncaught
* remove extra spike file
* fix formatting file
* pr feedback and a little bit of refactoring
* fix some types that were erroneously marked as invalid schema
* some comments
* add jsonschemamixin to interfaces
* update changelog
* bump version
* 5-step tutorial
* move
* tiny bit of editing
* Update tutorial
* update docs
* reset
* move files
* record selector, request options, and more links
* update
* update
* connector definition
* link
* links
* update example
* footnote
* typo
* document string interpolation
* note on string interpolation
* update
* fix code sample
* fix
* update sample
* fix
* use the actual config
* Update as per comments
* write as yaml
* typo
* Clarify options overloading
* clarify that docker must be running
* remove extra footnote
* use venv directly
* Apply suggestions from code review
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* signup instructions
* update
* clarify that both dot and bracket notations are interchangeable
* Clarify how check works
* create spec and config before updating connector definition
* clarify what now_local() is
* rename to yaml structure
* Go through tutorial and update end of section code samples
* fix link
* update
* update code samples
* Update code samples
* Update to bracket notation
* remove superfluous comments
* Update docs/connector-development/config-based/tutorial/2-install-dependencies.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/4-reading-data.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* fix path
* update
* motivation blurp
* warning
* warning
* fix code block
* update code samples
* update code sample
* update code samples
* small updates
* update yaml structure
* custom class example
* language annotations
* update warning
* Update tutorial to use dpath extractor
* Update record selector docs
* unit test
* link to contributing
* tiny update
* $ in front of commands
* $ in front of commands
* More readings
* link to existing config-based connectors
* index
* update
* delete broken link
* supported features
* update
* Add some links
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/record-selector.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* mention the unit
* headers
* remove mentions of interpolating on stream slice, etc.
* update
* exclude config-based docs
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* move components to definitions field
* Also update the references
* validate the top level fields and add version
* raise exception on unknown fields
* newline
* unit tests
* set version to 0.1.0
* newline
* fix parse and format methods
* define constant
* remove timestamp magic keyword
* comment
* test for ci
* uncomment test
* use timestamp()
* Bump cdk version
* bump to 0.1.72
* initial first pass converting every component to dataclasses
* replace the hackier options pass through
* get rid of the hackier way
* fix issues w/ type hints by making options required and lots of fixes to the language to fix compatability for dataclasses
* add dataclasses-jsonschema to setup
* fix oauth authenticator to avoid dataclass name collisions
* fix spacing for CI tests
* remove property from oauth and fix a interpolation bug
* pr feedback and cleaning up the code a bit, attempt at avoiding renaming
* fix templates and bugs surfaced during greenhouse spec testing
* fix tests
* fix missing options in some declarative components
* fix tests related to pulling latest master
* fix issue w/ passing state, slice, and token to subcomponents
* switch name back to get_access_token() since no name collision anymore
* requester is a request options provider
* get request options from slicer
* remove prints
* share interface
* actual fix with test
* small fix
* missing tests
* missing *
* simplify intersection logic
* bump cdk version
* Emit a state message even if no records were read
* newline
* merge
* comment
* implement logic in the abstract source
* remove logic from declarative source
* comment
* bump cdk version
