* don't update cursor for log messages and and default schema path coming from connector builder
* replace check for connector_builder module with a basic default file path
* update changelog and patch version
* catch the correct exception when pkgutil can't load json file
* method yielding airbytemessage
* move to Stream
* update abstract source
* reset
* missing file
* Yield request and response as log messages
* only emit request and responses if the debug flag is on
* add test docker image
* script to run acceptance tests with local cdk
* Update conftest to use a different image
* extract to method
* dont use a different image tag
* Always install local cdk
* break the cdk
* get path from current working directory
* or
* ignore unit test
* debug log
* Revert "AMI change: ami-0f23be2f917510c26 -> ami-005924fb76f7477ce (#18689)"
This reverts commit bf06decf73.
* build from the top
* Update source-acceptance-test
* fix
* copy setup
* some work on the gradle plugin
* reset to master
* delete unused file
* delete unused file
* reset to master
* optional argument
* delete dead code
* use latest cdk with sendgrid
* fix sendgrid dockerfile
* break the cdk
* use local file
* Revert "break the cdk"
This reverts commit 600c195541.
* dont raise an exception
* reset to master
* unit tests
* missing test
* more unit tests
* remove deprecated comment
* newline
* reset to master
* remove files
* reset
* Update abstract source
* remove method from stream
* convert to airbytemessage
* unittests
* Update
* unit test
* remove debug logs
* Revert "remove debug logs"
This reverts commit a1a139ef37.
* Revert "Revert "remove debug logs""
This reverts commit b1d62cdb60.
* Revert "reset to master"
This reverts commit 3fa6a004c1.
* fix
* slightly better test
* typing
* extract method
* Revert "Revert "reset to master""
This reverts commit 5dac7c2804.
* reset to master
* reset to master
* Revert "reset to master"
This reverts commit 3fa6a004c1.
* Comment
* operate on the message
* Revert "Revert "reset to master""
This reverts commit 5833c84d0a.
* comment
* test
* Revert "test"
This reverts commit 2f91b803b0.
* test
* Revert "test"
This reverts commit 62d95ebbb5.
* test
* Revert "test"
This reverts commit 27150ba341.
* format
* format
* symlink
* Update setup
* update path
* reset to master
* update
* Add local files
* greenhouse
* format
* symlink
* try reordering
* better error message
* better log message
* reset to master
* Revert "merge for qa"
This reverts commit ad7128f2c5, reversing
changes made to 7196c22a73.
* reset to master
* reset to master
* reset to master
* format
* gradlew format
* right type hints
* reset to master
* reset to master
* gradlew format
* a bunch of small fixes
* Update output format
* fixes from feedback
* fixme comment
* streams cannot return AirbyteRecordMessage
* fix
* format
* only return logs when running on debug mode
* move branching
* update typing
* remove dead code
* fix simpleretriever name
* i think this is better
* log response.text
* debug flag
* comment
* pass config
* comments
* run SATs
* fix most of the unit tests
* fix unit test
* reset to master
* runFromPath
* Revert "runFromPath"
This reverts commit 85979a801a.
* Revert "run SATs"
This reverts commit a8a8a2da95.
* no need to convert to dict
* fix test
* method yielding airbytemessage
* move to Stream
* update abstract source
* reset
* missing file
* add test docker image
* script to run acceptance tests with local cdk
* Update conftest to use a different image
* extract to method
* dont use a different image tag
* Always install local cdk
* break the cdk
* get path from current working directory
* or
* ignore unit test
* debug log
* Revert "AMI change: ami-0f23be2f917510c26 -> ami-005924fb76f7477ce (#18689)"
This reverts commit bf06decf73.
* build from the top
* Update source-acceptance-test
* fix
* copy setup
* some work on the gradle plugin
* reset to master
* delete unused file
* delete unused file
* reset to master
* optional argument
* delete dead code
* use latest cdk with sendgrid
* fix sendgrid dockerfile
* break the cdk
* use local file
* Revert "break the cdk"
This reverts commit 600c195541.
* dont raise an exception
* reset to master
* unit tests
* missing test
* more unit tests
* newline
* reset to master
* remove files
* reset
* Update abstract source
* remove method from stream
* convert to airbytemessage
* unittests
* Update
* unit test
* remove debug logs
* Revert "remove debug logs"
This reverts commit a1a139ef37.
* Revert "Revert "remove debug logs""
This reverts commit b1d62cdb60.
* Revert "reset to master"
This reverts commit 3fa6a004c1.
* Revert "Revert "reset to master""
This reverts commit 5dac7c2804.
* reset to master
* reset to master
* test
* Revert "test"
This reverts commit 2f91b803b0.
* test
* Revert "test"
This reverts commit 62d95ebbb5.
* test
* Revert "test"
This reverts commit 27150ba341.
* format
* allow for spec to be defined in the source.yaml manifest instead of an external file
* make spec a component within the language to get schema validation and rework the code for better testing
* fix formatting and extra method
* pr feedback and add some more test
* pr feedback
* bump airbyte-cdk version
* bump version
* gradle format
* remove from manifest spec
* replace emptySchemaLoader with DefaultSchemaLoader
* fix test name
* fix test
* add logging for when we default to the empty schema
* increment patch version
* fix formatting
* update changelog
* use mockschemaloader as the default and rename JsonSchema to JsonFileSchemaLoader
* rename mock to EmptySchemaLoader
* retain existing JsonSchema while hacktoberfest is still in progress
* bump version and changelog
* support custom error messaging for error response + retryable errors
* remove changed backoff i was using for testing
* refactor filter to construct response status internally
* pr feedback
* bump version and update changelog
* schema helpers raises a config_error
* traced exceptions can be turned into connection status messages
* add tests for schema helpers
* return a failed status message rather than throwing in check command
* remove unused imports
* add comment
* bump version / update changelog
* Added new "filters" python file, along with a "hash" filter. This can be extended to include other custom filters in the future.
* Added additional comments
* Moved usage of the hash_obj inside the conditional that confirms it exists
* Moved the hash function call inside a condition to ensure that it exists
* Fixed the application of the salt , so that it does not modify the hash unless it is actually passed in.
* Added unit tests to validate new jinja hash functionality
* Updated unit test to pass numeric value as a float instead of string
* Removed unreferenced import to pytest
* Updated version
* format
* format
* format
* format
* format
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
* Improve airbyte cdk invalid message data type error message
* Test cdk invalid message data type custom error is raised
* Fix test to pass stream as a string
* Add valid record message data input type test
* Add object type and value to AirbyteRecordMessage validator message
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
* input can only be a string
* remove debug print
* cleanup tests
* fix in the factory
* Add a unit test
* fix for lists
* Update changelog
* Update changelog
* update abstract source and connector state manager to update shared and stream state
* clean up some extra comments and extra lines
* a few changes based on PR feedback
* remove separate legacy map from state manager and simplify mapping to descriptor -> state blob
* rename a few paramets and add testing for state update via stream.state override
* replace shared_state processing with an explicit error and fix a few comments and pr feedback
* add some polish and additional test cases
* Add create_state_message to manager and emit per-stream state for connectors
* pr feedback and restructuring parts of the connector state manager initialization
* fix unfinished comment
* Update airbyte-cdk/python/unit_tests/sources/test_abstract_source.py
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* use pytest params to annotate tests better
* change to fix changed class name
* format test params a little better
* Update airbyte-cdk/python/airbyte_cdk/sources/connector_state_manager.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* a few bits of pr feedback
* do not emit namespace in the outbound message if it is None
* fix some issues w/ prior merge from master and pr feedback
* add new tests to ensure we never send None namespace to the platform
* bump CDK version number
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* update abstract source and connector state manager to update shared and stream state
* clean up some extra comments and extra lines
* a few changes based on PR feedback
* remove separate legacy map from state manager and simplify mapping to descriptor -> state blob
* rename a few paramets and add testing for state update via stream.state override
* replace shared_state processing with an explicit error and fix a few comments and pr feedback
* add some polish and additional test cases
* pr feedback and restructuring parts of the connector state manager initialization
* fix unfinished comment
* Update airbyte-cdk/python/unit_tests/sources/test_abstract_source.py
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* use pytest params to annotate tests better
* change to fix changed class name
* Update airbyte-cdk/python/airbyte_cdk/sources/connector_state_manager.py
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* a few bits of pr feedback
* pr feedback and cleaning up some comments and variable renames
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* interpret legacy and new per-stream format into AirbyteStateMessages
* add ConnectorStateManager stubs for future work
* remove frozen for the time being until we need to hash descriptors
* add validation that AirbyteStateMessage has at least one of stream, global, or data fields
* pr feedback and clean up of the code
* remove changes to airbyte_protocol and perform validation in read_state()
* fix import formatting
* replace file retrieval with pkgutil to fix getting schema files
* slightly better error handling on missing files
* filter our schema gen warnings for some classes that cannot generate schemas
* add comment for todo
* add changelog and setup before publish
* draft: first pass at complete schema language generation and factory validator
* actually a working validator and fixes to the schema that went uncaught
* remove extra spike file
* fix formatting file
* Add method to generate the complete JSON schema of the low code declarative language
* add testing of a few components during schema gen
* pr feedback and a little bit of refactoring
* test for schema version
* fix some types that were erroneously marked as invalid schema
* some comments
* add jsonschemamixin to interfaces
* update tests now that interfaces are jsonschemamixin
* accidentally removed a mixin
* remove unneeded test
* make comment a little more clear
* update changelog
* bump version
* generic enum not enum class
* Add method to generate the complete JSON schema of the low code declarative language
* add testing of a few components during schema gen
* test for schema version
* update tests now that interfaces are jsonschemamixin
* accidentally removed a mixin
* remove unneeded test
* make comment a little more clear
* generic enum not enum class
* add generated json file and update docs to reference it
* verbage
* draft: first pass at complete schema language generation and factory validator
* actually a working validator and fixes to the schema that went uncaught
* remove extra spike file
* fix formatting file
* pr feedback and a little bit of refactoring
* fix some types that were erroneously marked as invalid schema
* some comments
* add jsonschemamixin to interfaces
* update changelog
* bump version
* 5-step tutorial
* move
* tiny bit of editing
* Update tutorial
* update docs
* reset
* move files
* record selector, request options, and more links
* update
* update
* connector definition
* link
* links
* update example
* footnote
* typo
* document string interpolation
* note on string interpolation
* update
* fix code sample
* fix
* update sample
* fix
* use the actual config
* Update as per comments
* write as yaml
* typo
* Clarify options overloading
* clarify that docker must be running
* remove extra footnote
* use venv directly
* Apply suggestions from code review
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* signup instructions
* update
* clarify that both dot and bracket notations are interchangeable
* Clarify how check works
* create spec and config before updating connector definition
* clarify what now_local() is
* rename to yaml structure
* Go through tutorial and update end of section code samples
* fix link
* update
* update code samples
* Update code samples
* Update to bracket notation
* remove superfluous comments
* Update docs/connector-development/config-based/tutorial/2-install-dependencies.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* Update docs/connector-development/config-based/tutorial/4-reading-data.md
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
* fix path
* update
* motivation blurp
* warning
* warning
* fix code block
* update code samples
* update code sample
* update code samples
* small updates
* update yaml structure
* custom class example
* language annotations
* update warning
* Update tutorial to use dpath extractor
* Update record selector docs
* unit test
* link to contributing
* tiny update
* $ in front of commands
* $ in front of commands
* More readings
* link to existing config-based connectors
* index
* update
* delete broken link
* supported features
* update
* Add some links
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/record-selector.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* Update docs/connector-development/config-based/overview.md
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
* mention the unit
* headers
* remove mentions of interpolating on stream slice, etc.
* update
* exclude config-based docs
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
Co-authored-by: Augustin <augustin.lafanechere@gmail.com>
Co-authored-by: Brian Lai <51336873+brianjlai@users.noreply.github.com>
