* Dockerfile to 3.9
* Python version
* More python updates
* 3.9 on GitHub actions and lint updates
* Test out 3.9.11 on GitHub actions
* install python with an action
* formatting: newline
* Also has python code
* only check first level for changed modules
Previous example (source-google-search-console/credentials)
* Test failure: there is no logger.trace
* add response text and message to backoff error handling
* add response text and message to backoff error handling
* add response text case http error
* change response text to before raise error
* apply suggestions
* bump cdk version
* Pass worker metadata to connector
* Fix compilation
* Pass in job id and image from worker
* Remove application version
* Add default job environment variables
* Add back removed comment
* Rename env map to job metadata
* Fix env configs
* Read connector from application
* Use empty string
* Remove println
* Fix unit test
* Fix compilation error
* Introduce constants for worker env
* Add worker env to ENV_VARS_TO_TRANSFER
* Pass into getWorkerMetadata map to all constructions
* Format code
* Format octavia cli
* Fix test compilation
* Fix typos
* Ensure AirbyteLogger is thread-safe
- Introduce a global lock to ensure `AirbyteLogger` is thread-safe.
- The `logging` module is thread-safe, however `print` is not, and is currently used. This means that messages sent to stdout can clash if connectors use threading. This is obviously a huge problem when the IPC between the source/destination is stdout!
- A `multiprocessing.Lock` could have been introduced however given that `logging` module is not multiprocess-safe I thought that thread-safety should be first goal.
- IMO the `AirbyteLogger` should be a subclass of the `logging.Logger` so you have thread-safety automatically, however I didn't want to make a huge wholesale change here.
* Revert lock and add deprecation warning instead
* add support for streams with state attribute
* fix pre-commit and format
* update state attribute docs and logic
* added IncrementalMixin
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
* CDK #9500 - fixing type errors
* CDK #9500 - exclude singer from check
* CDK #9500 - type ignore in utils.schema_helpers.py
* CDK #9500 - type ignore in utils.sentry.py
* CDK #9500 - type ignore in singer.singer_helpers.py
* CDK #9500 - type ignore in singer.source
* CDK #9500 - fix according to PR review
* CDK #9500 - fix according to PR review
* CDK #9500 - bump version
* fix typing, drop AirbyteLogger
* format
* bump the version
* use logger instead of fixture logger
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: auganbay <auganenu@gmail.com>
* [9044] Destination-gcs\destination-bigquery(gcs) - updated check() method to handle that user has both storage.objects.create and storage.multipartUploads.create roles
This commit reverts #9348 (9bb28939ee) because it does not work. The `test_docker_runner[standard]` and `test_docker_runner[waiting]` test cases still fail transiently.
* WIP - Adding unit tests to SAT full refresh test
* unit tests to SAT full refresh test
* Format
* Update airbyte-integrations/bases/source-acceptance-test/unit_tests/test_test_full_refresh.py
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
* use symmetric_difference to check for differences between expected/actual when testing stream read on SAT
* bump version
* bump version
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
* Integrate Sentry for performance and errors tracking.
* Add sentry sensitive data scrubbing.
* updated cdk version and changelog
* Integrate Sentry for performance and errors tracking
Add `SENTRY_DSN` environment variable
* Integrate Sentry for performance and errors tracking
Add `sentry_sdk` to install requirements
* format cdk
* enable Sentry for google-search-console
* updated connector version
* update spec and source yamls
Co-authored-by: auganbay <auganenu@gmail.com>
Co-authored-by: Sergei Solonitcyn <sergei.solonitcyn@zazmic.com>
* Log http status code and content in default backoff handler
* Log http status code and content in usef defined backoff handler
* updated cdk version and changelog
* make it clear: exc.response
Co-authored-by: auganbay <auganenu@gmail.com>
* add base spec model
* fix usage of state_checkpoint_interval in case it is dynamic
* add schema base models, fix spelling, signatures and polishing
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
* Secure logger implementation minus still broken new tests
* Secure logger implementation and unit tests
* code review changes
* filter text on uncaught exceptions
* auto-formatting
* Mistaken change left in code
* filter text on uncaught exceptions
* Simplify code, remove LoggingFilter and move obfuscation functionality to Formatter
* Update airbyte-cdk/python/airbyte_cdk/entrypoint.py
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
* Obfuscate Secrets in Logging, code review changes
* Obfuscate Secrets in Logging, code review changes, unit test fixes
* CHANGELOG.md
* Format and flake8
* Fix build error/bump version/format
* Add airbyte prefix on test logger
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
* Secure logger implementation minus still broken new tests
* Secure logger implementation and unit tests
* code review changes
* filter text on uncaught exceptions
* auto-formatting
* Mistaken change left in code
* filter text on uncaught exceptions
* Simplify code, remove LoggingFilter and move obfuscation functionality to Formatter
* Update airbyte-cdk/python/airbyte_cdk/entrypoint.py
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
* Obfuscate Secrets in Logging, code review changes
* Obfuscate Secrets in Logging, code review changes, unit test fixes
* CHANGELOG.md
Co-authored-by: Eugene Kulak <widowmakerreborn@gmail.com>
* Change OAuth API
* Change protocol for new OAuthConfigSpecification
* Refactor OAuth classes and tests
* Remove webbackend source/destination creation
* Change from webback to normal API
* Implement new protocol change with OAuth specs
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* format
* format
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Change OAuth API
* Change protocol for new OAuth Spec (#7827)
* Add examples
* Add protocol object to api too
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* adding google sheets oauth flow to server
* fix oauth type in protocol yaml
* bump sheets version in definitions
* added GDrive scope
* update sheets to master changes
* update protocol incl. cdk
* protocol typing for oauth rootobject
* format
* Add support for headers to OAuth2Authenticator
Send custom headers in `refresh_access_token()`.
* Bump version + update CHANGELOG.md
* Add tests
* Update tests for refresh_access_token()
* Assert that mock_refresh_token_call was called
* Remove init file
