Scripts to
* Run CATs against the local CDK for one connector
* Run CATs against the local CDK for multiple connectors
* Create a connecter image with the local CDK
---------
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Expected Records to `.jsonl` format
* fix formatting template
* remove endline
* update templates
* Update docs/connector-development/testing-connectors/source-acceptance-tests-reference.md
Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
* Dockerfile to 3.9
* Python version
* More python updates
* 3.9 on GitHub actions and lint updates
* Test out 3.9.11 on GitHub actions
* install python with an action
* formatting: newline
* Also has python code
* only check first level for changed modules
Previous example (source-google-search-console/credentials)
* Test failure: there is no logger.trace
* change license in generator base files
* run gradlew scaffold
* format eof
* add eof singer base
* remove license from test_* http source
* format test_incremental http source
source-acceptance-test framework not longer required json_schema parameters
from catalog file. This parameter is verbose, makes reading config file
complicated and could be misleading when debugging acceptance test issues.
Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
See bluedenim/log4j-s3-search#101 for a summary.
Because this is upstream, I built the jar locally and uploaded it to a temp gcs bucket serving as a maven repository.
I've added this location and the new version in the build.gradle.
I want to merge this in now since there are several users waiting on this. Created #4915 to follow up with the OSS PR and clean up work.
* add AIRBYTE_ENTRYPOINT for kubernetes support
* bump versions
* bump version in seed
* Update generic template
* keep scaffold sources at 0.1.0
* add missing newline
* handle python base versions correctly
* re-bump mysql and postgres sources
* re-bump snowflake destination
* add skip tests option
* switch to running tests
* reverse conditional to make it safer
* fix publish to include the test running
* fix iterable version
* fix file generation
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
As the title suggests.
Some intricacies around how various directories have diverged.
The HTTP templates now use integration_tests to house catalogs and config files instead of sample_files. After this PR, everything should be updated to use integration_tests for all files except config files. Config files use secrets since they often contain secrets. Anything, besides actual implementations that have yet to be updated, not following this is my mistake.
Even though it diverges from the generated templates, I've decided to leave the tutorial code and the tutorial as using sample_files for now. I think it's more straightforward than telling people to create a new secrets directory. We can change this at a later date.
Confirmed the checked in http python tutorial code works by running ./gradlew :airbyte-integrations:connectors:source-python-http-tutorial:standardSourceTestFile.
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.
General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.
Checked in and added tests for the generated modules.
