* try to publish new normalization version
* default to using ssl in postgres destinatoin
* tidy up
* Run normalization tests using postgres DB with SSL support
* bump version
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* updated mysql tests
* updated mysql tests
* added mysql ssh tunnel tests by key
* fixed remarks
* fixed remarks
* updated DatabricksStreamCopier
* switch to custom file for ssh config in normalization
* updated MySQL SSH tests
* bump version
* get local port properly
* updated assertSameValue for MySQL ssh tunnel
* updated image version and documentation
* updated code style
* updated CI credentials
* updated normalization documentation
Co-authored-by: George Claireaux <george@claireaux.co.uk>
* oracle normalization
* correct dbt_project function for oracle
* unit tests
* run format
* correct ephemeral tests
* add gradle dependency for oracle destination
* run int tests
* add oracle in settings.gradle for normalization run[
* use default airbyte columns
* format
* test all destinatoin ephemeral
* correct unit test
* correct unit test
* destination docs update
* correct mypy
* integration test all dest
* refactor oracle function
* merge master
* run all destinations
* flake8 escape regex
* surrogate key function
* correct few minor comments
* refactor scd sql function
* refactor scd function
* revert test
* refactor minor details
* revert tests
* revert ephemeral test
* revert unit test table_registry
* revert airbyte_protocol format
* format
* bump normalization version in worker
* minor chnages
* minor chages
* correct json_column for other destinations
* gradlew format
* revert tests
* remove comments
* add Oracle destination explicit in safe_cast_str
* add quote_in_parenthesis inside if clause
* gradlew format
* Add mysql dbt package
* Add mysql normalization support in java
* Add mysql normalization support in python
* Fix unit tests
* Update readme
* Setup mysql container in integration test
* Add macros
* Depend on dbt-mysql from git repo
* Remove mysql limitation test
* Test normalization
* Revert protocol format change
* Fix mysel json macros
* Fix two more macros
* Fix table name length
* Fix array macro
* Fix equality test macro
* Update replace-identifiers
* Add more identifiers to replace
* Fix unnest macro
* Fix equality macro
* Check in mysql test output
* Update column limit test for mysql
* Escape parentheses
* Remove unnecessary mysql test
* Remove mysql output for easier code review
* Remove unnecessary mysql test
* Remove parentheses
* Update dependencies
* Skip mysql instead of manually write out types
* Bump version
* Check in unit test for mysql name transformer
* Fix type conversion
* Use json_value to extract scalar json fields
* Move dbt-mysql to Dockerfile (#4459)
* Format code
* Check in mysql dbt output
* Remove unnecessary quote
* Update mysql equality test to match 0.19.0
* Check in schema_test update
* Update readme
* Bump base normalization version
* Update document
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Speed normalization unit tests by dropping hubspot catalog (too heavy, will be covering it in integration tests instead
* Add integration tests for normalization
* Add dedup test case
* adjust build.gradle
* add readme for normalization
* Share PATH env variable with subprocess calls
* Handle git non-versionned tests vs versionned ones
* Format code
* Add tests check to normalization integration tests
* Add docs
* complete docs on normalization integration tests
* format code
* Normalization integration tests output (#3026)
* Version generated/output files from normalization integration tests
* simplify cast of float columns to string when used as partition key (#3027)
* bump version of normalization image
* Apply suggestions from code review
Co-authored-by: Jared Rhizor <jared@dataline.io>
* Apply suggestions from code review
Co-authored-by: Jared Rhizor <jared@dataline.io>
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.
General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.
Checked in and added tests for the generated modules.
Add a step that generates config files and SQL files in order to run DBT which takes care of converting JSON blob data (from catalog.json JSON Schema) into normalized tables
Co-authored-by: cgardens <giardina.charles@gmail.com>
Co-authored-by: Sherif Nada <snadalive@gmail.com>
