* added supported_sync_modes for AirbyteStream initialization to fic full_refresh test
* fix unit test with supported_sync_modes argument in AirbyteStream initialization
* Added ability to run the incremental sync within slicing, by leveraging the cached_state
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
* fixed s3 destination field naming for Parquet and Avro formats
* pull request number update
* updated resources for parquet
* snowflake s3 destination COPY is writing records from different table in the same raw table fix
* fixed s3 destination name transformer
* updated s3 destination name transformer
* updated snowflake s3 file name
* updated snowflake documentation
* updated snowflake documentation
* updated snowflake documentation
* updated code style
* updated code style
* updated code style
* updated redshift destination
* added test data for test filed with bad first char
* updated s3 documentation
* fixed remarks
* fixed code style
* fixed s3 tests
* New abstraction for NoSql database sources
* New MongoDbSource: partial impl
* Added MongoDataType
* Improved MongoDatabase and fixed read method
* code review changes;
* merge clean up;
* Renamed NoSqlDatabase to AbstractDatabase
* formatter changes;
* code review changes: changed mongodb-new to mongodb-v2; left only new connector info in all docs
* code review changes: changed mongodb-new to mongodb-v2; left only new connector info in all docs
* updated spec.json and toDatabaseConfig() method
* updated doc accordingly to spec.json changes
Co-authored-by: Iryna Kruk <iryna.o.kruk@globallogic.com>
* fix negative backoff
* correct incremental logic for comments
* remove test logs
* bump version
* fix spec
* remove old non-relevant test
* correct backoff default value
* correction a test
* save addl export cursor field
* fix incorrect optinal date-time
* add format date-time again
* correction after review
* fix integration test
* fix unit test
Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
* oracle normalization
* correct dbt_project function for oracle
* unit tests
* run format
* correct ephemeral tests
* add gradle dependency for oracle destination
* run int tests
* add oracle in settings.gradle for normalization run[
* use default airbyte columns
* format
* test all destinatoin ephemeral
* correct unit test
* correct unit test
* destination docs update
* correct mypy
* integration test all dest
* refactor oracle function
* merge master
* run all destinations
* flake8 escape regex
* surrogate key function
* correct few minor comments
* refactor scd sql function
* refactor scd function
* revert test
* refactor minor details
* revert tests
* revert ephemeral test
* revert unit test table_registry
* revert airbyte_protocol format
* format
* bump normalization version in worker
* minor chnages
* minor chages
* correct json_column for other destinations
* gradlew format
* revert tests
* remove comments
* add Oracle destination explicit in safe_cast_str
* add quote_in_parenthesis inside if clause
* gradlew format
* Fixed (StackOverflow) destination bigquery denormalized destination to handle the case when received schema doesn't contain a data type for Array type
* new mongo db destination
* fix remarks
* updated documentation and added loggers
* updated documentation
* added hashCode field to mongodb document and fix minor remarks
* fix code style
* updated mongodb data hash from integer to UUID string
* Added the DynamoDB destination connector.
Implemented getConsumer and check methods.
Signed-off-by: Jinni Gu <jinnigu@uw.edu>
* Added auto-generated project files.
Signed-off-by: Yiqing Wang <yiqing@wangemail.com>
* Added config related files and output table helper.
Signed-off-by: Yiqing Wang <yiqing@wangemail.com>
* Added document for DynamoDB destination.
Signed-off-by: Jinni Gu <jinnigu@uw.edu>
* Implemented DynamodbWriter.
Added integration tests and unit tests.
Signed-off-by: qtz123 <qiutingzhi1995@gmail.com>
* Added DynamoDB in the SUMMARY.md.
Signed-off-by: qtz123 <qiutingzhi1995@gmail.com>
* Formatted code using ./gradlew format.
Signed-off-by: Jinni Gu <jinnigu@uw.edu>
* Added changelog to the doc.
Signed-off-by: qtz123 <qiutingzhi1995@gmail.com>
* Used PAY_PER_REQUEST instead of provisioned for DynamoDB.
Gave the value a name batchSize.
Removed unnecessary logs.
Signed-off-by: Yiqing Wang <yiqing@wangemail.com>
Co-authored-by: Yiqing Wang <yiqing@wangemail.com>
Co-authored-by: qtz123 <qiutingzhi1995@gmail.com>
* fix \u0000(NULL) value processing for Postgres + move postgres impl of SqlOperations to PostgresSqlOperations.
* changelog + format
* incr release version
* Add generic solution to adopt messages for a destination + remove unnecessary serialization
* revert version for build
* minor review fixes
* format
* add comments
* format
* incr version
