* added supported_sync_modes for AirbyteStream initialization to fic full_refresh test
* fix unit test with supported_sync_modes argument in AirbyteStream initialization
* Added ability to run the incremental sync within slicing, by leveraging the cached_state
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
* fixed s3 destination field naming for Parquet and Avro formats
* pull request number update
* updated resources for parquet
* snowflake s3 destination COPY is writing records from different table in the same raw table fix
* fixed s3 destination name transformer
* updated s3 destination name transformer
* updated snowflake s3 file name
* updated snowflake documentation
* updated snowflake documentation
* updated snowflake documentation
* updated code style
* updated code style
* updated code style
* updated redshift destination
* added test data for test filed with bad first char
* updated s3 documentation
* fixed remarks
* fixed code style
* fixed s3 tests
* New abstraction for NoSql database sources
* New MongoDbSource: partial impl
* Added MongoDataType
* Improved MongoDatabase and fixed read method
* code review changes;
* merge clean up;
* Renamed NoSqlDatabase to AbstractDatabase
* formatter changes;
* code review changes: changed mongodb-new to mongodb-v2; left only new connector info in all docs
* code review changes: changed mongodb-new to mongodb-v2; left only new connector info in all docs
* updated spec.json and toDatabaseConfig() method
* updated doc accordingly to spec.json changes
Co-authored-by: Iryna Kruk <iryna.o.kruk@globallogic.com>
* fix negative backoff
* correct incremental logic for comments
* remove test logs
* bump version
* fix spec
* remove old non-relevant test
* correct backoff default value
* correction a test
* save addl export cursor field
* fix incorrect optinal date-time
* add format date-time again
* correction after review
* fix integration test
* fix unit test
Co-authored-by: Maksym Pavlenok <maksym.pavlenok@globallogic.com>
* Add a variant of the CDK speedrun that uses the PokeAPI
* Make a lot of clarifications to improve the learning experience.
* Gradle format
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
* docker-compose split of scheduler and worker
* fix heartbeat location bug + add support for kubernetes
* use two workers in integration tests
* capture logs in AirbyteTestContainer
* add waiting
* rename to make it easier to review
* rename module
* fix remaining conflicts
* allow configuring max workers of each type and document usage
* fix build
* remove comment
* add worker resource requiremetns
* try to fix for connector build
* fix regression in biuld
* add env comments for SUBMITTER_NUM_THREADS
* Update airbyte-workers/src/main/java/io/airbyte/workers/WorkerApp.java
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Update airbyte-workers/src/main/java/io/airbyte/workers/temporal/TemporalPool.java
Co-authored-by: Davin Chia <davinchia@gmail.com>
* merge temporalpool into workerapp
* output docker system info
* move check to before
* remove unnecessary parts of the patch
* could this be the problem? i thought i added this
* show disk usage
* add print statements
* add pruning
* fix prune option
* use force
Co-authored-by: Davin Chia <davinchia@gmail.com>
* oracle normalization
* correct dbt_project function for oracle
* unit tests
* run format
* correct ephemeral tests
* add gradle dependency for oracle destination
* run int tests
* add oracle in settings.gradle for normalization run[
* use default airbyte columns
* format
* test all destinatoin ephemeral
* correct unit test
* correct unit test
* destination docs update
* correct mypy
* integration test all dest
* refactor oracle function
* merge master
* run all destinations
* flake8 escape regex
* surrogate key function
* correct few minor comments
* refactor scd sql function
* refactor scd function
* revert test
* refactor minor details
* revert tests
* revert ephemeral test
* revert unit test table_registry
* revert airbyte_protocol format
* format
* bump normalization version in worker
* minor chnages
* minor chages
* correct json_column for other destinations
* gradlew format
* revert tests
* remove comments
* add Oracle destination explicit in safe_cast_str
* add quote_in_parenthesis inside if clause
* gradlew format
* Fixed (StackOverflow) destination bigquery denormalized destination to handle the case when received schema doesn't contain a data type for Array type
* new mongo db destination
* fix remarks
* updated documentation and added loggers
* updated documentation
* added hashCode field to mongodb document and fix minor remarks
* fix code style
* updated mongodb data hash from integer to UUID string
