1
0
mirror of synced 2026-01-08 12:03:02 -05:00
Commit Graph

9 Commits

Author SHA1 Message Date
Liren Tu
61c67bada9 Databricks destination: introduce personal staging location (#18557)
* Update s3 parquet writer

- Rename gcsFileLocation.
- Merge local variables.

* Implement staging location getter

* Add integration test and fix post request

* Add comment and format code
2022-10-27 09:42:59 -07:00
Edward Gao
f90ffb0a12 Destination Redshift: handle empty s3 bucket_path (#18434)
* make bucket_path required

* check for nonempty path

* not required

* handle null/empty path

* version bump + changelog

* fix comment

* auto-bump connector version

* format

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-26 11:04:10 -07:00
Edward Gao
ba21d9048c fix formatting + imports (#18457) 2022-10-25 15:41:19 -07:00
N-o-Z
9b57d972af 🐛 Fix S3 destination integration tests to use bucket path (#18031)
* Fix S3 destination tests

* Add Changelog

* CR Fixes

* add unit test

* version bump

* fix dockerfile

* auto-bump connector version

Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-25 15:21:32 -07:00
Augustin
710c432f00 fix Connector Base build (#18351) 2022-10-24 11:19:21 +02:00
Augustin
b564f3eb78 Protocol: make supported_sync_modes a required not empty list on AirbyteStream (#15591) 2022-10-19 15:22:25 +02:00
Ryan Fu
1d956df434 🚨 Add SSL documentation and check logic for S3 Destination 🚨 (#17340)
* Adds logic to fail upon non-deterministic custom S3 endpoint and documentation for insecure settings

* Reused config factory settings to a single static variable

* Updated error message and example in the spec.json to match expectation of secured endpoint

* Added validation check within the base s3

* Integrated AdaptiveDestinationRunner with S3Destination

* Reduced visibility for testing and fixed AdaptiveDestinationRunner issue

* Adds speicifc secure protocol with S3 and empty endpoint check

* Bumps docker version and adds comments and clearer string methods

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-03 13:56:58 -07:00
Liren Tu
d14b4423ea Publish R2 destination (#17122)
* Format code

* Update doc

* Add r2 to build doc

* Update readme

* Add r2 to destination definitions

* Move test resource to base-java-s3

* Remove redundant dependency

* Add missing dependency

* Format json

* Remove s3-destination-base-integration-test setting.gradle root

* Add missing lzo dependency

* Fix import in destination jdbc
2022-09-25 21:45:00 -07:00
Nugusbayev Kanagat
3494405078 🎉 New Destination: R2 Cloudflare (#15296)
* added destination-r2 spec

added base-java-s3 module

updated common s3 lib

local

changed dependency of s3 in all realted modules

minor style fixes

added common module for s3 integration tests

* fix configuration for r2 integration tests

* fix configuration for r2 integration tests

* added docs for destination-r2

* minor import fixes

* mark test as disabled

* fixed imports in destination-snowflake

* restored styling task

* added tests for S3DEstinationConfig

* added upload threads count configuration for r2(due to limitations)

* deleted parquet format type

* Fix import in gcs

* Fix import in redshift

* Fix import in snowflake

* Fix one more import in gcs

* Fix one more import in redshift

* Fix import in databricks

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
2022-09-25 19:20:47 -07:00