1
0
mirror of synced 2025-12-21 11:01:41 -05:00
Commit Graph

163 Commits

Author SHA1 Message Date
Abhi Vaidyanatha
3b2009723e Add note about collocating Redshift cluster and Airbyte server in the same security group (#4490)
Co-authored-by: Abhi Vaidyanatha <abhivaidyanatha@Abhis-MacBook-Pro.local>
2021-07-02 12:42:35 -07:00
Marcos Marx
7c70eb02cd GitBook: [master] 84 pages and 72 assets modified 2021-05-03 12:14:16 +00:00
Christophe Duong
77ffd74b32 Ignore records that are too big in Redshift destinations (instead of failing) (#2988)
* Abort sync if one of the part fails to copy to temp table

* Check for record size when copying data from s3 to redshift

* Handle big record in RedshiftInsertDestination too
2021-04-30 21:04:03 +02:00
Davin Chia
e63ab8490a Make Redshift part size configurable. (#3053)
The Redshift Copy strategy currently has it's part size set to 10 MB. Since S3 allows a file to be broken into max of 10k parts, this results in a 100GB table limit. A user is trying to sync a table of 115GB and running into this issue.

This makes the part size configurable so users can increase this size if needed.
2021-04-27 13:35:35 +08:00
Davin Chia
42686add8a Release connectors with namespace change. (#2990)
Release all connectors affected by namespace change. Includes all JDBC sources and destinations.

Also add documentation for normalisation. Prerequisite to actually releasing 0.21.0-alpha.
2021-04-21 11:35:08 +08:00
Michel Tricot
e4326aa21e Revert "GitBook: [master] 25 pages and 46 assets modified"
This reverts commit 0b4eddc7cf.
2021-04-16 14:20:42 -07:00
Michel Tricot
0b4eddc7cf GitBook: [master] 25 pages and 46 assets modified 2021-04-16 20:29:16 +00:00
Davin Chia
e8c13cf0fe Update documentation to include Redshift COPY strategy. (#2728) 2021-04-06 09:10:19 +08:00
Christophe Duong
194d6cb5d0 Add metadata prefix to destination internal columns (#1708)
* Add metadata prefix to destination internal columns

* Bumpversion all destinations for new normalization behavior

* Fix acceptance tests
2021-01-19 17:21:15 +01:00
Charles
01cdeafdb1 release v0.8.0-alpha (#1339) 2020-12-16 14:01:57 -08:00
John Lafleur
6668a514f4 GitBook: [master] 10 pages and 335 assets modified 2020-12-04 06:42:08 +00:00
Christophe Duong
d06392e900 Redshift Destination & refactoring to introduce destination-jdbc
Closes #193
Closes #1126
2020-12-03 18:07:46 +01:00
Christophe Duong
206d3cbea8 Naming conventions managed in destinations (#1060)
Handled field and table identifier conversions as necessary in order to successfully in destinations
2020-11-25 18:53:23 +01:00