1
0
mirror of synced 2025-12-21 19:11:14 -05:00
Commit Graph

64 Commits

Author SHA1 Message Date
Audrey Maldonado
b9690ba76e Wrapping changelog in details for a collapsible view (#38801) 2024-06-04 13:40:53 -04:00
Natik Gadzhi
bb33e908ec destination-aws-datalake: [autopull] base image + poetry + up_to_date (#38413) 2024-05-21 09:22:30 -07:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
menishak
3b5c44aaf3 🐛 Destination AWS Datalake: Enable STS Role Credential Refresh for Long Syncs (#33853)
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2024-04-29 10:05:31 -03:00
Henri Blancke
2cb16f41e9 🐛 Destination AWS Datalake: Update to support new state protocol (#36386)
Signed-off-by: Henri Blancke <blanckehenri@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2024-03-27 20:08:34 -03:00
Evan Tahler
64cd1ce789 AWS Destinations: New AWS regions (#33924) 2024-01-05 10:51:10 -06:00
Henri Blancke
8d60177786 🐛 Destination AWS Datalake: Upgrade AWSWrangler (#29221)
Signed-off-by: Henri Blancke <blanckehenri@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2023-10-25 14:00:35 -03:00
Henri Blancke
af3db20f64 🐛 Destination AWS Datalake: fix airbyte type in complex data types (#24642)
* [UPD] prefer airbyte type in complex data types

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] bump version

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] only for integers

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] changelog table

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] cast objects that can have additional props to string

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [ADD] tests

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* auto-bump connector version

---------

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>
Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-04-19 15:12:41 -04:00
Greg Solovyev
efcec106f5 Greg/community pr 17193 (#23855)
* [UPD] Add format and partitioning spec

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] Add parquet format, compression and partitioning

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [ADD] Integration and unit tests

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] Update docs and add bootstrap

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] bump version to 0.1.2

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [ADD] Changelog entry

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] typo

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] cast arrays with mixed types to json string

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] issues when casting athena to pandas types

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] cleanup

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] flush interval to reduce memory usage

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] allow state reset per stream

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] capitalize AWS

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [ADD] decimal support and default db LF-tags

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] account for type error

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] partition field duplication

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] bump awswrangler (fixes json compression issue)

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] refactor, infer pandas and glue types from json schema

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] default for items get

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] account for mixed type properties

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] bad complex types to json string

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] drop top keys when not in json schema

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] fix partitioning, add airbyte type, fix keyerror concurrent partitioning

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] make table type configurable

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] fix obvious type violations

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] add missing columns to create correct schema

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] integration test

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] formatting and flake

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] overwrite partial flush bug

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] integration tests

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* fix formatting

* [FIX] check and typing

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] rmv fillna

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] warn on reset

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] log on failed reset

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] cast bool

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] cast pandas columns bool casting

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [FIX] required spec and format defaults

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [ADD] icon

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* [UPD] address review comments

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>

* Automated Change

* auto-bump connector version

---------

Signed-off-by: Henri Blancke <blanckehenri@gmail.com>
Co-authored-by: Henri Blancke <blanckehenri@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Sunny <6833405+sh4sh@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-27 10:52:04 -07:00
Stéphane Kattoor
5b6a2256bb Destination AWS Datalake: documentation update to match Airbyte template (#13716)
* Documentation update to match Airbyte template

* Update AWS Datalake doc
2022-07-27 17:29:05 -03:00
Topher Lubaway
9c6c092a22 Revert "Improving docusaurus sidebar generation (#1927) (#14369)" (#14596)
This reverts commit a2c194a11f.
2022-07-11 15:27:14 -05:00
Mykyta Serbynevskiy
a2c194a11f Improving docusaurus sidebar generation (#1927) (#14369)
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
2022-07-08 14:18:27 -05:00
Stéphane Kattoor
66d760955d Fix typo in spec.json for destination AWS Datalake & fix python license plugin setup (#11811) 2022-05-06 01:28:00 -07:00
Stéphane Kattoor
e3523e0a70 🎉 New Destination: AWS Datalake (#10760)
* Implementation of a destination connector for AWS Datalake (governed tables)

* refactor

* state message

* correct source config

* add connector spec config

* update docs

* modify spec json

* run seed config

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-04-04 19:17:34 -03:00