jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-22 03:21:25 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

321 Commits

Author SHA1 Message Date
Augustin
2de6895bc6 destination-bigquery: Use airbyte/java-connector-base:1.0.0 (#49902) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-01-09 16:19:05 +01:00
Stephane Geneix
fea5ca8414 destination-bigquery: update all dependencies (#45433) 2024-09-18 14:14:41 -07:00
Evan Tahler
d141d18a86 Destinations - Increase Jackson parse limit to 100MB (#44476) 2024-08-29 12:03:02 -05:00
Edward Gao
ee532e2f79 Destinations bigquery+snowflake+redshift: hard revert refreshes thing (#44526) 2024-08-22 13:26:32 -07:00
Edward Gao
b20365a390 Destination bigquery: pull in cdk update for refreshes bugfix (#42504) 2024-08-20 08:11:56 -07:00
Juan
ea919ce5cb Update bigquery.md (#44120) 2024-08-15 14:30:47 -04:00
Vee
35458f4508 BigQueryDestination: Added a copy operation to validate copy permissions in the check function (#42511) 2024-08-01 10:25:16 -07:00
Evan Tahler
6c07b5fd95 destination-bigquery: batched standard inserts is default loading mode (#42407) 2024-07-22 15:55:45 -07:00
Edward Gao
b7d7db69ee Destinations bigquery, snowflake, redshift: Don't hang forever on empty stream list; shorten error message on INCOMPLETE stream status (#41968) 2024-07-15 17:57:15 -07:00
Edward Gao
25d0030481 Destinations bigquery+snowflake: upgrade cdk (#41674) 2024-07-12 14:45:12 -04:00
Edward Gao
b2e7895ed3 Destination Bigquery+Snowflake: Fix resuming truncate refresh (#41041) 2024-07-08 21:37:52 -04:00
kabeer27
ee1c21c2b1 [destination-bigquery] Removing excess permission requirements when dataset already exists (#39379) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: davinchia <davinchia@users.noreply.github.com>
Co-authored-by: Roman Yermilov [GL] <86300758+roman-yermilov-gl@users.noreply.github.com>
Co-authored-by: Christo Grabowski <108154848+ChristoGrab@users.noreply.github.com>
Co-authored-by: Audrey Maldonado <audrey.maldonado@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
Co-authored-by: Baz <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Anatolii Yatsuk <35109939+tolik0@users.noreply.github.com>
Co-authored-by: Anjay Goel <anjay.goel@gmail.com>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: lazebnyi <lazebnyi@users.noreply.github.com>
Co-authored-by: Cristina Mariscal <166420606+cmm-airbyte@users.noreply.github.com>
Co-authored-by: cristina.mariscal <cristina.mariscal@cristina.mariscal--MacBook-Pro---DFJ27FJFXX>
Co-authored-by: Juan <80164312+jnr0790@users.noreply.github.com>
Co-authored-by: alafanechere <alafanechere@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Anton Karpets <anton.karpets@globallogic.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
 2024-06-26 10:08:24 -07:00
Stephane Geneix
62042cbe80 destination-bigquery: convert prod code to kotlin (#39904) 
## What
<!--
* Describe what the change is solving. Link all GitHub issues related to this change.
-->

## How
<!--
* Describe how code changes achieve the solution.
-->

## Review guide
<!--
1. `x.py`
2. `y.py`
-->

## User Impact
<!--
* What is the end result perceived by the user?
* If there are negative side effects, please list them. 
-->

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [ ] YES 💚
- [ ] NO ❌
 2024-06-21 10:20:05 -07:00
Edward Gao
9458fd22fb Destination bigquery: Throw INCOMPLETE stream status as TRANSIENT error (#39526) 2024-06-18 16:03:13 -04:00
Edward Gao
9b36fa0cb6 Destination bigquery: Implement refreshes logic (#38713) 2024-06-17 16:19:18 +00:00
Edward Gao
9410f42941 Destination bigquery: Bump cdk again (#38331) 2024-06-10 11:35:00 -07:00
Edward Gao
f0c85c7d39 Destination bigquery: increase json deserialization limit (#38764) 2024-06-07 19:29:30 -04:00
Audrey Maldonado
b9690ba76e Wrapping changelog in details for a collapsible view (#38801) 2024-06-04 13:40:53 -04:00
Edward Gao
7ddca9758c Destination bigquery: switch cdk back to artifact (#38770) 2024-05-30 07:48:33 -07:00
Edward Gao
fc205e4065 Destination bigquery: airbyte_meta/sync_id/generation_id (#38359) 2024-05-29 14:43:19 -04:00
Gireesh Sreepathi
95e701bbd6 Destination BigQuery: Include forward-slash prefix in the staging rootpath (#38591) 2024-05-22 17:34:20 -07:00
Gireesh Sreepathi
5b7873a0de Destination BigQuery: Adapt to newer interface for Sync operations (#38132) 2024-05-22 15:34:51 -07:00
Gireesh Sreepathi
e0225c18b2 Destination BigQuery: Consolidation of objects to StreamConfig, cleanup (#38131) 
## What
Removing redundant references and duplicate information passed around using `WriteConfig` objects. No functional changes and resurrected all the information needed through `StreamConfig` and adapted changes accordingly. 

This PR should be in a mergeable state with no functional changes after the ones down the stack are published. 


## Review guide
* Removed references of `BigQueryWriteConfig` and reused already built `StreamConfig`
* Removing unnecessary `StagingOperations` interface and made concrete class, this will help for later adding a shim on this and refactoring without large changes 
* Removed other unnecessary references of getting dynamic schema, `WriteDispostion` etc. Probably remnant of bigquery-denormalized bespoke connector. 

## User Impact
<!--
* What is the end result perceived by the user?
* If there are negative side effects, please list them. 
-->

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌
 2024-05-13 11:09:19 -07:00
Stephane Geneix
eadbd41b95 destination-bigquery: bump CDK version (#38125) 
the latest CDK should prevent the destination from hanging when encountering an error in the exception handler.
 2024-05-10 12:38:54 -07:00
Gireesh Sreepathi
1188221473 Destination BigQuery: Nuking old remnants (#38111) 2024-05-10 10:54:42 -07:00
Edward Gao
e950c56cc5 Destination bigquery: switch to artifact cdk (#38098) 2024-05-09 17:26:52 -04:00
Edward Gao
47c86a9ea6 Destination Bigquery: Remove reference to TDValve (#37714) 2024-05-08 14:22:40 -04:00
Edward Gao
813510a71f Destination bigquery: Update to latest CDK (#34611) 2024-05-07 18:21:03 -04:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Joe Bell
994e8c9015 fix admonition rendering for bq (#37730) 2024-04-30 22:41:31 +00:00
Evan Tahler
e8859b2121 Exclude zookeeper dependencies (#37584) 2024-04-26 16:28:26 -04:00
Stephane Geneix
c4ad3d9660 convert destination-bigquery to kotlin CDK (#36899) 2024-04-25 14:46:47 -07:00
Edward Gao
160ec72937 Destination bigquery: upgrade cdk (#35315) 
Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
 2024-03-04 11:43:43 -08:00
Marius Posta
084671b580 destination-bigquery: bump patch version following #35569 (#35574) 
Co-authored-by: Stephane Geneix <stephane@airbyte.io>
 2024-02-23 00:11:27 -04:00
Subodh Kant Chaturvedi
a951624991 destination-async-framework: move the state emission logic into GlobalAsyncStateManager (#35240) 2024-02-16 00:36:48 +05:30
Marius Posta
4a2bf02656 java CDK: improve blobstore module structure (#35285) 2024-02-15 13:09:30 -04:00
Subodh Kant Chaturvedi
0c228f4a64 destination-async-framework: make emission of state from FlushWorkers synchronized (#35144) 2024-02-12 23:57:19 +05:30
Subodh Kant Chaturvedi
a15bd21714 destination-async-framework: use the value from stats counter for global state conversion (#35111) 2024-02-12 14:18:54 +05:30
Edward Gao
5a1f3595f5 Destination bigquery: update test fixtures (#34575) 2024-02-09 18:22:34 -04:00
Marius Posta
796b2e8dad java CDK: clean up dependencies, refactor modules (#34745) 2024-02-08 19:46:51 -06:00
Subodh Kant Chaturvedi
e691b66ffa async-destination-framework: make GlobalAsyncStateManager entirely thread safe (#35027) 2024-02-09 02:45:59 +05:30
Gireesh Sreepathi
b36bcc3c1a Destination BigQuery: CDK updates for latest changes (#34728) 
## What
* Upgrade to latest CDK to fix flaky tests

## How
* See [CDK Updates](https://github.com/airbytehq/airbyte/pull/34728)
 2024-02-01 12:14:22 -08:00
Edward Gao
96b9f5a059 Destinations bigquery, snowflake, redshift: log better message when failing to parse an AirbyteMessage (#34451) 2024-01-25 18:33:33 -04:00
Cynthia Yin
7d89fc9112 📚 Destinations V2: update typing & deduping doc link (#34505) 2024-01-24 17:50:42 -06:00
Edward Gao
a0345ae732 Destinations snowflake, bigquery, redshift: improve error reporting (#34458) 2024-01-24 19:20:45 -04:00
Gireesh Sreepathi
c42c5e6889 Bigquery Destination: CDK Update (#34468) 
## What
* Upgrade to latest CDK with signature changes.
 2024-01-24 13:37:38 -08:00
Cynthia Yin
13e32a6e87 ✨ Destination Redshift: GA Destinations V2 (#34077) 
Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
 2024-01-23 16:02:00 -08:00
Edward Gao
699360cb8c Destination bigquery: concat transactions together (#34023) 2024-01-22 14:49:21 -04:00
Gireesh Sreepathi
59df0cf302 Destination Bigquery: Clean up dependencies with TD/CDK (#34226) 
Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>
 2024-01-12 17:04:45 -04:00
Evan Tahler
dcbec4a65b 📝 Fix BigQuery Destination: fail when using Google Default Application Credentials (#34073) 2024-01-10 16:36:27 -08:00