jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-23 21:03:15 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,121 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

1127 Commits

Author SHA1 Message Date
Edward Gao
4c26d4cceb DV2: Fix column name collision detection (#30739) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-09-27 15:57:43 -07:00
Aaron ("AJ") Steers
da32fc86bf 🚀 Launch Java CDK v0.1.0 (PR #30383) 
- Launch Java CDK `v0.1.0`. 🚀
- Remove `airbyte-docker` gradle plugin from java connectors.
- Add `airbyte-java-connector` gradle plugin.
- Remove `airbyte-commons` from global build.gradle.
- Add `/publish-java-cdk` slash command.
- Initial launch of `/airbyte-ci` slash command to test connectors.
 2023-09-27 15:46:02 -07:00
Joe Bell
05c7269a50 Destination BigQuery - Try unsafe operations with exception handling (#30696) 2023-09-27 20:57:44 +00:00
Edward Gao
8abda48f14 🐛 Destination snowflake: support concurrent syncs with identical stream name but different namespace (#30599) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-09-27 15:57:11 -05:00
Edward Gao
1dc11b05b2 DV2 tests: support metadata column name mappings (#30356) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-09-22 20:53:34 +00:00
Edward Gao
7a8150eb25 Destination bigquery: soft reset correctly handles previous sync unclean exit (#30697) 2023-09-22 19:05:14 +00:00
Edward Gao
15c8ce9a67 Destination bigquery: handle case where stream name and namespace are identical (#30640) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-09-22 10:51:28 -05:00
Ben Church
5d8278900f Github Action: Add format.yml workflow (#30604) 
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
Co-authored-by: octavia-approvington <octavia-approvington@users.noreply.github.com>
 2023-09-21 18:11:41 -05:00
Benoit Moriceau
f599c1c3c9 Reduce the snowflake async buffer size (#30671) 2023-09-21 16:47:13 -05:00
Ryan Br
317f7acc22 ✨ Bigquery uses async framework (#30069) 
Co-authored-by: tryangul <tryangul@users.noreply.github.com>
Co-authored-by: benmoriceau <benoit@airbyte.io>
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
 2023-09-20 16:34:07 -05:00
Ben Church
5c56ac1d84 Airbyte-ci: Remove gradle task connectorAcceptanceTest (#30326) 2023-09-19 15:16:37 -05:00
Benoit Moriceau
12fb75331b Make sure that the buffers are push at least every 5 minutes (#30546) 
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
 2023-09-19 08:33:04 -07:00
Cynthia Yin
839fb26e09 🐛 Destination Snowflake - support reserved column names (#30319) 
Co-authored-by: cynthiaxyin <cynthiaxyin@users.noreply.github.com>
 2023-09-19 02:27:20 -07:00
Benoit Moriceau
a7c13dc6f1 Improve the memory usage of the async destination (#30479) 
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
 2023-09-18 16:31:57 -07:00
Duy Nguyen
f226503bd1 [Source-MySQL] Enhanced Standard sync with PK initial load -> Cursor based switch over (#30270) 2023-09-18 17:17:00 -05:00
Edward Gao
8805edcea6 DV2: Better errors in the UI (#30491) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-09-18 15:58:56 +00:00
Marius Posta
a5eb24e2fc bump testcontainers version to latest (#30493) 2023-09-18 03:50:43 -05:00
Edward Gao
68c6b01937 Destinations v2: threadsafe setup (#30439) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-09-14 11:56:12 -05:00
Edward Gao
70ef3b4d1d 🐛 Destination snowflake: Handle namespace/stream name/columns containing ${ (#30416) 2023-09-14 08:17:24 -07:00
Subodh Kant Chaturvedi
6f0c4e2594 mysql-source-cdc: enable compression logic for schema history (#30308) 
Co-authored-by: subodh1810 <subodh1810@users.noreply.github.com>
Co-authored-by: Prateek Mukhedkar <prateek@airbyte.io>
 2023-09-13 13:14:01 -05:00
Subodh Kant Chaturvedi
5a4cc0d8b0 source-mysql-cdc: introduce logic to compress schema history if greater than 3MB (#30289) 
Co-authored-by: subodh1810 <subodh1810@users.noreply.github.com>
Co-authored-by: Prateek Mukhedkar <prateek@airbyte.io>
 2023-09-13 12:41:28 -05:00
Edward Gao
0910f3a626 DV2: add log messages for name collisions (#30364) 2023-09-12 22:47:08 +00:00
Edward Gao
e0ce2ac4d0 destinations v2: snowflake: threadsafe typing and deduping (#29878) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: evantahler <evantahler@users.noreply.github.com>
 2023-09-12 17:28:50 +00:00
Subodh Kant Chaturvedi
ad47884297 postgres+mysql sources: use constants to keep checkpointing behaviour same (#30197) 
Co-authored-by: subodh1810 <subodh1810@users.noreply.github.com>
 2023-09-12 02:43:23 -05:00
Marius Posta
51c67d7eaa gradle: remove airbyteDocker.outputs dependencies (#30314) 2023-09-11 17:16:27 -07:00
Benoit Moriceau
48b752fa86 Avoid failing because of an NPE if the flush failure doesn't provide … (#30266) 
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2023-09-08 07:40:10 -07:00
Denys Davydov
5248854f2e CAT: changing format or airbyte_type is a breaking change (#30172) 
Co-authored-by: davydov-d <davydov-d@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
 2023-09-08 05:55:52 +00:00
Marius Posta
ef2849e35e gradle: fix airbyteDocker task inputs (#30187) 2023-09-07 03:46:31 -07:00
Subodh Kant Chaturvedi
4749441695 postgres-cdc: fix flaky tests (#30195) 
Co-authored-by: subodh1810 <subodh1810@users.noreply.github.com>
 2023-09-07 01:48:17 -05:00
Joe Bell
11068c1c4e Destinations V2 - T&D Streams in parallel (#30020) 
Co-authored-by: jbfbell <jbfbell@users.noreply.github.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
 2023-09-06 10:04:34 -07:00
Evan Tahler
7fc7db7ea4 T&D frequency: stream start, 6h(*) (#30117) 2023-09-06 01:22:20 +00:00
Ben Church
4d5e17bc90 [skip ci] Update test_incremental to be unaware when source defines the cursor (#27872) 
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
 2023-09-05 15:57:18 -07:00
Marius Posta
be1e1adabd gradle: cleanup (#30060) 2023-09-05 14:05:40 -05:00
Edward Gao
9231c044d9 🐛 Destination snowflake: Create final tables with uppercase naming (#30056) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-09-05 11:54:53 -05:00
Augustin
d2a915fbed cat: fix skip_backward_compatibility fixture for discovery (#30141) 2023-09-04 15:20:53 -04:00
Evan Tahler
02a0cca67a Build better snowflake exception when PK is missing (#30065) 
Co-authored-by: evantahler <evantahler@users.noreply.github.com>
 2023-09-01 17:50:51 +00:00
Edward Gao
2d941ba37e DV2 sqlgenerator test: use generator.createTable instead of custom createTable method (#30037) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
 2023-08-31 18:36:31 +00:00
Lake Mossman
b19ed6d8b3 Support display_type property for oneOf fields and adjust CDC connectors to use it (#29821) 
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
 2023-08-31 12:19:02 -05:00
Augustin
da3d20e1f1 cat: fix fixture skipping backward compatiblity testing (#29583) 2023-08-31 17:58:33 +02:00
Edward Gao
44f840ddb5 ✨ Destinations bigquery+snowflake: Release DV2 (#29783) 
Co-authored-by: edgao <edgao@users.noreply.github.com>
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: evantahler <evantahler@users.noreply.github.com>
 2023-08-29 14:06:50 -05:00
Jonathan Pearlin
13ef239ccc ✨ Source MongoDB Internal POC: CDC Metadata Injector (#29730) 
Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
Co-authored-by: Jose Pefaur <jose.pefaur@gmail.com>
Co-authored-by: subodh <subodh1810@gmail.com>
 2023-08-28 14:01:32 -05:00
Jonathan Pearlin
0724132a1a ✨ Source MongoDB Internal POC: Debezium Configuration Properties (#29727) 2023-08-28 08:44:39 -05:00
Edward Gao
4584f1c8ce Destination snowflake: prevent data loss, by only running T+D once at end of sync (#29881) 2023-08-26 01:49:15 +00:00
Ben Church
ba7ef7b30a 📚 Update the docs to be more clear about how and when you run CAT (#29733) 2023-08-25 19:33:43 +00:00
Marius Posta
3e680675a4 github workflows: repo-wide auto-format (#29798) 
Co-authored-by: postamar <postamar@users.noreply.github.com>
 2023-08-25 10:20:41 -07:00
Charles
c85cc89344 🐛 update test namespace convention (#29769) 2023-08-24 17:55:10 -07:00
Joe Bell
2215d0f8e5 Destinations V2 Remove Soft Reset (#29805) 2023-08-24 14:19:50 -05:00
Jonathan Pearlin
fcb23c867a ✨ Source MongoDB Internal POC: Debezium target position (#29710) 
Co-authored-by: jdpgrailsdev <jdpgrailsdev@users.noreply.github.com>
 2023-08-24 13:11:56 -04:00
Joe Bell
ea3a07f03d 🐛 Destination Bigquery - don't soft reset overwrite syncs (#29774) 2023-08-23 22:31:32 +00:00
Jonathan Pearlin
d281eb34d4 ✨ Source MongoDB Internal POC: Construct Initial Debezium State (#29676) 2023-08-23 17:04:23 -04:00