jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-19 18:14:56 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

2353 Commits

Author SHA1 Message Date
Edward Gao
54605dcb8f Bulk load cdk: set different uuid per record (#61530) 2025-06-11 18:58:01 +00:00
Edward Gao
25dd360f9d Bulk load CDK: fix testTruncateRefresh; fix direct-load generation_id fetch logic (#61518) 2025-06-11 18:08:22 +00:00
Jonathan Pearlin
61d4aadf26 feat: improve uuid generation performance (#61504) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-11 13:54:25 -04:00
Benoit Moriceau
97a2dc1435 chore: re-enable union test for clickhouse (#61515) 2025-06-11 08:59:18 -07:00
Maxime Carbonneau-Leclerc
a5f0091986 feat: add Data Activation fields to stream (#61478) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-11 09:23:18 -04:00
Maxime Carbonneau-Leclerc
6744d0f4eb feat: bulk toolkit load-http (#61421) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-11 09:11:00 -04:00
Jimmy Ma
1077944930 feat: add load-dlq toolkit (#61493) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-10 14:18:37 -07:00
Edward Gao
5aaaca4a1f Bulk load CDK: Behave correctly in truncate refresh when receiving INCOMPLETE stream status (#61382) 2025-06-10 19:50:03 +00:00
Edward Gao
dca64bf1c2 Bulk load CDK: better exception handling in input loop (#61492) 2025-06-10 12:41:23 -07:00
Ryan Br...
8989f43758 Make various direct load interface methods that do IO suspend. (#61494) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-10 10:19:35 -07:00
Ryan Br...
7c70247b7a Rbroughan/rebase master ch (#61485) 
Co-authored-by: benmoriceau <benoit@airbyte.io>
Co-authored-by: Francis Genet <francis.genet@airbyte.io>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-09 16:40:44 -07:00
Edward Gao
005bc9adf4 Destination bigquery: fix build; really roll back (#61480) 2025-06-09 16:17:44 +00:00
Johnny Schmidt
84d1e7a069 [Load-CDK] Reduced log volume, few s3 parts (#61424) 2025-06-07 00:01:38 +00:00
Jimmy Ma
63aab18316 feat: enable destination discover (#61395) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-06 15:21:49 -07:00
Edward Gao
56a8be4e22 bulk load cdk: direct-load orchestration (#58641) 
Co-authored-by: Francis Genet <francis.genet@airbyte.io>
 2025-06-06 09:58:07 -07:00
Johnny Schmidt
144298e062 [Load-CDK] Speed: Unbreak what speed broke (#61420) 2025-06-06 16:35:39 +00:00
Johnny Schmidt
17d9874562 [Load-CDK] Speed: Dummy Stats additionalProperties fix, namespace mapping bug fix, broken test fixes (#61406) 
Co-authored-by: subodhchaturvedi <subodh1810@gmail.com>
 2025-06-06 15:57:40 +00:00
Johnny Schmidt
650f2188e3 [Load-CDK]: Speed namespace mapping in destination (#61401) 2025-06-05 20:14:44 +00:00
Subodh Kant Chaturvedi
c4e33ab0e0 fix: use counting stream to compute byte size correctly (#61404) 2025-06-05 13:01:16 -07:00
Johnny Schmidt
dba9eec7d3 [Load-CDK] Fix broken speed stats DI (#61378) 2025-06-04 21:03:16 +00:00
Johnny Schmidt
ece60cac0e [Load-CDK] Tolerate source sending end-of-stream on only one socket (#61357) 2025-06-04 19:49:46 +00:00
Subodh Kant Chaturvedi
9e44cb7bc0 feat: add total records and bytes info in state message in dest cdk + impl dummy stats emitter (#60972) 2025-06-04 21:49:08 +05:30
Edward Gao
948477f913 Destination Bigquery: throw useful error when crashing in setup (#61018) 2025-06-04 08:47:08 -07:00
Yarden Carmeli
f643734dc5 old CDK: update logging to avoid duplicate messages (mssql & postgres sources) + revert initial load timeout back to 4h (#60251) 2025-06-03 12:55:21 -07:00
Johnny Schmidt
9c1724c693 Load-CDK: Allow source to send probe messages in socket mode (#61330) 2025-06-03 19:14:19 +00:00
Johnny Schmidt
1d18c2c546 [Load-CDK] Speed: Protobuf support for sockets (#61008) 2025-06-03 17:15:16 +00:00
Johnny Schmidt
5df88a7145 [Load-CDK]: Speed state record count fix (#61327) 2025-06-03 01:41:08 +00:00
Edward Gao
75f028d574 Destination Bigquery: use default namespace in CHECK (#60986) 2025-05-29 23:14:22 +00:00
Edward Gao
88181cbd55 Destination MSSQL: fix build (#60897) 
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Francis Genet <francis.genet@airbyte.io>
 2025-05-29 20:56:27 +00:00
Rodi Reich Zilberman
cfc1243bd2 concurrent mysql read (#60218) 2025-05-29 13:38:46 -07:00
Jason T Brown
2bda109503 [source-bigquery] use GoogleCredentials to get cred from configured JSON (#59744) 
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
 2025-05-29 15:10:38 -03:00
Johnny Schmidt
f319bf32e8 [Load-CDK] Speed: Sockets are multi-threaded (#60935) 2025-05-28 22:40:44 +00:00
Johnny Schmidt
b06466dc3a Load-CDK: Fix flaky integration tests (#60956) 2025-05-28 15:30:13 -07:00
Johnny Schmidt
6b917664be [Load-CDK] Speed: prepare state handling for new protocol (#60886) 2025-05-28 13:27:53 -07:00
Francis Genet
7468253b94 Not displaying a literal null if the namespace is null (#60951) 2025-05-28 11:00:01 -07:00
Jonathan Pearlin
1303d3cf93 feat: install flatbuffers compiler for cdk bulk builds (#60856) 
Co-authored-by: Davin Chia <davinchia@gmail.com>
 2025-05-27 08:32:32 -04:00
Ryan Br...
85fb60c0c1 Revert "Don't filter if files stream." (#60895) 2025-05-23 14:16:14 -07:00
Ryan Br...
75b63a65d9 Don't filter if files stream. (#60799) 2025-05-23 13:31:58 -07:00
Francis Genet
25e3573846 [BigQuery] Add a random element to the check namespace (#60865) 2025-05-22 20:29:18 +00:00
Johnny Schmidt
d11f649db9 [Load CDK] Speed: Sockets first cut (single-threaded, JSONL-only) (#60307) 2025-05-22 02:32:32 +03:00
Johnny Schmidt
cf44842830 [Load-CDK] Reduce max flush interval; heartbeat flushes checkpoints (#60833) 2025-05-21 13:07:37 -07:00
Matt Bayley
181759453f Split DebeziumOperations interface (#59768) 2025-05-20 14:42:22 -07:00
Ryan Br...
effe9e9744 Enable file perf test in CI so we catch files regressions. (#60331) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-05-19 14:10:05 -07:00
Edward Gao
c731bd2d51 Destinations bigquery, mssql: handle streams with no columns (#60791) 2025-05-19 18:22:48 +00:00
Edward Gao
15b5614d6d bulk load cdk: turn supportsDedup into an object (#60286) 2025-05-16 17:27:13 -04:00
Ryan Br...
229aa9851d Rbroughan/fix num workers (#60327) 2025-05-16 14:09:18 -07:00
Edward Gao
d0e98198ad Destinations S3 + MSSQL: fix builds (#60322) 2025-05-16 19:04:08 +00:00
Ryan Br...
a6818d7eb9 Rbroughan/fix timed flush for files (#60313) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
 2025-05-16 11:33:20 -07:00
Johnny Schmidt
777041368d [Load-CDK] Speed Prep: DI Wiring to make input configurable (#60260) 2025-05-16 15:57:43 +00:00
Edward Gao
6fad1d44b2 Destination Bigquery: RC2 - fix column name handling (#60302) 2025-05-15 21:24:23 +00:00