jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-21 02:51:29 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

127 Commits

Author SHA1 Message Date
Edward Gao
335acd4052 Bulk CDK: move connector cdk version decl into property file (#66484) 2025-09-22 10:14:55 -07:00
Rodi Reich Zilberman
22eb524942 Fix mysql proto encoding of types (#66179) 2025-09-10 17:27:40 -07:00
Rodi Reich Zilberman
cbf42c3333 Socket mode in bulk extract CDK (#60993) 2025-07-07 12:54:08 -07:00
Rodi Reich Zilberman
3e878bf5ed remove local in mysql cdk version (#61014) 2025-06-02 10:15:38 -07:00
Rodi Reich Zilberman
cfc1243bd2 concurrent mysql read (#60218) 2025-05-29 13:38:46 -07:00
Davin Chia
112cb7b909 chore: move source postgres and mysql to new build flow. (#60214) 2025-05-13 11:13:13 -07:00
Rodi Reich Zilberman
dbd76645a3 CDK version bump (#59683) 2025-05-06 12:36:10 -07:00
Rodi Reich Zilberman
e97ed2611e Improve BigDecimal to json conversion to prevent scientific notation (#59223) 2025-05-06 10:35:31 -07:00
Yue Li
bcf7dcc23d [source-mysql] Bump up CDK for source-mysql (#58644) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-04-24 12:12:14 -07:00
Yue Li
ad7aa7af4b [cdk, source-postgres, source-mysql, source-mongodb-v2] Fixing vulnerabilities in dependencies (#58132) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-04-24 08:28:30 -07:00
Rodi Reich Zilberman
af0ced7f62 Set mysql CDK version to latest (#58623) 2025-04-23 14:38:19 -07:00
Rodi Reich Zilberman
f8c945442c Extend the amount of time for debezium shutdown (#58599) 2025-04-23 09:25:38 -07:00
Matt Bayley
5ea3ede279 [source-mysql] pin to cdk 0.342 (#55754) 2025-03-13 11:44:21 -07:00
Wenqi Hu
f0a96b3c1c [source volcano] Decorate stream with cursor field for trigger based CDC stream (#53152) 2025-02-12 22:31:22 +02:00
Marius Posta
8c2ed63cf4 source-mysql: adopt bulk-cdk-toolkit-extract-cdc API changes (#52039) 2025-01-30 23:06:08 +02:00
Marius Posta
36bafca764 source-mysql: pin CDK version (#52038) 2025-01-21 10:59:09 -08:00
Rodi Reich Zilberman
b3f56fb349 Set cdc metadata fields types to match legacy connectors (#51029) 2025-01-13 22:24:18 +02:00
Marius Posta
7f98173dae source-mysql: use CDK version 257 (#50987) 2025-01-08 16:21:51 -05:00
Marius Posta
ac054a641e source-mysql: fix converter bug (#50965) 2025-01-07 22:27:17 +02:00
Marius Posta
e01e6a21fa source-mysql: fix indentation level for rules defs (#50868) 2025-01-03 15:23:21 -08:00
Marius Posta
1d6b34cfba source-mysql: datatype fixes (#49918) 2024-12-20 19:44:03 +02:00
Marius Posta
2910475443 source-mysql: simplify MySqlSelectQuerier (#49948) 2024-12-20 09:24:46 -05:00
Marius Posta
cc383cf360 source-mysql: normalize class names (#49939) 2024-12-19 15:44:33 -05:00
Rodi Reich Zilberman
29a9e986b5 Set fetch size on a jdbc resultset rather than on a statement (#48840) 2024-12-09 08:23:43 -08:00
Rodi Reich Zilberman
e42d056050 merge mysql v2 into v1 (#48369) 2024-11-07 10:39:55 -08:00
Yue Li
2e62443f1c [source-postgres/mysql/mssql/mongodb-v2] Use latest CDK to adopt latest apache sshd mina to handle tcpkeepalive requests (#45639) 2024-09-18 14:46:10 +03:00
Yue Li
af98a810b5 [certified db sources] adopt latest CDK to use error translation bug fixes (#44841) 2024-08-28 22:35:16 +03:00
Yue Li
ed44c093b3 [source-mysql] Upgrade Debezium and mysql connector to 2.7.1.Final (#44013) 2024-08-14 10:15:05 -07:00
Akash Kulkarni
b43b1111a8 [cdk] : More logging for count mismatch (#43410) 2024-08-09 05:45:27 +03:00
Xiaohan Song
97d822f820 [cdk, source-mysql/postgres/mssql/mongodb-v2] log record count to state message (#42869) 
Co-authored-by: Yue Li <yue.li@airbyte.io>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
 2024-08-07 04:35:22 +03:00
Xiaohan Song
ab005c6465 [source-mssql/postgres/mysql] fix rfr losing stream state (#42550) 
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
 2024-08-02 20:02:19 +03:00
Yue Li
d34d7e627a [source-postgres/mssql/mysql] use the latest CDK to use the latest fix of error translation (#42852) 2024-07-29 10:42:44 -07:00
Yue Li
81cd8f2005 [cdk] handle null message inside exception (#42417) 
Fixes airbytehq/oncall#5904, error message inside an exception could be null.
 2024-07-23 17:27:33 -07:00
Xiaohan Song
af28105efa [source-mssql/mysql/postgres] Fix and cleanup oc map (#42024) 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Antonio Papa <antoniogpapa@gmail.com>
Co-authored-by: Adam Marcus <marcua@marcua.net>
Co-authored-by: Bryce Groff <bgroff@hawaii.edu>
Co-authored-by: Akash Kulkarni <113392464+akashkulk@users.noreply.github.com>
Co-authored-by: Patrick Nilan <nilan.patrick@gmail.com>
 2024-07-22 11:32:51 -07:00
Akash Kulkarni
000dc477cf [WASS] : Enhancements in logging & analytics (#42122) 2024-07-21 23:07:18 -04:00
Akash Kulkarni
e53b8e245b Toggle useLocalCdk = false (#42120) 2024-07-19 16:08:59 -04:00
Yue Li
4417769038 [cdk, source-postgres, source-mysql] a new error handling and translation framework (#40208) 
Fixes airbytehq/airbyte-internal-issues#8516

This set of changes mainly moves error translation to be part of each connector.

In general, each connector needs to implement its own error translation class that inherits from the abstract class ConnectorExceptionTranslator, which is part of the CDK. By implementing, it means the connector developer or our support will populate the error dictionary with error samples with matching rules (e.g., regex). See the example we created for the Postgres source.
 2024-07-17 17:19:13 -07:00
Akash Kulkarni
508638fcb7 [Source-mysql] : Adopt latest CDK + WASS fixes (#42043) 2024-07-17 13:33:05 -07:00
Akash Kulkarni
310e6bd315 [Source-mysql] : Implement WASS algo (#38240) 
Co-authored-by: Evan Tahler <evan@airbyte.io>
 2024-07-11 11:54:54 -04:00
Akash Kulkarni
3a3e058ae8 [DB-sources] : Improve heartbeat logic (#40516) 2024-07-01 12:51:33 -07:00
Xiaohan Song
b80a72888c [source-postgres/mssql/mysql] Send state and count for full refresh (#39349) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Anjay Goel <anjay.g@dashtoon.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Baz <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Natalie Kwong <38087517+nataliekwong@users.noreply.github.com>
Co-authored-by: Audrey Maldonado <audrey.maldonado@gmail.com>
Co-authored-by: gosusnp <gosusnp@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Abdul Rahman Zantout <abed-zantout@live.com>
Co-authored-by: Anatolii Yatsuk <35109939+tolik0@users.noreply.github.com>
Co-authored-by: Dhroov Makwana <pabloescoder@gmail.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Ben Church <ben@airbyte.io>
Co-authored-by: Justin Flannery <juftin@juftin.com>
Co-authored-by: Mal Hancock <mallory@archangelic.space>
Co-authored-by: Enrique Alcázar Garzás <enrique.alcazar@omniloy.com>
Co-authored-by: Daniela García Nistor <58893405+gndaniela@users.noreply.github.com>
Co-authored-by: Carlo Nuccio <carlonuccio91@gmail.com>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: Charlie Duong <charlie.duong@hey.com>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
Co-authored-by: Cristina Mariscal <166420606+cmm-airbyte@users.noreply.github.com>
Co-authored-by: cristina.mariscal <cristina.mariscal@cristina.mariscal--MacBook-Pro---DFJ27FJFXX>
Co-authored-by: Gonzalo Villafañe Tapia <gvillafanetapia@gmail.com>
Co-authored-by: Jérémy Denquin <jdenquin@users.noreply.github.com>
Co-authored-by: Maxime Carbonneau-Leclerc <3360483+maxi297@users.noreply.github.com>
Co-authored-by: williammcguinness <wmcguinness@findoctave.com>
Co-authored-by: Marius Posta <marius@airbyte.io>
Co-authored-by: Stephane Geneix <147216312+stephane-airbyte@users.noreply.github.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Akash Kulkarni <113392464+akashkulk@users.noreply.github.com>
Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
 2024-06-17 13:30:56 -04:00
Akash Kulkarni
fcc2940d20 [DB sources] : Debezium properties accept list of streams (#39405) 2024-06-13 15:11:10 -04:00
Yue Li
30cd6f1a3d source-mysql: upgrade debezium to 2.5.4 (#39144) 
This patch attempts to fix the following issues:

airbytehq/oncall#5332
#32348
 2024-06-12 16:25:21 -07:00
Xiaohan Song
2d194fa359 Set isResumeable flag in catalog (#38584) 
Co-authored-by: Dhroov Makwana <pabloescoder@gmail.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
Co-authored-by: Maxime Carbonneau-Leclerc <3360483+maxi297@users.noreply.github.com>
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Ben Church <ben@airbyte.io>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Christo Grabowski <108154848+ChristoGrab@users.noreply.github.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Catherine Noll <clnoll@users.noreply.github.com>
Co-authored-by: Audrey Maldonado <audrey.maldonado@gmail.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
 2024-05-30 10:23:09 -07:00
Akash Kulkarni
a2bcd4904c [DB sources] : Exit when encountering a config error (#38538) 2024-05-29 17:44:21 -04:00
Rodi Reich Zilberman
7bd9423216 mysql stream statuses (#38198) 
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
 2024-05-23 19:25:51 -04:00
Subodh Kant Chaturvedi
073b94062c db-sources: disable counts for state messages for FULL_REFRESH streams (#38208) 
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
 2024-05-15 11:15:26 -07:00
Akash Kulkarni
fcbfaa376f [DB source errors] : Handle common transient errors (#38104) 2024-05-13 20:27:07 -04:00
Xiaohan Song
d3864c29d2 [source-mysql] Fix NPE on cursor based full refresh (#37824) 2024-05-03 19:24:18 -04:00
Xiaohan Song
b9dc2053cf Mysql to emit stream initial loader (#36932) 
Co-authored-by: Aaron ("AJ") Steers <aj@airbyte.io>
 2024-05-03 12:30:53 -07:00