jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-20 18:39:31 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

281 Commits

Author SHA1 Message Date
Rodi Reich Zilberman
14aee8b1f5 Make sample query compatible with views (#50437) 
Co-authored-by: Marius Posta <marius@airbyte.io>
 2025-01-07 21:57:08 +02:00
Augustin
d274352d95 source-mysql: Use airbyte/java-connector-base:1.0.0 (#49892) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-01-07 14:30:06 -05:00
Marius Posta
e01e6a21fa source-mysql: fix indentation level for rules defs (#50868) 2025-01-03 15:23:21 -08:00
Xiaohan Song
afb589435c [bulk-cdk] state manager only reports stream with count in it (#48587) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2024-12-23 15:25:04 -05:00
Marius Posta
1d6b34cfba source-mysql: datatype fixes (#49918) 2024-12-20 19:44:03 +02:00
Marius Posta
c5a12bada2 source-mysql: clean up config (#49950) 2024-12-20 09:44:54 -05:00
Marius Posta
2910475443 source-mysql: simplify MySqlSelectQuerier (#49948) 2024-12-20 09:24:46 -05:00
Marius Posta
cc383cf360 source-mysql: normalize class names (#49939) 2024-12-19 15:44:33 -05:00
Rodi Reich Zilberman
8819bd143e [source-mysql] 7084 source mysql regression in 39x date parsing errors (#49932) 2024-12-19 17:26:10 +00:00
Rodi Reich Zilberman
dfcdea8ade Send a correct fallback value for autogenerated boolean (#49830) 2024-12-17 06:36:51 -08:00
Catherine Noll
c4cb39d63c fix: bump version to re-release source mysql (#49456) 2024-12-12 17:50:30 -08:00
github-actions[bot]
be03d32669 🐙 source-mysql: release 3.9.0 (#49423) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2024-12-12 18:15:16 -05:00
Rodi Reich Zilberman
c6b7ac1450 10684 source mysql beta stream with id of type binary shows an error when queried (#48501) 2024-11-14 18:18:22 -08:00
Rodi Reich Zilberman
d184317acc quote compressed db schema history (#48456) 2024-11-11 15:35:41 -08:00
Rodi Reich Zilberman
b03023d78e Don't emit state message with null stream status (#48441) 2024-11-11 06:46:09 -08:00
Rodi Reich Zilberman
63cd395fbe handle pk value from saved state (#48423) 2024-11-08 19:47:59 +00:00
Rodi Reich Zilberman
c589cb5e9e migrate CAT tests to new source-mysql location (#48414) 2024-11-07 19:18:07 -08:00
Rodi Reich Zilberman
e42d056050 merge mysql v2 into v1 (#48369) 2024-11-07 10:39:55 -08:00
Natalie Kwong
b0407dc16a docs: Add field anchors for top source docs (#45885) 2024-09-25 08:55:06 -05:00
Yue Li
2e62443f1c [source-postgres/mysql/mssql/mongodb-v2] Use latest CDK to adopt latest apache sshd mina to handle tcpkeepalive requests (#45639) 2024-09-18 14:46:10 +03:00
Xiaohan Song
03584d5afb [source-mysql/mssql] Fix state manager on determining non-resumable streams (#45181) 2024-09-10 16:13:04 -07:00
Yue Li
af98a810b5 [certified db sources] adopt latest CDK to use error translation bug fixes (#44841) 2024-08-28 22:35:16 +03:00
Yue Li
ed44c093b3 [source-mysql] Upgrade Debezium and mysql connector to 2.7.1.Final (#44013) 2024-08-14 10:15:05 -07:00
Akash Kulkarni
b43b1111a8 [cdk] : More logging for count mismatch (#43410) 2024-08-09 05:45:27 +03:00
Xiaohan Song
97d822f820 [cdk, source-mysql/postgres/mssql/mongodb-v2] log record count to state message (#42869) 
Co-authored-by: Yue Li <yue.li@airbyte.io>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
 2024-08-07 04:35:22 +03:00
Xiaohan Song
ab005c6465 [source-mssql/postgres/mysql] fix rfr losing stream state (#42550) 
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
 2024-08-02 20:02:19 +03:00
Yue Li
d34d7e627a [source-postgres/mssql/mysql] use the latest CDK to use the latest fix of error translation (#42852) 2024-07-29 10:42:44 -07:00
Yue Li
81cd8f2005 [cdk] handle null message inside exception (#42417) 
Fixes airbytehq/oncall#5904, error message inside an exception could be null.
 2024-07-23 17:27:33 -07:00
Akash Kulkarni
af0394150f [WASS] : Remove final transient error emitter iterators (#42421) 2024-07-23 14:27:15 -04:00
Xiaohan Song
af28105efa [source-mssql/mysql/postgres] Fix and cleanup oc map (#42024) 
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Artem Inzhyyants <36314070+artem1205@users.noreply.github.com>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Antonio Papa <antoniogpapa@gmail.com>
Co-authored-by: Adam Marcus <marcua@marcua.net>
Co-authored-by: Bryce Groff <bgroff@hawaii.edu>
Co-authored-by: Akash Kulkarni <113392464+akashkulk@users.noreply.github.com>
Co-authored-by: Patrick Nilan <nilan.patrick@gmail.com>
 2024-07-22 11:32:51 -07:00
Yue Li
03c57ea472 [source-postgres, source-mysql] Translating top opaque and unfriendly error messages (#42087) 
This patch includes multiple translation rules for additional top opaque and system error messages that can be converted to transient or config errors. Translation/error profiles are added to the exception handler class for Postgres and MySQL sources.
 2024-07-21 23:24:17 -07:00
Akash Kulkarni
000dc477cf [WASS] : Enhancements in logging & analytics (#42122) 2024-07-21 23:07:18 -04:00
Yue Li
4417769038 [cdk, source-postgres, source-mysql] a new error handling and translation framework (#40208) 
Fixes airbytehq/airbyte-internal-issues#8516

This set of changes mainly moves error translation to be part of each connector.

In general, each connector needs to implement its own error translation class that inherits from the abstract class ConnectorExceptionTranslator, which is part of the CDK. By implementing, it means the connector developer or our support will populate the error dictionary with error samples with matching rules (e.g., regex). See the example we created for the Postgres source.
 2024-07-17 17:19:13 -07:00
Akash Kulkarni
508638fcb7 [Source-mysql] : Adopt latest CDK + WASS fixes (#42043) 2024-07-17 13:33:05 -07:00
Akash Kulkarni
310e6bd315 [Source-mysql] : Implement WASS algo (#38240) 
Co-authored-by: Evan Tahler <evan@airbyte.io>
 2024-07-11 11:54:54 -04:00
Akash Kulkarni
3a3e058ae8 [DB-sources] : Improve heartbeat logic (#40516) 2024-07-01 12:51:33 -07:00
Yue Li
42124f6772 [source-mysql]: support Planetscale MySQL's per-query row limit (#40561) 
Fixes airbytehq/oncall#5051

Planet MySQL has defined multiple system limits. One of our customers has been hitting the 100K per-query row limit, which causes our connector to emit system errors. This patch defines a chunk size limit for our connector. By default, it is the largest long, and if we find it is a Planetscale MySQL, we will adjust to 100K.
 2024-06-26 17:29:02 -07:00
Xiaohan Song
b80a72888c [source-postgres/mssql/mysql] Send state and count for full refresh (#39349) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Anjay Goel <anjay.g@dashtoon.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Baz <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Natalie Kwong <38087517+nataliekwong@users.noreply.github.com>
Co-authored-by: Audrey Maldonado <audrey.maldonado@gmail.com>
Co-authored-by: gosusnp <gosusnp@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Abdul Rahman Zantout <abed-zantout@live.com>
Co-authored-by: Anatolii Yatsuk <35109939+tolik0@users.noreply.github.com>
Co-authored-by: Dhroov Makwana <pabloescoder@gmail.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Ben Church <ben@airbyte.io>
Co-authored-by: Justin Flannery <juftin@juftin.com>
Co-authored-by: Mal Hancock <mallory@archangelic.space>
Co-authored-by: Enrique Alcázar Garzás <enrique.alcazar@omniloy.com>
Co-authored-by: Daniela García Nistor <58893405+gndaniela@users.noreply.github.com>
Co-authored-by: Carlo Nuccio <carlonuccio91@gmail.com>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: Charlie Duong <charlie.duong@hey.com>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
Co-authored-by: Cristina Mariscal <166420606+cmm-airbyte@users.noreply.github.com>
Co-authored-by: cristina.mariscal <cristina.mariscal@cristina.mariscal--MacBook-Pro---DFJ27FJFXX>
Co-authored-by: Gonzalo Villafañe Tapia <gvillafanetapia@gmail.com>
Co-authored-by: Jérémy Denquin <jdenquin@users.noreply.github.com>
Co-authored-by: Maxime Carbonneau-Leclerc <3360483+maxi297@users.noreply.github.com>
Co-authored-by: williammcguinness <wmcguinness@findoctave.com>
Co-authored-by: Marius Posta <marius@airbyte.io>
Co-authored-by: Stephane Geneix <147216312+stephane-airbyte@users.noreply.github.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Akash Kulkarni <113392464+akashkulk@users.noreply.github.com>
Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
 2024-06-17 13:30:56 -04:00
Akash Kulkarni
fcc2940d20 [DB sources] : Debezium properties accept list of streams (#39405) 2024-06-13 15:11:10 -04:00
Yue Li
30cd6f1a3d source-mysql: upgrade debezium to 2.5.4 (#39144) 
This patch attempts to fix the following issues:

airbytehq/oncall#5332
#32348
 2024-06-12 16:25:21 -07:00
Audrey Maldonado
b9690ba76e Wrapping changelog in details for a collapsible view (#38801) 2024-06-04 13:40:53 -04:00
Xiaohan Song
2d194fa359 Set isResumeable flag in catalog (#38584) 
Co-authored-by: Dhroov Makwana <pabloescoder@gmail.com>
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: Gireesh Sreepathi <gisripa@gmail.com>
Co-authored-by: Yue Li <61070669+theyueli@users.noreply.github.com>
Co-authored-by: Augustin <augustin@airbyte.io>
Co-authored-by: Natik Gadzhi <natik@respawn.io>
Co-authored-by: Danylo Jablonski <150933663+DanyloGL@users.noreply.github.com>
Co-authored-by: Maxime Carbonneau-Leclerc <3360483+maxi297@users.noreply.github.com>
Co-authored-by: Oleksandr Bazarnov <oleksandr.bazarnov@globallogic.com>
Co-authored-by: Ben Church <ben@airbyte.io>
Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Christo Grabowski <108154848+ChristoGrab@users.noreply.github.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Catherine Noll <clnoll@users.noreply.github.com>
Co-authored-by: Audrey Maldonado <audrey.maldonado@gmail.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <serhii.lazebnyi@globallogic.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
 2024-05-30 10:23:09 -07:00
Akash Kulkarni
a2bcd4904c [DB sources] : Exit when encountering a config error (#38538) 2024-05-29 17:44:21 -04:00
Rodi Reich Zilberman
7bd9423216 mysql stream statuses (#38198) 
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
 2024-05-23 19:25:51 -04:00
Evan Tahler
a914027d74 [docs] Vendor-Specific limitations added to troubleshooting sections (#38351) 
Co-authored-by: Stephane Geneix <147216312+stephane-airbyte@users.noreply.github.com>
 2024-05-21 14:42:22 +00:00
Subodh Kant Chaturvedi
073b94062c db-sources: disable counts for state messages for FULL_REFRESH streams (#38208) 
Co-authored-by: Xiaohan Song <xiaohan@airbyte.io>
 2024-05-15 11:15:26 -07:00
Akash Kulkarni
fcbfaa376f [DB source errors] : Handle common transient errors (#38104) 2024-05-13 20:27:07 -04:00
Xiaohan Song
533cea8bb8 Only check sourcedefined pk for eligibility of rfr (#38046) 2024-05-07 16:54:08 -04:00
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Xiaohan Song
d3864c29d2 [source-mysql] Fix NPE on cursor based full refresh (#37824) 2024-05-03 19:24:18 -04:00