jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-20 18:39:31 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

482 Commits

Author SHA1 Message Date
Aaron ("AJ") Steers
a7be6e6a0f chore: Sort externalDocumentationUrls by type then title (do not merge) (#69742) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2025-11-18 21:51:00 -08:00
Aaron ("AJ") Steers
d9b08c06d9 feat: Add API release history URLs to 44 certified connectors (#69741) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2025-11-18 20:20:34 -08:00
Aaron ("AJ") Steers
f589f89217 feat: Add external documentation URLs to Group H source connectors (source-microsoft-entra-id through source-oura) (#69735) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2025-11-18 18:09:21 -08:00
sophiecuiy
4846a52f60 add table filtering to mysql (#69228) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-11-14 10:35:55 -08:00
Rodi Reich Zilberman
0780b35684 MySql bump to the latest CDK (#69284) 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-11-11 16:37:34 -08:00
Rodi Reich Zilberman
55bb5d8dbe Set lower bound included flag in snapshot partitions (#69177) 2025-11-05 15:19:25 -08:00
Rodi Reich Zilberman
e8721949a8 9851 partition mysql source cdc syncs and refreshes are both taking a very long time (#69104) 2025-11-03 16:35:15 -08:00
Rodi Reich Zilberman
947b162e6b Bump MySql to latest CDK (#68652) 2025-10-28 11:27:31 -07:00
Rodi Reich Zilberman
4afd43d9a6 Mysql extract proto temporal representation (#68094) 
Co-authored-by: subodhchaturvedi <subodh1810@gmail.com>
 2025-10-21 10:24:13 -07:00
Rodi Reich Zilberman
79b0c93291 14652 source mysql cdc meta fields encoded incorrectly to protobuf (#67151) 
## What
Among other CDC_* fields, CDC_UPDATED_AT and CDC_DELETED_AT fields are
sent as part of CDC snapshot and incremental records.
These two are unique because their catalog definition
(AirbyteSchemaType) is a String, but the values they hold are date time.
Json encoding relied on the fact that all dates encode to string in json
but in protobuf we encoded these values using
`.setTimestampWithTimezone` which prevented destination from pulling the
value out of protobuf. Destination relies on the catalog type to know
how to decode messages.
We need to ensure that these fields are encoded to protobuf using
`.setString` instead in order to match the catalog type definition.


#### Note: The CDK portion of this PR was separated to
https://github.com/airbytehq/airbyte/pull/67152. Once the CDK PR is
merged this PR will only include connector changes.

*** Please ignore the CDK changes on this PR ***

## How
Updated CDC Meta field type to support a unique type that accepts an
`OffsetDateTime` but encodes the value in protobuf as a string.

## Review guide
<!--
1. `x.py`
2. `y.py`
-->

## User Impact
Users who previously used Mysql CDC in speed mode will need to refresh
those streams.

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌


Resolves https://github.com/airbytehq/airbyte/issues/66735
 2025-10-07 13:12:12 -07:00
Yarden Carmeli
63d68d7cf6 Source-MySQL: Prevent division by zero when split boundaries are empty (#66515) 2025-09-24 09:03:13 -07:00
Edward Gao
335acd4052 Bulk CDK: move connector cdk version decl into property file (#66484) 2025-09-22 10:14:55 -07:00
Rodi Reich Zilberman
22eb524942 Fix mysql proto encoding of types (#66179) 2025-09-10 17:27:40 -07:00
Jim Ruppert
a61f616127 moved db version logging from mysql connector to CDK for all JDBC sou… (#64569) 
moved db version logging from mysql connector to CDK , will handle all JDBC sources
logs only during check operation
 2025-08-12 17:08:22 -07:00
Rodi Reich Zilberman
8ed6f3e14e CDC state fixes (#63377) 2025-07-30 18:01:08 -07:00
Jim Ruppert
b793d217e4 log of db version (#64134) 2025-07-30 17:23:12 -07:00
Rodi Reich Zilberman
8724d83401 Migrate concurrency setting (#63349) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-22 15:06:37 -07:00
Charles
50a97db804 chore: migrate connectors from MIT to ELv2 (#63723) 2025-07-22 14:26:58 -07:00
Rodi Reich Zilberman
4dc9bbce6e Rodi/cdc state socket mode (#62932) 2025-07-17 11:28:15 -07:00
Rodi Reich Zilberman
045491d8fe Rodi/refactor output consumer (#62862) 2025-07-10 12:59:25 -07:00
octavia-bot[bot]
ca0cdb3a20 🐙 source-mysql: release 3.50.0 (#62863) 
Co-authored-by: octavia-bot[bot] <108746235+octavia-bot[bot]@users.noreply.github.com>
 2025-07-08 15:05:52 -07:00
Rodi Reich Zilberman
cbf42c3333 Socket mode in bulk extract CDK (#60993) 2025-07-07 12:54:08 -07:00
Rodi Reich Zilberman
3e878bf5ed remove local in mysql cdk version (#61014) 2025-06-02 10:15:38 -07:00
Rodi Reich Zilberman
cfc1243bd2 concurrent mysql read (#60218) 2025-05-29 13:38:46 -07:00
Davin Chia
112cb7b909 chore: move source postgres and mysql to new build flow. (#60214) 2025-05-13 11:13:13 -07:00
Rodi Reich Zilberman
a6a52d6429 Fix partition creation to avoid a loop when primary key is a string (#59732) 2025-05-09 11:29:41 -07:00
Rodi Reich Zilberman
dbd76645a3 CDK version bump (#59683) 2025-05-06 12:36:10 -07:00
Rodi Reich Zilberman
e97ed2611e Improve BigDecimal to json conversion to prevent scientific notation (#59223) 2025-05-06 10:35:31 -07:00
Matt Bayley
b486f268d2 Mitigate MySQL state with sci notation issue (#59150) 2025-04-29 23:13:22 -07:00
Yarden Carmeli
e4ddfdc4dd source-mysql: Update default ssl mode to required (#59144) 2025-04-29 12:46:25 -07:00
Aaron ("AJ") Steers
b9cdbfba1c chore: add poe support for gradle (java/kotlin) connectors (#58619) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2025-04-24 15:37:02 -07:00
devin-ai-integration[bot]
e9922c2737 chore(source-mysql): Fix vulnerabilities in dependencies (#58646) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yue.li@airbyte.io <yue.li@airbyte.io>
 2025-04-24 23:32:36 +03:00
Yue Li
bcf7dcc23d [source-mysql] Bump up CDK for source-mysql (#58644) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-04-24 12:12:14 -07:00
Yue Li
ad7aa7af4b [cdk, source-postgres, source-mysql, source-mongodb-v2] Fixing vulnerabilities in dependencies (#58132) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-04-24 08:28:30 -07:00
Rodi Reich Zilberman
af0ced7f62 Set mysql CDK version to latest (#58623) 2025-04-23 14:38:19 -07:00
Rodi Reich Zilberman
f8c945442c Extend the amount of time for debezium shutdown (#58599) 2025-04-23 09:25:38 -07:00
Matt Bayley
c21c460301 fix(cdc) no streams means include no tables (#55891) 2025-03-20 20:36:39 -07:00
Przemysław Dąbek
7c51ec118d [source-mysql] More meaningful errors for variables used in extra checks for CDC (#55731) 2025-03-14 13:09:42 -03:00
Yue Li
c26d2ae755 [source-mysql] don't do sampling for source-mysql (#55761) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-03-14 00:33:22 +00:00
Wenqi Hu
ae6db5ad2c Fix reversed assertions in MySQL source tests (#55756) 2025-03-13 13:28:54 -07:00
Matt Bayley
5ea3ede279 [source-mysql] pin to cdk 0.342 (#55754) 2025-03-13 11:44:21 -07:00
Matt Bayley
e52d188761 pass streams to debezium sources on cold start (#55734) 2025-03-13 10:58:37 -07:00
Matt Bayley
43858a9b08 [source-mysql] Fix fetching binlog status for version >=8.4 (#55237) 
Co-authored-by: matt.bayley <matt.bayley@matt.bayley--MacBook-Pro---G4WV0GW6J3>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yue.li@airbyte.io <yue.li@airbyte.io>
 2025-03-06 15:01:24 -08:00
devin-ai-integration[bot]
3cf3c3d2fa [source-mysql, mssql, postgres, mongodb-v2] chore: update base image version for certified DB source connectors (#55234) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yue.li@airbyte.io <yue.li@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-03-06 12:15:07 -08:00
Yarden Carmeli
d74e8a5794 [source-mysql] Set 'required' as the default encryption method in spec. (#55214) 2025-03-05 17:12:31 -08:00
Yarden Carmeli
eeda92acf3 [source-mysql] Improve Error Handling and Regex Detection (#54156) 2025-02-24 09:32:55 -08:00
Wenqi Hu
fc80e61358 [source mysql] Fix integration test (#54183) 2025-02-20 12:36:11 -08:00
Wenqi Hu
f0a96b3c1c [source volcano] Decorate stream with cursor field for trigger based CDC stream (#53152) 2025-02-12 22:31:22 +02:00
Yue Li
c288e1276a [source-mysql] fix typo in CDC configruation (#53195) 2025-02-06 20:09:30 +02:00
Marius Posta
8c2ed63cf4 source-mysql: adopt bulk-cdk-toolkit-extract-cdc API changes (#52039) 2025-01-30 23:06:08 +02:00