jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-20 10:32:35 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

281 Commits

Author SHA1 Message Date
sophiecuiy
4846a52f60 add table filtering to mysql (#69228) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-11-14 10:35:55 -08:00
Rodi Reich Zilberman
0780b35684 MySql bump to the latest CDK (#69284) 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-11-11 16:37:34 -08:00
Rodi Reich Zilberman
55bb5d8dbe Set lower bound included flag in snapshot partitions (#69177) 2025-11-05 15:19:25 -08:00
Rodi Reich Zilberman
e8721949a8 9851 partition mysql source cdc syncs and refreshes are both taking a very long time (#69104) 2025-11-03 16:35:15 -08:00
Rodi Reich Zilberman
947b162e6b Bump MySql to latest CDK (#68652) 2025-10-28 11:27:31 -07:00
devin-ai-integration[bot]
096023bb8f docs(source-mysql): Improve type mapping accuracy in documentation (#68636) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ian.alton@airbyte.io <ian.alton@airbyte.io>
 2025-10-24 11:43:55 -07:00
devin-ai-integration[bot]
179b36ff43 docs(source-mysql): Fix broken link to datatype test file (#68635) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ian.alton@airbyte.io <ian.alton@airbyte.io>
 2025-10-24 18:14:32 +00:00
Rodi Reich Zilberman
4afd43d9a6 Mysql extract proto temporal representation (#68094) 
Co-authored-by: subodhchaturvedi <subodh1810@gmail.com>
 2025-10-21 10:24:13 -07:00
Ian Alton
b159bdb179 docs: clean up IP allow list URLs and security info re: data planes (#67074) 
## What
<!--
* Describe what the change is solving. Link all GitHub issues related to
this change.
-->

This is minor docs cleanup after we released the Iowa region on GCP.
There's not much to update, but it did prompt me to take a look at a few
things, update some URLs, and reframe the discussion of how Airbyte is
deployed to Cloud.

Companion change to this UI update, but there is no dependency between
them. https://github.com/airbytehq/airbyte-platform-internal/pull/17923

## How
<!--
* Describe how code changes achieve the solution.
-->

## Review guide
<!--
1. `x.py`
2. `y.py`
-->

## User Impact
<!--
* What is the end result perceived by the user?
* If there are negative side effects, please list them. 
-->

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-10-09 13:22:10 -07:00
Rodi Reich Zilberman
79b0c93291 14652 source mysql cdc meta fields encoded incorrectly to protobuf (#67151) 
## What
Among other CDC_* fields, CDC_UPDATED_AT and CDC_DELETED_AT fields are
sent as part of CDC snapshot and incremental records.
These two are unique because their catalog definition
(AirbyteSchemaType) is a String, but the values they hold are date time.
Json encoding relied on the fact that all dates encode to string in json
but in protobuf we encoded these values using
`.setTimestampWithTimezone` which prevented destination from pulling the
value out of protobuf. Destination relies on the catalog type to know
how to decode messages.
We need to ensure that these fields are encoded to protobuf using
`.setString` instead in order to match the catalog type definition.


#### Note: The CDK portion of this PR was separated to
https://github.com/airbytehq/airbyte/pull/67152. Once the CDK PR is
merged this PR will only include connector changes.

*** Please ignore the CDK changes on this PR ***

## How
Updated CDC Meta field type to support a unique type that accepts an
`OffsetDateTime` but encodes the value in protobuf as a string.

## Review guide
<!--
1. `x.py`
2. `y.py`
-->

## User Impact
Users who previously used Mysql CDC in speed mode will need to refresh
those streams.

## Can this PR be safely reverted and rolled back?
<!--
* If unsure, leave it blank.
-->
- [x] YES 💚
- [ ] NO ❌


Resolves https://github.com/airbytehq/airbyte/issues/66735
 2025-10-07 13:12:12 -07:00
Yarden Carmeli
63d68d7cf6 Source-MySQL: Prevent division by zero when split boundaries are empty (#66515) 2025-09-24 09:03:13 -07:00
Rodi Reich Zilberman
22eb524942 Fix mysql proto encoding of types (#66179) 2025-09-10 17:27:40 -07:00
Jim Ruppert
a61f616127 moved db version logging from mysql connector to CDK for all JDBC sou… (#64569) 
moved db version logging from mysql connector to CDK , will handle all JDBC sources
logs only during check operation
 2025-08-12 17:08:22 -07:00
Rodi Reich Zilberman
8ed6f3e14e CDC state fixes (#63377) 2025-07-30 18:01:08 -07:00
Jim Ruppert
b793d217e4 log of db version (#64134) 2025-07-30 17:23:12 -07:00
Rodi Reich Zilberman
8724d83401 Migrate concurrency setting (#63349) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-22 15:06:37 -07:00
Rodi Reich Zilberman
4dc9bbce6e Rodi/cdc state socket mode (#62932) 2025-07-17 11:28:15 -07:00
Rodi Reich Zilberman
045491d8fe Rodi/refactor output consumer (#62862) 2025-07-10 12:59:25 -07:00
octavia-bot[bot]
ca0cdb3a20 🐙 source-mysql: release 3.50.0 (#62863) 
Co-authored-by: octavia-bot[bot] <108746235+octavia-bot[bot]@users.noreply.github.com>
 2025-07-08 15:05:52 -07:00
Rodi Reich Zilberman
cbf42c3333 Socket mode in bulk extract CDK (#60993) 2025-07-07 12:54:08 -07:00
Rodi Reich Zilberman
3e878bf5ed remove local in mysql cdk version (#61014) 2025-06-02 10:15:38 -07:00
Rodi Reich Zilberman
cfc1243bd2 concurrent mysql read (#60218) 2025-05-29 13:38:46 -07:00
devin-ai-integration[bot]
f719e8dafe docs: add Amazon RDS MySQL binlog retention configuration information (#60883) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yue.li@airbyte.io <yue.li@airbyte.io>
 2025-05-23 19:31:14 -07:00
Davin Chia
112cb7b909 chore: move source postgres and mysql to new build flow. (#60214) 2025-05-13 11:13:13 -07:00
Rodi Reich Zilberman
a6a52d6429 Fix partition creation to avoid a loop when primary key is a string (#59732) 2025-05-09 11:29:41 -07:00
Rodi Reich Zilberman
dbd76645a3 CDK version bump (#59683) 2025-05-06 12:36:10 -07:00
Rodi Reich Zilberman
e97ed2611e Improve BigDecimal to json conversion to prevent scientific notation (#59223) 2025-05-06 10:35:31 -07:00
Matt Bayley
b486f268d2 Mitigate MySQL state with sci notation issue (#59150) 2025-04-29 23:13:22 -07:00
Yarden Carmeli
e4ddfdc4dd source-mysql: Update default ssl mode to required (#59144) 2025-04-29 12:46:25 -07:00
devin-ai-integration[bot]
e9922c2737 chore(source-mysql): Fix vulnerabilities in dependencies (#58646) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yue.li@airbyte.io <yue.li@airbyte.io>
 2025-04-24 23:32:36 +03:00
Yue Li
ad7aa7af4b [cdk, source-postgres, source-mysql, source-mongodb-v2] Fixing vulnerabilities in dependencies (#58132) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-04-24 08:28:30 -07:00
Ian Alton
01cd16654e 11059 multi-instance, versioned docs (#58095) 
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
 2025-04-24 02:58:09 +03:00
Rodi Reich Zilberman
af0ced7f62 Set mysql CDK version to latest (#58623) 2025-04-23 14:38:19 -07:00
Rodi Reich Zilberman
f8c945442c Extend the amount of time for debezium shutdown (#58599) 2025-04-23 09:25:38 -07:00
Przemysław Dąbek
7c51ec118d [source-mysql] More meaningful errors for variables used in extra checks for CDC (#55731) 2025-03-14 13:09:42 -03:00
Yue Li
c26d2ae755 [source-mysql] don't do sampling for source-mysql (#55761) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-03-14 00:33:22 +00:00
Matt Bayley
e52d188761 pass streams to debezium sources on cold start (#55734) 2025-03-13 10:58:37 -07:00
Matt Bayley
43858a9b08 [source-mysql] Fix fetching binlog status for version >=8.4 (#55237) 
Co-authored-by: matt.bayley <matt.bayley@matt.bayley--MacBook-Pro---G4WV0GW6J3>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yue.li@airbyte.io <yue.li@airbyte.io>
 2025-03-06 15:01:24 -08:00
devin-ai-integration[bot]
3cf3c3d2fa [source-mysql, mssql, postgres, mongodb-v2] chore: update base image version for certified DB source connectors (#55234) 
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yue.li@airbyte.io <yue.li@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-03-06 12:15:07 -08:00
Yarden Carmeli
d74e8a5794 [source-mysql] Set 'required' as the default encryption method in spec. (#55214) 2025-03-05 17:12:31 -08:00
Yarden Carmeli
eeda92acf3 [source-mysql] Improve Error Handling and Regex Detection (#54156) 2025-02-24 09:32:55 -08:00
Yue Li
c288e1276a [source-mysql] fix typo in CDC configruation (#53195) 2025-02-06 20:09:30 +02:00
Marius Posta
8c2ed63cf4 source-mysql: adopt bulk-cdk-toolkit-extract-cdc API changes (#52039) 2025-01-30 23:06:08 +02:00
Tyler B
42c359665b Update mysql.md to clarify CDC (#51561) 2025-01-15 16:50:08 -08:00
github-actions[bot]
f6093fa1a0 🐙 source-mysql: release 3.11.0 (#51545) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-01-15 08:55:28 -06:00
Rodi Reich Zilberman
b3f56fb349 Set cdc metadata fields types to match legacy connectors (#51029) 2025-01-13 22:24:18 +02:00
Augustin
915a5e62e8 source-mysql: Use airbyte/java-connector-base:2.0.0 (#51510) 2025-01-13 08:02:57 -08:00
github-actions[bot]
c60625e5b5 🐙 source-mysql: release 3.10.0 (#51008) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2025-01-09 10:14:50 -06:00
Marius Posta
7f98173dae source-mysql: use CDK version 257 (#50987) 2025-01-08 16:21:51 -05:00
Marius Posta
ac054a641e source-mysql: fix converter bug (#50965) 2025-01-07 22:27:17 +02:00