jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-19 18:14:56 -05:00
Code Issues Packages Projects Releases Wiki Activity
41,480 Commits 5,109 Branches 518 Tags
yarden/source-mongodb-support-dynamic-arrays
Commit Graph

2353 Commits

Author SHA1 Message Date
Ryan Br...
4c1ec8b2b8 Rbroughan/dataflow state (#64514) 2025-08-07 19:48:00 -07:00
Subodh Kant Chaturvedi
1ee8cb5edb fix: cap socket part size (#64545) 2025-08-08 00:45:03 +05:30
Francis Genet
8957930ad2 Allow us to dynamically set the log level via an env variable (#64148) 2025-08-05 17:31:03 +00:00
Benoit Moriceau
e708affbc2 Cdk data flow spike (#64163) 
Co-authored-by: tryangul <ryan.broughan@gmail.com>
Co-authored-by: benoit.m <benoit.m@benoit.m--MacBook-Pro---GMG2HJ9HJQ>
 2025-08-05 10:03:45 -07:00
Subodh Kant Chaturvedi
94ad36e465 chore: get azure blob storage on the latest cdk (#64487) 
Co-authored-by: Maxime Carbonneau-Leclerc <3360483+maxi297@users.noreply.github.com>
 2025-08-05 16:33:00 +05:30
Rodi Reich Zilberman
8ed6f3e14e CDC state fixes (#63377) 2025-07-30 18:01:08 -07:00
Maxime Carbonneau-Leclerc
369e12eaa6 fix: discover operation failures (#63756) 2025-07-29 18:21:54 -04:00
Subodh Kant Chaturvedi
0ca89e9a04 feat: implement proto to parquet formatter (#64111) 2025-07-29 19:59:42 +05:30
Subodh Kant Chaturvedi
88c1e1ef69 fix: implement proper handling of state messages in speed mode (#63758) 
Co-authored-by: Jonathan Pearlin <jonathan@airbyte.io>
 2025-07-28 21:35:47 +05:30
Benoit Moriceau
dd12f774f7 CDK version use the SemVer format (#63740) 
This PR is making the CDK to use SemVer.

It requires the contributor to manually set the CDK version allowing to define patch/minor/major version.

This is a first step into improving the way we manage the CDK versioning.

The changes are the following ones:

java-bulk-cdk-publish.yml: remove the autogeneration of the version. Add a check step to ensure that we don't override the latest version
java-cdk-tests.yml: add a step in the CDK CI to ensure that the we don't override existing version.
README.md: Update the doc
build.gradle: remove the task which generate the build number; add a task to check the version doesn't conflict with the latest version.


Co-authored-by: Davin Chia <davinchia@gmail.com>
 2025-07-24 12:16:29 -07:00
Edward Gao
885cc13c4f Destination S3 Data Lake: remove unnecessary properties on table (#63746) 2025-07-23 20:51:20 +00:00
Matt Bayley
df1ee16459 Bulk CDK: No supplier for trigger-based connectors (#63729) 2025-07-23 11:07:51 -07:00
Charles
f3a5cb388d chore: migrate licenses to elv2 (#63730) 2025-07-23 08:54:38 -07:00
Rodi Reich Zilberman
8724d83401 Migrate concurrency setting (#63349) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-22 15:06:37 -07:00
Edward Gao
789c0cb29d Bulk load CDK: Move CDK tests into mock test (#62895) 2025-07-22 08:27:48 -07:00
Maxime Carbonneau-Leclerc
d2da528a31 fix: do not return operation if matching key predicate but none found (#63699) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Jimmy Ma <gosusnp@users.noreply.github.com>
 2025-07-22 09:47:45 -04:00
Edward Gao
ce2d87f0a0 Bulk Load CDK: stop testing undeclared fields (#62882) 2025-07-21 22:36:39 +00:00
Edward Gao
7fe15a7003 Destination Bigquery: GCS mode supports null chars in strings (#63312) 2025-07-18 17:01:43 +00:00
Maxime Carbonneau-Leclerc
6658d74f67 feat: dynamic discovery in low-code (#63288) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-18 07:32:36 -04:00
Rodi Reich Zilberman
4dc9bbce6e Rodi/cdc state socket mode (#62932) 2025-07-17 11:28:15 -07:00
Subodh Kant Chaturvedi
97f4b06d00 feat: implement proto to avro fast route for S3 (#63332) 2025-07-17 13:49:32 +05:30
Subodh Kant Chaturvedi
f322e4edba fix: use toS3SafeCharacters instead of toAlphanumericAndUnderscore (#63333) 2025-07-17 13:47:09 +05:30
Maxime Carbonneau-Leclerc
14a7cb2e9a fix: format (#63284) 2025-07-14 12:34:56 -04:00
Maxime Carbonneau-Leclerc
78f5eaf541 feat: load-http: improve retrying policies (#62907) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-14 11:58:29 -04:00
Matt Bayley
a4d4746020 Move source test framework to cdk (#62929) 2025-07-11 16:51:52 -07:00
Francis Genet
c388d1877e [S3-Data-Lake] Handle compaction and file delete on truncate refresh (#62888) 2025-07-11 16:46:23 -07:00
Maxime Carbonneau-Leclerc
2716ebb581 feat: allow for kotlin like access operator in string interpolation (#62908) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-11 11:42:29 -04:00
Rodi Reich Zilberman
045491d8fe Rodi/refactor output consumer (#62862) 2025-07-10 12:59:25 -07:00
Francis Genet
f748f52ace Removing testBizarrePkSchemaEvolution since this is not a scenario that we are supposed to handle (#62904) 2025-07-10 19:40:24 +00:00
Subodh Kant Chaturvedi
30c736f15a chore: add comments to StatsEmitter to explain the usage of record message (#62892) 2025-07-10 18:11:29 +05:30
Subodh Kant Chaturvedi
a1a67c0dab feat: implement proto to csv fast writer (#62880) 2025-07-10 14:55:46 +05:30
Edward Gao
da56fa8b65 Destination bigquery: more schema evolution fixes (#62448) 2025-07-08 20:47:43 +00:00
Subodh Kant Chaturvedi
e580033e75 Revert "fix: nullability when assembling to JsonNode" (#62854) 2025-07-08 22:00:00 +05:30
Maxime Carbonneau-Leclerc
81e36697fb feat: set dlq checker in low-code and apply to destination-customer-io (#62843) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-08 06:35:23 -04:00
Jimmy Ma
21c9c3bdc9 fix: dlq should support empty lists (#62841) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-07 15:48:33 -07:00
Maxime Carbonneau-Leclerc
92c9c1162b feat: destination customer io first iteration (#62083) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-07 18:36:57 -04:00
Ryan Br...
1ff3468b71 Rbroughan/cleanup stringify unions (#62509) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-07 12:56:36 -07:00
Rodi Reich Zilberman
cbf42c3333 Socket mode in bulk extract CDK (#60993) 2025-07-07 12:54:08 -07:00
Maxime Carbonneau-Leclerc
b091f64313 feat: PoC check as declarative component (#61720) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-07 14:00:10 -04:00
Jimmy Ma
222fc46431 chore: dlq-toolkit updates rejected records counts (#62502) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-07-03 14:08:46 -07:00
Edward Gao
7cd8dd2b78 Bulk load CDK: add integration test for weird global state message (#62477) 2025-07-03 11:20:54 -07:00
Jonathan Pearlin
844f15bcbb chore: prepare rc release (#62068) 
Co-authored-by: Subodh Kant Chaturvedi <subodh1810@gmail.com>
 2025-07-02 11:19:06 -04:00
Rodi Reich Zilberman
16e7780b2e Revert recent change to CDK function signature (#62052) 2025-07-01 14:39:14 -07:00
Yue Li
7c3b0d1d23 [source-mssql] Support logical primary key for initial snapshot and CDC (#61729) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
 2025-07-01 13:55:51 -07:00
Benoit Moriceau
483a3c141c Clickhouse: Support Json data type (#62100) 
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Kat Wilson <40400595+katmarkham@users.noreply.github.com>
 2025-07-01 11:49:15 -07:00
Benoit Moriceau
2b364cc008 ch fix temp table (#62447) 2025-06-30 14:09:38 -07:00
Jimmy Ma
0dcf664ec8 chore: add merge option to DlqRecordBuilder (#62111) 2025-06-30 13:13:57 -07:00
Maxime Carbonneau-Leclerc
f2ae90c7cd feat: reducing number of assembling between layers (#62450) 
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
 2025-06-30 16:12:23 -04:00
Jimmy Ma
dc5c21c839 chore: make dlq config optional (#62120) 2025-06-30 10:29:42 -07:00
Jonathan Pearlin
72ba2830d3 refactor: push path fix down into readers (#62116) 2025-06-30 09:56:44 -04:00