1
0
mirror of synced 2025-12-21 19:11:14 -05:00
Commit Graph

31 Commits

Author SHA1 Message Date
Edward Gao
335acd4052 Bulk CDK: move connector cdk version decl into property file (#66484) 2025-09-22 10:14:55 -07:00
Wenqi Hu
8a6f866c1c [Snowflake source] Change CDK version to 0.1.31 instead of latest (#66311) 2025-09-16 14:10:25 -07:00
Wenqi Hu
5c1a2f2f69 Fix sampling bug for DefaultJdbcCursorIncrementalPartition (#66200) 2025-09-15 16:59:59 -07:00
Matt Bayley
a4d4746020 Move source test framework to cdk (#62929) 2025-07-11 16:51:52 -07:00
Wenqi Hu
9bb0f19b96 [Snowflake] Create certified source connector (#61535) 2025-07-09 12:23:49 -07:00
Xiaohan Song
4370bf3c01 [source-snowflake] Upgrade snowflake jdbc driver (#48073) 2024-10-31 14:56:39 -07:00
Marius Posta
b59098fe15 source-snowflake: adopt CDK 0.20.4 (#35220) 2024-02-14 11:54:57 -08:00
Subodh Kant Chaturvedi
b2eda46c7e sources(bigquery, redshift, snowflake): migrate to PER-STREAM state (#33484) 2023-12-19 01:04:44 +05:30
Marius Posta
30a5127168 enable groovy formatting in airbyte-ci format config, remove spotless from gradle build (#33013)
Co-authored-by: postamar <postamar@users.noreply.github.com>
2023-12-01 09:27:16 -08:00
Stephane Geneix
c642d81b84 Stephane/enable javac errors 1 (#32467) 2023-11-13 18:07:58 -06:00
Stephane Geneix
9c6ed93a9d Revert "update all connectors for new CDK version and more stringent javac checks"
This reverts commit 18ad87bbd1.
2023-11-02 15:42:28 -07:00
Stephane Geneix
18ad87bbd1 update all connectors for new CDK version and more stringent javac checks 2023-11-02 15:24:58 -07:00
Marius Posta
7cd8020ac8 java CDK: hoist top-level gradle projects into CDK (#31960)
Co-authored-by: postamar <postamar@users.noreply.github.com>
2023-10-30 12:03:06 -07:00
Xiaohan Song
4aff41087c Source Snowflake: Upgrade snowflake JDBC version to 3.14.1 (#31252) 2023-10-11 14:50:51 -07:00
Aaron ("AJ") Steers
da32fc86bf 🚀 Launch Java CDK v0.1.0 (PR #30383)
- Launch Java CDK `v0.1.0`. 🚀
- Remove `airbyte-docker` gradle plugin from java connectors.
- Add `airbyte-java-connector` gradle plugin.
- Remove `airbyte-commons` from global build.gradle.
- Add `/publish-java-cdk` slash command.
- Initial launch of `/airbyte-ci` slash command to test connectors.
2023-09-27 15:46:02 -07:00
Ben Church
5c56ac1d84 Airbyte-ci: Remove gradle task connectorAcceptanceTest (#30326) 2023-09-19 15:16:37 -05:00
Marius Posta
51c67d7eaa gradle: remove airbyteDocker.outputs dependencies (#30314) 2023-09-11 17:16:27 -07:00
Akash Kulkarni
2f807440da Fix issue in streaming JDBC database (#27212)
* Fix issue in streaming JDBC database

* Bump + publish

* 🤖 Auto format source-db2 code [skip ci]

* 🤖 Auto format source-mssql code [skip ci]

* 🤖 Auto format source-postgres-strict-encrypt code [skip ci]

* 🤖 Auto format source-postgres code [skip ci]

* 🤖 Auto format source-mssql-strict-encrypt code [skip ci]

* 🤖 Auto format source-oracle code [skip ci]

* 🤖 Auto format source-tidb code [skip ci]

* 🤖 Auto format source-snowflake code [skip ci]

* 🤖 Auto format source-redshift code [skip ci]

* 🤖 Auto format source-mysql code [skip ci]

* 🤖 Auto format source-mysql-strict-encrypt code [skip ci]

---------

Co-authored-by: octavia-squidington-iii <octavia-squidington-iii@users.noreply.github.com>
2023-06-20 12:08:47 -05:00
Peter Hu
e5eac0a4cc use published protocol models jar (#22498)
* add airbyte-protocol to deps.toml

* use published protocol jar for platform

* use published protocol jar for connectors

* point at published jar

* fix dep

* bump gcs storage

* fix build failures in standard-source-test

* fix deps

* downgrade alloy db because it is missing strictness tests

* Revert "downgrade alloy db because it is missing strictness tests"

This reverts commit cc6089d053.

---------

Co-authored-by: cgardens <charles@airbyte.io>
2023-02-13 12:50:43 -06:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846)
Rename source acceptance test to connector acceptance test
2023-02-02 11:38:19 +01:00
Akash Kulkarni
cbb2e986fb Set app_id in snowflake source connector based on oss/cloud (#19314)
* Set app_id in snowflake source connector based on oss/cloud

Set app_id in snowflake source connector based on

* Bump version + docs

* auto-bump connector version

* Formatting

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-10 17:24:10 -08:00
Andrii Leonets
58e4ef6912 Enable SPEC SAT for Java sources (#18779)
* enable py spec compatibility tests

* add missing properties

* use expected spec file instead of the source spec.json + use dummy config file

* add missing files for the clickhouse

* move test files to the integration-test from the unit test folder

* add missing files to cockroachdb

* add missing files to db2 + fix spec format

* add missing files to elasticsearch + fix spec format

* add missing files to jdbc

* add missing files to mongodb_v2 + fix spec format

* add missing files to mssql + fix spec format

* add missing files to mysql + fix spec format

* add missing files to postgres + fix spec format

* add missing files to oracle + fix spec format

* add missing files to redshift

* add missing files to sftp

* add missing files to snowflake + fix spec format

* add missing files to tidb

* add missing files to kafka - fix spec format

* airbyte-source-acceptance-test added

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>

* add missing import

* Delete acceptance-test-config.yml

* Delete acceptance-test-docker.sh

* Update build.gradle

* Update build.gradle

* format

* revert changes

* manual .sh files format

* upd expected spec

* format

* fix SAT after master merge

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-11-09 21:45:46 +02:00
Akash Kulkarni
44ef1c4ad6 🎉 Source Snowflake : Update JDBC driver for Snowflake to 3.13.22 (#16766)
Update JDBC driver for Snowflake

Co-authored-by: Liren Tu <tuliren.git@outlook.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 16:07:34 -07:00
Davin Chia
83a89aa843 Fat Jar: Rename Dir Part 1 (#13476)
Part 1 of #13122.

Rename airbyte-db:lib to airbyte-db:db-lib.
Rename airbyte-metrics:lib to airbyte-metrics:metrics-lib
Rename airbyte-protocol:models to airbyte-protocol:protocol-models.

Explanation for what is happening:

Identically named subprojects have the following issues:
- publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
- deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
- the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See Projects with same name lead to unintended conflict resolution gradle/gradle#847 for more info.
- given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.
2022-06-06 00:35:43 +08:00
Yurii Bidiuk
45a212c4c5 🎉 Source Snowflake: support oauth (#10953)
* add oauth flow to source-snowflake

* Add unit test for oauth flow

* add docs to method

* format code

* change configs

* fixed remarks

* fixed comments

* fixed PR remark

* update with master

* format code

* fix PR remmakrs

* add test for backward compatibility

* bump version

* small fix for test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-27 18:52:33 +03:00
Lake Mossman
3d8a0dc048 Add ExitOnOutOfMemoryError to java connectors and bump versions (#10256) 2022-02-14 15:49:15 -08:00
Yurii Bidiuk
986eb588b4 🐛 Source Snowflake: fix tests (#9304)
* fixed tests, upgrade jdbc driver version

* bump vesion
2022-01-11 15:58:26 +02:00
Subodh Kant Chaturvedi
7591324351 introduce jvm flag MaxRAMPercentage for java connectors (#6001)
* introduce jvm flag MaxRAMPercentage for java connectors

* temporary commit to test this out on GKE via kube acceptance test

* undo temp commit
2021-09-22 19:59:39 +05:30
LiRen Tu
b9e1997d2f Split airbyte-db and move db dev commands to gradle (#5616)
# Summary

- A follow-up PR for #5543.
- This PR separates the `airbyte-db` project to two modules:
  - `lib` is the original `airbyte-db`.
  - `jooq` is for jOOQ code generation.
- This is necessary because the jOOQ generator requires a custom database implementation that can run Flyway migration. So the code generator logic needs to depend on the compilation of the original `airbyte-db` project.

# Commits
* Separate db to lib and jooq modules
* Update dependencies
* Add jobs db migrator test
* Fix compose build
* Add migration dev center
* Add schema dump task
* Update airbyte-db/lib/README.md
  * Co-authored-by: Davin Chia <davinchia@gmail.com>
* Update readme
* Remove bom dependency
* Update readme
* Use jooq code in db config persistence
* Remove AirbyteConfigsTable

Co-authored-by: Davin Chia <davinchia@gmail.com>
2021-08-26 10:44:09 -07:00
Andrii Leonets
107f5b8d61 🎉 Abstract level for SQL relational database sources (#4123)
Abstract level for SQL relational database sources
2021-07-05 17:18:07 +03:00
irynakruk
446ec0407f 🎉 New Source: Snowflake DB (#3844) 2021-06-10 12:52:47 -07:00