1
0
mirror of synced 2025-12-22 19:38:29 -05:00
Commit Graph

57 Commits

Author SHA1 Message Date
Aaron ("AJ") Steers
a7be6e6a0f chore: Sort externalDocumentationUrls by type then title (do not merge) (#69742)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2025-11-18 21:51:00 -08:00
Aaron ("AJ") Steers
16aaee0b9e feat: Add external documentation URLs to all destination connectors (Group A) (#69353)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2025-11-17 21:11:07 -08:00
Davin Chia
2844fbfeb8 chore: Update these destinations. (#61359)
After this PR all JVM destinations will be on the new non-dagger flow.

Note on the failed builds:
destination-csv - docker only, which is why the tests are failing. I am going to deprecate this if I cannot convince Junie to fix this for me.
destination-local-json - same as destination csv.
destination-postgres-strict-encrypt - passing for me locally, looks like a certificate error
destination-postgres - passing for me locall
destination-oracle - these are failing on container errors - the test containers they are trying to download aren't able to start up
destination-oracle-strict-encrypt - same as base oracle
destination-starburst-galaxy - actual test failures
destination-yellowbrick - actual test failures
destination-redis - actual test failures
The failed tests mirror master so we aren't adding any new failures here.
2025-06-11 19:42:38 -07:00
Aaron ("AJ") Steers
05965c781e chore: add poe_tasks.toml for all java connectors (#59670) 2025-05-05 17:28:27 -07:00
Marius Posta
dd2d443572 gradle: remove deps.toml (#45160) 2024-09-05 08:44:44 -05:00
Augustin
601c3c455c Rename registries to registryOverrides in connector metadata (#43757) 2024-08-13 16:10:56 +02:00
Augustin
31c95da499 [skip ci] Add connectorTestSuitesOptions to metadata (#38188) 2024-05-15 15:04:10 -05:00
Joe Bell
b0d0dcd297 oracle_x_norm (#36048)
The testing framework for oracle is not functional and will not be fixed for this release
2024-04-11 16:01:50 -07:00
Stephane Geneix
c642d81b84 Stephane/enable javac errors 1 (#32467) 2023-11-13 18:07:58 -06:00
Augustin
368ba78b64 🧹doc: update connectors README and remove acceptance-test-docker.sh (#32209) 2023-11-06 12:36:07 -06:00
Marius Posta
7cd8020ac8 java CDK: hoist top-level gradle projects into CDK (#31960)
Co-authored-by: postamar <postamar@users.noreply.github.com>
2023-10-30 12:03:06 -07:00
Augustin
ac6f44cf2c remove Dockerfile and .dockerignore for Java connectors (#31769) 2023-10-24 15:08:13 -05:00
Marius Posta
f8edc18039 airbyte-ci,gradle: replace airbyte-docker with airbyte-ci (#30743) 2023-10-04 08:38:17 -07:00
Aaron ("AJ") Steers
da32fc86bf 🚀 Launch Java CDK v0.1.0 (PR #30383)
- Launch Java CDK `v0.1.0`. 🚀
- Remove `airbyte-docker` gradle plugin from java connectors.
- Add `airbyte-java-connector` gradle plugin.
- Remove `airbyte-commons` from global build.gradle.
- Add `/publish-java-cdk` slash command.
- Initial launch of `/airbyte-ci` slash command to test connectors.
2023-09-27 15:46:02 -07:00
Marius Posta
a5eb24e2fc bump testcontainers version to latest (#30493) 2023-09-18 03:50:43 -05:00
Marius Posta
51c67d7eaa gradle: remove airbyteDocker.outputs dependencies (#30314) 2023-09-11 17:16:27 -07:00
Marius Posta
ef2849e35e gradle: fix airbyteDocker task inputs (#30187) 2023-09-07 03:46:31 -07:00
Edward Gao
ab04c3b2da DATs: Fix spurious failures (#29734) 2023-08-22 17:44:04 -05:00
Evan Tahler
9f515a867f DAT tearDown should cleanup *all* schemas used (#29312)
* DAT `tearDown` should cleanup *all* schemas used

* redshift too

* push all method signature changes

* Automated Commit - Format and Process Resources Changes

* use hashset

* TEST_SCHEMAS on setup as well

* fix call

* Automated Commit - Format and Process Resources Changes

---------

Co-authored-by: evantahler <evantahler@users.noreply.github.com>
2023-08-11 11:40:49 -07:00
Evan Tahler
f455c1288d Java DB Destination connector licenses to Elv2 (#27781)
* Java DB Destination connector licenses to Elv2

* PR id for docs

* fix redshift tagging
2023-06-29 12:26:24 -05:00
Ben Church
1dabc6208e Metadata: add tags field (#26320)
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-05-26 16:13:09 -07:00
Augustin
fd3655707e connectors-ci: add finalize_build logic to handle custom Dockerfiles (#26489) 2023-05-25 09:32:19 +02:00
Ben Church
248bbf94c1 HACKDAY: Icon CDN (#26158)
* Move icons to connector folder

* Delete old icons

* Update upload logic

* Add icon url to definitions

* Update registry model

* Populate cdn url

* DNC butcher the pipeline

* Low hanging fruit fixes

* Fix bucket name

* Merge old and new approaches

* Fix metadata upload step

* Format

* Fix test
2023-05-24 17:25:41 -07:00
Ben Church
efbdb73e24 Bnchrch/metadata/add missing metadata files (#25899)
* Add source fullstory metadata

* Add source merge metadata

* Add source yotpo metadata

* Update normalization tag

* Update oss diffs

* Add aircall to cloud

* Pin Source sales force

* Ensure strict encrypt matches

* Remove zurora pin

* Make metadata validation mandatory

* Remove top level third party folders
2023-05-08 16:55:11 -05:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254)
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-04-18 22:15:11 +02:00
Ryan Fu
0dcc653bd1 Shuffled destination-jdbc into bases folder (#25120)
* Testing changes for reshuffle

* Resolves dependency/race-condition with base classes and docker creation

* Removes dockerfile

* Updates java connectors gradle dependencies
2023-04-17 12:29:03 -07:00
Augustin
aa43bfc196 fix strict-encrypt metadata dockerRepository (#25159) 2023-04-13 20:48:23 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
Sherif A. Nada
e85eda088e Remove ExtendedNameTrasformer (#23655) 2023-03-07 17:22:08 -08:00
Peter Hu
e5eac0a4cc use published protocol models jar (#22498)
* add airbyte-protocol to deps.toml

* use published protocol jar for platform

* use published protocol jar for connectors

* point at published jar

* fix dep

* bump gcs storage

* fix build failures in standard-source-test

* fix deps

* downgrade alloy db because it is missing strictness tests

* Revert "downgrade alloy db because it is missing strictness tests"

This reverts commit cc6089d053.

---------

Co-authored-by: cgardens <charles@airbyte.io>
2023-02-13 12:50:43 -06:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Augustin
6c2b9a164b destination acceptance tests: get normalization and dbt support from definition (#21299) 2023-01-18 20:00:47 +01:00
Jimmy Ma
094aff1520 Java connectors use protocol v0 objects (#20404)
* Dupl v0 objects

* v0 namespace swap

* Introduce some object conversions in the tests

* Update source-java-jdbc template

* Use v0 objects in db-lib

* Fix more protocol object references

* Fix arg capture in tests
2022-12-15 13:39:50 -08:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
Oleksandr Sheheda
3751be47cb [10719] Destination Oracle: custom JDBC parameters (#13841)
* [10719] Destination Oracle: custom JDBC parameters

* [10719] Destination Oracle: custom JDBC parameters
fixed tests

* [10719] Destination Oracle: custom JDBC parameters
fixed tests

* [10719] Destination Oracle: custom JDBC parameters
fix for SSH oracle tests

* [10719] Destination Oracle: custom JDBC parameters
fixed test

* [10719] Destination Oracle: custom JDBC parameters
updated image tag

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-27 13:37:44 +03:00
Ryan Fu
063b01f51c De-duplicate config parameters (#14781)
* Migrated 'ssl' string to become a constant in JdbcUtils

* Migrated 'database' string to become a constant in JdbcUtils

* Migrated 'username' string to become a constant in JdbcUtils

* Migrated 'password' string to become a constant in JdbcUtils

* Migrated 'schema' string to become a constant in JdbcUtils and updated corresponding config params

* Migrated 'jdbc_url' string to become a constant in JdbcUtils

* Migrated 'connection_properties' string to become a constant in JdbcUtils

* Migrated list of 'host_key' string to become a constant in JdbcUtils

* Migrated list of 'port_key' string to become a constant in JdbcUtils

* Migrated 'host_key' string to become a constant in JdbcUtils and updated corresponding config params

* Migrated 'port_key' string to become a constant in JdbcUtils

* Migrated 'tls' string to become a constant in JdbcUtils

* Migrated 'schemas' string to become a constant in JdbcUtils

* Migrated 'encryption' string to become a constant in JdbcUtils

* Migrated 'jdbc_url_params' string to become a constant in JdbcUtils and fixed uncaught parenthesis

* Fixed linter issues with comments on AcceptanceTest and failed import statement
2022-07-21 16:05:21 -07:00
Greg Solovyev
6673dc4034 Align strict-encrypt connector version with the actual connector (#14775) 2022-07-18 15:39:30 -07:00
VitaliiMaltsev
095b403c7b Remove additionalProperties: false from JDBC destination connectors (#14618)
* Remove additional properties from JDBC destinations spec

* Remove additional properties from JDBC destinations spec

* removed additionalProperties from Databricks destination and MSSql Destination

* updated Changelog

* bump version

* bump Oracle Strict encrypt

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-07-15 01:00:41 +03:00
Jonathan Pearlin
2b31011bce Separate platform and connector testcontainer versions (#13642)
* Separate platform and connector testcontainer versions

* Fix dependency

* Fix dependency

* Fix dependency usage

* Prevent leaking testcontainer dependencies
2022-06-10 09:34:31 -04:00
Greg Solovyev
f0cfaf6b35 Update version in Dockerfile to match docs (#13586)
We already have version 0.1.5 in the docs, but we failed to push the 0.1.5 image to Docker Hub
2022-06-07 19:24:50 -07:00
Davin Chia
83a89aa843 Fat Jar: Rename Dir Part 1 (#13476)
Part 1 of #13122.

Rename airbyte-db:lib to airbyte-db:db-lib.
Rename airbyte-metrics:lib to airbyte-metrics:metrics-lib
Rename airbyte-protocol:models to airbyte-protocol:protocol-models.

Explanation for what is happening:

Identically named subprojects have the following issues:
- publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
- deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
- the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See Projects with same name lead to unintended conflict resolution gradle/gradle#847 for more info.
- given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.
2022-06-06 00:35:43 +08:00
Alexandre Girard
3894134d11 Bump year in license short to 2022 (#13191)
* Bump to 2022

* format
2022-05-25 17:56:49 -07:00
Oleksandr Sheheda
44a43f0338 Improve check connection performance for all JDBC-related DBs (#12820)
* removed redundant read permission check for destination

* bump docker image versions, update definitions and changelog

* bump docker image versions, update definitions and changelog

* update changelog

* fix for MSSQL and Oracle tests

* fix for MSSQL strict encrypt

* fixed conflicts

* revert versions that were not published
2022-05-24 16:06:01 +03:00
Topher Lubaway
013a886f4f Fixes Spotless and runs spotless (#13040)
zipped files with JSON extension made this task sad
2022-05-20 07:26:55 -05:00
Jonathan Pearlin
ebb9f3e1ac Prepare Database Access Layer for Dependency Injection (#12546)
* Prepare database access objects for dependency injection

* Replace duplicate code

* Remove unused imports

* Remove redundant validation call

* Remove unused imports

* Use constants

* Disable fast fail during connection pool initialization

* Remove typo

* Add missing test dependency

* Add missing test dependency

* Add missing test dependency

* Fix issue caused by rebase

* Add method for cloud

* Autoclose DSL context during migration

* Better connection close handling

* Fix typo in dependency

* Fix SpotBugs issue

* React to rebase

* Fix typo

* Update JavaDoc

* Fix database close calls

* Pass configs to getServer

* Fix typo

* Fix call to removed method

* Fix typo

* Use catalog to manage versions

* PR feedback

* Centralize shutdown hook

* Fix rebase issues

* Document test cases

* Document test cases

* Formatting

* Properly close database resources

* Rebase cleanup
2022-05-09 15:26:54 -04:00
LiRen Tu
e15ae56389 Close all unsafe queries (#12495)
* Add helper methods to return lists of json nodes

* Close all unsafe queries

* Add one more helper method

* Simplify helper names

* Format code
2022-05-03 13:45:02 -07:00
LiRen Tu
35f2aa9aed 🎉 Jdbc sources: publish new version with adaptive fetch size (#12480)
* Default scaffold to use adaptive streaming config

* Switch more connectors to use adaptive streaming config

* Bump version for cockroach db

* Bump version for db2

* Bump mssql version

* Bump mysql version

* Bump oracle version

* Bump postgres version

* Bump redshift version

* Bump snowflake version

* Bump tidb version

* auto-bump connector version

* Fix db2 findbug issue

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* Fix more findbug issues

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* Fix findbug issue for mysql-strict-encrypt

* Fix findbugs issue for oracle source

* auto-bump connector version

* Remove suppress warnings annotation

* Fix oracle encrypt tests

* Fix oracle encrypt acceptance test

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-29 23:14:58 -07:00
LiRen Tu
e5ae3b3990 Rename jdbc db methods that require manual closure (#11300)
* Rename jdbc db methods to raise awareness of potential connection leak

* Format code
2022-03-21 15:00:10 -07:00
Subodh Kant Chaturvedi
c14260aa8d close ssh tunnel in case of exception in destination consumer (#10686)
* close ssh tunnel in case of exception

* format
2022-03-02 03:56:32 +05:30
girarda
5d8cb41150 Refactor to enable support for optional JDBC parameters for all JDBC destinations (#10421)
* refactoring to allow testing

* MySQLDestination uses connection property map instead of url arguments

* Update jdbc destinations

* A little more generic

* reset to master

* reset to master

* move to jdbcutils

* Align when multiline

* Align when multiline

* Update postgres to use property map

* Move tests to AbstractJdbcDestinationTest

* clean

* Align when multiline

* return property map

* Add postgres tests

* update clickhouse

* reformat

* reset

* reformat

* fix test

* reformat

* fix bug

* Add mssql tests

* refactor test

* fix oracle destination test

* oracle tests

* fix redshift acceptance test

* Pass string

* Revert "Pass string"

This reverts commit 697821738c.

* Double deserialization

* Revert "Double deserialization"

This reverts commit ee8d75245b.

* try updating json_operations

* Revert "try updating json_operations"

This reverts commit c8022c2994.

* json parse

* Revert "json parse"

This reverts commit 11a6725eaa.

* Revert "Revert "Double deserialization""

This reverts commit 213f47acc4.

* Revert "Revert "Revert "Double deserialization"""

This reverts commit 66822454af.

* move to constant

* Add comment

* map can be constant

* Add comment

* move map

* hide in method

* no need to create new map

* no need to create new map

* no need to create new map

* enably mysql test

* Update changelogs

* Update changelog

* update changelog

* Bump versions

* bump version

* disable dbt support

* update spec

* update other oracle tests

* update doc

* bump seed

* fix source test

* update seed spec file

* fix expected spec
2022-02-28 09:25:04 -08:00