mirror of synced 2026-01-10 09:04:48 -05:00

Files

Davin Chia 83a89aa843 Fat Jar: Rename Dir Part 1 (#13476 )

Part 1 of #13122.

Rename airbyte-db:lib to airbyte-db:db-lib.
Rename airbyte-metrics:lib to airbyte-metrics:metrics-lib
Rename airbyte-protocol:models to airbyte-protocol:protocol-models.

Explanation for what is happening:

Identically named subprojects have the following issues:
- publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
- deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
- the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See Projects with same name lead to unintended conflict resolution gradle/gradle#847 for more info.
- given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.

2022-06-06 00:35:43 +08:00

src

Bump year in license short to 2022 (#13191 )

2022-05-25 17:56:49 -07:00

build.gradle

Fat Jar: Rename Dir Part 1 (#13476 )

2022-06-06 00:35:43 +08:00

Dockerfile

Use multi-stage builds in dockerfiles to reduce java images (#9077 )

2021-12-23 22:27:05 +02:00

README.md

CI: Add action to check broken doc links (#5254 )

2021-08-07 14:28:02 -07:00

README.md

Source Scaffold Java Jdbc

This is the repository for the Scaffold Java Jdbc source connector in Java. For information about how to use this connector within Airbyte, see the User Documentation.

Local development

Building via Gradle

From the Airbyte repository root, run:

./gradlew :airbyte-integrations:connectors:source-scaffold-java-jdbc:build

Create credentials

If you are a community contributor, generate the necessary credentials and place them in secrets/config.json conforming to the spec file in src/main/resources/spec.json. Note that the secrets directory is git-ignored by default, so there is no danger of accidentally checking in sensitive information.

If you are an Airbyte core member, follow the instructions to set up the credentials.

Locally running the connector docker image

Build

Build the connector image via Gradle:

./gradlew :airbyte-integrations:connectors:source-scaffold-java-jdbc:airbyteDocker

When building via Gradle, the docker image name and tag, respectively, are the values of the io.airbyte.name and io.airbyte.version LABELs in the Dockerfile.

Run

Then run any of the connector commands as follows:

docker run --rm airbyte/source-scaffold-java-jdbc:dev spec
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-scaffold-java-jdbc:dev check --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-scaffold-java-jdbc:dev discover --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-scaffold-java-jdbc:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json

Testing

We use JUnit for Java tests.

Unit and Integration Tests

Place unit tests under src/test/... Place integration tests in src/test-integration/...

Acceptance Tests

Airbyte has a standard test suite that all source connectors must pass. Implement the TODOs in src/test-integration/java/io/airbyte/integrations/sources/scaffold_java_jdbcSourceAcceptanceTest.java.

Using gradle to run tests

All commands should be run from airbyte project root. To run unit tests:

./gradlew :airbyte-integrations:connectors:source-scaffold-java-jdbc:unitTest

To run acceptance and custom integration tests:

./gradlew :airbyte-integrations:connectors:source-scaffold-java-jdbc:integrationTest

Dependency Management

Publishing a new version of the connector

You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?

Make sure your changes are passing unit and integration tests.
Bump the connector version in Dockerfile -- just increment the value of the LABEL io.airbyte.version appropriately (we use SemVer).
Create a Pull Request.
Pat yourself on the back for being an awesome contributor.
Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.