1
0
mirror of synced 2026-01-10 09:04:48 -05:00
Commit Graph

13 Commits

Author SHA1 Message Date
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
George Claireaux
c5783aa799 connectors that published (#13932) 2022-06-20 12:42:56 +01:00
Davin Chia
eb99f47746 Fat Jar: Rename Dir Part 2 (#13478)
## What
Part 2 of https://github.com/airbytehq/airbyte/pull/13122.

Follow up to #13476 .

Explanation for what is happening:

Identically named subprojects have the following issues:

* publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
* deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
* the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See https://github.com/gradle/gradle/issues/847 for more info.
* given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.

* Rename airbyte-config:models to airbyte-config:config-models.
* Rename airbyte-config:persistence to airbyte-config:config-persistence.
2022-06-06 02:21:54 +08:00
Davin Chia
83a89aa843 Fat Jar: Rename Dir Part 1 (#13476)
Part 1 of #13122.

Rename airbyte-db:lib to airbyte-db:db-lib.
Rename airbyte-metrics:lib to airbyte-metrics:metrics-lib
Rename airbyte-protocol:models to airbyte-protocol:protocol-models.

Explanation for what is happening:

Identically named subprojects have the following issues:
- publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
- deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
- the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See Projects with same name lead to unintended conflict resolution gradle/gradle#847 for more info.
- given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.
2022-06-06 00:35:43 +08:00
Alexandre Girard
3894134d11 Bump year in license short to 2022 (#13191)
* Bump to 2022

* format
2022-05-25 17:56:49 -07:00
LiRen Tu
0c909ded6a Switch json libraries to jitpack (#12531)
* Switch json-avro-converter to jitpack

* Switch jsongenerator to jitpack

* Fix default config

* Fix one more default config use case

* Fix jitpack dependency

* Move jitpack repo to root build.gradle
2022-05-03 17:45:57 -07:00
LiRen Tu
81417e6728 Add connector metadata as sentry tags (#10475)
* Pass worker metadata to connector

* Fix compilation

* Pass in job id and image from worker

* Remove application version

* Add default job environment variables

* Add back removed comment

* Rename env map to job metadata

* Fix env configs

* Read connector from application

* Use empty string

* Remove println

* Fix unit test

* Fix compilation error

* Introduce constants for worker env

* Add worker env to ENV_VARS_TO_TRANSFER

* Pass into getWorkerMetadata map to all constructions

* Format code

* Format octavia cli

* Fix test compilation

* Fix typos
2022-03-09 07:36:03 -08:00
LiRen Tu
39049cbf24 🎉 End-to-end test source: support stream duplication (#10298)
* Add support to duplicate one stream multiple times

* Update cloud version

* Bump version

* Remove unused logger

* Add trace for each stream in mock source

* Fix cloud version

* Bump version in seed
2022-02-12 00:50:48 -08:00
LiRen Tu
8518767f81 🐛 Mock source: fix schema validation and json generation (#9954)
* Remove legacy modes

* Use patched json generator library

* Update doc

* Fix unit test

* Resolve merge conflict

* Add unit tests for the generator

* Bump version

* Go back to master branch

* Fix cloud version

* Format code

* Bump version in seed
2022-02-01 03:31:51 -08:00
Alexander Tsukanov
479f0d7c8d [MVP] Integrate sentry to all java-based connectors (#9745)
* airbyte-9328: Added Sentry integration to BigQuery and BigQuery denormalized connector.

* airbyte-5050: Added strategy for INSERT ROW.

* airbyte-9328: Added Sentry integration to Snowflake.

* airbyte-9328: Fix Sentry config.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fix PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Small changes.

* airbyte-9328: Small changes.

* airbyte-9328: Move SENTRY DSN keys to Dockerfiles.

* Use new dsn

* Revert format

* Remove sentry dsn from compose temporarily

* Log sentry event id

* Move sentry to java base

* Remove sentry code from bigquery

* Update dockerfiles

* Fix build

* Update release tag format

* Bump version

* Add env to dockerfiles

* Fix e2e test connector dockerfil

* Fix snowflake bigquery dockerfile

* Mark new versions as unpublished

Co-authored-by: LiRen Tu <tuliren@gmail.com>
Co-authored-by: Liren Tu <tuliren.git@outlook.com>
2022-01-29 16:58:35 -08:00
LiRen Tu
e4661fb92a Remove regex check from Java source acceptance test (#9829)
* Move getRegexTests to python source acceptance test

* Remove unused imports

* Update test template
2022-01-26 17:51:37 -08:00
LiRen Tu
d0b226bb1b Address review comments for mock source (#9773)
* Address review comments

* Add comments

* Format code

* Update user documentation
2022-01-25 13:39:28 -08:00
LiRen Tu
b269b9fb1e 🎉 Source e2e test: support custom catalog (#9720)
* Add continuous feed mode to source e2e-test

* Update connector catalog doc

* Fix sonar qube issues

* Add cloud variant

* Format code

* Add testing source connector to seed
2022-01-23 21:09:03 -08:00