1
0
mirror of synced 2026-01-04 09:04:47 -05:00
Commit Graph

28 Commits

Author SHA1 Message Date
Teal Larson
9243c6cc3b fix outdated docs urls in metadata files (#27420)
Co-authored-by: Ben Church <ben@airbyte.io>
2023-06-15 15:45:12 -07:00
Ben Church
1dabc6208e Metadata: add tags field (#26320)
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-05-26 16:13:09 -07:00
Augustin
fd3655707e connectors-ci: add finalize_build logic to handle custom Dockerfiles (#26489) 2023-05-25 09:32:19 +02:00
Ben Church
248bbf94c1 HACKDAY: Icon CDN (#26158)
* Move icons to connector folder

* Delete old icons

* Update upload logic

* Add icon url to definitions

* Update registry model

* Populate cdn url

* DNC butcher the pipeline

* Low hanging fruit fixes

* Fix bucket name

* Merge old and new approaches

* Fix metadata upload step

* Format

* Fix test
2023-05-24 17:25:41 -07:00
terencecho
bab1f2dd0d Fix and unpin destination-dev-null (#26106)
* Fix destination-dev-null

* Fix destination-dev-null part 2

* Bump version

* Fix modify spec

* Update acceptance test

* Revert "Bump version"

This reverts commit c25747c49e.

* Unpin destination-dev-null
2023-05-18 09:47:03 -07:00
Ben Church
58f4f15ed0 Pin metadata to 0.2.7 (#26104) 2023-05-15 19:40:47 -05:00
terencecho
14823becdd Add icon to destination-dev-null (#26094)
Co-authored-by: Ben Church <ben@airbyte.io>
2023-05-15 17:08:39 -07:00
terencecho
780903ff80 🚨🚨 e2e-test: Standardize e2e testing destination spec (#25776)
* Standardize e2e testing destination

* Change casing

* Fix implementation

* Update versions and docs

* Do major version bump

* Update acceptance tests

* use correct name

* Fix implementation

* Automated Change

* Revert "Automated Change"

This reverts commit 816b66229f.

* Automated Change

* Fix unit test

---------

Co-authored-by: terencecho <terencecho@users.noreply.github.com>
2023-05-15 14:01:54 -07:00
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Evan Tahler
b11657ac62 remove ENABLE_SENTRY from connector Dockerfiles (#25442) 2023-04-25 22:13:39 +00:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254)
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-04-18 22:15:11 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
Rodi Reich Zilberman
0bab1756b8 Rename airbyte-config module (#24885)
* rename airbyte-config module

* Automated Commit - Formatting Changes

* sanity

* update import

* update import

* update script

* update script

* update script

* update script

* Automated Change

* Automated Change

* Automated Change

* Automated Change

* update awsdatalake icon

* point slash commands to new path

* sanity

* Automated Commit - Formatting Changes

* sanity

* Automated Change

* Automated Change

* sanity

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-04-06 10:47:30 -07:00
Peter Hu
e5eac0a4cc use published protocol models jar (#22498)
* add airbyte-protocol to deps.toml

* use published protocol jar for platform

* use published protocol jar for connectors

* point at published jar

* fix dep

* bump gcs storage

* fix build failures in standard-source-test

* fix deps

* downgrade alloy db because it is missing strictness tests

* Revert "downgrade alloy db because it is missing strictness tests"

This reverts commit cc6089d053.

---------

Co-authored-by: cgardens <charles@airbyte.io>
2023-02-13 12:50:43 -06:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Jimmy Ma
094aff1520 Java connectors use protocol v0 objects (#20404)
* Dupl v0 objects

* v0 namespace swap

* Introduce some object conversions in the tests

* Update source-java-jdbc template

* Use v0 objects in db-lib

* Fix more protocol object references

* Fix arg capture in tests
2022-12-15 13:39:50 -08:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
George Claireaux
c5783aa799 connectors that published (#13932) 2022-06-20 12:42:56 +01:00
Davin Chia
eb99f47746 Fat Jar: Rename Dir Part 2 (#13478)
## What
Part 2 of https://github.com/airbytehq/airbyte/pull/13122.

Follow up to #13476 .

Explanation for what is happening:

Identically named subprojects have the following issues:

* publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
* deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
* the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See https://github.com/gradle/gradle/issues/847 for more info.
* given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.

* Rename airbyte-config:models to airbyte-config:config-models.
* Rename airbyte-config:persistence to airbyte-config:config-persistence.
2022-06-06 02:21:54 +08:00
Davin Chia
83a89aa843 Fat Jar: Rename Dir Part 1 (#13476)
Part 1 of #13122.

Rename airbyte-db:lib to airbyte-db:db-lib.
Rename airbyte-metrics:lib to airbyte-metrics:metrics-lib
Rename airbyte-protocol:models to airbyte-protocol:protocol-models.

Explanation for what is happening:

Identically named subprojects have the following issues:
- publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
- deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
- the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See Projects with same name lead to unintended conflict resolution gradle/gradle#847 for more info.
- given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.
2022-06-06 00:35:43 +08:00
Alexandre Girard
3894134d11 Bump year in license short to 2022 (#13191)
* Bump to 2022

* format
2022-05-25 17:56:49 -07:00
George Claireaux
f7d3a706b2 Publish connector command now auto bumps version (#11712)
* using yq + more success/failure comments

* added gradle process + git push

* dummy bump to test publish

* dummy bump to test publish

* dummy bump to test publish

* bump version

* fix connector variable

* bump version

* only git add necessary files

* remove git config

* bump version

* making octavia user

* version bump

* auto-bump connector version

* added docs and auto-bumo flag

* bump version

* separate IMAGE_NAME and IMAGE_VERSION env vars from sentry prep

* version bump

* auto-bump connector version

* added entry to apify changelog for clarity

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-05 23:21:55 +01:00
LiRen Tu
81417e6728 Add connector metadata as sentry tags (#10475)
* Pass worker metadata to connector

* Fix compilation

* Pass in job id and image from worker

* Remove application version

* Add default job environment variables

* Add back removed comment

* Rename env map to job metadata

* Fix env configs

* Read connector from application

* Use empty string

* Remove println

* Fix unit test

* Fix compilation error

* Introduce constants for worker env

* Add worker env to ENV_VARS_TO_TRANSFER

* Pass into getWorkerMetadata map to all constructions

* Format code

* Format octavia cli

* Fix test compilation

* Fix typos
2022-03-09 07:36:03 -08:00
Lake Mossman
3d8a0dc048 Add ExitOnOutOfMemoryError to java connectors and bump versions (#10256) 2022-02-14 15:49:15 -08:00
Alexander Tsukanov
479f0d7c8d [MVP] Integrate sentry to all java-based connectors (#9745)
* airbyte-9328: Added Sentry integration to BigQuery and BigQuery denormalized connector.

* airbyte-5050: Added strategy for INSERT ROW.

* airbyte-9328: Added Sentry integration to Snowflake.

* airbyte-9328: Fix Sentry config.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fix PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Fixed PR comments.

* airbyte-9328: Small changes.

* airbyte-9328: Small changes.

* airbyte-9328: Move SENTRY DSN keys to Dockerfiles.

* Use new dsn

* Revert format

* Remove sentry dsn from compose temporarily

* Log sentry event id

* Move sentry to java base

* Remove sentry code from bigquery

* Update dockerfiles

* Fix build

* Update release tag format

* Bump version

* Add env to dockerfiles

* Fix e2e test connector dockerfil

* Fix snowflake bigquery dockerfile

* Mark new versions as unpublished

Co-authored-by: LiRen Tu <tuliren@gmail.com>
Co-authored-by: Liren Tu <tuliren.git@outlook.com>
2022-01-29 16:58:35 -08:00
Serhii Chvaliuk
844dd93122 Use multi-stage builds in dockerfiles to reduce java images (#9077)
* use multi-stage to reduce image size

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2021-12-23 22:27:05 +02:00
LiRen Tu
6ee29b5b82 🐞 Destination e2e test: fix documentation url (#8905)
* Fix documentation url

* Update pr url

* Update doc

* Bump version
2021-12-19 19:34:26 -08:00
LiRen Tu
0de30f5e4d 🎉 Testing destination: multiple logging modes (#8824)
* Implement destination null

* Update existing testing destinations

* Merge in logging consumer

* Remove old destination null

* Add documentation

* Add destination to build and summary

* Fix test

* Update acceptance test

* Log state message

* Remove unused variable

* Remove extra statement

* Remove old null doc

* Add dev null destination

* Update doc to include changelog for dev null

* Format code

* Fix doc

* Register e2e test destination in seed
2021-12-19 00:33:42 -08:00