1
0
mirror of synced 2025-12-20 02:23:30 -05:00
Commit Graph

28 Commits

Author SHA1 Message Date
Natik Gadzhi
cb80e6922a [tools] prettier rules for .md + formatting cleanup 2024-05-07 08:19:33 -07:00
Stephane Geneix
75ac5a98b4 remove all dependencies to lombok (#37627)
We only use lombok for super trivial stuff and it tends to not play well with Micronauts. Lets' get rid of it!
2024-04-26 12:01:29 -07:00
Stephane Geneix
6c8ca12e4c enable spotbugs for CDK core and dependencies submodule (#36612) 2024-04-03 11:43:30 -07:00
Marius Posta
2495575795 java-cdk: re-export airbyte-api dependency (#36759) 2024-04-03 10:43:05 -07:00
Marius Posta
dc088bc3a9 gradle: split off python cdk (#35306) 2024-02-16 04:39:14 -08:00
Marius Posta
796b2e8dad java CDK: clean up dependencies, refactor modules (#34745) 2024-02-08 19:46:51 -06:00
Stephane Geneix
49043b570f 🐛 📝 re-update all connectors for new CDK version and more stringent javac checks (#32125) 2023-11-08 18:21:14 -04:00
Stephane Geneix
9c6ed93a9d Revert "update all connectors for new CDK version and more stringent javac checks"
This reverts commit 18ad87bbd1.
2023-11-02 15:42:28 -07:00
Stephane Geneix
18ad87bbd1 update all connectors for new CDK version and more stringent javac checks 2023-11-02 15:24:58 -07:00
Marius Posta
7cd8020ac8 java CDK: hoist top-level gradle projects into CDK (#31960)
Co-authored-by: postamar <postamar@users.noreply.github.com>
2023-10-30 12:03:06 -07:00
Xiaohan Song
3624c865d8 Support mongoDB in speed perf harness (#31633) 2023-10-20 10:30:12 -07:00
Xiaohan Song
99b435ff71 Support incremental sync in performance harness tool (#31588) 2023-10-20 10:29:52 -07:00
Xiaohan Song
a946a3840b Integrate source performance harness with datadog (#31410)
Co-authored-by: xiaohansong <xiaohansong@users.noreply.github.com>
2023-10-17 09:57:24 -07:00
Xiaohan Song
bc670da427 Fix connector harness performance ci run (#31304)
Co-authored-by: xiaohansong <xiaohansong@users.noreply.github.com>
2023-10-13 22:02:10 +00:00
Marius Posta
f8edc18039 airbyte-ci,gradle: replace airbyte-docker with airbyte-ci (#30743) 2023-10-04 08:38:17 -07:00
Marius Posta
7ae97175a6 gradle: fix repo wide behaviour (#30607) 2023-09-28 05:01:13 -07:00
Aaron ("AJ") Steers
da32fc86bf 🚀 Launch Java CDK v0.1.0 (PR #30383)
- Launch Java CDK `v0.1.0`. 🚀
- Remove `airbyte-docker` gradle plugin from java connectors.
- Add `airbyte-java-connector` gradle plugin.
- Remove `airbyte-commons` from global build.gradle.
- Add `/publish-java-cdk` slash command.
- Initial launch of `/airbyte-ci` slash command to test connectors.
2023-09-27 15:46:02 -07:00
Marius Posta
51c67d7eaa gradle: remove airbyteDocker.outputs dependencies (#30314) 2023-09-11 17:16:27 -07:00
Marius Posta
ef2849e35e gradle: fix airbyteDocker task inputs (#30187) 2023-09-07 03:46:31 -07:00
Charles
480a43b9ab grab bag of non-controversial clean up tasks (#26702) 2023-05-31 08:53:27 -07:00
Ryan Fu
c87ffb3149 Adds the bottleneck datasources and respective catalogs and updated description for the harness (#26503)
* Adds the bottleneck datasources and respective catalogs and updated description for the harness

* Automated Commit - Formatting Changes

---------

Co-authored-by: ryankfu <ryankfu@users.noreply.github.com>
2023-05-24 12:17:34 -05:00
Ryan Fu
8849669420 Destination Performance Harness: Adds support for incremental syncs with GLOBAL state message emission (#26443)
* Adds support for incremental syncs with GLOBAL state message emission

* Adds args to Source to not break, future work is to add same features to Source perf-harness and ./gradlew :spotlessJavaApply
2023-05-23 15:13:31 -07:00
Ryan Fu
3689ff3abb Performance Harness: Support for pseudo-parallel streams (#26219)
* Adds support for pseudo-parallel datasets

* Ran ./gradlew :spotlessJavaApply

* Automated Change

* Fixes issue with parallel datasets credentials

* Fixes filter for parallel credentials

* Adds a new configurable property  to build a pseudo-parallel catalog

* Fixes Github Actions variable to be processed properly with the K8s harness yaml

* Adds unit test for random streams and generating streams within the same configured catalog

* Ran ./gradlew :spotlessJavaApply

* Added additional description for GitHub Actions

* Update connector-performance-command.yml

Moved text up to connect with other argument discussion

* Fixes spotBugs issue

* Automated Commit - Formatting Changes

* Update GitHub Action description

---------

Co-authored-by: ryankfu <ryankfu@users.noreply.github.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
2023-05-22 14:13:38 -05:00
Rodi Reich Zilberman
6374f4780f destination harness (#25030)
* destination harness

* Automated Commit - Formatting Changes

* refactor to support multiple connectors

* better byte counting

* Automated Change

* fix main cp

* harness type

* harness type

* harness type

* harness type

* harness type

* harness type

* harness type

* speed up harness build

* harness type

* harness type

* harness type

* harness type

* 1m catalog

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* Automated Commit - Formatting Changes

* Automated Change

* add some additional logging for debugging

* test

* test

* debugging code

* Automated Commit - Formatting Changes

* debugging code

* debugging code

* debugging code

* Automated Commit - Formatting Changes

* Checkpoint: This is working!.

* Automated Commit - Formatting Changes

* Updates plumbing for Snowflake with static records - Working

* Automated Commit - Formatting Changes

* working 10m

* Automated Commit - Formatting Changes

* add destination datasource

* Automated Commit - Formatting Changes

* sanity

* sanity

* Delete 1m_catalog.json

* sanity

* Update airbyte-integrations/connectors-performance/destination-harness/README.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* sanity

* Update airbyte-integrations/connectors/destination-e2e-test/src/main/java/io/airbyte/integrations/destination/e2e_test/TestingDestinations.java

Co-authored-by: Davin Chia <davinchia@gmail.com>

* Splits performance harness into separated methods

* Automated Commit - Formatting Changes

* Updates test description to force Github builds

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
Co-authored-by: Jeff Cowan (Airbyte) <4992320+jcowanpdx@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: davinchia <davinchia@users.noreply.github.com>
Co-authored-by: ryankfu <ryan.fu@airbyte.io>
Co-authored-by: ryankfu <ryankfu@users.noreply.github.com>
2023-05-16 06:37:00 +00:00
Rodi Reich Zilberman
5ff0cfd588 refactor source harness and GitHub action to add source-mysql (#25036)
* refactor to add source-mysql

* refactor to add source-mysql

* refactor to add source-mysql

* test

* Automated Change

* fix if condition

* try multiline

* new db

* add mysql catalogs

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-04-13 13:47:20 -07:00
Rodi Reich Zilberman
32b9760cff Update seeds in new location under config-oss (#24966)
* update seeds in new location under config-oss

* Automated Change

* force change

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-04-06 13:27:04 -05:00
Rodi Reich Zilberman
0bab1756b8 Rename airbyte-config module (#24885)
* rename airbyte-config module

* Automated Commit - Formatting Changes

* sanity

* update import

* update import

* update script

* update script

* update script

* update script

* Automated Change

* Automated Change

* Automated Change

* Automated Change

* update awsdatalake icon

* point slash commands to new path

* sanity

* Automated Commit - Formatting Changes

* sanity

* Automated Change

* Automated Change

* sanity

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-04-06 10:47:30 -07:00
Rodi Reich Zilberman
0a38d4e1ca Source connector performance harness (#23786)
* remove arguments

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* Source Performance harness (#23922)

* test

* initial commit

* Working dev

* src-platform-dst

* Integrate secrets

* Update .dockerignore

* Update build.gradle

* Update base.sh

* Build harness module and use it to measure

* Build harness module and use it to measure

* Build harness module and use it to measure

* Automated Change

* Integrate secrets

* Integrate secrets

* Integrate secrets

* Integrate secrets

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* Automated Change

* test

* Automated Change

* better error handling

* temp

* Automated Change

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* add test name argument

* add test name argument

* add test name argument

* add test name argument

* add test name argument

* add test name argument

* Automated Change

* add bottleneck_stream1 dataset

* Automated Change

* fix dependency

* fix dependency

* test downgrade of commons to 41.1

* Automated Change

* test downgrade of commons to 41.0

* test

* test

* test

* test

* test

* exit condition

* exit condition

* exit condition

* test

* test

* test

* test

* test

* test

* sanity

* readme

* change command name

* update gradle settings

* sanity

* architecture aware docker build

* sanity

* put run harness step in a script

* sanity

* sanity

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-03-30 10:26:31 -07:00