1
0
mirror of synced 2025-12-23 21:03:15 -05:00
Commit Graph

10 Commits

Author SHA1 Message Date
Stephane Geneix
75ac5a98b4 remove all dependencies to lombok (#37627)
We only use lombok for super trivial stuff and it tends to not play well with Micronauts. Lets' get rid of it!
2024-04-26 12:01:29 -07:00
Xiaohan Song
3624c865d8 Support mongoDB in speed perf harness (#31633) 2023-10-20 10:30:12 -07:00
Xiaohan Song
99b435ff71 Support incremental sync in performance harness tool (#31588) 2023-10-20 10:29:52 -07:00
Xiaohan Song
a946a3840b Integrate source performance harness with datadog (#31410)
Co-authored-by: xiaohansong <xiaohansong@users.noreply.github.com>
2023-10-17 09:57:24 -07:00
Xiaohan Song
bc670da427 Fix connector harness performance ci run (#31304)
Co-authored-by: xiaohansong <xiaohansong@users.noreply.github.com>
2023-10-13 22:02:10 +00:00
Ryan Fu
8849669420 Destination Performance Harness: Adds support for incremental syncs with GLOBAL state message emission (#26443)
* Adds support for incremental syncs with GLOBAL state message emission

* Adds args to Source to not break, future work is to add same features to Source perf-harness and ./gradlew :spotlessJavaApply
2023-05-23 15:13:31 -07:00
Ryan Fu
3689ff3abb Performance Harness: Support for pseudo-parallel streams (#26219)
* Adds support for pseudo-parallel datasets

* Ran ./gradlew :spotlessJavaApply

* Automated Change

* Fixes issue with parallel datasets credentials

* Fixes filter for parallel credentials

* Adds a new configurable property  to build a pseudo-parallel catalog

* Fixes Github Actions variable to be processed properly with the K8s harness yaml

* Adds unit test for random streams and generating streams within the same configured catalog

* Ran ./gradlew :spotlessJavaApply

* Added additional description for GitHub Actions

* Update connector-performance-command.yml

Moved text up to connect with other argument discussion

* Fixes spotBugs issue

* Automated Commit - Formatting Changes

* Update GitHub Action description

---------

Co-authored-by: ryankfu <ryankfu@users.noreply.github.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
2023-05-22 14:13:38 -05:00
Rodi Reich Zilberman
6374f4780f destination harness (#25030)
* destination harness

* Automated Commit - Formatting Changes

* refactor to support multiple connectors

* better byte counting

* Automated Change

* fix main cp

* harness type

* harness type

* harness type

* harness type

* harness type

* harness type

* harness type

* speed up harness build

* harness type

* harness type

* harness type

* harness type

* 1m catalog

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* 1m catalog

* Automated Commit - Formatting Changes

* 1m catalog

* Automated Commit - Formatting Changes

* Automated Change

* add some additional logging for debugging

* test

* test

* debugging code

* Automated Commit - Formatting Changes

* debugging code

* debugging code

* debugging code

* Automated Commit - Formatting Changes

* Checkpoint: This is working!.

* Automated Commit - Formatting Changes

* Updates plumbing for Snowflake with static records - Working

* Automated Commit - Formatting Changes

* working 10m

* Automated Commit - Formatting Changes

* add destination datasource

* Automated Commit - Formatting Changes

* sanity

* sanity

* Delete 1m_catalog.json

* sanity

* Update airbyte-integrations/connectors-performance/destination-harness/README.md

Co-authored-by: Davin Chia <davinchia@gmail.com>

* sanity

* Update airbyte-integrations/connectors/destination-e2e-test/src/main/java/io/airbyte/integrations/destination/e2e_test/TestingDestinations.java

Co-authored-by: Davin Chia <davinchia@gmail.com>

* Splits performance harness into separated methods

* Automated Commit - Formatting Changes

* Updates test description to force Github builds

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
Co-authored-by: Jeff Cowan (Airbyte) <4992320+jcowanpdx@users.noreply.github.com>
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: davinchia <davinchia@users.noreply.github.com>
Co-authored-by: ryankfu <ryan.fu@airbyte.io>
Co-authored-by: ryankfu <ryankfu@users.noreply.github.com>
2023-05-16 06:37:00 +00:00
Rodi Reich Zilberman
5ff0cfd588 refactor source harness and GitHub action to add source-mysql (#25036)
* refactor to add source-mysql

* refactor to add source-mysql

* refactor to add source-mysql

* test

* Automated Change

* fix if condition

* try multiline

* new db

* add mysql catalogs

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-04-13 13:47:20 -07:00
Rodi Reich Zilberman
0a38d4e1ca Source connector performance harness (#23786)
* remove arguments

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* Source Performance harness (#23922)

* test

* initial commit

* Working dev

* src-platform-dst

* Integrate secrets

* Update .dockerignore

* Update build.gradle

* Update base.sh

* Build harness module and use it to measure

* Build harness module and use it to measure

* Build harness module and use it to measure

* Automated Change

* Integrate secrets

* Integrate secrets

* Integrate secrets

* Integrate secrets

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* test

* Automated Change

* test

* Automated Change

* better error handling

* temp

* Automated Change

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* use connector image name

* add test name argument

* add test name argument

* add test name argument

* add test name argument

* add test name argument

* add test name argument

* Automated Change

* add bottleneck_stream1 dataset

* Automated Change

* fix dependency

* fix dependency

* test downgrade of commons to 41.1

* Automated Change

* test downgrade of commons to 41.0

* test

* test

* test

* test

* test

* exit condition

* exit condition

* exit condition

* test

* test

* test

* test

* test

* test

* sanity

* readme

* change command name

* update gradle settings

* sanity

* architecture aware docker build

* sanity

* put run harness step in a script

* sanity

* sanity

---------

Co-authored-by: rodireich <rodireich@users.noreply.github.com>
2023-03-30 10:26:31 -07:00