3 Commits

Author	SHA1	Message	Date
Ryan Fu	8849669420	Destination Performance Harness: Adds support for incremental syncs with GLOBAL state message emission (#26443) ... * Adds support for incremental syncs with GLOBAL state message emission * Adds args to Source to not break, future work is to add same features to Source perf-harness and ./gradlew :spotlessJavaApply	2023-05-23 15:13:31 -07:00
Ryan Fu	3689ff3abb	Performance Harness: Support for pseudo-parallel streams (#26219) ... * Adds support for pseudo-parallel datasets * Ran ./gradlew :spotlessJavaApply * Automated Change * Fixes issue with parallel datasets credentials * Fixes filter for parallel credentials * Adds a new configurable property to build a pseudo-parallel catalog * Fixes Github Actions variable to be processed properly with the K8s harness yaml * Adds unit test for random streams and generating streams within the same configured catalog * Ran ./gradlew :spotlessJavaApply * Added additional description for GitHub Actions * Update connector-performance-command.yml Moved text up to connect with other argument discussion * Fixes spotBugs issue * Automated Commit - Formatting Changes * Update GitHub Action description --------- Co-authored-by: ryankfu <ryankfu@users.noreply.github.com> Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>	2023-05-22 14:13:38 -05:00
Rodi Reich Zilberman	6374f4780f	destination harness (#25030) ... * destination harness * Automated Commit - Formatting Changes * refactor to support multiple connectors * better byte counting * Automated Change * fix main cp * harness type * harness type * harness type * harness type * harness type * harness type * harness type * speed up harness build * harness type * harness type * harness type * harness type * 1m catalog * 1m catalog * Automated Commit - Formatting Changes * 1m catalog * Automated Commit - Formatting Changes * 1m catalog * Automated Commit - Formatting Changes * 1m catalog * 1m catalog * Automated Commit - Formatting Changes * 1m catalog * Automated Commit - Formatting Changes * Automated Change * add some additional logging for debugging * test * test * debugging code * Automated Commit - Formatting Changes * debugging code * debugging code * debugging code * Automated Commit - Formatting Changes * Checkpoint: This is working!. * Automated Commit - Formatting Changes * Updates plumbing for Snowflake with static records - Working * Automated Commit - Formatting Changes * working 10m * Automated Commit - Formatting Changes * add destination datasource * Automated Commit - Formatting Changes * sanity * sanity * Delete 1m_catalog.json * sanity * Update airbyte-integrations/connectors-performance/destination-harness/README.md Co-authored-by: Davin Chia <davinchia@gmail.com> * sanity * Update airbyte-integrations/connectors/destination-e2e-test/src/main/java/io/airbyte/integrations/destination/e2e_test/TestingDestinations.java Co-authored-by: Davin Chia <davinchia@gmail.com> * Splits performance harness into separated methods * Automated Commit - Formatting Changes * Updates test description to force Github builds --------- Co-authored-by: rodireich <rodireich@users.noreply.github.com> Co-authored-by: Jeff Cowan (Airbyte) <4992320+jcowanpdx@users.noreply.github.com> Co-authored-by: Davin Chia <davinchia@gmail.com> Co-authored-by: davinchia <davinchia@users.noreply.github.com> Co-authored-by: ryankfu <ryan.fu@airbyte.io> Co-authored-by: ryankfu <ryankfu@users.noreply.github.com>	2023-05-16 06:37:00 +00:00