* Rename to connector test report
* Get all test outputs
* Get last 10 output for all connectors
* Get runs by connector
* Crudely write badge and summary
* Add render functions
* Update connector report badge and link
* Fix url link
* Fix qa report tests
* update qa-engine tests
* Move regex to resource
* Add cron
* Run format
* New line
* Make badge green if last is successful
* Fix test
* Remove report.sh and build_report.py
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
* Adds support for incremental syncs with GLOBAL state message emission
* Adds args to Source to not break, future work is to add same features to Source perf-harness and ./gradlew :spotlessJavaApply
* Adds support for pseudo-parallel datasets
* Ran ./gradlew :spotlessJavaApply
* Automated Change
* Fixes issue with parallel datasets credentials
* Fixes filter for parallel credentials
* Adds a new configurable property to build a pseudo-parallel catalog
* Fixes Github Actions variable to be processed properly with the K8s harness yaml
* Adds unit test for random streams and generating streams within the same configured catalog
* Ran ./gradlew :spotlessJavaApply
* Added additional description for GitHub Actions
* Update connector-performance-command.yml
Moved text up to connect with other argument discussion
* Fixes spotBugs issue
* Automated Commit - Formatting Changes
* Update GitHub Action description
---------
Co-authored-by: ryankfu <ryankfu@users.noreply.github.com>
Co-authored-by: Rodi Reich Zilberman <867491+rodireich@users.noreply.github.com>
* remove arguments
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* Source Performance harness (#23922)
* test
* initial commit
* Working dev
* src-platform-dst
* Integrate secrets
* Update .dockerignore
* Update build.gradle
* Update base.sh
* Build harness module and use it to measure
* Build harness module and use it to measure
* Build harness module and use it to measure
* Automated Change
* Integrate secrets
* Integrate secrets
* Integrate secrets
* Integrate secrets
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* test
* Automated Change
* test
* Automated Change
* better error handling
* temp
* Automated Change
* use connector image name
* use connector image name
* use connector image name
* use connector image name
* use connector image name
* use connector image name
* use connector image name
* use connector image name
* add test name argument
* add test name argument
* add test name argument
* add test name argument
* add test name argument
* add test name argument
* Automated Change
* add bottleneck_stream1 dataset
* Automated Change
* fix dependency
* fix dependency
* test downgrade of commons to 41.1
* Automated Change
* test downgrade of commons to 41.0
* test
* test
* test
* test
* test
* exit condition
* exit condition
* exit condition
* test
* test
* test
* test
* test
* test
* sanity
* readme
* change command name
* update gradle settings
* sanity
* architecture aware docker build
* sanity
* put run harness step in a script
* sanity
* sanity
---------
Co-authored-by: rodireich <rodireich@users.noreply.github.com>
* Updates docusaurus
* Small docs fixes
deploy_docusaurus info messages are changed to
reflect reality
updates a secret to exist
* Go back to the right branch after deploy
dev runs of this pr can be found here: https://github.com/airbytehq/airbyte/actions/workflows/release-airbyte-os.yml
I've tested the `create_release` step for both `airbyte` and `airbyte-platform`.
What I haven't tested are:
- publishing docker images step for airbyte-platform
- the auto commit steps for both.
What also should be noted: the artifacts from the `run-ab-platform.sh` script need to be commited into the repo for them to be included in the release.
* delete platform code and workflows
* delete more platform workflows
* sunset kustomize (#23187)
* sunset kustomize
* remove build steps that use kustomize
* remove k8s kustomize gke-kube-acceptance-tests
* delete docker-compose files
* remove platform from check_images_exist.sh
* Working replacement script for docker
* rename start script and add download files to gitignore
* add args and download only option
* integration test downloads docker compose files
* remove helm charts
* remove more platform files
* add flags.yml
* use version from gradle.properties instead of .env
* Working replacement script for docker (#23227)
* Working replacement script for docker
* Working debugging line
* rename start script and add download files to gitignore
* add args and download only option
* Made flags cleaner and more cleanup
i've never used this flag pattern, its much
easier than my usual way
* warn about a sharp edge
* Don't run when we don't know a command
* better help spacing
* add flags.yml
---------
Co-authored-by: cpdeethree <conor@airbyte.io>
Co-authored-by: Conor <cpdeethree@users.noreply.github.com>
* remove more-secrets env
---------
Co-authored-by: Charles <charles@airbyte.io>
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
* Refactor the job log json to include the docker_version
* Output to versioned folder
* Handle the case where people call the action without connector prefixed
* Retrieve status of each connector
* Use build report statuses in the QA Engine
* Cast build status as an enum
* Add Airbyte Protocol V1 support.
* Fix VersionedAirbyteStreamFactoryTest
* Remove AirbyteMessageMigrationV0 example
* Add Protocol Version constants
* 🎉Updated normalization to handle new datatypes (#19721)
* Updated normalization simple stream processing to handle new datatypes
* Updated normalization nested stream processing to handle new datatypes
* Updated normalization nested stream processing to handle new datatypes
* Updated normalization drop_scd_catalog processing to handle new datatypes
* Updated normalization ephemeral test processing to handle new datatypes
* fixed more tests for normalization
* fixed more tests for normalization
* fixed more tests for normalization
* fixed more tests for normalization
* fixed more issues
* fixed more issues (clickhouse)
* fixed more issues
* fixed more issues
* fixed more issues
* added binary type processing for some DBs
* cleared commented code and moved some hardcodes to processing as macro
* fixed codestyle and cleared commented code
* minor refactor
* minor refactor
* minor refactor
* fixed bool cast error
* fixed dict->str cast error
* fixed is_combining_node cast py check
* removed commented code
* removed commented code
* committed autogenerated normalization_test_output files
* committed autogenerated normalization_test_output files (new files)
* refactored utils.py
* Updated utils.py to use Callable functions and get rid of property_type in is_number and is_bool functions
* committed autogenerated normalization_test_output files (new files)
* fixed typo in TIMESTAMP_WITH_TIMEZONE_TYPE
* updated stream_processor to handle string type first as a wider type
* fixed arrays normalization by updating is_simple_property method as per new approaches
* format
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
* Update airbyte protocol migration (#20745)
* Extract MigrationContainer from AirbyteMessageMigrator
* Add ConfiguredAirbyteCatalogMigrations
* Add ConfiguredAirbyteCatalog to AirbyteMessageMigrations
* Enable ConfiguredAirbyteCatalog migration
* Fix tests
* Remove extra this.
* Add missing docs
* Typo
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
* Data types update: Implement protocol message migrations (#19240)
* Extract MigrationContainer from AirbyteMessageMigrator
* Add ConfiguredAirbyteCatalogMigrations
* Add ConfiguredAirbyteCatalog to AirbyteMessageMigrations
* Enable ConfiguredAirbyteCatalog migration
* set up scaffolding
* [wip] more scaffolding, basic unit test
* minimal green code
* [wip] add failing test for other primitive types
* correct version number
* handle basic primitive type decls
* add implicit cases
* add recursive schema
* formatting
* comment
* support not
* fix indentation
* handle all nested schema cases
* handle boolean schemas
* verify empty schema handling
* cleanup
* extract map
* code organization
* extract method
* reformat
* [wip] more tests, minor fix type array handling
* corrected test
* cleanup
* reformat
* switch to v1
* add support for multityped fields
* missed test case
* nested test class
* basic record upgrade
* implement record upgrades
* slight refactor
* comments+clarificationso
* extract constants
* (partly) correct model classes
* add de/ser
* formatting
* extract constants
* fix json reference
* update docs
* switch to v1 models
* fix compile+test
* add base64 handling
* use vnull
* Data types update: Implement protocol message downgrade path (#19909)
* rough skeleton for passing catalog into migration
* basic test
* more scaffolding
* basic implementation
* add primitives test
* add in other tests (nested fields currently failing)
* add formats
* impleent oneOf handling
* formatting
* oneOf handling
* better tests
* comments + organization
* progress
* basic test case
* downgrade objects, ish
* basic array implementation
* handle numeric failure
* test for new type
* handle array items
* empty schema handling
* first pass at oneof handling
* add more tests+handling
* more tests
* comments
* add empty oneof test case
* format + reorganize
* more reorganize
* fix name
* also downgrade binary data
* only import vnull
* move migrations into v1 package
* extract schema mutation code
* comment
* extract schema migration to new class
* extract record downgrade logic for future use
* format
* fix build after rebase
* rename private method for consistency
* also implement configuredcatalog migrations >.>
* quick and dirty tests
* slight cleanup
* fix tests
* pmd
* pmd test
* null check on message objects
* maybe fix acceptance tests?
* fix name
* extract constants
* more fixes
* tmp
* meh
* fix cdc acc tests
* revert to master source-postgres
* remove log messages
* revert other misc hacks
* integers are valid cursors
* remove unrelated change
* fix build
* fix build more?
* [MUST REVERT] use dev normalization
* capture kube logs
* also here?
* no debug logs?
* delete dup from merging
* add final everywhere
* revert test changes
Co-authored-by: Jimmy Ma <jimmy@airbyte.io>
* On-the-fly migrations of persisted catalogs (#21757)
* On the fly catalog migration for normalization activity
* On the fly catalog migration for job persistence
* On the fly migration for standard sync persistence
* On the fly migration for airbyte catalogs
* Refactor code to share JsonSchema traversal
* Add V0 Data type search function
* PMD and Format
* Fix getOrInsertActorCatalog and ConfigRepositoryE2E tests
* Null-proofing CatalogMigrationV1Helper
* More null checks
* Fix test
* Format
* Add data type v1 support to the FE
* Changes AC test check to check exited ps (#21672)
some docker compose changes no longer show exited
processes. this broke out test
this change should fix master
tested in a runner that failed
* Move wellknown types mapping to the utility function
* use protocolv1 normalization
---------
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
* Update protocol support range (#21996)
* bump normalization version to 0.3.0
* Add version check on normalization (#22048)
* Add normalization min version check
* Add visible for testing
---------
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Eugene <etsybaev@gmail.com>
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
