* Add instructions to install reqs and run tests locally
* add passing tests with old behavior
* Make it like test strictness level
* Remove expected teams from non-tracked change test since they are not asserted against
* Make changed GA connector list object same object format as regex checker to be able to compare them. Fix tests
* Refactor and align naming
* Update all find methods to return Connector objects
* Bump setup.py
* Updates docusaurus
* Small docs fixes
deploy_docusaurus info messages are changed to
reflect reality
updates a secret to exist
* Go back to the right branch after deploy
dev runs of this pr can be found here: https://github.com/airbytehq/airbyte/actions/workflows/release-airbyte-os.yml
I've tested the `create_release` step for both `airbyte` and `airbyte-platform`.
What I haven't tested are:
- publishing docker images step for airbyte-platform
- the auto commit steps for both.
What also should be noted: the artifacts from the `run-ab-platform.sh` script need to be commited into the repo for them to be included in the release.
* delete platform code and workflows
* delete more platform workflows
* sunset kustomize (#23187)
* sunset kustomize
* remove build steps that use kustomize
* remove k8s kustomize gke-kube-acceptance-tests
* delete docker-compose files
* remove platform from check_images_exist.sh
* Working replacement script for docker
* rename start script and add download files to gitignore
* add args and download only option
* integration test downloads docker compose files
* remove helm charts
* remove more platform files
* add flags.yml
* use version from gradle.properties instead of .env
* Working replacement script for docker (#23227)
* Working replacement script for docker
* Working debugging line
* rename start script and add download files to gitignore
* add args and download only option
* Made flags cleaner and more cleanup
i've never used this flag pattern, its much
easier than my usual way
* warn about a sharp edge
* Don't run when we don't know a command
* better help spacing
* add flags.yml
---------
Co-authored-by: cpdeethree <conor@airbyte.io>
Co-authored-by: Conor <cpdeethree@users.noreply.github.com>
* remove more-secrets env
---------
Co-authored-by: Charles <charles@airbyte.io>
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* cloud-availability-updater: enable it
* tools changes only
* normalize line to lower
* improve url prefix ignore
* more robust tests
* ignore acceptance_tests_logs
* qa-checks: validate the connector code does not contain http:// urls
* qa-checks: validate the connector code does not contain http:// urls
* Refactor the job log json to include the docker_version
* Output to versioned folder
* Handle the case where people call the action without connector prefixed
* Retrieve status of each connector
* Use build report statuses in the QA Engine
* Cast build status as an enum
* Github Action to enforce allowed hosts are set for GA connectors
* include beta connectors too
* fix keys and add github
* Update tools/ci_connector_ops/ci_connector_ops/allowed_hosts_checks.py
Co-authored-by: Augustin <augustin@airbyte.io>
* remove comments
* bump version
* add docs
---------
Co-authored-by: Augustin <augustin@airbyte.io>
* Add Airbyte Protocol V1 support.
* Fix VersionedAirbyteStreamFactoryTest
* Remove AirbyteMessageMigrationV0 example
* Add Protocol Version constants
* 🎉Updated normalization to handle new datatypes (#19721)
* Updated normalization simple stream processing to handle new datatypes
* Updated normalization nested stream processing to handle new datatypes
* Updated normalization nested stream processing to handle new datatypes
* Updated normalization drop_scd_catalog processing to handle new datatypes
* Updated normalization ephemeral test processing to handle new datatypes
* fixed more tests for normalization
* fixed more tests for normalization
* fixed more tests for normalization
* fixed more tests for normalization
* fixed more issues
* fixed more issues (clickhouse)
* fixed more issues
* fixed more issues
* fixed more issues
* added binary type processing for some DBs
* cleared commented code and moved some hardcodes to processing as macro
* fixed codestyle and cleared commented code
* minor refactor
* minor refactor
* minor refactor
* fixed bool cast error
* fixed dict->str cast error
* fixed is_combining_node cast py check
* removed commented code
* removed commented code
* committed autogenerated normalization_test_output files
* committed autogenerated normalization_test_output files (new files)
* refactored utils.py
* Updated utils.py to use Callable functions and get rid of property_type in is_number and is_bool functions
* committed autogenerated normalization_test_output files (new files)
* fixed typo in TIMESTAMP_WITH_TIMEZONE_TYPE
* updated stream_processor to handle string type first as a wider type
* fixed arrays normalization by updating is_simple_property method as per new approaches
* format
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
* Update airbyte protocol migration (#20745)
* Extract MigrationContainer from AirbyteMessageMigrator
* Add ConfiguredAirbyteCatalogMigrations
* Add ConfiguredAirbyteCatalog to AirbyteMessageMigrations
* Enable ConfiguredAirbyteCatalog migration
* Fix tests
* Remove extra this.
* Add missing docs
* Typo
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
* Data types update: Implement protocol message migrations (#19240)
* Extract MigrationContainer from AirbyteMessageMigrator
* Add ConfiguredAirbyteCatalogMigrations
* Add ConfiguredAirbyteCatalog to AirbyteMessageMigrations
* Enable ConfiguredAirbyteCatalog migration
* set up scaffolding
* [wip] more scaffolding, basic unit test
* minimal green code
* [wip] add failing test for other primitive types
* correct version number
* handle basic primitive type decls
* add implicit cases
* add recursive schema
* formatting
* comment
* support not
* fix indentation
* handle all nested schema cases
* handle boolean schemas
* verify empty schema handling
* cleanup
* extract map
* code organization
* extract method
* reformat
* [wip] more tests, minor fix type array handling
* corrected test
* cleanup
* reformat
* switch to v1
* add support for multityped fields
* missed test case
* nested test class
* basic record upgrade
* implement record upgrades
* slight refactor
* comments+clarificationso
* extract constants
* (partly) correct model classes
* add de/ser
* formatting
* extract constants
* fix json reference
* update docs
* switch to v1 models
* fix compile+test
* add base64 handling
* use vnull
* Data types update: Implement protocol message downgrade path (#19909)
* rough skeleton for passing catalog into migration
* basic test
* more scaffolding
* basic implementation
* add primitives test
* add in other tests (nested fields currently failing)
* add formats
* impleent oneOf handling
* formatting
* oneOf handling
* better tests
* comments + organization
* progress
* basic test case
* downgrade objects, ish
* basic array implementation
* handle numeric failure
* test for new type
* handle array items
* empty schema handling
* first pass at oneof handling
* add more tests+handling
* more tests
* comments
* add empty oneof test case
* format + reorganize
* more reorganize
* fix name
* also downgrade binary data
* only import vnull
* move migrations into v1 package
* extract schema mutation code
* comment
* extract schema migration to new class
* extract record downgrade logic for future use
* format
* fix build after rebase
* rename private method for consistency
* also implement configuredcatalog migrations >.>
* quick and dirty tests
* slight cleanup
* fix tests
* pmd
* pmd test
* null check on message objects
* maybe fix acceptance tests?
* fix name
* extract constants
* more fixes
* tmp
* meh
* fix cdc acc tests
* revert to master source-postgres
* remove log messages
* revert other misc hacks
* integers are valid cursors
* remove unrelated change
* fix build
* fix build more?
* [MUST REVERT] use dev normalization
* capture kube logs
* also here?
* no debug logs?
* delete dup from merging
* add final everywhere
* revert test changes
Co-authored-by: Jimmy Ma <jimmy@airbyte.io>
* On-the-fly migrations of persisted catalogs (#21757)
* On the fly catalog migration for normalization activity
* On the fly catalog migration for job persistence
* On the fly migration for standard sync persistence
* On the fly migration for airbyte catalogs
* Refactor code to share JsonSchema traversal
* Add V0 Data type search function
* PMD and Format
* Fix getOrInsertActorCatalog and ConfigRepositoryE2E tests
* Null-proofing CatalogMigrationV1Helper
* More null checks
* Fix test
* Format
* Add data type v1 support to the FE
* Changes AC test check to check exited ps (#21672)
some docker compose changes no longer show exited
processes. this broke out test
this change should fix master
tested in a runner that failed
* Move wellknown types mapping to the utility function
* use protocolv1 normalization
---------
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
* Update protocol support range (#21996)
* bump normalization version to 0.3.0
* Add version check on normalization (#22048)
* Add normalization min version check
* Add visible for testing
---------
Co-authored-by: Edward Gao <edward.gao@airbyte.io>
Co-authored-by: Eugene <etsybaev@gmail.com>
Co-authored-by: Topher Lubaway <asimplechris@gmail.com>
* Revert "Revert "Add dependencies to docker-compose (#19257)" (#19306)"
This reverts commit b16f28fd4d.
* Bump docker-compose version
* test
* Revert "test"
This reverts commit c3b28c9ec3.
* Bump docker compose version to 3.8
* test
* test
* Use a newer docker for platform build
* test
* Use docker compose v2
* Update docker for frontend e2e
* Use docker compose v2 for e2e tests
* Update tools to use docker compose v2
* Update more docker compose references to v2
* Update docker compose usage in docs
* Update deploying airbyte docs
* Fix permission
* Case insensistive check
* Add extra mention for docker compose in upgrading airbyte
* Improve upgrade docker note
