* Move icons to connector folder
* Delete old icons
* Update upload logic
* Add icon url to definitions
* Update registry model
* Populate cdn url
* DNC butcher the pipeline
* Low hanging fruit fixes
* Fix bucket name
* Merge old and new approaches
* Fix metadata upload step
* Format
* Fix test
* rename catalog to registry in metadata service
* rename catalog to registry in metadata files
* Run generate models
* Fix missed renames
* Add github personal access token
* Run black
* Automated Change
---------
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
* add airbyte-protocol to deps.toml
* use published protocol jar for platform
* use published protocol jar for connectors
* point at published jar
* fix dep
* bump gcs storage
* fix build failures in standard-source-test
* fix deps
* downgrade alloy db because it is missing strictness tests
* Revert "downgrade alloy db because it is missing strictness tests"
This reverts commit cc6089d053.
---------
Co-authored-by: cgardens <charles@airbyte.io>
Part 1 of #13122.
Rename airbyte-db:lib to airbyte-db:db-lib.
Rename airbyte-metrics:lib to airbyte-metrics:metrics-lib
Rename airbyte-protocol:models to airbyte-protocol:protocol-models.
Explanation for what is happening:
Identically named subprojects have the following issues:
- publishing as is leads to classpath confusion when the jars with the same names are placed in the Java distribution. This leads to NoClassDefFound errors on runtime.
- deconflicting the jar names without changing directory names leads to dependency errors as the OSS jar pom files are generated using project dependencies (suggesting a dependency a sibling subproject in the same repo) that use subprojects group and name as a reference. This means the generated jars look for Jars that do not exists (as their names have been changed) and cannot compile.
- the workaround to changing a subproject's name involves resetting the subproject's name in the settings.gradle and depending on the new name in each build.gradle. This increases configuration burden and decreases the ease of reading, since one will have to check the settings.gradle to know what the right subproject name is. See Projects with same name lead to unintended conflict resolution gradle/gradle#847 for more info.
- given that Gradle itself doesn't have support for identically named subprojects (see the linked issue), the simplest solution is to not allow duplicated directories. I've only renamed conflicting directories here to keep things simple. I will create a follow up issues to enforce non-identical subproject names in our builds.
* Switch json-avro-converter to jitpack
* Switch jsongenerator to jitpack
* Fix default config
* Fix one more default config use case
* Fix jitpack dependency
* Move jitpack repo to root build.gradle
* Pass worker metadata to connector
* Fix compilation
* Pass in job id and image from worker
* Remove application version
* Add default job environment variables
* Add back removed comment
* Rename env map to job metadata
* Fix env configs
* Read connector from application
* Use empty string
* Remove println
* Fix unit test
* Fix compilation error
* Introduce constants for worker env
* Add worker env to ENV_VARS_TO_TRANSFER
* Pass into getWorkerMetadata map to all constructions
* Format code
* Format octavia cli
* Fix test compilation
* Fix typos
* Add support to duplicate one stream multiple times
* Update cloud version
* Bump version
* Remove unused logger
* Add trace for each stream in mock source
* Fix cloud version
* Bump version in seed
* Remove legacy modes
* Use patched json generator library
* Update doc
* Fix unit test
* Resolve merge conflict
* Add unit tests for the generator
* Bump version
* Go back to master branch
* Fix cloud version
* Format code
* Bump version in seed
* add test connectors and bump versions
* add failure timeout acceptance test
* run gw format
* include exception in runtime exception
* mark as disabled and add comment
* use gentle close instead of gentle close with heartbeat when closing source
* also lower destination process gentle close duration
* add test connectors
# Summary
- A follow-up PR for #5543.
- This PR separates the `airbyte-db` project to two modules:
- `lib` is the original `airbyte-db`.
- `jooq` is for jOOQ code generation.
- This is necessary because the jOOQ generator requires a custom database implementation that can run Flyway migration. So the code generator logic needs to depend on the compilation of the original `airbyte-db` project.
# Commits
* Separate db to lib and jooq modules
* Update dependencies
* Add jobs db migrator test
* Fix compose build
* Add migration dev center
* Add schema dump task
* Update airbyte-db/lib/README.md
* Co-authored-by: Davin Chia <davinchia@gmail.com>
* Update readme
* Remove bom dependency
* Update readme
* Use jooq code in db config persistence
* Remove AirbyteConfigsTable
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Migrate BufferedStreamConsumer users (e.g. all JDBC destinations, MeiliSearch) (#3473)
* Add checkpointing test cases in Acceptance Tests (#3473)
* Add testing for emitting state in Destination Standard Test (#3546)
* Migrate BQ to support checkpointing (#3546)
* Migrate copy destinations support checkpointing (#3547)
* Checkpointing: Migrate CSV and JSON destinations (#3551)
