21ba32cd66
* Copy schedule schema to standard sync * Add config model readme * Remove references of standard sync schedule * Add migration * Fix unit test * Remove standard sync schedule schema * Fix unit test * Remove unnecessary migration resource * Add migration test * Apply spotless format * Add minor updates * Update readme
2.0 KiB
2.0 KiB
Airbyte Config Migration
This module migrates configs specified in airbyte-config to new versions.
Change Airbyte Configs
- Update the config json schema in
airbyte-config/models. - Add the changed json schema to the main resources.
- If a migration is needed, create a migration file under
io.airbyte.migrate.migrations. - Register the migration in
Migrations.java. - If needed, write a migration unit test under
io.airbyte.migrate.migrations. - Test the migration locally in IDE or commandline (see below).
Test Migration Locally
IDE
Run MigrationRunner.java with arguments (--input, --output, --target-version).
Command line
Run the following command in project root:
# Get the current version
BUILD_VERSION=$(cat .env | grep VERSION | awk -F"=" '{print $2}')
# Build the migration bundle file
./gradlew airbyte-migration:build
# Extract the bundle file
tar xf ./airbyte-migration/build/distributions/airbyte-migration-${BUILD_VERSION}.tar --strip-components=1
# Run the migration
bin/airbyte-migration \
--input <input_config_archive.tar.gz> \
--output <output_config_archive.tar.gz>
See MigrationRunner for details.
Run migration in production
BUILD_VERSION=$(cat .env | grep VERSION | awk -F"=" '{print $2}')
INPUT_PATH=<path to directory containing downloaded airbyte_archive.tar.gz>
OUTPUT_PATH=<path to where migrated archive will be written (should end in .tar.gz)>
TARGET_VERSION=<version you are migrating to or empty for latest>
docker run --rm -v ${INPUT_PATH}:/config airbyte/migration:${BUILD_VERSION} -- \
--input /config/airbyte_archive.tar.gz \
--output ${OUTPUT_PATH} \
[ --target-version ${TARGET_VERSION} ]
See Upgrading Airbyte for details.