* Implement database migrator
* Add unit tests
* Add RUN_FLYWAY_MIGRATION variable
* Run flyway migration in server
* Add db migration info api
* Add db migration migrate api
* Add unit test
* Remove base airbyte migration
* Implement migration dev helper
* Dry and format code
* Fix url
* Use camel case
* Add db migration page
* Add button to run migration
* Update migration table
* Fix resource warning
* Update readme
* Revert package-lock changes
* Update readme
* Address simple frontend review comments
* Add java migration template (not completed yet)
* Add method to generate migration file
* Set up jooq code generation
* Check in generated code
* Move generated code to build directory
* Exclude db dev center methods in gradle
* Update airbyte-db/README.md
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Mark getMigrator as private
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Address review comments
* Format code
* Fix format output column name
* Remove config persistence builder
* Remove dumpSchemaToFile method
* Run baseline in server
* Rename info to list
Co-authored-by: Charles <giardina.charles@gmail.com>
* Rename executeDbMigrationInfo to listMigrations
Co-authored-by: Charles <giardina.charles@gmail.com>
* Rename RUN_FLYWAY_MIGRATION
* Clean up migration apis
* Remove redundant version comparison
* Refactor db migrator
* Add migration file location parameter back
This is necessary because other databases may exist in a different module and follow different patterns.
* Fix build
* Generate jooq code in gradle
* Remove frontend changes
* Remove testing migration
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Charles <giardina.charles@gmail.com>
* Add paged results to job history retrieval.
* Make job histories come back in a sort order controlled by the sql query
* Increased default job history to show, to make better defaults for UI before paging
* Code review cleanup, constants.
* Update airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.java
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Update airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.java
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Use a Pagination object for pagesize and offset
* NPE fix when pagination is not sent with the request and it falls back to defaults
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Standardize error structure in json responses. Use named types for raising exceptions instead of http status codes peppered throughout the code.
* Whitespace fix
* Rename fix
* Consistent naming for things related to KnownException, for traceability.
* For input field validation errors, list the affected fields as separate json objects. Also uses cleaner helpers for exception mappers.
* Whitespace
* Rename to clarify exception class hierarchy
* Wrap not found responses in a schema and adjusted mapper to match
* Include issue id in comments.
* Add select namespace dropdown
* Rename frequency to schedule. Add normalization support
* Update styling for namespace
* remove waitFor for events
* Add special variables notification
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Set namespaceFormat to by default
* Add proper links
* Rename withOperations to operations field
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Add normalization block
* Add transformation block
* Add operations resource
* Move transformation form to separate component
* Rename operations field on backend. Improve transformations usage
* Mock listOperations operation in unit test
* Make basic a default normalization
* Add errors for forms
* Select basic normalization by default
* Add support to optionally display normalization
* Add supportsDbt and supportsNormalization destination fields
* add useInitialValues hook
* Reinit form on change
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Fix onChange to onSelect renaming
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Migrate BufferedStreamConsumer users (e.g. all JDBC destinations, MeiliSearch) (#3473)
* Add checkpointing test cases in Acceptance Tests (#3473)
* Add testing for emitting state in Destination Standard Test (#3546)
* Migrate BQ to support checkpointing (#3546)
* Migrate copy destinations support checkpointing (#3547)
* Checkpointing: Migrate CSV and JSON destinations (#3551)
