* tombstone added to schema dump
* read/write actor_definition.tombstone in config persistence
* add tombstone field when loading from yaml seed persistence
* add tombstone to normalized_tables_schema.txt
* add test to ensure tombstone defaults to false
* add release_date and release_stage to actor_definition
- migration to add columns
- update json schema to add new fields
- update API schema to include new fields in definition reads
- update tests
* rename definition creation methods to include custom
* add format:date for releaseDate, and define shared releaseStage enum
* handle null releaseDate
* add attempt failure info to api yml
* populate failureSummary in JobConverter
* tests and formatting
* add Partial Success to glossary section of docs
* failure summary naming/enum tweaks
* tweak enum values and add retryable boolean instead of transient failureType
* Allow updating workspace names
* Add additional unit test
* Fix code styling
* Update slug as well
* Update indentations
* Pull name update into separate endpoint
* add optional tombstone field to StandardSourceDefinition and update repository methods to use it
add new ConfigRepository unit tests to cover includeTombstone logic
whitespace nits
remove accidental import
let autoformat decide whitespace
set tombstone false when creating a SourceDefinition
set tombstone in a couple missed places
remove includeTombstone param from get repository methods
check for tombstone sourceDefinition in ConfigDumpImporter
add tombstone to DestinationDefinition as well
add sourceDefinition delete endpoint, handler, and tests. also fix up some variable names (source -> sourceDefinition) because of confusion and conflicts
add api endpoint to delete destination definitions
filter via stream instead of if-continue
fix comment formatting
* gradle format
After a sync is completed, we tag all the workspace with related to this sync as having a first sync completed.
This will then be forwarded to the UI in another PR.
This is part of #5884
* Enhance API for use by cloud to provide per-connector billing information
* Add listAllConnectionsForWorkspace to include deleted
* Config for list all connections
* Merged master, and formatting
* Formatting
* Name change per PR suggestion
* Name change per PR suggestion
* Formatting again
* Change OAuth API
* Change protocol for new OAuthConfigSpecification
* Refactor OAuth classes and tests
* Remove webbackend source/destination creation
* Change from webback to normal API
* Implement new protocol change with OAuth specs
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* format
* format
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Change OAuth API
* Change protocol for new OAuth Spec (#7827)
* Add examples
* Add protocol object to api too
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* adding google sheets oauth flow to server
* fix oauth type in protocol yaml
* bump sheets version in definitions
* added GDrive scope
* update sheets to master changes
* update protocol incl. cdk
* protocol typing for oauth rootobject
* format
* Add API endpoints for connections, sources, and destinations
* Updating tests
* Tests for source and destination handlers
* Reformat code
* Update airbyte-server/src/main/java/io/airbyte/server/handlers/helpers/ConnectionMatcher.java
Co-authored-by: LiRen Tu <tuliren@gmail.com>
* Adding tests
* Format code
Co-authored-by: LiRen Tu <tuliren@gmail.com>
* Implement database migrator
* Add unit tests
* Add RUN_FLYWAY_MIGRATION variable
* Run flyway migration in server
* Add db migration info api
* Add db migration migrate api
* Add unit test
* Remove base airbyte migration
* Implement migration dev helper
* Dry and format code
* Fix url
* Use camel case
* Add db migration page
* Add button to run migration
* Update migration table
* Fix resource warning
* Update readme
* Revert package-lock changes
* Update readme
* Address simple frontend review comments
* Add java migration template (not completed yet)
* Add method to generate migration file
* Set up jooq code generation
* Check in generated code
* Move generated code to build directory
* Exclude db dev center methods in gradle
* Update airbyte-db/README.md
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Mark getMigrator as private
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Address review comments
* Format code
* Fix format output column name
* Remove config persistence builder
* Remove dumpSchemaToFile method
* Run baseline in server
* Rename info to list
Co-authored-by: Charles <giardina.charles@gmail.com>
* Rename executeDbMigrationInfo to listMigrations
Co-authored-by: Charles <giardina.charles@gmail.com>
* Rename RUN_FLYWAY_MIGRATION
* Clean up migration apis
* Remove redundant version comparison
* Refactor db migrator
* Add migration file location parameter back
This is necessary because other databases may exist in a different module and follow different patterns.
* Fix build
* Generate jooq code in gradle
* Remove frontend changes
* Remove testing migration
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Charles <giardina.charles@gmail.com>
* Add paged results to job history retrieval.
* Make job histories come back in a sort order controlled by the sql query
* Increased default job history to show, to make better defaults for UI before paging
* Code review cleanup, constants.
* Update airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.java
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Update airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.java
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Use a Pagination object for pagesize and offset
* NPE fix when pagination is not sent with the request and it falls back to defaults
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Standardize error structure in json responses. Use named types for raising exceptions instead of http status codes peppered throughout the code.
* Whitespace fix
* Rename fix
* Consistent naming for things related to KnownException, for traceability.
* For input field validation errors, list the affected fields as separate json objects. Also uses cleaner helpers for exception mappers.
* Whitespace
* Rename to clarify exception class hierarchy
* Wrap not found responses in a schema and adjusted mapper to match
* Include issue id in comments.
* Add select namespace dropdown
* Rename frequency to schedule. Add normalization support
* Update styling for namespace
* remove waitFor for events
* Add special variables notification
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Set namespaceFormat to by default
* Add proper links
* Rename withOperations to operations field
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Add normalization block
* Add transformation block
* Add operations resource
* Move transformation form to separate component
* Rename operations field on backend. Improve transformations usage
* Mock listOperations operation in unit test
* Make basic a default normalization
* Add errors for forms
* Select basic normalization by default
* Add support to optionally display normalization
* Add supportsDbt and supportsNormalization destination fields
* add useInitialValues hook
* Reinit form on change
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Fix onChange to onSelect renaming
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Migrate BufferedStreamConsumer users (e.g. all JDBC destinations, MeiliSearch) (#3473)
* Add checkpointing test cases in Acceptance Tests (#3473)
* Add testing for emitting state in Destination Standard Test (#3546)
* Migrate BQ to support checkpointing (#3546)
* Migrate copy destinations support checkpointing (#3547)
* Checkpointing: Migrate CSV and JSON destinations (#3551)
