After a sync is completed, we tag all the workspace with related to this sync as having a first sync completed.
This will then be forwarded to the UI in another PR.
This is part of #5884
* Enhance API for use by cloud to provide per-connector billing information
* Add listAllConnectionsForWorkspace to include deleted
* Config for list all connections
* Merged master, and formatting
* Formatting
* Name change per PR suggestion
* Name change per PR suggestion
* Formatting again
* Change OAuth API
* Change protocol for new OAuthConfigSpecification
* Refactor OAuth classes and tests
* Remove webbackend source/destination creation
* Change from webback to normal API
* Implement new protocol change with OAuth specs
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* format
* format
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Change OAuth API
* Change protocol for new OAuth Spec (#7827)
* Add examples
* Add protocol object to api too
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* adding google sheets oauth flow to server
* fix oauth type in protocol yaml
* bump sheets version in definitions
* added GDrive scope
* update sheets to master changes
* update protocol incl. cdk
* protocol typing for oauth rootobject
* format
* Add API endpoints for connections, sources, and destinations
* Updating tests
* Tests for source and destination handlers
* Reformat code
* Update airbyte-server/src/main/java/io/airbyte/server/handlers/helpers/ConnectionMatcher.java
Co-authored-by: LiRen Tu <tuliren@gmail.com>
* Adding tests
* Format code
Co-authored-by: LiRen Tu <tuliren@gmail.com>
We do not need authentication on the health check route since this is meant for monitoring.
Disable this by specifying an empty security block in the route.
* Implement database migrator
* Add unit tests
* Add RUN_FLYWAY_MIGRATION variable
* Run flyway migration in server
* Add db migration info api
* Add db migration migrate api
* Add unit test
* Remove base airbyte migration
* Implement migration dev helper
* Dry and format code
* Fix url
* Use camel case
* Add db migration page
* Add button to run migration
* Update migration table
* Fix resource warning
* Update readme
* Revert package-lock changes
* Update readme
* Address simple frontend review comments
* Add java migration template (not completed yet)
* Add method to generate migration file
* Set up jooq code generation
* Check in generated code
* Move generated code to build directory
* Exclude db dev center methods in gradle
* Update airbyte-db/README.md
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Mark getMigrator as private
Co-authored-by: Davin Chia <davinchia@gmail.com>
* Address review comments
* Format code
* Fix format output column name
* Remove config persistence builder
* Remove dumpSchemaToFile method
* Run baseline in server
* Rename info to list
Co-authored-by: Charles <giardina.charles@gmail.com>
* Rename executeDbMigrationInfo to listMigrations
Co-authored-by: Charles <giardina.charles@gmail.com>
* Rename RUN_FLYWAY_MIGRATION
* Clean up migration apis
* Remove redundant version comparison
* Refactor db migrator
* Add migration file location parameter back
This is necessary because other databases may exist in a different module and follow different patterns.
* Fix build
* Generate jooq code in gradle
* Remove frontend changes
* Remove testing migration
Co-authored-by: Davin Chia <davinchia@gmail.com>
Co-authored-by: Charles <giardina.charles@gmail.com>
* Add paged results to job history retrieval.
* Make job histories come back in a sort order controlled by the sql query
* Increased default job history to show, to make better defaults for UI before paging
* Code review cleanup, constants.
* Update airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.java
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Update airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.java
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Use a Pagination object for pagesize and offset
* NPE fix when pagination is not sent with the request and it falls back to defaults
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Standardize error structure in json responses. Use named types for raising exceptions instead of http status codes peppered throughout the code.
* Whitespace fix
* Rename fix
* Consistent naming for things related to KnownException, for traceability.
* For input field validation errors, list the affected fields as separate json objects. Also uses cleaner helpers for exception mappers.
* Whitespace
* Rename to clarify exception class hierarchy
* Wrap not found responses in a schema and adjusted mapper to match
* Include issue id in comments.
* Add select namespace dropdown
* Rename frequency to schedule. Add normalization support
* Update styling for namespace
* remove waitFor for events
* Add special variables notification
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Set namespaceFormat to by default
* Add proper links
* Rename withOperations to operations field
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Add normalization block
* Add transformation block
* Add operations resource
* Move transformation form to separate component
* Rename operations field on backend. Improve transformations usage
* Mock listOperations operation in unit test
* Make basic a default normalization
* Add errors for forms
* Select basic normalization by default
* Add support to optionally display normalization
* Add supportsDbt and supportsNormalization destination fields
* add useInitialValues hook
* Reinit form on change
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Fix onChange to onSelect renaming
Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
* Making single port 8000 default and update NetworkErrorBoundary.tsx response code 502
* add .env files
* Updating instructions in docs/deploying-airbyte/*
* Migrate BufferedStreamConsumer users (e.g. all JDBC destinations, MeiliSearch) (#3473)
* Add checkpointing test cases in Acceptance Tests (#3473)
* Add testing for emitting state in Destination Standard Test (#3546)
* Migrate BQ to support checkpointing (#3546)
* Migrate copy destinations support checkpointing (#3547)
* Checkpointing: Migrate CSV and JSON destinations (#3551)
Fixes issue #2122
Added icon data
Added icon filenames to connector description yaml files
Added plumbing to transfer and load data properly
Updated tests
This PR is step 5 of this tech spec - https://docs.google.com/document/d/1qFk4YqnwxE4MCGeJ9M2scGOYej6JnDy9A0zbICP_zjI/edit.
The first of (at least) 2 PRs to implement this on the source side. I made some headway before deciding to break the changes into one PR implementing this for discover schema job, and another PR implementing this for read. The combined PR would have been too big otherwise.
Also refactor MoreResources as the test method was attempting to write to the location classes where loaded out from - the issue is we cannot guarantee where the class is loaded from can be written to. Changing this to write to a random folder in the temp directory.
