7bb3241781
API changes to support the progress bar. - The eventual idea is for the save_stats route to be called by the workers during replication. Workers will save stats for a job id and attempt number. - Make modifications to the /jobs/list and the /jobs/get_debug_info routes to also return estimated bytes/records. We need both estimated metadata, as well as running states to calculate progress bar and throughput. - add the save_stats route. This is the route that will be called by workers. I've done my best to reuse existing openapi bodies to reduce duplication. - add the estimatedRecords and estimatedBytes fields to the AttemptStats body. This is part of the AttemptRead and the AttemptStreamStats objects. This eventually filters up to the jobs/list and jobs/get_debug_info objects. This also adds these to all the endpoints that were previously returning stats information. I think the duplicated data is a small issue and don't think it's worth splitting out a new api objects, though I will gladly do so if folks feel strongly. minor changes to the AttemptApiController to support the new route. - I've stubbed out the handlers for now since the backend is not yet implemented.
airbyte-api
Defines the OpenApi configuration for the Airbyte Configuration API. It also is responsible for generating the following from the API spec:
- Java API client
- Java API server - this generated code is used in
airbyte-serverto allow us to implement the Configuration API in a type safe way. SeeConfigurationApi.javainairbyte-server - API docs
Key Files
- src/openapi/config.yaml - Defines the config API interface using OpenApi3
- AirbyteApiClient.java - wraps all api clients so that they can be dependency injected together
- PatchedLogsApi.java - fixes generated code for log api.