What:
Swagger UI has bad-UI and extensibility. It's not intuitive and often displays information badly (especially for more complicated structs), which leads to initial API confusion.
How:
Switch over to use Rapidoc. This is our new api docs!
Created the airbyte-public-api-docs bucket and uploaded our new html docs. Set up a CloudFront distribution to enable HTTPS.
A copy of our html docs is checked in in this PR. As you can see, this is a minimal file that relies on a React component published by Rapidoc. It is pointed to the api docs in our master branch and will automatically update itself; do not expect us to need to touch this api file unless we want to make display changes.
I consolidated all files involving api documentation into the docs/api folder for simplicity. I also moved the generated html file to the generated-api-html folder within the api folder to make it clear it was generated. Finally, added a README to document our set up.
Benefits:
UI is much cleaner.
We control the html source file, which means it'll be easier for us to make changes in the future.
Expose the latest connector version to users for better connector UX experience. The airbyte webapp currently displays the docker tag. Users can specify a custom tag to use within using the same cell. However, users do not know what the latest connector image tag is without manually checking the Airbyte github or change log. These changes will provide an endpoint for the webapp to retrieve the relevant information.
The main change is adding a list_latest endpoint to the SourceDefinition and DestinationDefinition resources. Some minor refactors here and there for code readability and maintenance. Note, we use Github as a source of truth for the latest master list.
See this google document to better understand the thought behind this: https://docs.google.com/document/d/1CeqSw9ooWDZP0yvVSmi28DgIHrlW97Tlbm9lIDSzipQ/edit
* Previously if the job failed at all the API would return a 4XX or 500. This prevented the UI from being able to display logs from the failed job.
* Instead, if the underlying job fails the API will still return a 200. The schema field will be empty however and the consumer must check the jobInfo.status field to determine if the job succeeded or not.
* Will still return 422 on invalid inputs.
* remove syncMode from connections (sync mode is already controlled at the stream level, this is just an artifact of the original design.)
* make supportedSyncModes required in the API. (functionally it was already never null because of how we did conversions. it still can be empty.)
* Remove onboardingComplete flag (no longer used).
* Add news, anonymousDataCollection, and securityUpdates to the read struct so that we can continue to do PUT instead of PATCH when updating workspaces. workspaces/update was prevously only called once at when Airbyte was started up, but with import / export it can now be called more frequently.
* add scaffolding
* add backend api handling
* add other fields for merging
* keep defaults separate
* add good unit tests and fix bug
* remove sout
* add tests for discovery combination and fix lack of field selection propagation
* address some pr comments
* address some more pr comments
* remove wait for completion
* use first field
* fmt
* fix tests that expected sync sync jobs
* fmt
* Change api calls for source create in UI
* Change api calls for destination create in UI
* Change api calls for source and destination update in UI
* Update api calls for source and destination check connection
* Fix connector data
* Add failed logs to create source view
* Add failed logs to update source and destination view
* Add failed logs to create destination view
* Create util function for error responce
* Add failed logs to create source and destination in onboarding
* Add failed logs to connection schema view
* Change 'incomplete' status message to 'retrying' message
* Trim the spaces before and after the text fields in JSON configs
* Fix status
* Remove unused webBackend methods
