6f3a9d89f6
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Ian Alton <ian.alton@airbyte.io> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
3.8 KiB
deepset AI Platform
deepset AI Platform is a SaaS platform for building LLM applications and managing them across the whole lifecycle - from early prototyping to large-scale production. For details, see deepset documentation.
Data Integration with Airbyte
To make it possible to synchronize data to deepset AI Platform using Airbyte, we've added an Airbyte deepset destination connector. You can use it to stream data into deepset from any Airbyte source that emits records matching the document file type. The synchronized data are available in deepset on the Files page as Markdown files.
Note: The deepset destination connector writes data to your deepset workspace, but does not delete any data from the workspace. If a file with the same name already exists in the destination workspace, it is overwritten.
Supported Sync Modes
The deepset destination connector supports the following sync modes:
- Full refresh - append
- Full refresh - overwrite
- Incremental sync - append
- Incremental sync - append + deduped
Syncing Data to deepset AI Platform
To use the deepset destination in Airbyte:
-
Log in to deepset AI Platform.
-
Generate the deepset Cloud API key:
- Click your initials in the top right corner and choose Connections.
- Scroll down the Connections page to the API Keys section and click Add new key. If you need help, see Generate an API Key.
-
Set up the destination connector in Airbyte providing the following details:
Base URL: This is the URL for the deepset environment with your account. Possible options are:https://api.cloud.deepset.ai(default) for EU users ,https://api.us.deepset.aifor US users, or custom URL for on-premise deployments.API key: Your deepset API key (generated in step 2 above).Workspace name: The name of the deepset workspace where you want to store the data.Retry count: The number of times to retry syncing a record before marking it as failed. Defaults to 5 times.
After you connect a source and the first stream synchronization succeeds, your records are available in deepset AI Platform on the Files page as Markdown files.
Changelog
Expand to review
| Version | Date | Pull Request | Subject |
|---|---|---|---|
| 0.1.8 | 2025-05-17 | 60635 | Update dependencies |
| 0.1.7 | 2025-05-10 | 59834 | Update dependencies |
| 0.1.6 | 2025-05-03 | 58717 | Update dependencies |
| 0.1.5 | 2025-04-19 | 58229 | Update dependencies |
| 0.1.4 | 2025-04-12 | 57619 | Update dependencies |
| 0.1.3 | 2025-04-05 | 57176 | Update dependencies |
| 0.1.2 | 2025-03-29 | 56599 | Update dependencies |
| 0.1.1 | 2025-03-22 | 56097 | Update dependencies |
| 0.1.0 | 2025-01-10 | 48875 | Initial release |