mirror of synced 2026-01-06 15:03:36 -05:00

Files

Emma Forman Ling 360e2f9261 🐛 Update Convex destination connector to fix overwrite sync mode (#26103 )

* auto-generated destination connector template

* add config and health check

* support full_refresh overwrite

* support full refresh overwrite, full refresh append, and incremental append

* write works!

* lint

* update readme

* add test messages

* list destination??

* surface error messages

* add integration test

* add destination docs

* update source docs

* change integration test to unit test

* add check unit test

* use a map for streams

* make compatible with API changes

* add indexes for primary keys

* poll for indexes to be backfilled

* simplified stream API

* support append_dedup sync mode

* cleanup

* add changelog

* fix changelog

* use indexes ready endpoint

* update types

* rename to include primary key in API, handle namespaces

* fix types

* update docs

* add better test cases

* fix docs link

* update API and headers

* fix formatting

* update API name to streaming_import

* improve request exception and add a test for error formatting

* remove secret config

* use /replace_tables endpoint instead of /clear_tables

* partial revert of replace_tables

* fix

* formatting and fix table names

* update docs

* update metadata and dockerfile

---------

Co-authored-by: Lee Danilek <lee@convex.dev>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

2023-05-24 12:12:35 -03:00

2.8 KiB

Raw Blame History

Convex

This page contains the setup guide and reference information for the Convex destination connector.

Get started with Convex at the Convex website. See your data on the Convex dashboard.

Overview

The Convex destination connector supports Full Refresh Overwrite, Full Refresh Append, Incremental Append, and Incremental Dedup. Note that for Incremental Dedup, Convex does not store a history table like some other destinations that use DBT, but Convex does store a deduped snapshot.

Output schema

Each stream will be output into a table in Convex. Convex's table naming rules apply - table names can only contain letters, digits, and underscores and may not start with an underscore.

Each record is a document in Convex and is assigned _id and _creationTime fields during sync.

Features

Feature	Supported?
Full Refresh Sync	Yes
Incremental - Append Sync	Yes
Incremental - Dedupe Sync	Yes
Replicate Incremental Deletes	Yes
Change Data Capture	Yes
Namespaces	Yes

Performance considerations

Take care to use the appropriate sync method and frequency for the quantity of data streaming from the source. Performance may suffer with large, frequent syncs with Full Refresh. Prefer Incremental modes when they are supported and especially for large tables. If you see performance issues, please reach via email to Convex support or on Discord.

Getting started

Requirements

Convex Account
Convex Project
Deploy key

Setup guide

Airbyte integration is available to all Convex developers.

On the Convex dashboard, navigate to the project and deployment that you want to sync.

Navigate to the Settings tab.
Copy the "Deployment URL" from the settings page to the deployment_url field in Airbyte.
Click "Generate a deploy key".
Copy the generated deploy key into the access_key field in Airbyte.

Changelog

Version	Date	Pull Request	Subject
0.2.0	2023-05-15	26103	🐛 Update Convex destination connector to fix overwrite sync mode
0.1.0	2023-01-05	21287	🎉 New Destination: Convex

2.8 KiB Raw Blame History