mirror of synced 2025-12-30 03:02:21 -05:00

Files

danilo-dti 7ca371b26d 🎉 New Source: Microsoft Dataverse [python cdk] (#18646 )

* Connector files

* Add test files

* Add integration test config files

* Multiple changes to make it on Airbyte standards

* Cleaning up

* More clean ups

* More clean ups

* Removed max pages

* Remove unused variable

* Correctly separating Full refresh and incremental

* Removed unused variables

* Fix full_refresh class

* Better code for creating stream classes

* Fixing review comments

* Update docs and Enum class

* Update type conversion function

* Fix enum class and update docs

* Update discover

* Implemented some unit tests

* Update discover

* Update test_source

* Increase discovery test timeout

* Update configured_catalog

* Fix default_cursor_field

* Adding final unit tests

* Update spec: set client_id and tenant_id as secrets

* Update discover to deal with Lookup and Picklist types

* Fix Lookup data type conversion

* add microsoft dataverse to source def

* run format

* auto-bump connector version

Co-authored-by: Marcelo Pio de Castro <marcelopiocastro@gmail.com>
Co-authored-by: daniloss99 <danilosiqueira99@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>

2022-11-14 09:02:05 -03:00

3.3 KiB

Raw Blame History

Microsoft Dataverse

Sync overview

This source can sync data for the Microsoft Dataverse API to work with Microsoft Dataverse.

This connector currently uses version v9.2 of the API

Output schema

This source will automatically discover the schema of the Entities of your Dataverse instance using the API https:///api/data/v9.2/EntityDefinitions?$expand=Attributes

Data type mapping

Integration Type	Airbyte Type	Notes
`String`	`string`
`UniqueIdentifier`	`string`
`DateTime`	`timestamp with timezone`
`Integer`	`integer`
`BigInt`	`integer`
`Money`	`number`
`Boolean`	`boolean`
`Double`	`number`
`Decimal`	`number`
`Status`	`integer`
`State`	`integer`
`Virtual`	None	We skip virtual types

Other types are defined as string.

Features

Feature	Supported?(Yes/No)	Notes
Full Refresh Sync	Yes
Incremental Sync	Yes
CDC	Yes	Not all entities support it. Deleted data only have the ID
Replicate Incremental Deletes	Yes
SSL connection	Yes
Namespaces	No

Getting started

Requirements

Application (client) ID
Directory (tenant) ID
Client secrets

Setup guide

The Microsoft Dataverse API uses OAuth2 for authentication. We need a 'client_credentials' type, that we usually get by using an App Registration. https://learn.microsoft.com/en-us/power-apps/developer/data-platform/authenticate-oauth

The procedure to generate the credentials and setup the necessary permissions is well described in this post from Magnetism blog: https://blog.magnetismsolutions.com/blog/paulnieuwelaar/2021/9/21/setting-up-an-application-user-in-dynamics-365

CHANGELOG

Version	Date	Pull Request	Subject

3.3 KiB Raw Blame History