jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2025-12-25 02:09:19 -05:00
Code Issues Packages Projects Releases Wiki Activity

fix: remove duplicate breaking changes from destination-mssql metadata (#55718)

Browse Source
This commit is contained in:
Jonathan Pearlin
2025-03-12 10:06:29 -04:00
committed by GitHub
parent 691f8a82d1
commit a8a4ae8fff
3 changed files with 4 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
airbyte-integrations/connectors/destination-mssql/metadata.yaml
View File

@@ -16,7 +16,7 @@ data:
type: GSM
connectorType: destination
definitionId: 37a928c1-2d5c-431a-a97d-ae236bd1ea0c
dockerImageTag: 2.0.0
dockerImageTag: 2.0.1
dockerRepository: airbyte/destination-mssql
documentationUrl: https://docs.airbyte.com/integrations/destinations/mssql
githubIssueLabel: destination-mssql
@@ -30,17 +30,6 @@ data:
enabled: true
releases:
breakingChanges:
1.0.0:
message:
'This version removes the option to use "normalization" with MSSQL.
It also changes the schema and database of Airbyte''s "raw" tables to be
compatible with the new [Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2)
format. These changes will likely require updates to downstream dbt / SQL
models. Selecting `Upgrade` will upgrade **all** connections using this
destination at their next sync.
'
upgradeDeadline: "2024-05-25"
2.0.0:
message: >
'This version introduces new configuration options (bulk insert via Azure Cloud Storage) and removes

68
docs/integrations/destinations/mssql-migrations.md
View File

@@ -2,7 +2,7 @@
## Upgrading to 2.0.0
This version removes the Airbyte "raw" tables introduced in version [1.0.0](#upgrading-to-100). As such,
This version removes the Airbyte "raw" tables introduced in version 1.0.0. As such,
any attempt to upgrade an existing connection will fail unless a "truncate refresh" is first executed. It is
recommended that you should create a new connection using this upgraded destination and delete the existing
connection and generated "raw" tables in the destination after performing a successful sync via the new connection.
@@ -10,68 +10,4 @@ connection and generated "raw" tables in the destination after performing a succ
In addition to removing the Airbyte "raw" tables, this version also introduces a new insert mode: bulk insert via
Azure Blob Storage. This mode is only supported with Microsoft SQL Server 2017 (14.x) and newer and in scenarios where
the database instance has access to Azure Cloud. This is a net-new configuration option and may be opted in to at any point after upgrading to
the new connection version.
## Upgrading to 1.0.0
This version removes the option to use "normalization" with Microsoft SQL Server. It also changes
the schema and database of Airbyte's "raw" tables to be compatible with the new
[Destinations V2](https://docs.airbyte.com/release_notes/upgrading_to_destinations_v2/#what-is-destinations-v2)
format. These changes will likely require updates to downstream dbt / SQL models. After this update,
Airbyte will only produce the "raw" v2 tables, which store all content in JSON. These changes remove
the ability to do deduplicated syncs with Microsoft SQL Server.
If you are interested in the Microsoft SQL Server destination gaining the full features
of Destinations V2 (including final tables), click [[https://github.com/airbytehq/airbyte/discussions/37010]]
to register your interest.
This upgrade will ignore any existing raw tables and will not migrate any data to the new schema.
For each stream, you should perform the following query to migrate the data from the old raw table
to the new raw table:
```sql
-- assumes your schema was 'default'
-- replace `{{stream_name}}` with replace your stream name
CREATE TABLE airbyte_internal.default_raw__stream_{{stream_name}} (
_airbyte_raw_id VARCHAR(64) PRIMARY KEY,
_airbyte_data NVARCHAR(MAX),
_airbyte_extracted_at DATETIMEOFFSET(7) DEFAULT SYSDATETIMEOFFSET(),
_airbyte_loaded_at DATETIMEOFFSET(7),
_airbyte_meta NVARCHAR(MAX)
);
INSERT INTO airbyte_internal.default_raw__stream_{{stream_name}}
SELECT
_airbyte_ab_id AS _airbyte_raw_id,
_airbyte_data as _airbyte_data,
_airbyte_emitted_at as _airbyte_extracted_at,
NULL as _airbyte_loaded_at,
NULL as _airbyte_meta
FROM airbyte._airbyte_raw_{{stream_name}}
```
**Airbyte will not delete any of your v1 data.**
### Schema and the Internal Schema
We have split the raw and final tables into their own schemas. For the Microsoft SQL Server destination, this means that
we will only write into the raw table which will live in the `airbyte_internal` schema.
The tables written into this schema will be prefixed with either the default database provided in
the `DB Name` field when configuring Microsoft SQL Server (but can also be overridden in the connection). You can
change the "raw" database from the default `airbyte_internal` by supplying a value for
`Raw Table Schema Name`.
For Example:
- Schema: `default`
- Stream Name: `my_stream`
Writes to `airbyte_intneral.default_raw__stream_my_stream`
Where as:
- Schema: `default`
- Stream Name: `my_stream`
- Raw Table Schema Name: `raw_data`
Writes to `raw_data.default_raw__stream_my_stream`
the new connection version.

1
docs/integrations/destinations/mssql.md
View File

@@ -158,6 +158,7 @@ See the [Getting Started: Configuration section](#configuration) of this guide f
| Version | Date | Pull Request | Subject |
|:-----------|:-----------|:-----------------------------------------------------------|:-------------------------------------------------------|
| 2.0.1 | 2025-03-12 | [55718](https://github.com/airbytehq/airbyte/pull/55718) | Fix breaking change information in metadata.yaml |
| 2.0.0 | 2025-03-11 | [55684](https://github.com/airbytehq/airbyte/pull/55684) | Release 2.0.0 |
| 2.0.0.rc13 | 2025-03-07 | [55252](https://github.com/airbytehq/airbyte/pull/55252) | RC13: Bugfix for OOM on Bulk Load |
| 2.0.0.rc12 | 2025-03-05 | [54159](https://github.com/airbytehq/airbyte/pull/54159) | RC12: Support For Bulk Insert Using Azure Blob Storage |