1
0
mirror of synced 2025-12-22 19:38:29 -05:00
Commit Graph

196 Commits

Author SHA1 Message Date
Liren Tu
baebe1619c Release new postgres strict encrypt version (#14331)
* Bump postgres strict encrypt version

* Update changelogs

* Update doc
2022-06-30 21:50:15 -07:00
Tuhai Maksym
a147b6f453 6339: error when attempting to use azure sql database within an elastic pool as source for cdc based replication (#14121)
* 6339: implementation

* 6339: changelog updated

* 6339: definitions updated

* 6339: definitions reverted

* 6339: still struggling with publishing

* auto-bump connector version

* 6339: definitions reverted - correct

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-29 14:44:38 +03:00
Tuhai Maksym
1b28dce3ed Revert "6339: error when attempting to use azure sql database within an elastic pool as source for cdc based replication (#13866)" (#14011)
This reverts commit 0d870bd37b.
2022-06-22 17:06:10 +03:00
Tuhai Maksym
0d870bd37b 6339: error when attempting to use azure sql database within an elastic pool as source for cdc based replication (#13866)
* 6339: debug info

* 6339: not using 'USE' on Azure SQL servers

* 6339: cleanup

* 6339: cleanup2

* 6339: cleanup3

* 6339: versions/changelogs updated

* 6339: merge from master (consolidation issue)

* 6339: dev connector version (for testing in airbyte cloud)

* 6339: code review implementation

* 6339: apply formatting
2022-06-22 13:08:45 +03:00
Andrii Leonets
8f602aee96 Source MsSql : incr ver to include changes #13854 (#13887)
* incr version

* put PR id

* docker ver
2022-06-20 11:54:40 +03:00
LiRen Tu
545a7a3eb6 🎉 JDBC source: adjust fetch size based on max memory and max row size (#13435)
* Switch to measure max row byte size

* Reduce fetch size change logs

* Update unit tests

* Determine jdbc buffer size based on max memory

* Bump postgres version

* Bump postgres version

* Bump mysql version

* Bump mssql version

* Format java code

* Increase hikari connection timeout

* Update data source default parameters

* auto-bump connector version

* Mark postgres 0.4.21 as not published

* Revert "Bump mysql version"

This reverts commit ad9135258c.

* Fix unit test

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-07 14:29:34 -07:00
Marcos Marx
b5b43d3fa1 Source MSSQL: correct enum Standard method (#13419)
* correct enum MSSQL source

* bump strict version and update doc

* correct expect spec test

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-02 14:16:47 -03:00
LiRen Tu
868ed76212 Patch for mssql source cdc update (#13168)
* MSSQL CDC feature to capture only changes option added

* MSSQL CDC feature to capture only changes option added

* added option to disable snapshot 

snapshot isolation setting is hardcode and I modified as user option

* recommitting

* docker version change

docker version change

* resolve conflict

* review 1 added

* review 1 added

* removed extra space

removed extra space

* Update spec.json

* Format java code

* Update doc and version

* Update is-cdc method and add test

* Update spec

* Update doc

* Update snapshot isolation check

* Add helper method to check snapshot level

* Introduce enums

* Format code

* Remove empty constant class

* Update expected spec

* Update spec

* Prevent npe

* Update changelog

* Use a different field name

* Fix missing return statement

Co-authored-by: Sivakumar Ramaswamy <sivakumar.ramaswamy@zip.co>
Co-authored-by: sivankumar86 <sivankumar86@users.noreply.github.com>
Co-authored-by: sivankumar86 <sivankumar86@gmail.com>
2022-05-25 08:06:44 -07:00
sivankumar86
90b757b229 🎉 Source MSSQL: add option to disable initial snapshot and change snapshot isolation level (#12759)
* MSSQL CDC feature to capture only changes option added

* MSSQL CDC feature to capture only changes option added

* added option to disable snapshot 

snapshot isolation setting is hardcode and I modified as user option

* recommitting

* docker version change

docker version change

* resolve conflict

* review 1 added

* review 1 added

* removed extra space

removed extra space
2022-05-25 07:58:39 -07:00
LiRen Tu
35f2aa9aed 🎉 Jdbc sources: publish new version with adaptive fetch size (#12480)
* Default scaffold to use adaptive streaming config

* Switch more connectors to use adaptive streaming config

* Bump version for cockroach db

* Bump version for db2

* Bump mssql version

* Bump mysql version

* Bump oracle version

* Bump postgres version

* Bump redshift version

* Bump snowflake version

* Bump tidb version

* auto-bump connector version

* Fix db2 findbug issue

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* Fix more findbug issues

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

* Fix findbug issue for mysql-strict-encrypt

* Fix findbugs issue for oracle source

* auto-bump connector version

* Remove suppress warnings annotation

* Fix oracle encrypt tests

* Fix oracle encrypt acceptance test

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-29 23:14:58 -07:00
Greg Solovyev
597d6ccbb8 Reconcile versions in git with versions in docker hub (#12368) 2022-04-26 11:41:59 -07:00
Jared Rhizor
14dfc1d417 fix parentheses and info/warning/danger formatting in docs (#11981)
* fix parens

* fix formatting for hints

* remove hint from non-docusaurus or gitbook doc
2022-04-13 12:08:34 -07:00
eric
57fa793c77 🐛 Fix the MSSQL Server Agent not querying in Chinese system (#11495)
* bug: Fix the SQL Server Agent not querying in Chinese system

* Update airbyte-integrations/connectors/source-mssql/src/main/java/io/airbyte/integrations/source/mssql/MssqlSource.java

Co-authored-by: LiRen Tu <tuliren@gmail.com>

* bump connector version

Co-authored-by: LiRen Tu <tuliren@gmail.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-04-04 19:13:29 -03:00
andriikorotkov
1d66672fad 🐛 Fixed cursor for old connectors that use non-microsecond format. Now connectors work with both formats (#10508)
* updated timestamp transformation with microseconds

* updated timestamp transformation with microseconds

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql tests

* removed extra logs

* fixed cursor for timestamp

* updated test

* updated type transformation

* updated type transformation

* updated type transformation

* updated tests

* updated oracle tests

* updated documentations and connectors versions

* updated documentations and connectors versions

* fix code style

* updated cursor for jdbc sources

* updated cursor for jdbc sources

* updated cursor for jdbc sources
2022-02-22 13:26:10 +02:00
andriikorotkov
9d401fc9f7 🎉 Updated timestamp transformation with microseconds (#10242)
* updated timestamp transformation with microseconds

* updated timestamp transformation with microseconds

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql datatype tests

* updated mysql tests

* removed extra logs

* fixed cursor for timestamp

* updated test

* updated type transformation

* updated type transformation

* updated type transformation

* updated tests

* updated oracle tests

* updated documentations and connectors versions

* updated documentations and connectors versions

* fix code style
2022-02-18 14:51:58 +02:00
Lake Mossman
3d8a0dc048 Add ExitOnOutOfMemoryError to java connectors and bump versions (#10256) 2022-02-14 15:49:15 -08:00
Andrii Leonets
4a290fa9ff [#9554 PR] Postgres/Mssql Source: Increase version (#9753)
* #9554 PR

* format

* incr version for Postgres

* Update docs/integrations/sources/postgres.md

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>

* test upd

* test upd

* incr ver for mssql as effected source

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-01-25 22:39:33 +02:00
Eugene
1877e2aba2 🐛Source-mssql: added support for missed data types (#9094)
* [7728] Source-mssql: added support for missed data types
2022-01-07 19:17:29 +02:00
Iryna Grankova
74cb28e0f6 Update fields in source-connectors specifications: mssql, mssql-strict-encrypt (#9206) 2022-01-05 15:47:41 +02:00
Yurii Bidiuk
cd30cf4fca bump version for affected sources from #8749 (#8958)
* bump versions

* fix tests for cockroachdb

* update changelog
2021-12-24 14:41:22 +02:00
oneshcheret
fcf7491fee 🐛 Validate incorrect handling '\n' symbols in ssh key (#8371)
* 🐛 Validate \n in ssh key

* bump versions for ssh key connectors

* update versions for ssh key connectors

* temporal fix for checking failed tests

* revert temp changes and destination oracle version

* bump versions in config for ssh key related connectors
2021-12-03 20:11:51 +02:00
oneshcheret
70ed18b89e 🐛 Fix handling binary data type for Mssql source (#7748)
* Fix data type tests Mssql source

* update test for boolean data type in  Mysql source

* dump versions for Mssql and Mysql source

* revert changes regarding handling bit data type

* revert changes regarding date and time data type tests Mssql source

* revert changes regarding date data type for Mssql source

* update message in mssql.md

* minor format changes

* format changes
2021-11-11 14:15:23 +02:00
Yurii Bidiuk
56db8065e9 🐛 Source MSSQL: fix data type (smalldatetime, smallmoney) conversion from mssql source (#5609) (#7386)
* Fix data type (smalldatetime, smallmoney) conversion from mssql source (#5609)

* Fixed code format

* Bumb new version

* Update documentation (mssql.md)

* formating

* fixed converter properties

* aligned converter utils with #7339

Co-authored-by: Andrii Leonets <30464745+DoNotPanicUA@users.noreply.github.com>
2021-11-05 12:21:07 +02:00
Abhi Vaidyanatha
ae32ecbb27 GitBook: [master] 186 pages and 77 assets modified 2021-10-08 21:17:47 +00:00
Abhi Vaidyanatha
276d914a57 Direct to Discourse tagged list instead of specific issues on Troubleshooting. (#6848) 2021-10-07 00:05:56 -07:00
Jenny Brown
2e5fbba434 Clarify ssh private key format for ssh tunnels (#6585)
* Clarify ssh private key format for ssh tunnels
* Improved SSH Tunnel key generation steps, fixed formatting
* Modified wording 'app' to 'connector' for consistency
* Ran format
2021-10-04 13:59:47 -05:00
Abhi Vaidyanatha
770bbf9e14 📚 Docs Refresh: Jira Source (#6699)
* Clean up MSSQL Troubleshooting

* Docs Refresh: Jira Source

* Update BQ troubleshooting

* Add discussion
2021-10-04 00:05:52 -07:00
Abhi Vaidyanatha
d383a1b3b2 📚 Docs Refresh: MSSQL Source (#6670)
* Docs Refresh: MSSQL Source

* Remove hesitancy.
2021-10-03 23:03:01 -07:00
Oleksandr Sheheda
59cd5ca308 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966) (#6318)
* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)

* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
updated image version

* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
correction of remarks

* 🎉 Source MSSQL: Added support for connection via SSH tunnels (#5966)
bumb image version
2021-09-21 17:25:04 +03:00
Eugene
a78efe090b 🎉 JAVA-Based connectors: Bumped version for some javabased connector to start using Config Validator from core module (#5398)
* Updated some java-based connectors version to start using new json config validator from java core
2021-08-17 22:30:16 +03:00
Marcos Marx
d91ff7dcf1 Documentation add data types to databases (#5008)
* add postgres data types

* add mysql data types

* add mssql data types

* add oracle data type
2021-07-26 23:02:46 -03:00
George Claireaux
43d86114a3 fixed broken links and styling (#4747) 2021-07-14 18:35:54 +01:00
Subodh Kant Chaturvedi
cc7b376044 🎉 Source MSSQL: implementation for CDC (#4689)
* first few classes for mssql cdc

* wip

* mssql cdc working against unit tests

* increment version

* add cdc acceptance test

* tweaks

* add file

* working on comprehensive tests

* change isolation from snapshot to read_committed_snapshot

* finalised type tests

* Revert "change isolation from snapshot to read_committed_snapshot"

This reverts commit 20c6768071.

* small docstring fix

* remove unused imports

* stress test fixes

* minor formatting improvements

* mssql cdc docs

* finish off cdc docs

* format fix

* update connector version

* add to changelog

* fix for sql server agent offline failing cdc enable on tables

* final structure

* few more updates

* undo unwanted changes

* add abstract test + more refinement

* remove CDC metadata to debezium

* use new cdc abstraction for mysql

* undo wanted change

* use cdc abstraction for postgres

* add files

* pull in latest changes

* ready

* rename class + add missing property

* use renamed class + move constants to MySqlSource

* use renamed class + move constants to PostgresSource

* move debezium to bases + upgrade debezium version + review comments

* downgrade version + minor fixes

* bring in latest changes from cdc abstraction

* reset to minutes

* bring in the latest changes

* format

* fix build

* address review comments

* bring in latest changes

* bring in latest changes

* use common abstraction for CDC via debezium for sql server

* remove debezium from build

* finalise PR

* should return Optional

* pull in latest changes

* pull in latest changes

* address review comments

* use common abstraction for CDC via debezium for mysql (#4604)

* use new cdc abstraction for mysql

* undo wanted change

* pull in latest changes

* use renamed class + move constants to MySqlSource

* bring in latest changes from cdc abstraction

* format

* bring in latest changes

* pull in latest changes

* use common abstraction for CDC via debezium for postgres (#4607)

* use cdc abstraction for postgres

* add files

* ready

* use renamed class + move constants to PostgresSource

* bring in the latest changes

* bring in latest changes

* pull in latest changes

* lower version for tests to run on CI

* format

* Update docs/integrations/sources/mssql.md

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* addressing review comments

* fix for testGetTargetPosition

* format changes

Co-authored-by: George Claireaux <george@claireaux.co.uk>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-07-14 17:15:55 +01:00
Marcos Marx
88d7953a18 Add CHANGELOG for Sources (#4215)
* add changelog for postres source

* add changelog for mysql source

* add changelog for mssql source

* add changelog for file source

* add changelog for google sheets

* add full changelog for file source

* add full changelog for google sheet source

* add full changelog for mssql source

* add changelog for http requests

* add changelog for salesforce source

* add changelog for facebook marketing source

* add changelog for google analytics source

* add changelog for shopify native source connector
2021-06-18 17:45:09 -03:00
Marcos Marx
7c70eb02cd GitBook: [master] 84 pages and 72 assets modified 2021-05-03 12:14:16 +00:00
Davin Chia
42686add8a Release connectors with namespace change. (#2990)
Release all connectors affected by namespace change. Includes all JDBC sources and destinations.

Also add documentation for normalisation. Prerequisite to actually releasing 0.21.0-alpha.
2021-04-21 11:35:08 +08:00
Christophe Duong
978e6d759b Add docs for new sync modes (#2636) 2021-03-30 10:18:42 +02:00
Shrif Nada
ff458bbd9d GitBook: [master] 111 pages modified 2021-01-26 04:14:55 +00:00
Jared Rhizor
4ab0464274 add postgres troubleshooting docs (#1653)
* add postgres troubleshooting docs

* update headings

* add to other db sources

* move connection troubleshooting to faq
2021-01-25 10:56:09 -08:00
John Lafleur
2acc3840d0 GitBook: [docs_feature_incremental] 33 pages modified (#1683)
Co-authored-by: John Lafleur <john@dataline.io>
2021-01-17 07:32:24 +11:00
Shrif Nada
82377407d0 GitBook: [master] one page modified 2021-01-16 17:15:49 +00:00
John Lafleur
5532aa370a Updating Incremental within docs pages for all sources to make it more explicit (#1381)
* GitBook: [incremental_update_docs] 79 pages modified

* GitBook: [incremental_update_docs] one page modified

Co-authored-by: John Lafleur <john@dataline.io>
2020-12-18 21:03:34 +11:00
Shrif Nada
06e98150a5 GitBook: [master] one page modified 2020-12-18 02:03:36 +00:00
John Lafleur
a5f509bf49 GitBook: [master] 15 pages and 4 assets modified 2020-11-25 19:01:56 +00:00
John Lafleur
ca864a588e Updating connector changelog with Mailchimp and next efforts (#1024) 2020-11-19 20:03:16 -08:00
Charles
c3293de232 MSSQL Source (#977) 2020-11-13 21:06:38 -08:00