1
0
mirror of synced 2025-12-21 11:01:41 -05:00
Commit Graph

482 Commits

Author SHA1 Message Date
Christophe Duong
6c6ea54bb8 Add SupportedDestinationSyncModes to destination specs objects (#2668)
* Add SupportedDestinationSyncModes to destination specs objects

* Bumpversions of destination connectors
2021-03-31 15:20:01 +02:00
Davin Chia
e8190ff860 🎉 Add NCHAR and NVCHAR support to DB and cursor type casting. (#2600) 2021-03-29 08:09:06 +08:00
Christophe Duong
af43c0bacf ☝ Add an optional jdbc_url_params to MySQL source connector to customize the jdbc_url string (#2611) 2021-03-26 22:10:36 +01:00
Christophe Duong
8a29584125 ☝🏼Destinations supports destination sync mode (#2460)
* Handle destination sync mode in destinations

* Source & Destination sync modes are required (#2500)

* Provide Migration script making sure it is always defined for previous sync configs
2021-03-26 20:23:48 +01:00
Sherif Nada
77b72dcf5b Revert "Revert "Merge branch 'master' of github.com:airbytehq/airbyte""
This reverts commit de66bf010d.
2021-03-23 15:12:01 -07:00
Sherif Nada
de66bf010d Revert "Merge branch 'master' of github.com:airbytehq/airbyte"
This reverts commit a3a27f2fa2, reversing
changes made to 5917feee28.
2021-03-23 15:09:44 -07:00
Christophe Duong
4bebeed60b Harmonize /test and /publish commands on github + Fix MySQL tests in CI (#2531)
* Harmonize /test and /publish commands on github

* Make normalization build on clean system without PyICU

see https://getdbt.slack.com/archives/C50NEBJGG/p1615803416053800?thread_ts=1615511262.026400&cid=C50NEBJGG

* Follow example from https://github.com/testcontainers/testcontainers-java/pull/2689 to modify MySQL container as root

* Add docs on new syntax for /test
2021-03-23 10:47:29 +01:00
Christophe Duong
41e8b6a824 Source support primary keys (#2488)
* Source support primary keys
2021-03-17 19:28:56 +01:00
Christophe Duong
070575ffdf Protocol allows future / unknown properties (#2238)
* Allow new extra properties in validation
* Create migration script to upgrade all connectors versions
* Bumpversion of all connectors
2021-03-09 13:36:36 +01:00
Charles
aadfae24bd Iterator-based JDBC Source (and Redshift bugfix) (#1887) 2021-02-02 17:14:14 -08:00
Charles
c34bcd230a Add db-specific quoting for non-simple names in AbstractJbdcSource (#1833) 2021-01-28 15:41:53 -08:00
Charles
f2f3b4ec37 Fix NPE in State Decorator (#1746) 2021-01-25 17:31:23 -08:00
Charles
3670545995 Fix JdbcSource handling of tables with same names in different schemas (#1724)
* Fix JdbcSource handling of tables with same names in different schemas

* Previously the JdbcSource was combining the columns of any tables with the same name across different schemas into a single stream in the catalog.

* This was caught because in those tables there were columns of the same name with different types which triggered a precondition to check for this.

* The fix makes sure we group by both schema name and table name.

* Adds test to the standard jdbc tests to catch this case.

* This test does NOT run for mysql as, mysql has no concept of schemas.
2021-01-19 18:45:53 -08:00
Charles
13c5eef93a Fix JdbcSource Incremental OOM (#1655) 2021-01-14 14:33:44 -08:00
Charles
102b432a5b Migrate Postgres and MySql to use new JdbcSource (#1307) 2021-01-08 14:15:34 -08:00
Charles
bbf851fe27 upgrade test containers (#1308) 2020-12-15 18:15:55 -08:00
Charles
86f2a659fc migrate mysql source to incremental (#1267) 2020-12-11 18:29:26 -08:00
Sherif A. Nada
93674f6b4d Respect sync mode regardless of input state in mailchimp (#1213) 2020-12-11 13:04:45 -08:00
Jared Rhizor
1bd19d1bae put all integration test tasks under integrationTest (#1231)
* always re-run standardSourceTestPython

* rename and regroup to integrationTest

* add comment
2020-12-07 10:10:26 -08:00
Sherif A. Nada
e8a332ae65 Standard source incremental tests (#1175) 2020-12-04 09:54:10 -08:00
Christophe Duong
a49e7834f8 Change jdbc sources to discover more than standard schemas (#1038)
Change jdbc sources to discover more than standard schemas
2020-11-30 17:36:54 +01:00
jrhizor
9418780c82 hotfix: add docker output dependency for integration tests in mysql and mssql 2020-11-20 14:53:29 -08:00
Charles
02819a4b87 Incremental Docs and Data Model Update (#1021) 2020-11-19 22:07:32 -08:00
Sherif A. Nada
f4c3ac70f9 annotate secret fields (#1012) 2020-11-19 15:13:23 -08:00
Charles
e7edb2c858 Adding incremental to the catalog data model (#998)
* Add ConfiguredAirbyteCatalog and ConfiguredAirbyteStream
2020-11-18 14:15:59 -08:00
Jared Rhizor
ae25781fd9 prevent NPEs when password isn't set for jdbc integrations (#927) 2020-11-11 20:48:52 -08:00
Charles
d507f0f95b Fix names for standard tests (#862) 2020-11-09 21:43:51 -08:00
jrhizor
484f5ba768 hotfix: fix remaining source test discrepancies 2020-11-09 14:05:02 -08:00
Jared Rhizor
e1d7add50a mostly incremental builds (#817)
* support cached builds for base -> base-java -> snowflake

* use plugin for image building

* fix matching on COPY from

* remove docker.gradle

* tmp commit

* update connectors

* finish rest of build files

* fix ide errors

* more build fixes

* clean up

* clean up for new soruces

* fix spotless

* fix flake problems

* add recommended empty file

* python caching

* fixes upon review

* clean up docker and build test files

* clean up python

* clean up

* fix integration test dependencies

* fix standard tests

* fix

* remove symlink

* re-add requirements to fix normalizatioon build

* fix symlink

* fix dumbest build problem of all

* add missing integration test def

* fix missing dep

* remove class exclusion

* move trim so null source versions are allowed

* rename map

* fix hardcoded value

* remove unnecessary dep

* use dashes for salesforce package name

* fix typo

* DRY and fix test image name

* Fix edit

* assert string is not empty

* build integration test image only for integrationTest

* move code generator to tools and rename docker build tasks

* make source test depend on integration test build, not the other way

* remove guard because the docker build should exist before the integrationtest is applied

* remove comment

* DRY up airbyte-source-test

* fix plugin compilation

* add missing dependency

* rename getTaggedImage to getDevTaggedImage

* fix test vs main docker build bug
2020-11-09 11:00:11 -08:00
Michel Tricot
4d1dc69b55 Hides SQL dialects + fix some bugs in the jdbc source (#767) 2020-10-31 11:18:53 -07:00
Charles
ae2f166019 add regex method (#765) 2020-10-30 15:56:06 -07:00
Charles
395aeca640 mysql source (#758) 2020-10-30 15:37:49 -07:00