1
0
mirror of synced 2025-12-20 10:32:35 -05:00
Commit Graph

238 Commits

Author SHA1 Message Date
Charles
13c5eef93a Fix JdbcSource Incremental OOM (#1655) 2021-01-14 14:33:44 -08:00
Charles
f9b63d0a9e Update Destination Abstractions (#1456) 2021-01-07 10:37:10 -08:00
Christophe Duong
6d25e65d57 Renaming some classes to be less specific to SQL destinations (#1283) 2020-12-21 15:16:57 +01:00
Charles
bbf851fe27 upgrade test containers (#1308) 2020-12-15 18:15:55 -08:00
Charles
473f52d6ff fix basic normalization link (#1282) 2020-12-10 16:48:52 -08:00
Jared Rhizor
1bd19d1bae put all integration test tasks under integrationTest (#1231)
* always re-run standardSourceTestPython

* rename and regroup to integrationTest

* add comment
2020-12-07 10:10:26 -08:00
Christophe Duong
4ca8760e51 Chris/snowflake incremental (#1194)
* Refactor Snowflake destination and support incremental (#1125)
2020-12-04 19:52:33 +01:00
Christophe Duong
b9fbf24ea6 Add Supports Incremental Flag to Destination Specs #1124 (#1193)
* Add Supports Incremental Flag to Destination Specs #1124
2020-12-04 18:19:09 +01:00
Christophe Duong
d06392e900 Redshift Destination & refactoring to introduce destination-jdbc
Closes #193
Closes #1126
2020-12-03 18:07:46 +01:00
Christophe Duong
e4db33ed71 BumpVersions of destinations (#1093) 2020-11-25 22:12:28 +01:00
Christophe Duong
206d3cbea8 Naming conventions managed in destinations (#1060)
Handled field and table identifier conversions as necessary in order to successfully in destinations
2020-11-25 18:53:23 +01:00
Charles
cf0476c822 bump version on some examples that use the title field (#1086) 2020-11-25 08:04:43 -08:00
Charles
eff4a9981c add title to some specs (#1075) 2020-11-24 15:53:40 -08:00
Christophe Duong
be83b18fa8 Destination-postgres should write data in a table from the specified schema (#1059)
* Destination-postgres should write data in a table from the specified schema (if any)
* Add tests when writing in different schemas than public
2020-11-23 19:42:18 +01:00
Charles
0d0eeff733 destination-postgres incremental (#1017)
* Adds tests to standard tests

* Need to add test to acceptance tests
2020-11-20 12:43:58 -08:00
Charles
02819a4b87 Incremental Docs and Data Model Update (#1021) 2020-11-19 22:07:32 -08:00
Sherif A. Nada
f4c3ac70f9 annotate secret fields (#1012) 2020-11-19 15:13:23 -08:00
Charles
e7edb2c858 Adding incremental to the catalog data model (#998)
* Add ConfiguredAirbyteCatalog and ConfiguredAirbyteStream
2020-11-18 14:15:59 -08:00
Sherif A. Nada
fba7f1f8bc correct timestamps in standard destination tests and assume millis input in BQ (#981) 2020-11-14 18:59:50 -08:00
Charles
f51fc05d5d fix basic normalization links (#945) 2020-11-12 09:50:51 -08:00
Charles
148d6592bd fix syntax error in hrefs in specs (#921) 2020-11-11 16:28:08 -08:00
Jared Rhizor
d92cd47cde fix postgres integration tests for stream casing/special characters (#916)
* fix integration tests except for normalization

* fix everything except lowercasing

* remove lowercasing of model names
2020-11-11 13:55:30 -08:00
Jared Rhizor
71e9484ead fix postgres table name quoting (#908) 2020-11-11 09:50:45 -08:00
Charles
dccd2f9a80 bump versions: all destinations and source exchange rate (#895) 2020-11-10 17:23:32 -08:00
Sherif A. Nada
ce40a0852b Add normalization tests to BQ (#875) 2020-11-10 14:39:44 -08:00
Christophe Duong
e4168fa6da Normalization integration test (#873) 2020-11-10 09:49:15 -08:00
Sherif A. Nada
9305461fec Add _raw suffix to non-normalized destination table names (#874) 2020-11-10 08:31:27 -08:00
Charles
c02f72e6d9 Add Basic Normalization Test to Standard Destination Test (#869) 2020-11-10 00:02:48 -08:00
Charles
d507f0f95b Fix names for standard tests (#862) 2020-11-09 21:43:51 -08:00
Jared Rhizor
e1d7add50a mostly incremental builds (#817)
* support cached builds for base -> base-java -> snowflake

* use plugin for image building

* fix matching on COPY from

* remove docker.gradle

* tmp commit

* update connectors

* finish rest of build files

* fix ide errors

* more build fixes

* clean up

* clean up for new soruces

* fix spotless

* fix flake problems

* add recommended empty file

* python caching

* fixes upon review

* clean up docker and build test files

* clean up python

* clean up

* fix integration test dependencies

* fix standard tests

* fix

* remove symlink

* re-add requirements to fix normalizatioon build

* fix symlink

* fix dumbest build problem of all

* add missing integration test def

* fix missing dep

* remove class exclusion

* move trim so null source versions are allowed

* rename map

* fix hardcoded value

* remove unnecessary dep

* use dashes for salesforce package name

* fix typo

* DRY and fix test image name

* Fix edit

* assert string is not empty

* build integration test image only for integrationTest

* move code generator to tools and rename docker build tasks

* make source test depend on integration test build, not the other way

* remove guard because the docker build should exist before the integrationtest is applied

* remove comment

* DRY up airbyte-source-test

* fix plugin compilation

* add missing dependency

* rename getTaggedImage to getDevTaggedImage

* fix test vs main docker build bug
2020-11-09 11:00:11 -08:00
Sherif A. Nada
03fb508820 cleanups (#847) 2020-11-06 20:59:24 -08:00
Christophe Duong
d042ef524a DBT project for Airbyte normalization (#802)
Add a step that generates config files and SQL files in order to run DBT which takes care of converting JSON blob data (from catalog.json JSON Schema) into normalized tables 

Co-authored-by: cgardens <giardina.charles@gmail.com>
Co-authored-by: Sherif Nada <snadalive@gmail.com>
2020-11-06 23:02:30 +01:00
Michel Tricot
4d1dc69b55 Hides SQL dialects + fix some bugs in the jdbc source (#767) 2020-10-31 11:18:53 -07:00
Charles
88052c3968 Add JDBC Base Source (#757) 2020-10-30 10:47:43 -07:00
Charles
5f45aef149 Add support for sources to java base (#749) 2020-10-30 10:33:51 -07:00
Charles
196674d49f remove ab protocol suffix (#677) 2020-10-26 09:40:38 -07:00
Charles
4d1a788177 standard source tests (#686) 2020-10-22 20:06:33 -07:00
Charles
8f7b559550 Integrations Reorganization: Connectors (#676) 2020-10-21 13:05:25 -07:00