1
0
mirror of synced 2026-01-19 00:05:14 -05:00
Commit Graph

22 Commits

Author SHA1 Message Date
Dmytro
db6daed922 Add autogenerated uittests for python http source (#5245)
Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
2021-08-12 10:33:50 +03:00
LiRen Tu
2906ec287a CI: Add action to check broken doc links (#5254)
* Add action to check broken doc links

* Ignore localhost

* Update config

* Fix broken links

* Use quiet mode

* Ignore PR link

* Fix more broken links

* Fix more broken links

* Fix more broken links

* Verify pattern

* Fix more broken links

* Separate full and pr check

* Update pattern

* Test invalid link

* Remove invalid link
2021-08-07 14:28:02 -07:00
vitaliizazmic
5700645946 🎉 New Source: PrestaShop (#4465)
* Source PrestaShop #2980 - init commit

* Source PrestaShop #2980 - schemas

* Source PrestaShop #2980 - full refresh sync

* Source PrestaShop #2980 - incremental sync, acceptance tests

* Source PrestaShop #2980 - incremental acceptance tests

* Source PrestaShop #2980 - incremental acceptance tests (remove configurations from incremental)

* Source PrestaShop #2980 - change Configurations stream to full refresh

* Source PrestaShop #2980 - docs

* format

* fix schemas, fill streams with data

* fix tests and future_state

* rename presta-shop -> prestashop

* fix mounting volumes inside a nested container

* special handle for NULL dates (MySQL)

* fix build URL

* fix URL and polishing

* fix header

* fix template

* format, update pre-commit to execute in the right order

* fix scaffolding

* bump and changelog for SAT

* register as the default source

* fix test timeout

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
2021-07-26 21:55:07 -04:00
Sherif A. Nada
b86ebf63fb Automatically pull latest SAT & build connector image in SAT runner (#4969) 2021-07-26 11:56:19 -07:00
Eugene Kulak
0f51edbd60 SAT: Replace validate_output_from_all_streams with empty_streams param (#4897)
* Replace validate_output_from_all_streams with empty_streams param

* update docs

* fix comment

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
2021-07-22 16:26:35 -04:00
Davin Chia
5f7d36efa6 🐛 Add missing dependencies for acceptance tests to run. (#4808) 2021-07-19 12:24:07 +08:00
Jared Rhizor
b4793b2510 add AIRBYTE_ENTRYPOINT for kubernetes support (#3973)
* add AIRBYTE_ENTRYPOINT for kubernetes support

* bump versions

* bump version in seed

* Update generic template

* keep scaffold sources at 0.1.0

* add missing newline

* handle python base versions correctly

* re-bump mysql and postgres sources

* re-bump snowflake destination

* add skip tests option

* switch to running tests

* reverse conditional to make it safer

* fix publish to include the test running

* fix iterable version

* fix file generation

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-06-09 13:01:45 -07:00
vovavovavovavova
21394c6dae Source-acceptance-test: improvements (#3888)
* rename state_path to abnormal_state_path

* make cursor_path optional

* apply 3rd fix

* format

* abnormal->future (requested)

* upd names

* upd wording (cursor absence fail message)

* bump version in dockerfile
2021-06-08 09:09:19 +03:00
Dmytro
a1035e007b 📚 Update all Python connectors READMEs with minimum required version (#3760) 2021-06-02 12:35:29 +03:00
Davin Chia
9a92da06b9 Update CDK Tutorial to use new CDK. (#3358)
As the title suggests.

Some intricacies around how various directories have diverged.

The HTTP templates now use integration_tests to house catalogs and config files instead of sample_files. After this PR, everything should be updated to use integration_tests for all files except config files. Config files use secrets since they often contain secrets. Anything, besides actual implementations that have yet to be updated, not following this is my mistake.

Even though it diverges from the generated templates, I've decided to leave the tutorial code and the tutorial as using sample_files for now. I think it's more straightforward than telling people to create a new secrets directory. We can change this at a later date.

Confirmed the checked in http python tutorial code works by running ./gradlew :airbyte-integrations:connectors:source-python-http-tutorial:standardSourceTestFile.
2021-05-11 16:52:13 +08:00
Davin Chia
94f6d17796 CDK: Update conceptual docs. Update template docs. (#3352) 2021-05-11 11:02:51 +08:00
Sherif A. Nada
ce4810e6a3 Use latest SAT image by default in generated templates (#3349) 2021-05-10 18:36:12 -07:00
Sherif A. Nada
d0b8e6ddc4 Comment out optional blocks in test config YML (#3347) 2021-05-10 18:28:20 -07:00
Sherif Nada
60e6f2fa6c hotfix: format 2021-05-10 17:08:00 -07:00
Davin Chia
945e46b53e Update generator templates to use CDK. (#3316)
* Update singer template.
* Update source python.
* Update http python.
* Fix a bug where the template tests were not previously running.
2021-05-11 06:26:22 +08:00
Eugene Kulak
752fe9172a add colors to tests when running inside docker (#3193)
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
2021-05-04 08:25:14 -04:00
Eugene Kulak
41ad3a9211 Standard Tests: fix incremental test and docs (#3161)
* fix cursor comparision in the tests

+ update docs and templates

* format

* format

* fix colors

* update acceptance-test-docker.sh script

* fix colors in docker

* Update airbyte-integrations/connector-templates/source-python-http-api/integration_tests/acceptance.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* Update airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/utils/json_schema_helper.py

Co-authored-by: Sherif A. Nada <snadalive@gmail.com>

* address comments from @sherifnada

Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2021-05-03 13:43:10 -04:00
Davin Chia
fd0d253373 Release CDK version. (#3130) 2021-04-29 07:27:28 -07:00
Sherif A. Nada
80ce46f357 Base-python: fix exception when reading schemas from shared/ folder (#3127) 2021-04-29 00:38:54 -07:00
Davin Chia
a187aa667a Rename unit_tests to unit_test so they are detected. Make sure the generated file does this too. (#3082)
Fix tests for the new CDK template generator. The previous test file was not recognised as it was named unit_tests instead of unit_test. Fix this.
2021-04-27 18:21:42 +08:00
Sherif A. Nada
4e9f4bdd06 CDK: tutorial for implementing an HTTP source (#3079) 2021-04-27 00:28:10 -07:00
Davin Chia
f660b0a946 Add template generation for Santa aka CDK. (#3034)
Template generation for new Source using the Santa CDK - provide basic scaffolding for someone implementing a new source.

General approach is to buff up comments in the original SDK, and add TODOs with secondary comments in the generated stub methods, as well as links to existing examples (e.g. Stripe or ExchangeRate api) users can look at.

Checked in and added tests for the generated modules.
2021-04-25 18:02:33 +08:00