e1d7add50a
* support cached builds for base -> base-java -> snowflake * use plugin for image building * fix matching on COPY from * remove docker.gradle * tmp commit * update connectors * finish rest of build files * fix ide errors * more build fixes * clean up * clean up for new soruces * fix spotless * fix flake problems * add recommended empty file * python caching * fixes upon review * clean up docker and build test files * clean up python * clean up * fix integration test dependencies * fix standard tests * fix * remove symlink * re-add requirements to fix normalizatioon build * fix symlink * fix dumbest build problem of all * add missing integration test def * fix missing dep * remove class exclusion * move trim so null source versions are allowed * rename map * fix hardcoded value * remove unnecessary dep * use dashes for salesforce package name * fix typo * DRY and fix test image name * Fix edit * assert string is not empty * build integration test image only for integrationTest * move code generator to tools and rename docker build tasks * make source test depend on integration test build, not the other way * remove guard because the docker build should exist before the integrationtest is applied * remove comment * DRY up airbyte-source-test * fix plugin compilation * add missing dependency * rename getTaggedImage to getDevTaggedImage * fix test vs main docker build bug
Running Standard Source Tests for Python Integrations
How to
(note: This is all going to be moved into the template as soon as our templating engine is merged. This is doc is written as if that template exists already. So it assumes that the basic file structure and templated files exist.) In order to use the standard source tests, follow these steps:
- files for standard test
standardtest/__init__.pystandardtest/standard_source_test.pystandardtestresources/Dockerfile.testrequirements.standard-test.txt
- In
standardtest/standard_source_test.pyimplement the annotated methods. With information specific to your integration. - Any files placed in the
secretsandstandardtestdirectory will be accessible in the test container / test class viapkgutil.- e.g. you can access them will the following Python:
pkgutil.get_data(self.__class__.__module__.split(".")[0], "spec.json") - todo (cgardens) - we should standardize where
spec.jsongoes, so that we can reliably pull that as well. Right now we look for it in the main module and pull it if we can find it. Maybe we should move it to aresourcesfolder or something so that it is standardized.
- e.g. you can access them will the following Python:
- You can make modification to the test container in
Dockerfile.test - You can add any dependencies that the test code needs in
requirements.standard-test.txtfor local development and to thestandardtestrequirements insetup.pyfor when it runs in the docker container.- Note: by default, the test code does not depend on the main code.