6ca4017d28
* WIP Per-stream state support for Postgres source * Fix failing test * Improve code coverage * Make global the default state manager * Add legacy adapter state manager * Formatting * Include legacy state for backwards compatibility * Add global state manager * Implement Global/CDC state handling * Fix test issues * Fix issue with updated method signature * Handle empty state case in global state manager * Adjust to protocol changes * Fix failing acceptance tests * Fix failing test * Fix unmodifiable list issue * Fix unmodifiable exception * PR feedback * Abstract global state manager selection * Handle conversion between different state types * Handle invalid conversion * Rename parameter * Refactor state manager creation * Fix failing tests * Fix failing integration tests * Add CDC test * Fix failing integration test * Revert change * Fix failing integration test * Use per-stream for postgres tests * Formatting * Correct stream descriptor validation * Correct permalink * PR feedback
Standard Source Test
Overview
These tests are designed to do the following:
- Test basic functionality for any Airbyte source. Think of it as an "It works!" test.
- Each test should test functionality that is expected of all Airbyte sources.
- Require minimum effort from the author of the integration.
- Think of these are "free" tests that Airbyte provides to make sure the integration works.
- To be run for ALL Airbyte sources.
These tests are not designed to do the following:
- Test any integration-specific cases.
- Test corner cases for specific integrations.
- Replace good unit testing and integration testing for the integration.
We will not merge sources that cannot pass these tests unless the author can provide a very good reason™.
What tests do the standard tests include?
Check out each function in SourceAcceptanceTest. Each function annotated with @Test is a single test. Each of these functions is proceeded by comments that document what they are testing.
How to run them from your integration?
- If writing a source in Python, check out this readme
What do I need to provide as input to these tests?
- The name of the image of your integration (with the
:devtag at the end). - A handful of json inputs, e.g. a valid configuration file and a catalog that will be used to try to run the
readoperation on your integration. These are fully documented in SourceAcceptanceTest. Each method that is marked asabstractare methods that the user needs to implement to provide the necessary information. Each of these methods are preceded by comments explaining what they need to return. - Optionally you can run before and after methods before each test.
Do I have to write java to use these tests?
No! Our goal is to allow you to write your integration entirely in your language of choice. If you are writing an integration in Python, for instance, you should be able to interact with this test suite in python and not need to write java. Right now, we only have a Python interface to reduce friction for interacting with these tests, and with time, we intend to make more language-specific helpers available. In the meantime, however, you can still use your language of choice and leverage this standard test suite.
If working in Python, the Python interface that you need to implement can be found here. Our python template and singer template also walk you through how to implement the interface.