d9fa24ffff
* checkout from alex/cac * checkout from alex/cac * checkout from alex/cac * Add missing tests * Add missing files * Add missing tests * add missing file * missing file * missing file * rename * doc * doc * remove broken test * rename * jinja dependency * Add comment * comment * comment * pyjq dependency * rename file * delete unused file * Revert "delete unused file" This reverts commit758e939367. * fix * rename * abstract property * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit4a79223944. * isort * update state * fix imports * update dependency * format * rename file * decoder * Use decoder * Update comment * dict_state is actually backed by a dict * Add a comment * update state takes kwargs * move state out of offset paginator * update jq parameter order * update * remove incremental mixin * delete comment * update comments * update comments * remove no_state * rename package * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit4a79223944. * format * decoder * better error handling * remove nostate * isort * remove print * move test * delete duplicates * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * pass parameters through kwargs * update interface to pass source in interface * update interface to pass source in interface * rename to stream_slicer * Allow passing a string or an enum * Define StateType enum * convert state_type if not of type type * convert state_type if not of type type * Low code connectors: string interpolation with jinja (#12852) * checkout from alex/cac * Add missing tests * Add missing files * missing file * rename * jinja dependency * Add comment * comment * comment * Revert "delete unused file" This reverts commit758e939367. * delete unused field * delete unused field * rename * pass kwargs directly * isort * Revert "isort" This reverts commit4a79223944. * format * decoder * better error handling * remove nostate * isort * delete dead code * Update mapping type to [str, Any] * add comment * Add comment * pass parameters through kwargs * move test to right module * Add missing test * Use authbase instead of deprecated class * leverage generator * Delete dead code * rename methods * rename to declarative * rename the classes too * Try to install packages to build jq * isort * only automake * Revert "only automake" This reverts commitc8fe154ffc. * remove git * format * Add jq dependency * Use request header provider * rename * rename field * remove get_context method * rename
67 lines
1.7 KiB
Python
67 lines
1.7 KiB
Python
#
|
|
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
|
|
#
|
|
|
|
import json
|
|
|
|
import requests
|
|
from airbyte_cdk.sources.declarative.decoders.json_decoder import JsonDecoder
|
|
from airbyte_cdk.sources.declarative.extractors.jq import JqExtractor
|
|
|
|
config = {"field": "record_array"}
|
|
decoder = JsonDecoder()
|
|
|
|
|
|
def test():
|
|
transform = ".data[]"
|
|
extractor = JqExtractor(transform, decoder, config)
|
|
|
|
records = [{"id": 1}, {"id": 2}]
|
|
body = {"data": records}
|
|
response = create_response(body)
|
|
actual_records = extractor.extract_records(response)
|
|
|
|
assert actual_records == records
|
|
|
|
|
|
def test_field_in_config():
|
|
transform = ".{{ config['field'] }}[]"
|
|
extractor = JqExtractor(transform, decoder, config)
|
|
|
|
records = [{"id": 1}, {"id": 2}]
|
|
body = {"record_array": records}
|
|
response = create_response(body)
|
|
actual_records = extractor.extract_records(response)
|
|
|
|
assert actual_records == records
|
|
|
|
|
|
def test_field_in_kwargs():
|
|
transform = ".{{ kwargs['data_field'] }}[]"
|
|
kwargs = {"data_field": "records"}
|
|
extractor = JqExtractor(transform, decoder, config, kwargs=kwargs)
|
|
|
|
records = [{"id": 1}, {"id": 2}]
|
|
body = {"records": records}
|
|
response = create_response(body)
|
|
actual_records = extractor.extract_records(response)
|
|
|
|
assert actual_records == records
|
|
|
|
|
|
def create_response(body):
|
|
response = requests.Response()
|
|
response._content = json.dumps(body).encode("utf-8")
|
|
return response
|
|
|
|
|
|
def test_default():
|
|
transform = ".{{kwargs['field']}}[]"
|
|
extractor = JqExtractor(transform, decoder, config)
|
|
|
|
records = [{"id": 1}, {"id": 2}]
|
|
response = create_response(records)
|
|
actual_records = extractor.extract_records(response)
|
|
|
|
assert actual_records == records
|