# Step 3: Connecting to the API We're now ready to start implementing the connector. Over the course of this tutorial, we'll be editing a few files that were generated by the code generator: - `source-exchange-rates-tutorial/source_exchange_rates_tutorial/manifest.yaml`: This is the connector manifest. It describes how the data should be read from the API source, as well as what inputs can be used to configure the connector. - `source-exchange_rates-tutorial/integration_tests/configured_catalog.json`: This is the connector's [catalog](../../../understanding-airbyte/beginners-guide-to-catalog.md). It describes what data is available in a source - `source-exchange-rates-tutorial/integration_tests/sample_state.json`: This is a sample state object to be used to test [incremental syncs](../../cdk-python/incremental-stream.md). We'll also be creating the following files: - `source-exchange-rates-tutorial/secrets/config.json`: This is the configuration file we'll be using to test the connector. Its schema should match the schema defined in the spec file. - `source-exchange-rates-tutorial/secrets/invalid_config.json`: This is an invalid configuration file we'll be using to test the connector. Its schema should match the schema defined in the spec file. - `source_exchange_rates_tutorial/schemas/rates.json`: This is the [schema definition](../../cdk-python/schemas.md) for the stream we'll implement. ## Updating the connector spec and config Let's populate the specification (`spec`) and the configuration (`secrets/config.json`) so the connector can access the access key and base currency. 1. We'll add these properties to the `spec` block in the `source-exchange-rates-tutorial/source_exchange_rates_tutorial/manifest.yaml` ```yaml spec: documentation_url: https://docs.airbyte.com/integrations/sources/exchangeratesapi connection_specification: $schema: http://json-schema.org/draft-07/schema# title: exchangeratesapi.io Source Spec type: object required: - access_key - base additionalProperties: true properties: access_key: type: string description: >- Your API Access Key. See here. The key is case sensitive. airbyte_secret: true base: type: string description: >- ISO reference currency. See here. examples: - EUR - USD ``` 2. We also need to fill in the connection config in the `secrets/config.json` Because of the sensitive nature of the access key, we recommend storing this config in the `secrets` directory because it is ignored by git. ```bash echo '{"access_key": "", "base": "USD"}' > secrets/config.json ``` ## Updating the connector definition Next, we'll update the connector definition (`source-exchange-rates-tutorial/source_exchange_rates_tutorial/manifest.yaml`). It was generated by the code generation script. More details on the connector definition file can be found in the [overview](../low-code-cdk-overview.md) and [connection definition](../understanding-the-yaml-file/yaml-overview.md) sections. Let's fill this out these TODOs with the information found in the [Exchange Rates API docs](https://apilayer.com/marketplace/exchangerates_data-api). 1. We'll first set the API's base url. According to the API documentation, the base url is `"https://api.apilayer.com"`. ```yaml definitions: <...> requester: url_base: "https://api.apilayer.com" ``` 2. Then, let's rename the stream from `customers` to `rates`, update the primary key to `date`, and set the path to "/exchangerates_data/latest" as per the API's documentation. This path is specific to the stream, so we'll set it within the `rates_stream` definition ```yaml rates_stream: $ref: "#/definitions/base_stream" $parameters: name: "rates" primary_key: "date" path: "/exchangerates_data/latest" ``` We'll also update the reference in the `streams` block ```yaml streams: - "#/definitions/rates_stream" ``` 3. Update the references in the `check` block ```yaml check: stream_names: - "rates" ``` Adding the reference in the `check` tells the `check` operation to use that stream to test the connection. 4. Next, we'll set up the authentication. The Exchange Rates API requires an access key to be passed as header named "apikey". This can be done using an `ApiKeyAuthenticator`, which we'll configure to point to the config's `access_key` field. ```yaml definitions: <...> requester: url_base: "https://api.apilayer.com" http_method: "GET" authenticator: type: ApiKeyAuthenticator header: "apikey" api_token: "{{ config['access_key'] }}" ``` 5. According to the ExchangeRatesApi documentation, we can specify the base currency of interest in a request parameter. Let's assume the user will configure this via the connector configuration in parameter called `base`; we'll pass the value input by the user as a request parameter: ```yaml definitions: <...> requester: <...> request_options_provider: request_parameters: base: "{{ config['base'] }}" ``` The full connector definition should now look like ```yaml version: "0.1.0" definitions: selector: extractor: field_path: [] requester: url_base: "https://api.apilayer.com" http_method: "GET" authenticator: type: ApiKeyAuthenticator header: "apikey" api_token: "{{ config['access_key'] }}" request_options_provider: request_parameters: base: "{{ config['base'] }}" retriever: record_selector: $ref: "#/definitions/selector" paginator: type: NoPagination requester: $ref: "#/definitions/requester" base_stream: retriever: $ref: "#/definitions/retriever" rates_stream: $ref: "#/definitions/base_stream" $parameters: name: "rates" primary_key: "date" path: "/exchangerates_data/latest" streams: - "#/definitions/rates_stream" check: stream_names: - "rates" spec: documentation_url: https://docs.airbyte.com/integrations/sources/exchangeratesapi connection_specification: $schema: http://json-schema.org/draft-07/schema# title: exchangeratesapi.io Source Spec type: object required: - access_key - base additionalProperties: true properties: access_key: type: string description: >- Your API Access Key. See here. The key is case sensitive. airbyte_secret: true base: type: string description: >- ISO reference currency. See here. examples: - EUR - USD ``` We can now run the `check` operation, which verifies the connector can connect to the API source. ```bash poetry run source-exchange-rates-tutorial check --config secrets/config.json ``` which should now succeed with logs similar to: ``` {"type": "LOG", "log": {"level": "INFO", "message": "Check succeeded"}} {"type": "CONNECTION_STATUS", "connectionStatus": {"status": "SUCCEEDED"}} ``` ## Next steps Next, we'll [extract the records from the response](4-reading-data.md) ## More readings - [Config-based connectors overview](../low-code-cdk-overview.md) - [Authentication](../understanding-the-yaml-file/authentication.md) - [Request options providers](../understanding-the-yaml-file/request-options.md) - [Schema definition](../../cdk-python/schemas.md) - [Connector specification reference](../../connector-specification-reference.md) - [Beginner's guide to catalog](../../../understanding-airbyte/beginners-guide-to-catalog.md)