airbyte/docs/connector-development/connector-metadata-file.md

# Connector Metadata.yaml File

The `metadata.yaml` file is a new addition to Airbyte's connector folders. This file is created with the goal of simplifying and enhancing how we manage information related to each connector. It is designed to replace the previous `source_definitions.yaml` and `destinations_definitions.yaml` files.

The `metadata.yaml` file contains crucial information about the connector, including its type, definition ID, Docker image tag, Docker repository, and much more. It plays a key role in the way Airbyte handles connector data and improves the overall organization and accessibility of this data.

## Structure

Below is an example of a `metadata.yaml` file for the Postgres source:

```yaml
data:
  allowedHosts:
    hosts:
      - ${host}
      - ${tunnel_method.tunnel_host}
  connectorSubtype: database
  connectorType: source
  definitionId: decd338e-5647-4c0b-adf4-da0e75f5a750
  dockerImageTag: 2.0.28
  maxSecondsBetweenMessages: 7200
  dockerRepository: airbyte/source-postgres
  githubIssueLabel: source-postgres
  icon: postgresql.svg
  license: MIT
  name: Postgres
  tags:
    - language:java
    - language:python
  registries:
    cloud:
      dockerRepository: airbyte/source-postgres-strict-encrypt
      enabled: true
    oss:
      enabled: true
  releaseStage: generally_available
  documentationUrl: https://docs.airbyte.com/integrations/sources/postgres
metadataSpecVersion: "1.0"
```

## The `registries` Section

The `registries` section within the `metadata.yaml` file plays a vital role in determining the contents of the `oss_registry.json` and `cloud_registry.json` files.

This section contains two subsections: `cloud` and `oss` (Open Source Software). Each subsection contains details about the specific registry, such as the Docker repository associated with it and whether it's enabled or not.

### Structure

Here's how the `registries` section is structured in our previous `metadata.yaml` example:

```yaml
  registries:
    cloud:
      dockerRepository: airbyte/source-postgres-strict-encrypt
      enabled: true
    oss:
      enabled: true
```

In this example, both `cloud` and `oss` registries are enabled, and the Docker repository for the `cloud` registry is overrode to `airbyte/source-postgres-strict-encrypt`.

### Updating Registries

When the `metadata.yaml` file is updated, this data is automatically uploaded to Airbyte's metadata service. This service then generates the publicly available `oss_registry.json` and `cloud_registry.json` registries based on the information provided in the `registries` section.

For instance, if a connector is listed as `enabled: true` under the `oss` section, it will be included in the `oss_registry.json` file. Similarly, if it's listed as `enabled: true` under the `cloud` section, it will be included in the `cloud_registry.json` file.

Thus, the `registries` section in the `metadata.yaml` file provides a flexible and organized way to manage which connectors are included in each registry.

## The `tags` Section

The `tags` field is an optional part of the `metadata.yaml` file. It is designed to provide additional context about a connector and improve the connector's discoverability. This field can contain any string, making it a flexible tool for adding additional details about a connector.

In the `metadata.yaml` file, `tags` is a list that may contain any number of string elements. Each element in the list is a separate tag. For instance:

```yaml
tags:
  - "language:java"
  - "keyword:database"
  - "keyword:SQL"
```
In the example above, the connector has three tags. Tags are used for two primary purposes in Airbyte:

1. **Denoting the Programming Language(s)**: Tags that begin with language: are used to specify the programming languages that are utilized by the connector. This information is auto-generated by a script that scans the connector's files for recognized programming languages. In the example above, language:java means that the connector uses Java.

2. **Keywords for Searching**: Tags that begin with keyword: are used to make the connector more discoverable by adding searchable terms related to it. In the example above, the tags keyword:database and keyword:SQL can be used to find this connector when searching for `database` or `SQL`.

These are just examples of how tags can be used. As a free-form field, the tags list can be customized as required for each connector. This flexibility allows tags to be a powerful tool for managing and discovering connectors.

## The `icon` Field
This denotes the name of the icon file for the connector. At this time the icon file is located in the `platform-internal` repository. So please ensure that the icon file is present in the `platform-internal` repository at [oss/airbyte-config/init/src/main/resources/icons](https://github.com/airbytehq/airbyte-platform-internal/tree/master/oss/airbyte-config/init/src/main/resources/icons) before adding the icon name to the `metadata.yaml` file.

### Future Plans
_⚠️ This property is in the process of being refactored to be a file in the connector folder_

You may notice a `icon.svg` file in the connectors folder.

This is because we are transitioning away from icons being stored in the `platform-internal` repository. Instead, we will be storing them in the connector folder itself. This will allow us to have a single source of truth for all connector-related information.

This transition is currently in progress. Once it is complete, the `icon` field in the `metadata.yaml` file will be removed, and the `icon.svg` file will be used instead.