🐙 Octavia CLI
Octavia CLI is a tool to manage Airbyte configuration in YAML. It has the following features:
- Scaffolding of a readable directory architecture that will host the YAML configs.
- Auto-generation of YAML config file that matches the resources' schemas.
- Manage Airbyte resources with YAML config files.
- Safe resources update through diff display and validation.
- Simple secret management to avoid versioning credentials.
The project is under development: readers can refer to our tech spec deck for an introduction to the tool.
Install
1. Install and run Docker
We are packaging this CLI as a Docker image to avoid dependency hell, please install and run Docker if you are not.
2.a If you are using ZSH / Bash:
curl -o- https://raw.githubusercontent.com/airbytehq/airbyte/master/octavia-cli/install.sh | bash
This script:
- Pulls the octavia-cli image from our Docker registry.
- Creates an
octaviaalias in your profile.
2.b If you want to directly run the CLI without alias in your current directory:
mkdir my_octavia_project_directory # Create your octavia project directory where YAML configurations will be stored.
docker run --rm -v ./my_octavia_project_directory:/home/octavia-project --network host -e AIRBYTE_URL="http://localhost:8000" airbyte/octavia-cli:dev
Current development status
Octavia is currently under development. You can find a detailed and updated execution plan here. We welcome community contributions!
Secret management
Sources and destinations configurations have credential fields that you do not want to store as plain text and version on Git.
octavia offers secret management through environment variables expansion:
configuration:
password: ${MY_PASSWORD}
If you have set a MY_PASSWORD environment variable, octavia apply will load its value into the password field.
Summary of achievements:
| Date | Milestone |
|---|---|
| 2022-03-09 | Implement secret management through environment variable expansion |
| 2022-03-09 | Implement octavia generate connection |
| 2022-03-09 | Implement octavia apply for connections |
| 2022-03-02 | Implement octavia apply (sources and destination only) |
| 2022-02-06 | Implement octavia generate (sources and destination only) |
| 2022-01-25 | Implement octavia init + some context checks |
| 2022-01-19 | Implement octavia list workspace sources, octavia list workspace destinations, octavia list workspace connections |
| 2022-01-17 | Implement octavia list connectors source and octavia list connectors destinations |
| 2022-01-17 | Generate an API Python client from our Open API spec |
| 2021-12-22 | Bootstrapping the project's code base |
Developing locally
- Install Python 3.8.12. We suggest doing it through
pyenv - Create a virtualenv:
python -m venv .venv - Activate the virtualenv:
source .venv/bin/activate - Install dev dependencies:
pip install -e .\[tests\] - Install
pre-commithooks:pre-commit install - Run the unittest suite:
pytest --cov=octavia_cli - Iterate: please check the Contributing for instructions on contributing.
Build
Build the project locally (from the root of the repo):
SUB_BUILD=OCTAVIA_CLI ./gradlew build # from the root directory of the repo
Contributing
- Please sign up to Airbyte's Slack workspace and join the
#octavia-cli. We'll sync up community efforts in this channel. - Read the execution plan and find a task you'd like to work on.
- Open a PR, make sure to test your code thoroughly.