ad78a287a7
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
18 KiB
products
| products |
|---|
| oss-community |
import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; import { FontAwesomeIcon } from "@fortawesome/react-fontawesome"; import { faWindows } from "@fortawesome/free-brands-svg-icons";
abctl
abctl is Airbyte's open source command line tool to create and manage local instances of Airbyte running in Docker.
Overview of abctl
Airbyte runs on Kubernetes. People run Airbyte in a diverse set of environments like a local computer, a bare metal server, or a virtual machine. However, you might not be running Kubernetes and might not even know much about it. abctl makes it easy to run Airbyte anywhere Docker is running.
When to use abctl
You use abctl to run Airbyte on a machine that isn't running a Kubernetes cluster, but is running Docker. Normally, you don't use abctl to manage enterprise deployments, because they use dedicated Kubernetes infrastructure. However, it's possible to use abctl this way if you want to.
What abctl does
abctl uses kind to create a Kubernetes cluster inside a Docker container. Then, it uses Helm to install the latest Airbyte and NGINX Ingress Controller Helm charts. It also helps you manage and understand that infrastructure.
Before you start
Before you use abctl, install Docker Desktop on your machine:
- Mac.
- Windows.
- Linux. If you're installing on a Linux headless virtual machine, you may have an easier time using Docker Engine instead of Docker Desktop. See the EC2 guide for an example.
Install abctl
To install abctl, follow the instructions for your operating system.
Use curl to install abctl.
-
Open a terminal and run the following command.
curl -LsfS https://get.airbyte.com | bash - -
If your terminal asks you to enter your password, do so.
When installation completes, you see abctl install succeeded.
Use go install to install abctl.
Open a terminal and run the following command.
go install github.com/airbytehq/abctl@latest
Use Homebrew to install abctl.
-
Install Homebrew, if you haven't already.
-
Run the following commands after Homebrew is installed.
brew tap airbytehq/tap brew install abctl -
Keep abctl up to date with Homebrew, too.
brew upgrade abctl
-
Verify your processor architecture.
uname -mIf the output is
x86_64, you'll download the linux-amd64 release. If the output isaarch64or similar, you'll download the linux-arm64 release. -
Download the file that is compatible with your machine's processor architecture
-
Extract the archive. This creates a directory named
abctl, which contains the executable and other needed files.tar -xvzf {name-of-file-downloaded.linux-*.tar.gz} -
Make the extracted executable accessible. This allows you to run
abctlas a command.chmod +x abctl/abctl -
Add
abctlto your PATH. This allows you to runabctlfrom any directory in your terminal.sudo mv abctl /usr/local/bin -
Verify the installation. If this command prints the installed version of abctl, you can now use it to manage a local Airbyte instance.
abctl version
-
Verify your processor architecture.
-
Press Windows + I.
-
Click System > About.
-
Next to Processor, if it says
AMD, you'll download the windows-amd64 release. If the output isARMor similar, you'll download the windows-arm64 release.
-
-
Download the latest release of
abctl. -
Extract the zip file to a destination of your choice. This creates a folder containing the abctl executable and other required files. Copy the filepath because you'll need this in a moment.
-
Add the executable to your
Pathenvironment variable.-
Click Start and type
environment. -
Click Edit the system environment variables. The System Properties opens.
-
Click Environment Variables.
-
Find the Path variable and click Edit.
-
Click New, then paste the filepath you saved in step 3.
-
Click OK, then click OK, then close the System Properties.
-
-
Open a new Command Prompt or PowerShell window. Changes to your Path variable only take effect in a new Window.
-
Verify abctl is installed correctly. If this command prints the installed version of abctl, you can now use it to manage a local Airbyte instance.
abctl version
Install and manage local Airbyte instances
This section shows you how to use abctl. It's not a step-by-step guide to deploy Airbyte. See Airbyte's Quickstart or deployment guides for a complete explanation.
Install or update Airbyte
To install or update Airbyte, run:
abctl local install
Optional flags are available and you can combine them in powerful ways. For example, you can create a Kubernetes secret and customize your Airbyte installation using a yaml file to make use of that secret.
abctl local install --secret YOUR_SECRET --values values.yaml
For a list of all flags, see the full reference.
:::note
Depending on your internet speed, abctl local install may take up to 30 minutes.
:::
Get information about your Airbyte instance
If an Airbyte installation exists, you can get critical information about that installation.
abctl local status
For example:
$ abctl local status
Existing cluster 'airbyte-abctl' found
Found helm chart 'airbyte-abctl'
Status: deployed
Chart Version: 0.422.2
App Version: 0.63.15
Found helm chart 'ingress-nginx'
Status: deployed
Chart Version: 4.11.1
App Version: 1.11.1
Airbyte should be accessible via http://localhost:8000
View and change credentials
If an Airbyte installation exists, you can view and update the credentials needed to access that installation.
To view credentials, run:
abctl local credentials
To update credentials, run:
abctl local credentials --email YOUR_EMAIL --password YOUR_PASSWORD
abctl returns something like this:
{
"email": "[YOUR EMAIL]",
"password": "[RANDOM PASSWORD]",
"client-id": "[RANDOM CLIENT-ID]",
"client-secret": "[RANDOM CLIENT-SECRET]"
}
Manage your local Kubernetes instance
To display Kubernetes deployment information, run:
abctl local deployments
If you need to restart Kubernetes, run:
abctl local deployments --restart
Uninstall Airbyte
To stop running all containers, but keep your data, run:
abctl local uninstall
To stop running containers and delete all data:
-
Uninstall Airbyte with the
--persistedflag.abctl local uninstall --persisted -
Clear any remaining information abctl created.
rm -rf ~/.airbyte/abctl
Manage Docker images
To get a manifest of the images used by Airbyte and abctl, run:
abctl images manifest
Get abctl version information
To display version information about the abctl tool, run abctl version.
$ abctl version
version: v0.19.0
Help and debugging
All abctl commands and sub-commands support two flags:
--help: Displays help information, describing the available options for this command.--verbose: Enables verbose/debug output. This is useful when debugging unexpected behavior.
Disable telemetry
You can turn off telemetry tracking on the abctl tool by setting the environment variable DO_NOT_TRACK to any value.
Full abctl reference
abctl has three commands: local, images, and version. Most commands have sub-commands and support various flags.
`local`
Local sub-commands are focused on managing the local Airbyte installation.
<details>
<summary>`credentials`</summary>
Display the credentials required to login to the local Airbyte installation.
When `abctl local install` is first executed, a random `password`, `client-id`, and `client-secret` are generated. Returns the `email`, `password`, `client-id`, and `client-secret` credentials. The `email` and `password` are required to login to Airbyte. The `client-id` and `client-secret` are necessary to create an [Access Token for interacting with the Airbyte API](https://reference.airbyte.com/reference/createaccesstoken).
`credentials` has the following flags.
| Name | Default | Description | Example |
| ---------- | ------- | ----------------------------------------- | ---------------- |
| --email | "" | Changes the authentication email address. | you@example.com |
| --password | "" | Changes the authentication password. | MyStrongPassword |
</details>
<details>
<summary>`deployments`</summary>
Display kubernetes deployment information and allows for restarting a kubernetes deployment.
`deployments` has the following flags.
| Name | Default | Description | Example |
| --------- | ------- | --------------------------------- | ------- |
| --restart | "" | Restarts the provided deployment. | webapp |
</details>
<details>
<summary>`install`</summary>
Installs a local Airbyte instance or updates an existing installation which was initially installed by `abctl`.
Depending on your internet speed, `abctl local install` may take up to 30 minutes.
`install` has the following flags.
| Name | Default | Description | Example |
| ------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------- |
| --chart | "" | Path to chart. | ./my-chart |
| --chart-version | latest | Which Airbyte helm-chart version to install. | 0.422.2 |
| --docker-email | "" | Docker email address to authenticate against `--docker-server`. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_EMAIL`. | user@example.com |
| --docker-password | "" | Docker password to authenticate against `--docker-server`. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_PASSWORD`. | mypassword |
| --docker-server | "" | Docker server to authenticate against. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_SERVER`. | docker.io |
| --docker-username | "" | Docker username to authenticate against `--docker-server`. Can also be specified by the environment-variable `ABCTL_LOCAL_INSTALL_DOCKER_USERNAME`. | myusername |
| --insecure-cookies | - | Disables secure cookie requirements. Only set if using `--host` with an insecure (non `https`) connection. | - |
| --low-resource-mode | false | Run Airbyte in low resource mode. | true |
| --host | "" | FQDN where the Airbyte installation will be accessed. Default is to allow for all incoming traffic on port `--port`. Set this if the Airbyte installation needs a more restricted host configuration. | airbyte.example.com |
| --no-browser | - | Disables launching the browser when installation completes. Useful to set in situations where no browser is available. | - |
| --port | 8000 | Port where the Airbyte installation will be accessed. Set this if port 8000 is already in use or if a different port is preferred. | 9000 |
| --secret | "" | **Can be set multiple times**. Creates a kubernetes secret based on the contents of the file provided. Useful when used in conjunction with `--values` for customizing installation. | ./my-secret.yaml |
| --values | "" | Helm values file to further customize the Airbyte installation. | ./values.yaml |
| --volume | "" | **Can be set multiple times**. Mounts additional volumes in the kubernetes cluster. Must be in the format of `<HOST_PATH>:<GUEST_PATH>`. | /host/path:/container/path |
</details>
<details>
<summary>`status`</summary>
If an Airbyte installation exists, returns information regarding that installation.
For example:
```
$ abctl local status
Existing cluster 'airbyte-abctl' found
Found helm chart 'airbyte-abctl'
Status: deployed
Chart Version: 0.422.2
App Version: 0.63.15
Found helm chart 'ingress-nginx'
Status: deployed
Chart Version: 4.11.1
App Version: 1.11.1
Airbyte should be accessible via http://localhost:8000
```
</details>
<details>
<summary>`uninstall`</summary>
Uninstalls a local Airbyte instance.
The data associated with the installed Airbyte instance will not be removed. This is done to allow Airbyte to be reinstalled at a later date with all the data preserved.
`uninstall` has the following flags.
| Name | Default | Description | Example |
| ----------- | ------- | ------------------------------------------------------------------------------ | ------- |
| --persisted | - | Will remove all data for the Airbyte installation. This cannot be undone. | - |
</details>
`images`
Manage images used by Airbyte and abctl.
<details>
<summary>`manifest`</summary>
Display a manifest of images used by Airbyte and abctl.
`manifest` has the following flags.
| Name | Default | Description | Example |
| --------------- | ------- | ---------------------------------------------------------------- | ------------- |
| --chart | "" | Path to chart. | ./my-chart |
| --chart-version | latest | Which Airbyte Helm chart version to install. | 0.422.2 |
| --values | "" | Helm values file to further customize the Airbyte installation. | ./values.yaml |
</details>
`version`
Displays version information about the `abctl` tool.
For example:
```
$ abctl version
version: v0.19.0
```