From 3c048aa14ed2c5ae3f96837a0554a2dbc5f4dbbd Mon Sep 17 00:00:00 2001 From: Michel Tricot Date: Sun, 13 Dec 2020 22:33:50 -0800 Subject: [PATCH] restore images --- README.md | 8 ++++++-- docs/architecture/high-level-view.md | 2 ++ docs/deploying-airbyte/on-aws-ec2.md | 12 ++++++++++++ docs/deploying-airbyte/on-gcp-compute-engine.md | 4 ++++ docs/integrations/sources/facebook-marketing-api.md | 5 ++++- docs/integrations/sources/google-sheets.md | 2 ++ docs/tutorials/airbyte-config-persistence.md | 12 ++++++++++++ docs/tutorials/getting-started.md | 8 ++++++++ 8 files changed, 50 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 8c28eeb3c5e..8064cf637f5 100644 --- a/README.md +++ b/README.md @@ -2,10 +2,14 @@ ![GitHub Workflow Status](https://img.shields.io/github/workflow/status/airbytehq/airbyte/Airbyte%20CI) ![License](https://img.shields.io/github/license/airbytehq/airbyte) +![](docs/.gitbook/assets/airbyte_horizontal_color_white-background.svg) + **Data integration made simple, secure and extensible.** The new open-source standard to sync data from applications, APIs & databases to warehouses. -[](docs/deploying-airbyte/on-your-workstation.md) [](docs/deploying-airbyte/on-aws-ec2.md) [](docs/deploying-airbyte/on-gcp-compute-engine.md) +[![](docs/.gitbook/assets/deploy-locally.svg)](docs/deploying-airbyte/on-your-workstation.md) [![](docs/.gitbook/assets/deploy-on-aws.svg)](docs/deploying-airbyte/on-aws-ec2.md) [![](docs/.gitbook/assets/deploy-on-gcp.svg)](docs/deploying-airbyte/on-gcp-compute-engine.md) + +![](docs/.gitbook/assets/airbyte-ui-for-your-integration-pipelines.png) Airbyte is on a mission to make data integration pipelines a commodity. @@ -26,7 +30,7 @@ docker-compose up Now visit [http://localhost:8000](http://localhost:8000) -Here is a [step-by-step guide]() showing you how to load data from a sample Postgres database into another database using Airbyte, all on your computer. +Here is a [step-by-step guide](docs/tutorials/getting-started.md) showing you how to load data from a sample Postgres database into another database using Airbyte, all on your computer. ## Features diff --git a/docs/architecture/high-level-view.md b/docs/architecture/high-level-view.md index f18afa60795..09b6dd1da00 100644 --- a/docs/architecture/high-level-view.md +++ b/docs/architecture/high-level-view.md @@ -4,6 +4,8 @@ description: A high level view of Airbyte's components. # High-level View +![3.048-Kilometer view](../.gitbook/assets/10-000-feet-view.png) + * `UI`: Acts as the control center for Airbyte. From the UI, you can configure new integration connections. You can also track the different syncing jobs and view logs. * `Config Store`: Stores all the connections information \(credentials, frequency...\). * `Scheduler Store`: Stores statuses and job information for the scheduler bookkeeping. diff --git a/docs/deploying-airbyte/on-aws-ec2.md b/docs/deploying-airbyte/on-aws-ec2.md index 27ee974c6fe..32af1b67ae6 100644 --- a/docs/deploying-airbyte/on-aws-ec2.md +++ b/docs/deploying-airbyte/on-aws-ec2.md @@ -8,12 +8,18 @@ The instructions have been tested on `Amazon Linux 2 AMI (HVM)` * Launch a new instance +![](../.gitbook/assets/aws_ec2_launch.png) + * Select instance AMI +![](../.gitbook/assets/aws_ec2_ami.png) + * Select instance type * For testing out Airbyte, a `t2.medium` instance is likely sufficient. * For long-running Airbyte installations, we recommend a `t2.large` instance. +![](../.gitbook/assets/aws_ec2_instance_type.png) + * `Next: Configure Instance Details` * You can tune parameters or keep the defaults * `Next: Add Storage` @@ -23,13 +29,19 @@ The instructions have been tested on `Amazon Linux 2 AMI (HVM)` * `Next: Configure Security Groups` * We are going to allow network for `ssh` +![](../.gitbook/assets/aws_ec2_security_group.png) + * `Review and Launch` * `Launch` * Create a ssh key so you can connect to the instance * Download the key \(and don't lose it or you won't be able to connect to the instance\) +![](../.gitbook/assets/aws_ec2_ssh_key.png) + * `Launch Instances` +![](../.gitbook/assets/aws_ec2_instance_view.png) + * Wait for the instance to become `Running` ## Install environment diff --git a/docs/deploying-airbyte/on-gcp-compute-engine.md b/docs/deploying-airbyte/on-gcp-compute-engine.md index 19f6abb0ad5..b7fdb01cc67 100644 --- a/docs/deploying-airbyte/on-gcp-compute-engine.md +++ b/docs/deploying-airbyte/on-gcp-compute-engine.md @@ -8,10 +8,14 @@ The instructions have been tested on `Debian GNU/Linux 10 (buster)` * Launch a new instance +![](../.gitbook/assets/gcp_ce_launch.png) + * Configure new instance * For testing out Airbyte, an `e2.medium` instance is likely sufficient. * For long-running Airbyte installations, we recommend a `n1-standard-2` instance. +![](../.gitbook/assets/gcp_ce_configure.png) + * `Create` ## Install environment diff --git a/docs/integrations/sources/facebook-marketing-api.md b/docs/integrations/sources/facebook-marketing-api.md index a8d0dd61456..5eab689b8e9 100644 --- a/docs/integrations/sources/facebook-marketing-api.md +++ b/docs/integrations/sources/facebook-marketing-api.md @@ -75,9 +75,12 @@ Visit the [Facebook Developers App hub](https://developers.facebook.com/apps/) a From the App's Dashboard screen \(seen in the screenshot below\) enable the Marketing API for your app if it is not already setup. +![](../../.gitbook/assets/screen-shot-2020-11-03-at-9.25.21-pm.png) + ### API Access Token In the App Dashboard screen, click Marketing API --> Tools on the left sidebar. Then highlight all the available token permissions \(`ads_management`, `ads_read`, `read_insights`\) and click "Get token". A long string of characters should appear in front of you; **this is the access token.** Copy this string for use in the Airbyte UI later. -With the Ad Account ID and API access token, you should be ready to start pulling data from the Facebook Marketing API. Head to the Airbyte UI to setup your source connector! +![](../../.gitbook/assets/screen-shot-2020-11-03-at-9.35.40-pm.png) +With the Ad Account ID and API access token, you should be ready to start pulling data from the Facebook Marketing API. Head to the Airbyte UI to setup your source connector! diff --git a/docs/integrations/sources/google-sheets.md b/docs/integrations/sources/google-sheets.md index e15814d4a8b..cdc6d93f911 100644 --- a/docs/integrations/sources/google-sheets.md +++ b/docs/integrations/sources/google-sheets.md @@ -74,6 +74,8 @@ Once you've created the Service Account, you need to explicitly give it access t Finally, you'll need the ID of the Spreadsheet you'd like to sync. To get it, navigate to the spreadsheet in your browser, then copy the portion of the URL which comes after "/d" and before "/edit" or "/view". This is the highlighted portion of the screenshot below: +![](../../.gitbook/assets/screen-shot-2020-10-30-at-2.44.55-pm.png) + ### Setting up in the Airbyte UI The Airbyte UI will ask for two things: diff --git a/docs/tutorials/airbyte-config-persistence.md b/docs/tutorials/airbyte-config-persistence.md index d7e430d8ba5..f383ce304b3 100644 --- a/docs/tutorials/airbyte-config-persistence.md +++ b/docs/tutorials/airbyte-config-persistence.md @@ -151,8 +151,12 @@ docker cp airbyte-server:/data $TUTORIAL_DIR/my-setup Head back to [http://localhost:8000/](http://localhost:8000/) and add more connectors. Here is an example of configuration from an API source: +![](../.gitbook/assets/airbyte_config_persistence_1.png) + and a local file destination: +![](../.gitbook/assets/airbyte_config_persistence_2.png) + ## Run a Sync job * once the source and destination are created @@ -160,6 +164,8 @@ and a local file destination: * then run the "Sync Now" button * finally inspect logs in the UI +![](../.gitbook/assets/airbyte_config_persistence_3.png) + ## Exploring Logs folders We can read from the lines reported in the logs the working directory that is being used to run the synchronization process from. @@ -356,10 +362,16 @@ Here are some examples of public API CSV: https://storage.googleapis.com/covid19-open-data/v2/latest/epidemiology.csv ``` +![](../.gitbook/assets/airbyte_config_persistence_4.png) + And a local Postgres Database: +![](../.gitbook/assets/airbyte_config_persistence_5.png) + After setting up the connectors, we can trigger the sync and study the logs: +![](../.gitbook/assets/airbyte_config_persistence_6.png) + Since we wiped the workspace volume and restarted the Airbyte Server, notice that the process ran in the `/tmp/workspace/5/0` as well but the logs for ExchangeRate are gone... ## Export and customize Normalization step with DBT diff --git a/docs/tutorials/getting-started.md b/docs/tutorials/getting-started.md index 6519c0d1ec1..bdabaf4c2a6 100644 --- a/docs/tutorials/getting-started.md +++ b/docs/tutorials/getting-started.md @@ -24,10 +24,14 @@ Once you see an Airbyte banner, the UI is ready to go at [http://localhost:8000/ You should see an onboarding page. Enter your email if you want updates about Airbyte and continue. +![](../.gitbook/assets/airbyte_get-started.png) + ## 2. Set up your first connection Now you will see a wizard that allows you choose the data you want to send through Airbyte. +![](../.gitbook/assets/02_set-up-sources.png) + As of our alpha launch, we have one database source \(Postgres\) and two API sources \(an exchange rate API and the Stripe API\). We're currently building an integration framework that makes it easy to create sources and destinations, so you should expect many more soon. Please reach out to us if you need a specific connector or would like to help build one. For now, we will start out with a Postgres source and destination. @@ -72,12 +76,16 @@ DB Name: postgres After adding the destination, you can choose what tables and columns you want to sync. +![](../.gitbook/assets/03_set-up-connection.png) + For this demo, we recommend leaving the defaults and selecting "Every 5 Minutes" as the frequency. Click `Set Up Connection` to finish setting up the sync. ## 3. Check the logs of your first sync You should now see a list of sources with the source you just added. Click on it to find more information about your connection. This is the page where you can update any settings about this source and how it syncs. There should be a `Completed` job under the history section. If you click on that run, it will show logs from that run. +![](../.gitbook/assets/04_source-details.png) + One of biggest problems we've seen in tools like Fivetran is the lack of visibility when debugging. In Airbyte, allowing full log access and the ability to debug and fix connector problems is one of our highest priorities. We'll be working hard to make these logs accessible and understandable. ## 4. Check if the syncing actually worked