GitBook: [master] 11 pages and 21 assets modified
This commit is contained in:
Michel Tricot
committed by
gitbook-bot
gitbook-bot
parent
6f817a855c
commit
f914670461
@@ -2,15 +2,15 @@
|
||||
|
||||
 
|
||||
|
||||
|
||||
|
||||
|
||||
**Data integration made simple, secure and extensible.**
|
||||
|
||||
The new open-source standard to sync data from applications, APIs & databases to warehouses.
|
||||
|
||||
[](docs/deploying-airbyte/on-your-workstation.md) [](docs/deploying-airbyte/on-aws-ec2.md) [](docs/deploying-airbyte/on-gcp-compute-engine.md)
|
||||
[](docs/deploying-airbyte/on-your-workstation.md) [](docs/deploying-airbyte/on-aws-ec2.md) [](docs/deploying-airbyte/on-gcp-compute-engine.md)
|
||||
|
||||
|
||||
|
||||
|
||||
Airbyte is on a mission to make data integration pipelines a commodity.
|
||||
|
||||
|
||||
BIN
docs/.gitbook/assets/02_set-up-sources (1) (1) (1) (1).png
Normal file
BIN
docs/.gitbook/assets/02_set-up-sources (1) (1) (1) (1).png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 185 KiB |
BIN
docs/.gitbook/assets/03_set-up-connection (1) (1) (1) (1).png
Normal file
BIN
docs/.gitbook/assets/03_set-up-connection (1) (1) (1) (1).png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 236 KiB |
BIN
docs/.gitbook/assets/04_source-details (1) (1) (1) (1).png
Normal file
BIN
docs/.gitbook/assets/04_source-details (1) (1) (1) (1).png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 214 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 19 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 155 KiB |
BIN
docs/.gitbook/assets/airbyte_get-started (1) (1) (1).png
Normal file
BIN
docs/.gitbook/assets/airbyte_get-started (1) (1) (1).png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 249 KiB |
11
docs/.gitbook/assets/airbyte_horizontal_dark (1) (1) (1).svg
Normal file
11
docs/.gitbook/assets/airbyte_horizontal_dark (1) (1) (1).svg
Normal file
@@ -0,0 +1,11 @@
|
||||
<svg width="1000" height="237" viewBox="0 0 1000 237" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M174.31 15.61C158.599 8.13269 140.887 5.95043 123.831 9.39048C106.775 12.8305 91.2943 21.7075 79.71 34.69C76.2614 38.6178 73.1974 42.8673 70.56 47.38L7.13 157.25C12.1719 160.155 18.1595 160.944 23.782 159.445C29.4046 157.946 34.2041 154.28 37.13 149.25L61.66 106.65C63.9539 116.453 68.0833 125.733 73.83 134L33.83 203.37C38.8719 206.275 44.8595 207.064 50.482 205.565C56.1046 204.066 60.9041 200.4 63.83 195.37L100.72 131.47C98.8124 129.769 97.0216 127.942 95.36 126C88.4067 117.87 83.8221 107.986 82.1073 97.4267C80.3925 86.8674 81.6134 76.04 85.6368 66.1279C89.6601 56.2158 96.3309 47.6004 104.92 41.2235C113.509 34.8467 123.686 30.9537 134.338 29.9703C144.99 28.9868 155.708 30.9506 165.32 35.647C174.931 40.3435 183.067 47.5918 188.837 56.5997C194.607 65.6077 197.79 76.0285 198.037 86.7232C198.285 97.4179 195.587 107.975 190.24 117.24L127.08 226.51C132.122 229.415 138.109 230.204 143.732 228.705C149.355 227.206 154.154 223.54 157.08 218.51L209.25 128.29C214.783 118.694 218.273 108.058 219.502 97.0501C220.731 86.0418 219.672 74.8983 216.391 64.3186C213.11 53.7389 207.679 43.9511 200.438 35.5693C193.196 27.1874 184.301 20.3922 174.31 15.61V15.61Z" fill="#36348F"/>
|
||||
<path d="M164.64 113.38C168.038 107.519 169.492 100.732 168.793 93.9926C168.094 87.2536 165.279 80.9092 160.75 75.87C156.221 70.8308 150.213 67.3558 143.586 65.9438C136.96 64.5317 130.057 65.2552 123.867 68.0103C117.677 70.7655 112.52 75.4107 109.134 81.2793C105.748 87.1479 104.309 93.9381 105.023 100.676C105.736 107.413 108.565 113.751 113.105 118.781C117.644 123.81 123.661 127.272 130.29 128.67L73.81 226.51C78.8519 229.415 84.8395 230.204 90.462 228.705C96.0845 227.206 100.884 223.54 103.81 218.51L164.64 113.38ZM128.23 92.38C128.885 91.2345 129.76 90.2299 130.805 89.4242C131.85 88.6184 133.044 88.0275 134.319 87.6854C135.593 87.3433 136.923 87.2569 138.231 87.431C139.539 87.6051 140.799 88.0364 141.94 88.7V88.7C143.647 89.693 145.021 91.169 145.89 92.9421C146.76 94.7152 147.084 96.7058 146.824 98.6632C146.563 100.62 145.729 102.457 144.426 103.941C143.124 105.425 141.411 106.49 139.504 107.002C137.597 107.514 135.581 107.449 133.71 106.817C131.839 106.185 130.198 105.013 128.992 103.45C127.786 101.886 127.07 100 126.935 98.0301C126.799 96.0602 127.25 94.0942 128.23 92.38V92.38Z" fill="#36348F"/>
|
||||
<path d="M388.57 115V158.38C388.57 165.91 384.35 170.43 377.27 170.43C370.34 170.43 366.12 165.91 366.12 158.38V141.22H316.42V158.69C316.42 166.07 312.2 170.44 305.42 170.44C298.49 170.44 294.42 166.07 294.42 158.69V115C294.42 84.12 311.42 66.8 341.42 66.8C371.42 66.8 388.57 84.13 388.57 115ZM366.12 114.1C366.12 96.62 357.39 87.29 341.27 87.29C325.15 87.29 316.42 96.62 316.42 114.1V121H366.12V114.1Z" fill="#36348F"/>
|
||||
<path d="M438.89 79.77V158.39C438.89 165.92 434.67 170.44 427.75 170.44C420.67 170.44 416.45 165.92 416.45 158.39V79.77C416.45 72.23 420.67 67.71 427.75 67.71C434.67 67.71 438.89 72.23 438.89 79.77Z" fill="#36348F"/>
|
||||
<path d="M553.82 151.61C555.55 153.799 556.503 156.5 556.53 159.29C556.53 166.22 552.01 170.44 545.08 170.44C541.17 170.44 538.15 168.63 535.74 165.44L512.25 134.26H490.56V158.51C490.56 165.89 486.34 170.41 479.41 170.41C472.33 170.41 468.11 165.89 468.11 158.51V80.67C468.11 73.14 472.63 68.67 480.16 68.67H520.68C543.43 68.67 555.68 79.52 555.68 100.15C555.68 115.51 549.05 125.45 536.54 129.67V129.97L553.82 151.61ZM518.27 113.81C527.61 113.81 532.73 109.29 532.73 101.45C532.73 93.61 527.61 89.11 518.27 89.11H490.56V113.81H518.27Z" fill="#36348F"/>
|
||||
<path d="M657.61 118.48C667.4 122.39 672.52 130.07 672.52 141.22C672.52 160.65 661.83 169.54 638.63 169.54H593.75C586.21 169.54 581.75 165.02 581.75 157.54V80.67C581.75 73.14 586.26 68.67 593.75 68.67H635.62C658.81 68.67 669.51 77.56 669.51 96.84C669.51 106.63 665.44 114.01 657.61 118.22V118.48ZM634.72 109.29C642.72 109.29 646.92 106.12 646.92 99.35C646.92 92.58 642.7 89.11 634.72 89.11H604.14V109.29H634.72ZM637.13 149.05C645.26 149.05 649.48 145.59 649.48 138.66C649.48 131.88 645.26 128.57 637.13 128.57H604.13V149.05H637.13Z" fill="#36348F"/>
|
||||
<path d="M761.1 72.08C762.157 70.7262 763.507 69.6297 765.048 68.8728C766.59 68.1159 768.283 67.7183 770 67.71C776.47 67.71 781 71.93 781 78.56C781.02 81.7383 779.837 84.8066 777.69 87.15L743 128.87V158.39C743 165.92 738.79 170.44 731.86 170.44C724.78 170.44 720.56 165.92 720.56 158.39V128.87L686.22 87.87C683.81 84.87 682.6 82.45 682.6 79.13C682.6 72.05 687.43 67.68 693.91 67.68C697.82 67.68 700.68 69.68 703.24 72.81L732 108.08L761.1 72.08Z" fill="#36348F"/>
|
||||
<path d="M880.65 68.62C887.43 68.62 891.5 72.53 891.5 78.86C891.5 85.19 887.43 89.11 880.65 89.11H854V158.39C854 165.92 849.78 170.44 842.85 170.44C835.77 170.44 831.56 165.92 831.56 158.39V89.11H804.89C798.11 89.11 794.04 85.19 794.04 78.86C794.04 72.53 798.11 68.62 804.89 68.62H880.65Z" fill="#36348F"/>
|
||||
<path d="M982 149.05C988.78 149.05 992.85 152.97 992.85 159.29C992.85 165.61 988.78 169.54 982 169.54H918.47C910.94 169.54 906.42 165.02 906.42 157.54V80.67C906.42 73.14 910.94 68.67 918.47 68.67H982C988.78 68.67 992.85 72.58 992.85 78.91C992.85 85.24 988.81 89.11 982 89.11H928.86V108.84H971.19C977.96 108.84 981.73 112.6 981.73 118.78C981.73 124.96 977.96 128.72 971.19 128.72H928.86V149.05H982Z" fill="#36348F"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 5.2 KiB |
BIN
docs/.gitbook/assets/aws_ec2_ami (1) (1) (1) (2) (1) (1) (1).png
Normal file
BIN
docs/.gitbook/assets/aws_ec2_ami (1) (1) (1) (2) (1) (1) (1).png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 47 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 76 KiB |
BIN
docs/.gitbook/assets/aws_ec2_ssh_key (1) (1) (1) (2) (1) (1).png
Normal file
BIN
docs/.gitbook/assets/aws_ec2_ssh_key (1) (1) (1) (2) (1) (1).png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 89 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 8.6 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 7.8 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 8.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 111 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 254 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 154 KiB |
@@ -4,7 +4,7 @@ description: Here is a high level view of Airbyte's components.
|
||||
|
||||
# Architecture
|
||||
|
||||
|
||||
|
||||
|
||||
* `UI`: Acts as the control center for Airbyte. From the UI, you can configure new integration connections. You can also track the different syncing jobs and view logs.
|
||||
* `Config Store`: Stores all the connections information \(credentials, frequency...\).
|
||||
|
||||
@@ -30,13 +30,11 @@ It's easy to add your own integrations to Airbyte! **Since Airbyte connectors ar
|
||||
|
||||
#### **Contributing sources:**
|
||||
|
||||
You can build a Source in any language. Since we frequently build integrations in Python or on top of Singer taps, we've created generator libraries to get you started quickly.
|
||||
See [Creating a new integration](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations#creating-a-new-integration) to get started.
|
||||
You can build a Source in any language. Since we frequently build integrations in Python or on top of Singer taps, we've created generator libraries to get you started quickly. See [Creating a new integration](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations#creating-a-new-integration) to get started.
|
||||
|
||||
#### **Contributing destinations:**
|
||||
|
||||
You can build a Destination in any language. Since we frequently build destinations with Java, we've created generator libraries to get you started quickly.
|
||||
See [Creating a new integration](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations#creating-a-new-integration) to get started.
|
||||
You can build a Destination in any language. Since we frequently build destinations with Java, we've created generator libraries to get you started quickly. See [Creating a new integration](https://github.com/airbytehq/airbyte/tree/master/airbyte-integrations#creating-a-new-integration) to get started.
|
||||
|
||||
### **Documentation**
|
||||
|
||||
|
||||
@@ -8,15 +8,15 @@ The instructions have been tested on `Amazon Linux 2 AMI (HVM)`
|
||||
|
||||
* Launch a new instance
|
||||
|
||||
|
||||
|
||||
|
||||
* Select instance AMI
|
||||
|
||||
|
||||
|
||||
|
||||
* Select instance type
|
||||
|
||||
|
||||
|
||||
|
||||
* `Next: Configure Instance Details`
|
||||
* You can tune parameters or keep the defaults
|
||||
@@ -27,18 +27,18 @@ The instructions have been tested on `Amazon Linux 2 AMI (HVM)`
|
||||
* `Next: Configure Security Groups`
|
||||
* We are going to allow network for `ssh`
|
||||
|
||||
|
||||
|
||||
|
||||
* `Review and Launch`
|
||||
* `Launch`
|
||||
* Create a ssh key so you can connect to the instance
|
||||
* Download the key \(and don't lose it or you won't be able to connect to the instance\)
|
||||
|
||||
|
||||
|
||||
|
||||
* `Launch Instances`
|
||||
|
||||
|
||||
|
||||
|
||||
* Wait for the instance to become `Running`
|
||||
|
||||
|
||||
@@ -8,11 +8,11 @@ The instructions have been tested on `Debian GNU/Linux 10 (buster)`
|
||||
|
||||
* Launch a new instance
|
||||
|
||||
|
||||
|
||||
|
||||
* Configure new instance
|
||||
|
||||
|
||||
|
||||
|
||||
* `Create`
|
||||
|
||||
|
||||
@@ -24,13 +24,13 @@ Once you see an Airbyte banner, the UI is ready to go at [http://localhost:8000/
|
||||
|
||||
You should see an onboarding page. Enter your email if you want updates about Airbyte and continue.
|
||||
|
||||
|
||||
|
||||
|
||||
## 2. Set up your first connection
|
||||
|
||||
Now you will see a wizard that allows you choose the data you want to send through Airbyte.
|
||||
|
||||
|
||||
|
||||
|
||||
As of our alpha launch, we have one database source \(Postgres\) and two API sources \(an exchange rate API and the Stripe API\). We're currently building an integration framework that makes it easy to create sources and destinations, so you should expect many more soon. Please reach out to us if you need a specific integration or would like to help build one.
|
||||
|
||||
@@ -76,7 +76,7 @@ DB Name: postgres
|
||||
|
||||
After adding the destination, you can choose what tables and columns you want to sync.
|
||||
|
||||
|
||||
|
||||
|
||||
For this demo, we recommend leaving the defaults and selecting "Every 5 Minutes" as the frequency. Click `Set Up Connection` to finish setting up the sync.
|
||||
|
||||
@@ -84,7 +84,7 @@ For this demo, we recommend leaving the defaults and selecting "Every 5 Minutes"
|
||||
|
||||
You should now see a list of sources with the source you just added. Click on it to find more information about your connection. This is the page where you can update any settings about this source and how it syncs. There should be a `Completed` job under the history section. If you click on that run, it will show logs from that run.
|
||||
|
||||
|
||||
|
||||
|
||||
One of biggest problems we've seen in tools like Fivetran is the lack of visibility when debugging. In Airbyte, allowing full log access and the ability to debug and fix integration problems is one of our highest priorities. We'll be working hard to make these logs accessible and understandable.
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
# Destinations
|
||||
|
||||
* [Contributing Destinations](../../contributing-to-airbyte/README.md#new-integrations)
|
||||
* [Contributing Destinations](../../contributing-to-airbyte/#new-integrations)
|
||||
* Please reach out to us for help develop destinations in other languages.
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ Note: Airbyte is not built on top of Singer, but is compatible with Singer's pro
|
||||
|
||||
## 10/30/2020
|
||||
|
||||
**New sources:** [Salesforce](sources/salesforce.md), [Google Analytics](sources/googleanalytics/README.md), [Hubspot](sources/hubspot/README.md), [GitHub](sources/github.md), [Google Sheets](sources/google-sheets.md), [Rest APIs](sources/rest-api/README.md), and [MySQL](sources/mysql.md)
|
||||
**New sources:** [Salesforce](sources/salesforce.md), [Google Analytics](sources/googleanalytics.md), [Hubspot](sources/hubspot.md), [GitHub](sources/github.md), [Google Sheets](sources/google-sheets.md), [Rest APIs](sources/rest-api.md), and [MySQL](sources/mysql.md)
|
||||
|
||||
## 10/21/2020
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
# Sources
|
||||
|
||||
* [Contributing Sources](../../contributing-to-airbyte/README.md#new-integrations)
|
||||
* [Contributing Sources](../../contributing-to-airbyte/#new-integrations)
|
||||
* Please reach out to us for help develop sources in other languages.
|
||||
|
||||
|
||||
@@ -75,13 +75,13 @@ Visit the [Facebook Developers App hub](https://developers.facebook.com/apps/) a
|
||||
|
||||
From the App's Dashboard screen \(seen in the screenshot below\) enable the Marketing API for your app if it is not already setup.
|
||||
|
||||
|
||||
|
||||
|
||||
### API Access Token
|
||||
|
||||
In the App Dashboard screen, click Marketing API --> Tools on the left sidebar. Then highlight all the available token permissions \(`ads_management`, `ads_read`, `read_insights`\) and click "Get token". A long string of characters should appear in front of you; **this is the access token.** Copy this string for use in the Airbyte UI later.
|
||||
|
||||
|
||||
|
||||
|
||||
With the Ad Account ID and API access token, you should be ready to start pulling data from the Facebook Marketing API. Head to the Airbyte UI to setup your source connector!
|
||||
|
||||
|
||||
@@ -72,7 +72,7 @@ Once you've created the Service Account, you need to explicitly give it access t
|
||||
|
||||
Finally, you'll need the ID of the Spreadsheet you'd like to sync. To get it, navigate to the spreadsheet in your browser, then copy the portion of the URL which comes after "/d" and before "/edit" or "/view". This is the highlighted portion of the screenshot below:
|
||||
|
||||
|
||||
|
||||
|
||||
### Setting up in the Airbyte UI
|
||||
|
||||
|
||||
Reference in New Issue
Block a user