jprdonnelly/airbyte
1
0
Fork 0
mirror of synced 2026-01-01 09:02:59 -05:00
Code Issues Packages Projects Releases Wiki Activity

🎉 New Source: Webflow (#13617)

Browse Source 
* Added webflow code

* Updated readme

* Updated README

* Added webflow to source_definitions.yaml

* Enhanced documentation for the Webflow source connector

* Improved webflow source connector instructions

* Moved Site ID to before API token in Spec.yaml (for presentation in the UI)

* Addressed comments in PR.

* Changes to address requests in PR review

* Removed version from config

* Minor udpate to spec.yaml for clarity

* Updated to pass the accept-version as a constant rather than parameter

* Updated check_connection to hit the collections API that requires both site id and the authentication token.

* Fixed the test_check_connection to use the new check_connection function

* Added a streams test for generate_streams

* Re-named "autentication" object to "auth" to be more consistent with the way it is created by the CDK

* Added in an explict line to instantiante an "auth" object from WebflowTokenAuthenticator, to make it easier to describe in the blog

* Fixed a typo in a comment

* Renamed some classes to be more intuitive

* Renamed class to be more intuitive

* Minor change to an internal method name

* Made _get_collection_name_to_id_dict staticmethod

* Fixed a unit-test error that only appeared when running " python -m pytest -s unit_tests".
This was caused by Mocked settings from test_source.py leaking into test_streams.py

* format: add double quotes and remove unused import

* readme: remove semantic version naming of connector in build commands

* Updated spec.yaml

* auto-bump connector version

* format files

* add changelog

* update dockerfile

* auto-bump connector version

Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
This commit is contained in:
Alexander Marquardt
2022-06-22 13:44:27 +02:00
committed by GitHub
parent aa28d448d8
commit e4d3d60ca8
30 changed files with 941 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/integrations/sources/webflow.md Normal file
View File

@@ -0,0 +1,38 @@
---
description: 'This connector extracts "collections" from Webflow'
---
# Sources
Webflow is used for publishing Airbyte's blogs, and provides several APIs. The APIs that are used by this connector to extract information from Webflow are described in [Webflow Developers documentation](https://developers.webflow.com/).
Webflow uses [Collections](https://developers.webflow.com/#collections) to store different kinds of information. A collection can be "Blog Posts", or "Blog Authors", etc. Collection names are not pre-defined, the number of collections is not known in advance, and the schema for each collection may be different. Therefore this connector dynamically figures our which collections are available and downloads the schema for each collection from Webflow. Each collection is mapped to an [Airbyte Streams](https://docs.airbyte.com/connector-development/cdk-python/full-refresh-stream/).
# Webflow credentials
You should be able to create a Webflow `API key` (aka `API token`) as described in [Intro to the Webflow API](https://university.webflow.com/lesson/intro-to-the-webflow-api).
Once you have the `API Key`/`API token`, you can confirm a [list of available sites](https://developers.webflow.com/#sites) and get their `_id` by executing the following:
```
curl https://api.webflow.com/sites \
-H "Authorization: Bearer <your API Key>" \
-H "accept-version: 1.0.0"
```
Which should respond with something similar to:
```
[{"_id":"<redacted>","createdOn":"2021-03-26T15:46:04.032Z","name":"Airbyte","shortName":"airbyte-dev","lastPublished":"2022-06-09T12:55:52.533Z","previewUrl":"https://screenshots.webflow.com/sites/<redacted>","timezone":"America/Los_Angeles","database":"<redacted>"}]
```
After retrieving your `site id`, you can create a file `secrets/config.json` conforming to the fields expected in `source_webflow/spec.yaml` file.
(Note that any directory named `secrets` is git-ignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information in this folder).
See `integration_tests/sample_config.json` for a sample config file that you can use as a template for entering in your `site id` and your `Webflow API Key`.
| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.1.0 | 2022-06-22 | [13617](https://github.com/airbytehq/airbyte/pull/13617) | Initial release |
| 0.1.1 | 2022-06-22 | [13617](https://github.com/airbytehq/airbyte/pull/13617) | Update Spec Documentation URL |