1
0
mirror of synced 2026-01-19 18:07:00 -05:00
Commit Graph

3798 Commits

Author SHA1 Message Date
Alexandre Girard
a1da5fc85b 🐛 Source Google Sheets: revert 'add row_id to rows and use as primary key' (#19574)
* revert PR 19215

* bump
2022-11-17 16:38:47 -08:00
akashpatl
5c7affcfb4 🎉 New Source: Copper [python cdk] (#18848)
* add younium source along with documentation

* init commit for new python source

* added streams for projects, companies and people

* breakout http_method

* add docs for Copper

* docs were not included in previous commit

* indentation error

* update schemas that failed acceptance test

* resolve merge conflict with docs

* add title to spec

* update doc url

* newlines

* add copper to source def

* add copper to source def

* run format

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-17 20:00:36 -03:00
Ivica Taseski
34d4a71875 🎉 New Destination: S3-Glue (#18695)
* destination-s3-glue

* reimplement schema generation to use recursion

* configure serialization library

* include data on update

* flatten data field

* improve location path

* generate s3-glue destination

* refactor s3-glue as separate connector

* add acceptance tests and cleanup

* check field presence

* override test image name

* add s3-glue readme

* format files

* add redpanda readme

* add s3 glue to source def

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-17 19:10:31 -03:00
Benoit Moriceau
90350c10fb Revert "Revert "Bmoric/add fetch source schema in oauth"" (#19513)
* Revert "Revert "Bmoric/add fetch source schema in oauth (#19392)" (#19512)"

This reverts commit f08b84f94f.

* Fix complete OAuth

* Use var
2022-11-17 13:18:43 -08:00
Amalia Advincula-Roye
d7e75c8acc Add Stripe API limitations to Events stream (#19497) 2022-11-17 14:34:02 -05:00
Amruta Ranade
6c0901817b Assorted fixes (#19570) 2022-11-17 14:31:54 -05:00
Johan Hammarstedt
b8ba5109dc 🎉 New Source: Recreation.gov (#18881)
* added Recreation.gov connector

* adding bootstrap

* fixing tests

* adding query params to not timeout on tests

* fix query for tests

* add recreation

* run format

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-17 16:22:34 -03:00
Yowan Ramchoreeter
1757b50a94 Note to call-out API restriction for event (#19515)
* Note to call-out API restriction for event

* Warning + Tweak wording
2022-11-17 13:51:01 -05:00
Shruti Kuber
b871485e70 Add restack to docs (#19345)
* add content to on-restack.md

* Add restack docs to sidebar and docs/readme.md

* adds changes requested by reviewer

* fixed typos

Co-authored-by: Amruta Ranade <11484018+Amruta-Ranade@users.noreply.github.com>
2022-11-17 13:46:25 -05:00
Amruta Ranade
5688f77f95 Update postgres.md 2022-11-17 13:29:11 -05:00
Scott Lee Chua
5ed632ac45 🎉 New Source: Plausible [low-code cdk] (#18657)
* initial commit

* Add authentication

* Add stats stream and schema

* Change overwrite to append, add primary key

* Make acceptance test executable

* Revert to overwrite

* Add documentation

* Update documentation

* Add titles to spec properties, update documentationUrl, remove EOF newlines

* Add bootstrap.md

* Update BOOTSTRAP.md

* Remove incremental sync to-dos

* fix: add source definition for plausible

* fix: format python files with flake8

* auto-bump connector version

Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-17 13:07:13 -05:00
Sherif A. Nada
b1dfeede2f Source Fauna: fix unit tests (#19534)
* Source Fauna: fix unit tests

* Bump version in dockerfile

* Update fauna.md

* auto-bump connector version

Co-authored-by: Akash Kulkarni <113392464+akashkulk@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-17 09:30:28 -08:00
Michael Siega
d87b38dc0b Add structured dbt cloud information to the operations api (#19395)
* add structured dbt cloud information to the operations api

* remove unused webhook features, test updates

* update tests to use structured dbt cloud operation api

* add missing webhook operator type
2022-11-17 17:49:27 +01:00
Himanshu
acea2a54e5 🎉 New source: Coda [python cdk] (#18675)
* init coda

* fix: Streams added

* fix: Common review comments

* fix: definition

* fix: Remove sample files

* fix: pagination

* integration test configs

* fix: remove validations

* fix: unit test

* fix: unit test removed

* fix: unit tests added

* linting

* fix: pytest to 6.2.5

* remove columns

* update connector

* add coda to source def

* remove coda from source def

* readd missing file

* rollback change to index.html

* correct some files

* rollback change in cart.com file

* fix sample files

* run format

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-17 13:18:50 -03:00
Arnaud Jeannin
0164355635 🎨 Add oss/cloud tags on doc for GA connectors (#19118)
* feat: add cloud and oss tags

* put headers back

* fix: rm prettier style

* fix: aws styles
2022-11-17 17:01:20 +01:00
Rodi Reich Zilberman
b1c9101b3a commit a change to publush strict-encrypt-source-postgres again (#19524) 2022-11-16 17:33:25 -08:00
Rodi Reich Zilberman
c739c4c36c Implement Debezium heartbeats for source-postgres (#19004)
* Initial working commit

* Code sanity. Provide no-on implementation to mysql, MSSql to allow compilation.

* Update test

* sanity

* sanity

* sanity

* sanity

* sanity

* changes per review comments

* Make heartbeat change waittime configurable.

* Trying to bypass test strictness test

* Trying to bypass test strictness test

* Trying to bypass test strictness test

* fix acceptance test config format

* add missing SAT test in config

* revert back changes in acceptance-test-config.yml

* Version and notes

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-16 16:20:04 -08:00
Marcos Marx
1421ba72aa New Connector Polygon (#19504)
* Weather API CDK Added

* Polygon-Stocks-API CDK Added

Polygon-Stocks-API is the API provided by https://polygon.io/ to get details about the stock

* spec.yaml updated

* Files Changes Resolved

* Review Changes Are Performed

* Some Requested Changes Are Done

Add title to your parameters is done
Change from to start_date and to to end_date is done

* correct schema

* add polygon to source def

* run format

* correct primary key

* auto-bump connector version

Co-authored-by: Harshil Khamar <harshilkhamar1@gmail.com>
Co-authored-by: Harshil Khamar <73790584+Harshil0512@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-16 19:55:50 -03:00
btkcodedev
310b518ece 🎉New Source: TMDb [low-code cdk] (#18561)
* Init: New Source: TMDb

* chore: Added documentation

* Chore: All Comments Resolved

* Added language config for filtering, All comomments resolved

* Update Doc URL

* add tmdb to source def and format files

* Removed unwanted authenticator, Added request params and configs

* cleansed lint

* feat: Added pagination, refactor: moved example to spec.yaml and minor corrections

* fix: change required in spec.yaml, corrections to docs

* removed unwanted file change

* remove movies_changes

* auto-bump connector version

Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-16 17:41:18 -03:00
Amruta Ranade
17932faebd Update low-code-cdk-overview.md 2022-11-16 14:02:12 -05:00
btkcodedev
8e9eaa8b66 🎉 New Source: Punk API [low-code cdk] (#18724)
* New source: punkapi

* chore: Added docs, fix: Streams, Test: Pass

* chore: Add doc URL

* All comments resolved

* Resolve Merge conflicts

* Resolve Merge conflicts

* feat: Added paginator, Introduced request params

* feat: fixed paginator

* add punk api to source def

* update sample file

* rebase

* auto-bump connector version

Co-authored-by: btkcodedev <btk.codedev@gmail.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-16 15:52:31 -03:00
swyx
7bec0d9460 📖 Minor docs sidebar updates (#19437)
* minor sidebar cleanup

* upates
2022-11-17 02:00:56 +08:00
VitaliiMaltsev
a1db24cf55 Postgres Source: use native Postgres timeout if it's not set by the user (#19291)
* Postgres Source: use native Postgres timeout if it's not set by the user

* refactoring

* updated connection timeout logic and added tests for different datasources creation

* fixed pmd

* refactoring

* bump version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-16 19:49:00 +02:00
Jimmy Ma
693f976efd Fix flaky tests (#19459)
* Expose WorkflowState in JobsDebugInfo

* Make attribute required

* Update the tests

* Protect more tests
2022-11-16 09:18:19 -08:00
Balaji Seetharaman
64cfb3806f 🎉 New Source: Twitter (#18863)
* 🎉 New Source: Twitter

* adhering to review checklist

* adding stream slicer

* adding stream slicer

* add twitter to source def and run format

* readd twitter to source def

* correct twitter spec

* run format

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-11-16 13:45:16 -03:00
Sherif A. Nada
88da35c6f9 Fix callout formatting in CDK docs 2022-11-16 08:14:08 -08:00
Samuel Christopher
a064fd64dd Update the LinkedIn Ads source connector doc (#19137)
* Update the LinkedIn Ads source connector doc

* make changes

* Update corrections

* Update changes
2022-11-16 10:48:25 -05:00
michizhou
4858fbaf3b 🎉 New Source: Tyntec SMS [low-code cdk] (#18883)
* Initial implementation: Support for SMS API streams

* Add connector documentation and icon

* Add new SMS stream with associated stream slicer

* add tyntec sms to source def

* format files

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-11-16 10:59:04 -03:00
Marcos Marx
a7ec584ba6 New Connector CallRail + format (#19407)
* init

* adding image and call rail record within source_definitions.yaml

* replacing data types in users.json

* converting text_messages.json  to conversations.json

* removing trackers.json

* renaming streams in abnormal_state.json and seperating incremental from non-incremental streams in caatalog files

* - moving stream slicer to retriever in definition
- renaming text_messages to conversations

* adding doc for callrail.md

* adjusting sepc.yaml based off of Marcos's comments.

* adjusting sepc.yaml

* cleaning up acceptance-test-config.
small adjustments to rest of files.

* adding pagination

* fix tests

* generate source spec

* solve conflict source spec and def

Co-authored-by: adhamsuliman <adhamsuliman93@gmail.com>
2022-11-16 10:28:23 -03:00
Andy
56342d28cd Lowcode CDK docs: Update with CTA for creating feature requests (#19462) 2022-11-15 21:25:01 -08:00
Evan Tahler
05c99c1693 Faker emits AirbyteEstimateTraceMessage (#19197)
* Faker emits `AirbyteEstimateTraceMessage`

* update pr

* fix SAT

* auto-bump connector version

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-15 15:58:22 -08:00
Sophia Wiley
5e2cea05f3 edited link (#19444) 2022-11-15 13:59:58 -05:00
Yowan Ramchoreeter
15c3d6253d Doc update: Read-only access should be sufficient (#19438) 2022-11-15 11:59:28 -05:00
Denys Davydov
217a651663 Source Salesforce: add more unsupported bulk entities, fix fallback to rest (#19286)
* #985 source salesforce: add more unsupported bulk entities, fix fallback to rest

* #985 source salesforce: upd changelog

* #985 source salesforce - add docstring for a test

* #985 source Salesforce: bump version

* #985 source salesforce: move SATs to high level strictness

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-15 13:40:27 +02:00
Serhii Chvaliuk
a3e83c306a Source Prestashop: Migrate connector from Alpha (Python) to Beta (YAML) (#18927)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-11-15 08:39:25 +02:00
Sherif A. Nada
33575dff44 Fix datadog connector docs title 2022-11-14 22:22:51 -08:00
Balaji Seetharaman
4fbd061b2b 🎉 New Source: Aha (#18893)
* 🎉 New Source: Aha

* addressing the comments

* addressing review comments

* addressing review comments & removing newly added EoL

* fixing end of file line

* fixing end of file line

* including paginator

* Fixing the pagination issue

* Removing the contents from sample_config.json

* fix schema, tests and add aha to source def

* format files

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 23:44:58 -03:00
Haithem SOUALA
b690004d8e 🎉 New Source: Rocket.chat [low-code cdk] (#18635)
* init commit

* add streams

* add streams

* add streams

* add streams

* solve conflict doc

* solve doc conflict

* updates to connector

* Update acceptance-test-config.yml

* Revert "Update acceptance-test-config.yml"

This reverts commit e79ea7c7db984b3abaf7686627b34f18ea42b526.

* solve source def conflict

* solve spec conflict

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Vincent Koc <25068+koconder@users.noreply.github.com>
2022-11-14 21:43:19 -03:00
Haithem SOUALA
c064db8ddf 🎉 New Source: getLago [low-code cdk] (#18727)
* initial commit

* add docs

* add doc

* lint

* add getlago to source def

* generate source spec

Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-11-14 21:34:08 -03:00
catpineapple
241cf4a6eb 🎉 New Destination: Apache Doris (#17884)
* first commit

* normalized code and integration test

* add bootstrap.md

* add doris to destination def

* auto-bump connector version

* format files

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 21:31:53 -03:00
Pierre Kerschgens
fae29ee2ab 🐛 Source Salesforce: Use Apparent Encoding (#19294)
* fix: prefer response body encoding over header encoding

* bump: source-salesforce version

* Update salesforce.md

* auto-bump connector version

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 13:42:33 -05:00
Mahdi Dibaiee
44ac470d28 Google Sheets: add row_id to rows and use as primary key (#19215)
* source-google-sheets: add row_id to rows and use as primary key

* Update Dockerfile

* Update google-sheets.md

* Update Dockerfile

* Update google-sheets.md

* auto-bump connector version

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 13:42:00 -05:00
Ivica Taseski
a2d24b5abf 🎉 New Source: Dynamodb (#18750)
* dynamodb source connector

* code cleanup

* code cleanup

* add date type filtering

* add doc

* revert integer schema primitive

* add dynamodb to source def

* auto-bump connector version

Co-authored-by: itaseski <itaseski@debian-BULLSEYE-live-builder-AMD64>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-11-14 12:21:50 -03:00
Ivica Taseski
d7e2b76d4a 🎉 New Destination: Redpanda (#18884)
* add Redpanda destination connector

* change cluster config

* improve test coverage

* add redpanda to destination def

Co-authored-by: itaseski <itaseski@debian-BULLSEYE-live-builder-AMD64>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-11-14 12:20:55 -03:00
Jared Rhizor
dffbeb3d20 🎉 New Source: RSS [Python CDK] (#18838)
* add source-rss

* update tests

* add docs

* fix doc link

* changes to pass tests

* update catalog

* Update test_streams.py

* fix time zone issue

* update source def

* auto-bump connector version

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-11-14 10:51:18 -03:00
danilo-dti
7ca371b26d 🎉 New Source: Microsoft Dataverse [python cdk] (#18646)
* Connector files

* Add test files

* Add integration test config files

* Multiple changes to make it on Airbyte standards

* Cleaning up

* More clean ups

* More clean ups

* Removed max pages

* Remove unused variable

* Correctly separating Full refresh and incremental

* Removed unused variables

* Fix full_refresh class

* Better code for creating stream classes

* Fixing review comments

* Update docs and Enum class

* Update type conversion function

* Fix enum class and update docs

* Update discover

* Implemented some unit tests

* Update discover

* Update test_source

* Increase discovery test timeout

* Update configured_catalog

* Fix default_cursor_field

* Adding final unit tests

* Update spec: set client_id and tenant_id as secrets

* Update discover to deal with Lookup and Picklist types

* Fix Lookup data type conversion

* add microsoft dataverse to source def

* run format

* auto-bump connector version

Co-authored-by: Marcelo Pio de Castro <marcelopiocastro@gmail.com>
Co-authored-by: daniloss99 <danilosiqueira99@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 09:02:05 -03:00
Subodh Kant Chaturvedi
5d2011d065 mysql-source:tinyint-unsigned should not be identified as boolean (#18956)
* mysql-source:tinyint-unsigned should not be identified as boolean

* upgrade version

* fix encrypted spec

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 17:10:49 +05:30
Haithem SOUALA
30d2621b86 🎉 New Source - Intruder [low-code cdk] (#18668)
* initial commit

* fix schema

* lint

* add docs

* add pagination

* add pagination

* add intruder to source def

* readd ip2whois

* add intruder to source def

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-11-14 08:05:39 -03:00
Denys Davydov
8ff8bece02 Source Tempo: migrate to lowcode + certify to beta (#18936)
* #18682 source tempo: migrate to lowcode + certify to beta

* #18936 source tempo: upd changelog

* #18682 source tempo: fix yaml

* #18682 source tempo: rm redundant config

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 10:53:47 +02:00
Elliot T
ebeef48295 🎉 New Source: Workramp (#18843)
* configure connector

* add docs

* add streams

* update PR number

* add workramp to source def

* auto-bump connector version

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
2022-11-13 21:31:54 -03:00