1
0
mirror of synced 2025-12-21 02:51:29 -05:00
Commit Graph

182 Commits

Author SHA1 Message Date
Evan Tahler
037a522f8c Google Source connector licenses to Elv2 (#27738)
* Google Source connector licenses to Elv2

* bump PR Ids

* Update metadata.yaml
2023-06-27 03:38:13 -04:00
Anatolii Yatsuk
592b0506e7 🐛 Source Google Sheets: migrate to advancedAuth (#26833)
Remove authSpecification in favour of advancedAuth in specification
2023-06-02 12:03:38 +03:00
Ben Church
1dabc6208e Metadata: add tags field (#26320)
* Add optional tags field

* Remove duplicate icons

* Add programming tags to all

* Update docs

* supportUrl -> documentationUrl

* Ensure one language tag is applied

* Add keyvalue check

* rebase and fix tests

* Format

* Add cache buster

* Improve test

* Automated Commit - Formatting Changes

* Update error

* Fix missing tags

* Fix scaffold

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@sers.noreply.github.com>
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-05-26 16:13:09 -07:00
Ben Church
248bbf94c1 HACKDAY: Icon CDN (#26158)
* Move icons to connector folder

* Delete old icons

* Update upload logic

* Add icon url to definitions

* Update registry model

* Populate cdn url

* DNC butcher the pipeline

* Low hanging fruit fixes

* Fix bucket name

* Merge old and new approaches

* Fix metadata upload step

* Format

* Fix test
2023-05-24 17:25:41 -07:00
Artem Inzhyyants
0eefa33bcb Source Google Sheets: handle config errors (#26097)
* Source Google Sheets: handle config errors

* Source Google Sheets: update docs

* Source Google Sheets: fix test

* Source Google Sheets: add unit tests

* auto-bump connector version

* Automated Change

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
2023-05-17 00:14:26 +02:00
Augustin
7310494846 qa-checks: check metadata version matches dockerfile version (#25661) 2023-05-04 16:08:19 -07:00
Ben Church
5563179782 Dagster: rename catalog to registry (#25254)
* rename catalog to registry in metadata service

* rename catalog to registry in metadata files

* Run generate models

* Fix missed renames

* Add github personal access token

* Run black

* Automated Change

---------

Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
2023-04-18 22:15:11 +02:00
Augustin
ebc907cdf7 create metadata files for all connectors (#24964) 2023-04-13 07:45:04 +02:00
midavadim
259b8a2766 🎉 Source Google Sheets - skip non-grid sheets (#23292)
* skip non grid sheets

* added  PR number to docs

* fixed Dockerfile

* fix doc

* fix doc

* updated seed file

* updated docs

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-02-27 19:01:51 +02:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084)
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-02-24 16:13:42 -05:00
Denys Davydov
623d33d568 Source google sheets: handle empty sheets gracefully (#23272)
* #11271 source google sheets: handle empty sheets gracefully

* #11271 source google sheets: upd changelog

* #11271 source google ads: upd allowed hosts

* auto-bump connector version

---------

Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-02-23 19:58:58 +02:00
Serhii Chvaliuk
a431a52aaa Source Google Sheets: slugify column names (#23057)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-02-23 18:10:46 +02:00
Serhii Lazebnyi
57c5a78c24 🐛Source Google Sheets: Handle shorter URL for Google sheet URL (#23071)
* Change min spreadsheet id size to 20 symbols

* Bump docker version

* Added allowedHosts

* Updated version

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-02-22 23:17:09 +00:00
Artem Inzhyyants
08a2d1c723 Source Google Sheets: handle authentication errors (#23278)
* Source Google Sheets: handle authentication errors

* Source Google Sheets: update docs

* Source Google Sheets: fix test connection expected status

* Source Google Sheets: all allowed hosts

* auto-bump connector version

---------

Co-authored-by: octavia-squidington-iii <octavia-bot@airbyte.io>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-02-22 23:38:08 +01:00
Augustin
c447923123 source-google-sheets: do not consume http spreadsheets (#22884)
* source-google-sheets: do not consume http spreadsheets

* source-google-sheets: do not consume http spreadsheets

* update tests

* table format

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-02-14 07:14:47 -06:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846)
Rename source acceptance test to connector acceptance test
2023-02-02 11:38:19 +01:00
midavadim
f49266fe7b 🎉 Source GoogleSheets - migrated SAT to strictness level (#21399)
* migrated SAT to strictness level

* fixed expected records

* revert file from another source

* changed extension to txt

* changed extension to txt
2023-01-19 00:45:39 +02:00
Alexandre Girard
a1da5fc85b 🐛 Source Google Sheets: revert 'add row_id to rows and use as primary key' (#19574)
* revert PR 19215

* bump
2022-11-17 16:38:47 -08:00
Mahdi Dibaiee
44ac470d28 Google Sheets: add row_id to rows and use as primary key (#19215)
* source-google-sheets: add row_id to rows and use as primary key

* Update Dockerfile

* Update google-sheets.md

* Update Dockerfile

* Update google-sheets.md

* auto-bump connector version

Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-14 13:42:00 -05:00
Brian Lai
3521802545 Update Python Source connectors to use the latest airbyte-cdk minor version (#18261)
* update all connectors currently only adhering to patch versions to pull latest minor version

* also add greenhouse

* fix scaffolds
2022-10-20 16:06:02 -04:00
Augustin
b564f3eb78 Protocol: make supported_sync_modes a required not empty list on AirbyteStream (#15591) 2022-10-19 15:22:25 +02:00
Liren Tu
a8461a54c3 🐞 Source google sheets: fix nonetype exception when no spreadsheet id is found (#17766)
* Check if the match is not none

* Bump version

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-11 12:14:28 -07:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
Augustin
7ab162a869 migrate source GA connectors to per-stream states (2) (#17410) 2022-09-30 16:44:11 +02:00
Augustin
0f618f3503 migrate GA source connectors to per-stream state (#17326) 2022-09-29 14:45:35 +02:00
Liren Tu
2e17e3f13a 🐛 Source google ads: mark custom query fields required (#15858)
* Mark query and table_name as required

* Bump version

* bypass backward compat tests

* bypass backward compat tests

* revert change on source-google-sheets/acceptance-test-config

* oncall 467 - source google ads: fix sats

* oncall #467 source google-ads: one more SATs fix

* auto-bump connector version [ci skip]

Co-authored-by: alafanechere <augustin.lafanechere@gmail.com>
Co-authored-by: Denys Davydov <davydov.den18@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-15 14:25:37 +03:00
JJ Nilbodee
4653af27a2 fix: Google Sheets Spec typo (#16343)
Co-authored-by: Sajarin <sajarindider@gmail.com>
2022-09-06 12:24:33 -04:00
Akul Goel
2a0c57b3ed Source Google Sheets: exposes row batch size config (#15107)
* exposes row batch size config to the connector

* review changes

* bump connector version

* auto-bump connector version [ci skip]

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-08-03 15:18:21 -03:00
Amruta Ranade
d4071a0ae5 source-salesforce,source-google-sheet,destination-snowflake: Updated field descriptions(#13729) 2022-07-07 15:55:52 +02:00
Alexandre Girard
19a20f0efc 🐛 Source Google Sheets: Retry on server errors (#13446)
* Retry on server errors

* retry on any 5XX

* bump version

* rename function for clarity

* reset

* delete unused code

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-06-02 18:28:45 -07:00
Baz
57a34b2251 🐛 Source Google Sheets: fixed issue with timeout (#13291) 2022-05-27 22:24:13 +03:00
Marcos Marx
dca2256a7c Bump 2022 license version (#13233)
* Bump year in license short to 2022

* remove protocol from cdk
2022-05-26 15:00:42 -03:00
Topher Lubaway
05ce3dcff0 results from gradle format on master (#13053) 2022-05-20 08:14:34 -05:00
Pedro S. Lopez
816fece247 bump GA connectors to latest version of python cdk (#12685)
* bump GA connectors to latest version of python cdk

* update changelogs

* fix changelogs

* more changelog fixes

* auto-bump connector version

* auto-bump connector version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-05-06 20:52:59 -04:00
Pedro S. Lopez
aa872d0016 update various connectors to use YAML specs (#12230)
* salesforce uses spec.yaml

* stripe uses spec.yaml

* revert unintended changes

* bump stripe version

* add yaml files to package_data

* hubspot uses a spec.yaml

* use double quotes in yaml files

* google sheets uses spec.yaml

* exhcange-rates uses spec.yaml

* remove usage of default and enum in spec

* bump hubspot cdk requirement

* auto-bump connector version

* auto-bump connector version

* manually bump versions

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-04-20 19:59:05 -07:00
Mahdi Dibaiee
45acd75bf0 Source Google Sheets: remove print statement breaking airbyte protocol (#11977)
* source-google-sheets: replace print statement with airbyte logger

* bump connector config file

* run format

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-04-15 11:33:49 -03:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Brian Leonard
0fa9f126d3 🎉 New Source: Pivotal Tracker (#11060)
* Generate source-pivotal-tracker

* Projects stream

* Adds tracker stories stream

* Story pagination

* labels

* Releases

* Shared schema with allOf

* Top level $ref didn’t work

* Be more verbose instead of $ref

* Project memberships

* epics

* activity

* Cleaning up todos

* Code formatting

* Unit tests

* small changes

* run format

* add doc

* change uuid value

* run seed config

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-04-05 13:05:13 -03:00
Greg Solovyev
1d12c7c6f7 Source google sheets: allow using spreadsheet URL instead of ID and update docs (#11404)
Update spec file
 * Add Authentication label to credentials section
 * Add doc link to spreadsheet ID
 * Add a link to setup guide
  * update description text for spreadsheet_id in spec
  * change spreadsheet_id label in spec to Spreadsheet Link per github issue 11406
 
Allow using sheet URL instead of sheet ID
  * add a parser helper that extracts Sheet ID from a URL
  * add a tests for parsing sheet ID out of a URL
  * add a test for backward compatibility (parser should recognize an ID)
  * Bump connector version in Dockerfile

Update documentation
    * Replace screenshot of sheet id with screenshot of sheet URL
    * Update connector version in changelog
    * Update links to docs in spec file
2022-03-28 09:13:17 -07:00
Greg Solovyev
721cb37f08 🐛 Source Google Sheets: add missing titles and descriptions to fields in connector spec (#11270)
* Remove copy-pasta comment from another connector

* Spec cleanup
  * Add title to credentials
  * Add text about where to get spreadsheet ID to spreadsheet_id
  * Add an example of spreadsheet id
2022-03-23 09:40:10 -07:00
ycherniaiev
7806dbce22 Update fields in source-connectors specifications: google-sheets (#9208)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-01-25 12:22:35 +02:00
Christophe Duong
0c57100b55 source-googlesheet: Service account json should be secret (#9356)
* Service account json should be secret

* Bumpversion of connector

* bumpversion
2022-01-10 10:36:01 +01:00
vitaliizazmic
487f9e77e0 🎉 Source Google Sheets: migrate to the CDK
* Source Google Sheets #8313 - migrate to the CDK

* Source Google Sheets #8313 - fixing unit tests

* Source Google Sheets #8313 - reformat

* Source Google Sheets #8313 - update ci_credentials

* Source Google Sheets #8313 - bump version and update docs
2021-12-07 12:38:03 +02:00
Christophe Duong
86ca36c5c0 Format code (#7978) 2021-11-15 14:51:10 +01:00
Christophe Duong
c4620559d7 🎉 Refactor Normalization docker images and upgrade to use dbt 0.21.0 (#6959)
* Split normalization docker images for some connectors with specifics dependencies

* Regenerate (#7003)
2021-10-14 20:29:16 +02:00
Arthur Galuza
bfe865aebf 🎉 Source Google Sheets: Support connecting via oAuth webflow (#6354)
* Add service account support

* Upd oauth support

* Upd auth creds selector
2021-10-13 23:02:22 +03:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
midavadim
6ffb682822 Source google sheets: fix full_refresh test by adding supported_sync_modes to Stream inistialization (#5951)
* added supported_sync_modes for AirbyteStream initialization to fic full_refresh test

* fix unit test with supported_sync_modes argument in AirbyteStream initialization
2021-09-13 22:39:44 +03:00
midavadim
fd52290f15 Source google spreadsheets: fix error during listing sheets with diagram only (#5233)
* Fix error during listing sheets with diagram only

* added PR number

* fixed formatting

* add dummy test

* added filter for grid sheets, added SAT tests.

* added new GOOGLE_SHEETS_TESTS_CREDS env

* fixed formatting
2021-08-11 21:25:19 +03:00