1
0
mirror of synced 2025-12-21 02:51:29 -05:00
Commit Graph

221 Commits

Author SHA1 Message Date
Mark Sumegi
b73d10bd33 Source Jira: Removed Connector Domain Pattern from Spec (#24636)
* Removing JIRA Connector Domain Pattern 

The pattern for the JIRA domain will not necessarily end in atlassian.net. (e.g. cloud hosted JIRA can also be jira.com and on premise installations can also deviate from this pattern). 

I propose removing this restriction. Our opinion is that it's too strict and brings little to no value. 

This PR fixes: https://github.com/airbytehq/airbyte/issues/7502

* Bump version

* Updated description

Co-authored-by: Willi Müller <willi.mueller@posteo.de>

* update examples

* update oss_catalog seed file

* update docker version

* Update docs

* auto-bump connector version

---------

Co-authored-by: Willi Müller <willi.mueller@posteo.de>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-04-10 17:15:42 -03:00
Serhii Chvaliuk
be87246b95 Source Jira: fix streams "IssuePropertyKeys", "ScreenTabFields" (#24890)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-04-07 11:52:35 +03:00
Serhii Lazebnyi
df1ec65154 Connector Health: Fixed SAT for marketo, close, chargebee, facebook marketing, paystack, hubspot, pipedrive and marketo (#24802)
* Fix marketo, close, chargebee, facebook, paystack

* Fix unittests

* Pass failed on extra columns

* Fix hubspot, marketo and pipedrive

* Fix marketo connector programs stream

* Fix CAT hubspot

* Fix programs for marketo

* Fix Chartmogul, Marketo, Paypal-transaction, sentry and trello

* Fix jira expected records

* Fix pipedrive

* Fix expected records hubspot

* Delete print from paypal

* Increase timeout for facebook marketing
2023-04-05 20:32:16 +02:00
Serhii Lazebnyi
03e1ec4cb8 Connector Health: Fixed SAT for close-com, facebook-marketing, freshsales, greenhouse, hubspot, intercom, jira, mixpanel, monday, pipedrive, recharge, sentry, slack, strava, stripe, trello, twitter, youtube-analytics, twillio, faker (#24361)
* Fix expected records for hubspot and intercom

* Fix typo for intercom

* Fix close com, fecebook marketing, freshsales, greenhouse expected records

* Fix intercom CAT

* Fix unittest for new pydentic version

* Fix facebook marketing unittests and jira CAT

* Fix formating for facebook marketing, jira and CAT for monday

* Fix pipedrive and recharge CAT

* Fix CAT for Slack, Strava, Stripe, Trello and Twitter

* Fix expected records for slack

* Fix CAT for Youtube and Sentry

* Fix unittests for mixpanel

* Added allowedHosts and fix facebook marketing CAT

* Fix CAT for Faker and Twillio

* Fix expected records for hubspot

* Fix expected records for Sentry

* Fix check status for GAv4
2023-03-28 20:31:41 +02:00
Ella Rohm-Ensing
900327726f Add bypasses for extra fields test (#24552)
* Add bypass for extra fields test

* Add bypass for extra fields test
2023-03-26 20:16:49 -05:00
Serhii Lazebnyi
c1a4a6f3d3 Fix expected records for jira and zenloop (#24182) 2023-03-17 12:55:11 +01:00
Serhii Lazebnyi
cfeabd3a4e Connector Health: Fixed SAT for Instagram, Facebook pages, Google Search Console, Stripe, Typeform, Hubspot, Sentry, Faker, Zendesk support, Slack, Jira (#23711)
* Fixed SAT

* Updated stripe and instagram

* Google search console add allowed hosts

* Fix hubspot, sentry, stripe

* Fix source faker and zendesk-support

* Fix google search console and hubspot

* Fix slack and jira acceptance tests

* Fix expected records for facebook marketing
2023-03-08 19:41:34 +01:00
Ana Loureiro
1c0faee5e8 Source Jira: Small fix in the board stream (#21524)
* fix board stream

* revert debug changes

* one more fix

* code improvements based on suggestion

---------

Co-authored-by: Nataly Merezhuk <65251165+natalyjazzviolin@users.noreply.github.com>
2023-03-02 17:32:25 -05:00
Catherine Noll
7da6a3bb77 Run CATs with local CDK (#23084)
Scripts to 

* Run CATs against the local CDK for one connector

* Run CATs against the local CDK for multiple connectors

* Create a connecter image with the local CDK

---------

Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
2023-02-24 16:13:42 -05:00
Artem Inzhyyants
fb9ad253dc Source Jira: Remove caching (#23006)
* Source Jira: Remove caching

* Source Jira: update docs

* Source Jira: ref

* Source Jira: add allowed hosts

* Source Jira: fix tests

* Source Jira: fix tests

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-02-15 05:15:19 -06:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Joe Reuter
6a10ae3e05 Rename source acceptance test to connector acceptance test (#21846)
Rename source acceptance test to connector acceptance test
2023-02-02 11:38:19 +01:00
Philippe Boyd
f3c8d83170 🐛 Source Jira: check_connection fails if no projects are defined (#20739)
Co-authored-by: Evan Tahler <evan@airbyte.io>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Co-authored-by: Serhii Chvaliuk <grubberr@gmail.com>
Co-authored-by: Augustin <augustin@airbyte.io>
2023-01-12 23:39:25 +02:00
Serhii Chvaliuk
6c1869bbfb Source Jira: Fixed pagination for streams issue_remote_links, sprints (#20859)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-01-04 10:59:13 +02:00
Evan Tahler
e39e1898c5 Expected Records to .jsonl format (#20850)
* Expected Records to `.jsonl` format

* fix formatting template

* remove endline

* update templates

* Update docs/connector-development/testing-connectors/source-acceptance-tests-reference.md

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>

Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
2023-01-03 15:55:36 -08:00
darynaishchenko
7ae05b16c6 Source Jira: added unit tests (#20525)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-12-16 21:05:54 +02:00
Serhii Chvaliuk
a5bbbc3005 Source Jira: Improved code to become BETA (#20128)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-12-16 20:35:23 +02:00
Filipp Balakin
9630633ef7 🚨🚨 Source Jira: add UserGroupsDetailed schema, fix Incremental normalization, add Incremental IssueComments, IssueWorklogs (#18901)
* Add UserGroupsDetailed schema, fix Incremental normalization, add Incremental support for IssueComments, IssueWorklogs

* update docs, version, specs and definitions

* revert requested changes

* bump version label

* fix pk

* add missing primarykey

* disable backwards compatibility test for 0.3.0

* fix

* disable compat test for discovery

* bump connector version
2022-11-10 12:37:01 -05:00
antonioneto-hotmart
3ffa11cd11 🐛 Source JIRA - Field correction bug causing stream connection to fail (#18505)
* max results field correction

* added version notes and up

* Update acceptance-test-config.yml

* auto-bump connector version

Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
Co-authored-by: Vincent Koc <koconder@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-11-07 10:10:01 -05:00
antonioneto-hotmart
80e77801dd Source Jira - Pagination fix on jira users stream (#16944)
* pagination fix on jira users stream

* Update spec.json

* Update spec.json

* Update streams.py

* bump docker version and changelog

* Update spec.json

* Update streams.py

* update jsons and source

* update source

* Update source.py

* auto-bump connector version

Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-10-24 13:58:10 -04:00
Pedro S. Lopez
938436bcc9 update connector specs and definitions with new .com documentation urls (#17585)
* update definitions with new .com docs urls

* update docs urls in specs

* update generators

* regenerate scaffold connectors

* remove unrelated changes

* update more urls

* update specs

* fix tests

* run `:airbyte-config:specs:generateSeedConnectorSpecs` to fix formatting

* revert docs changes to make pr more reviewable

* revert generator readme changes to make more reviewable

* fix mysql strict encrypt expected spec

* fix postgres expected spec
2022-10-11 11:04:23 -04:00
Stephen Wentling
123705c510 Source Jira: Added updates to include issue components and fixes to README files (#15135)
* solve readme conflict

* updated jira sources with open PR details

* correct additionalProperties test discover

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-08-09 17:30:14 -03:00
Mainara Cavalcanti
1d1476e493 Jira Source: Added resolutiondate to fields on issues stream (#13202)
* Added resolutiondate to fields on issues stream (jira source)

* updated jira source changelog

* updated dockerfile version

* correct jira ci

Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
2022-05-30 17:49:20 -03:00
Marcos Marx
dca2256a7c Bump 2022 license version (#13233)
* Bump year in license short to 2022

* remove protocol from cdk
2022-05-26 15:00:42 -03:00
Augustin
8ca229bb8f Source Jira: change wording for description of array fields (#10835) 2022-05-04 14:30:51 +02:00
Brian Leonard
c302af45ff Upgrade to Python 3.9 (#11763)
* Dockerfile to 3.9

* Python version

* More python updates

* 3.9 on GitHub actions and lint updates

* Test out 3.9.11 on GitHub actions

* install python with an action

* formatting: newline

* Also has python code

* only check first level for changed modules
Previous example (source-google-search-console/credentials)

* Test failure: there is no logger.trace
2022-04-11 20:51:37 -07:00
Christophe Duong
7727b86359 Include tcp port in Clickhouse destination configuration for normalization (#9340)
* Include tcp port in Clickhouse destination configuration for normalization

* revert custom dbt on clickhouse

* Bumpversion of destination connector

* Format code
2022-01-06 18:03:06 +01:00
Marcos Marx
62fb232191 Source Jira: Add pull requests stream (#8559)
* Add pull requests stream

* Add id field

* Fix to work with latest commits

* Add docs and prevent failures

* bump connector version

* run seed file

Co-authored-by: Chris Wu <chris@faros.ai>
2021-12-27 15:51:31 -03:00
oleh.zorenko
7f147315bd 🐛 Source Jira: Fix fetching and update schema for filters stream (#9079)
* 9017 Fix fetching and update schema for `filters` stream

* 9017 Bump connector's version

* 9017 Update docs

* 9017 Update docs

* 9017 Set additionalProperties to false

* 9017 Bump connector's version
2021-12-23 17:53:29 +02:00
Iryna Grankova
586f22d3f6 Update fields in source-connectors specifications: jira (#8999) 2021-12-23 14:19:24 +02:00
Marcos Marx
dcc72ba686 Source Jira: add options to stream (small fixes) (#8141)
* Add option to render fields in HTML format and fix sprint_issue ids

* Bump connector version

* run format

* run format

* run seed generaotr

* run format

Co-authored-by: Chris Wu <chris@faros.ai>
2021-11-19 17:42:55 -03:00
Chris Wu
15b85014db Source Jira: Add option to render fields in HTML format and fix sprint_issue ids (#7398)
* Add option to render fields in HTML format and fix sprint_issue ids

* Bump connector version
2021-11-19 17:14:52 -03:00
Christophe Duong
86ca36c5c0 Format code (#7978) 2021-11-15 14:51:10 +01:00
Vadym
9c78351d37 🐛 Source Jira: Fix normalization step error (#7408)
* Fix schemas.
Fix streams.py.
Fix acceptance test config.

* Delete sample_files dir.

* Remove `sample_files` references.

* Update `jira.md`.

* Fix build.gradle

* Fix stream_slice initialization

* Remove unnecessary build.gradle dependencies

* Bump docker version
2021-10-29 14:19:31 +03:00
Chris Wu
17961fccfc Source Jira: Make recently added configs optional for backwards compatibility (#7222)
* Make start_date optional

* Make projects optional

* Bump version
2021-10-21 22:46:27 -03:00
Marcos Marx
6c0a17304a Bump Source Jira Connector (#7202)
* Create separate cache file per stream and add projects and start_date config

* Add project id to issue record

* Add projectId to issue schema

* Add BoardIssues stream

* Add SprintIssues stream

* Add Epics stream and deduplicate state code

* Add EpicIssues stream and additional fields for Issues stream

* Add story points to sprint issues

* Add new streams to test catalog

* Update gitignore

* Rename cache boolean and fix test catalog

* Fix streams that depend on Issues stream

* Fix sprint_issues stream

* Add more fields to issues stream and format

* Add option to expand issue changelogs

* Remove epic_issues stream

* Expand project descriptions

* Show rendered fields for epics

* Include project key

* Include project key in issues stream

* Address comments

* Use CDK caching

* Remove extra changes

* Fix sprints stream reading from non-scrum boards

* Format

* small fix and bump version

* bump version

Co-authored-by: Chris Wu <chris@faros.ai>
2021-10-19 23:55:39 -03:00
Chris Wu
1fcd6c4671 Source Jira: Add Board, Epic, and Sprint-related streams and improve request caching (#6621)
* Create separate cache file per stream and add projects and start_date config

* Add project id to issue record

* Add projectId to issue schema

* Add BoardIssues stream

* Add SprintIssues stream

* Add Epics stream and deduplicate state code

* Add EpicIssues stream and additional fields for Issues stream

* Add story points to sprint issues

* Add new streams to test catalog

* Update gitignore

* Rename cache boolean and fix test catalog

* Fix streams that depend on Issues stream

* Fix sprint_issues stream

* Add more fields to issues stream and format

* Add option to expand issue changelogs

* Remove epic_issues stream

* Expand project descriptions

* Show rendered fields for epics

* Include project key

* Include project key in issues stream

* Address comments

* Use CDK caching

* Remove extra changes

* Fix sprints stream reading from non-scrum boards

* Format
2021-10-19 23:20:15 -03:00
Christophe Duong
a3196428a7 Forward destination location to dbt profiles (#6709)
* Forward destination location to dbt profiles

* Format code

* Update version
2021-10-06 19:20:15 +02:00
Marcos Marx
85a584b3f0 Publish and Bump Version Jira Connector (#6691)
* Add caching for Jira source and add Boards stream

* Fix draft version

* Add sprints stream

* Add vcr to setup.py

* Formatting

* Fix acceptance tests

* Fix acceptance tests

* bump version

Co-authored-by: Chris Wu <chris@faros.ai>
2021-10-05 13:54:41 -03:00
Davin Chia
5a3353dcf5 Bootstrap initial custom metrics library (#6529)
Wrapper around Prometheus lib to interface with Datadog.

We use prometheus because:
- Future-proofing as it's uses the general open metrics format.
- Prometheus makes it's metrics available to a scraper to it lends itself better to the OSS set up.
- Datadog automatically converts promethues metrics into dd metrics so we don't lose much.
2021-10-05 21:51:34 +08:00
Chris Wu
95bf6991d7 Source Jira: Add caching and more streams (#6523)
* Add caching for Jira source and add Boards stream

* Fix draft version

* Add sprints stream

* Add vcr to setup.py

* Formatting

* Fix acceptance tests

* Fix acceptance tests
2021-10-04 20:45:03 -03:00
Michel Tricot
1773e41e47 Shorten our headers + adds contributors file (#6478) 2021-09-27 10:45:50 -07:00
vovavovavovavova
7216de8b42 Jira structure: back to swagger schemas, but fix incorrect (#5523)
* revert schemas from response to previous

* fixing: newlines; permission_scheme normalization failure

* add nulls
2021-09-02 18:22:24 +03:00
vitaliizazmic
d3b2e8a46a 🐛 Source Jira: changed cursor field from created to updated
* Source Jira #4505 - change cursor field for Issues streams from fields.started to fields.updated, update IssuesWorklogs slicing

* Source Jira #4505 - reformat

* Jira #4505 - made issues worklog stream full refresh

* Jira #4505 - reformat

* Jira #4505 - bump version and updating documentation
2021-09-02 14:22:40 +03:00
LiRen Tu
2906ec287a CI: Add action to check broken doc links (#5254)
* Add action to check broken doc links

* Ignore localhost

* Update config

* Fix broken links

* Use quiet mode

* Ignore PR link

* Fix more broken links

* Fix more broken links

* Fix more broken links

* Verify pattern

* Fix more broken links

* Separate full and pr check

* Update pattern

* Test invalid link

* Remove invalid link
2021-08-07 14:28:02 -07:00
Dmytro
abde7c7727 Remove json schema parameter (#4907)
source-acceptance-test framework not longer required json_schema parameters
   from catalog file. This parameter is verbose, makes reading config file
   complicated and could be misleading when debugging acceptance test issues.

Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
2021-08-04 18:16:59 +03:00
vovavovavovavova
625a88cad8 Source Jira: fixing schemas accordinately to response. (#4947)
* pre-setup save

* save labels

* modify schemas (test)

* rm unused

* add newlines to schemas

* remove incremental as it was before

* fixing file

* format && fixes

* update docs
2021-07-28 19:35:32 +03:00
Eugene Kulak
9de53688e3 SAT: remove deprecated validate_output_from_all_streams from configs (#5010)
Co-authored-by: Eugene Kulak <kulak.eugene@gmail.com>
2021-07-27 03:02:32 -04:00
Sherif A. Nada
b86ebf63fb Automatically pull latest SAT & build connector image in SAT runner (#4969) 2021-07-26 11:56:19 -07:00
Davin Chia
350bb5876e Kube: Better Port Abstraction. (#4829)
Introduce a better port abstraction whose primary purpose is to confirm that ports are released when the Kube Pod Process is closed.

This prevents issues like #4660

I'm also opening more ports so we can run at least 10 syncs in parallel.
2021-07-21 11:07:00 +08:00