1
0
mirror of synced 2025-12-20 02:23:30 -05:00

97 Commits

Author SHA1 Message Date
David Gold
64f1ea20dc chore(source-github): Upgrade to Python 3.13 (#68637)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2025-10-29 15:43:14 -07:00
devin-ai-integration[bot]
667b2b6361 fix(source-github): Normalize reaction field names for Avro/Parquet compatibility (do not merge) (#68095)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Patrick Nilan <nilan.patrick@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2025-10-22 09:16:14 -07:00
Daryna Ishchenko
c628d28c7d fix(source-github): fix datetime converting in workflows stream (#67026)
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2025-10-03 16:51:41 +03:00
Daryna Ishchenko
8bfaa23b2f chore(source-github) update to airbyte cdk v7 (#66736) 2025-10-02 13:29:00 +03:00
Aldo Gonzalez
19243f4d28 bug(source-github): problem getting author for contributor_activity stream (#62054) 2025-06-25 16:27:38 -06:00
Aldo Gonzalez
9e6a874144 bug(source-github): fix case when repository is empty (#61742) 2025-06-23 14:41:42 -06:00
Aaron ("AJ") Steers
83ecbe0fc3 CI: apply pre-commit format fix from #49806 (#49852) 2024-12-18 14:05:43 -08:00
Anatolii Yatsuk
84f3b03f9c fix(source-github): fix state handling for stream WorkflowRuns (#44592) 2024-09-04 10:55:18 +03:00
Brian Lai
2aaf33e679 [source-github] Bump cdk version and enable RFR for all non-incremental streams (#42966) 2024-08-09 12:54:01 -04:00
Serhii Lazebnyi
d327ec4960 refactor(source-github): migrate to CDK v3 (#41677) 2024-07-18 20:03:57 +02:00
Christo Grabowski
074bc068f2 Source Github: Refactor incremental state handling (#39513) 2024-06-18 11:59:01 -04:00
Artem Inzhyyants
b7ae0907f6 feat(source-github): add max_waiting_time to configuration (#38758)
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
2024-05-30 19:55:05 +02:00
Denys Davydov
003fc6ef06 🐛 Source Github: increase discovery time (#36429) 2024-03-26 20:36:36 +02:00
Artem Inzhyyants
2ce32ab24d 🐛 Source Github: Handle rate limit exception as config error (#35986)
Signed-off-by: Artem Inzhyyants <artem.inzhyyants@gmail.com>
2024-03-13 10:09:11 +01:00
Artem Inzhyyants
18cfa24974 🐛 Source GitHub: Continue Sync on Stream failure (#34700) 2024-02-02 17:55:41 +01:00
Artem Inzhyyants
66cd528340 🐛 Source GitHub: Fix MultipleTokenAuthenticator sleep time (#34598) 2024-01-29 11:29:32 +01:00
Artem Inzhyyants
53851609f8 🐛 Source GitHub: Fix MultipleToken rotation logic (#34503) 2024-01-26 15:15:54 +01:00
Artem Inzhyyants
71783509f2 🐛 Source GitHub: Fix retry for 504 error in GraphQL based streams (#33783) 2024-01-03 11:22:42 +01:00
Daryna Ishchenko
6fce769ce3 🐛 Source Github: return AirbyteMessage if max retry exeeded for 202 status code (#32679)
Co-authored-by: darynaishchenko <darynaishchenko@users.noreply.github.com>
2023-11-23 07:06:38 -06:00
Artem Inzhyyants
c7d411061e Source GitHub: handle ContributorActivity continuous ACCEPTED response (#31386) 2023-10-13 18:35:36 +02:00
Artem Inzhyyants
f26bd9ed84 Source GitHub: update links to docs (#31307) 2023-10-12 19:00:57 +02:00
Artem Inzhyyants
a92ae4d39f 🐛 Source GItHub: Increase backoff for stream Contributor Activity (#31023) 2023-10-04 19:04:32 +02:00
Daryna Ishchenko
4e44cd03be Source Github: marked start date as optional field (#30971)
Co-authored-by: darynaishchenko <darynaishchenko@users.noreply.github.com>
2023-10-03 18:14:05 +03:00
Artem Inzhyyants
ce08bfd484 Source GitHub: add archived project cards (#30979)
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
2023-10-02 21:45:17 +02:00
Daryna Ishchenko
3c79da9ede Source GitHub: added Issue timeline events stream (#30823)
Co-authored-by: darynaishchenko <darynaishchenko@users.noreply.github.com>
2023-09-29 12:56:19 +03:00
Artem Inzhyyants
bc68d0a442 Source GitHub: ignore 204 error in stats response (#30824)
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
2023-09-28 17:49:53 +02:00
Artem Inzhyyants
61a63ec2ff Source GitHub: add stream ProjectsV2 (#30731)
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
2023-09-28 08:46:29 +02:00
Artem Inzhyyants
e763971bf1 Source GitHub: Skip 404 error in TeamMemberShips stream (#30693) 2023-09-25 19:56:36 +02:00
Artem Inzhyyants
32c08d73d5 Source GitHub: support self hosted instances (#30647)
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
Co-authored-by: Serhii Lazebnyi <53845333+lazebnyi@users.noreply.github.com>
2023-09-22 19:36:50 +02:00
Artem Inzhyyants
1c89aeaa5b Source GitHub: add new stream Contributor Activity (#30615)
Co-authored-by: artem1205 <artem1205@users.noreply.github.com>
2023-09-22 11:50:44 +02:00
Serhii Chvaliuk
df79780cb8 Source Github: deprecate "page size" input parameter (#25838)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-05-24 12:20:08 +03:00
Roman Yermilov [GL]
ce9eb8ab4c Source Github: better error explanation for 401 (#26025)
* Source Github: better error explanation for 401

* Source Github: bump version, update changelog

* Source Github: fix test name

* Source Github: fix unittests

* Source Github: bump version

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-05-16 22:35:22 +04:00
Serhii Chvaliuk
5053b278f5 Source Github: fix caching for get_starting_point in stream "Commits" (#24398)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-03-27 13:13:01 +03:00
midavadim
1b2b0392ef 🎉 Source Github - added user friendly messages, added AirbyteTracedException config_error (#23467)
* added user friendly messages, added AirbyteTracedException config_error

* added formatting

* fixed 'if' condition

* removed HTTPError words from error examples

* changed http to https for all schemas

* removed http adapter

* updated expected record file - one record for each stream

* updated expected record file

* updated expected record file

* bumped connector version

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-03-03 18:32:14 +00:00
Cole Snodgrass
2e099acc52 update headers from 2022 -> 2023 (#22594)
* It's 2023!

* 2022 -> 2023

---------

Co-authored-by: evantahler <evan@airbyte.io>
2023-02-08 13:01:16 -08:00
Ella Rohm-Ensing
8fc76a5d3c Turn off default httpavailabilitystrategy for source-github (GA) (#22039)
* Turn off default for source-github (GA)

* Update github.md

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-01-28 02:25:58 +00:00
Serhii Chvaliuk
65fc1bcdf0 🎉 Source Github: IssueReactions - re-implemented using GraphQL (#21457)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2023-01-27 02:45:39 +02:00
Roman Yermilov [GL]
d1688c7287 Source GitHub: #970 handle 502 Bad Gateway (#21481)
* Source Github: handle 502 Bad Gateway with proper log message

* Source Github: bump version

* Source Github: update changelog for new 0.3.12 version

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2023-01-19 19:27:30 +00:00
Ella Rohm-Ensing
8bb41282c5 CDK 0.15.0 and source-github 0.3.10 -- revert AvailabillityStrategy changes (#20523)
* Revert "source-github: move known error handling to GithubAvailabilityStrategy (#19978)"

This reverts commit f97db17ccc.

* Revert "🐛 Python CDK: fix `StopIteration` error for `check_availability` (#20429)"

This reverts commit 4e9b014277.

* Revert "CDK: `AbstractSource.read()` skips syncing stream if its unavailable (add `AvailabilityStrategy` concept) (#19977)"

This reverts commit 55a32886a3.

* Restore changelog entries

* bump CDK version

* Bump Github version

* Re-add removed dependencies

* auto-bump connector version

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-12-15 14:24:24 -05:00
Ella Rohm-Ensing
f97db17ccc source-github: move known error handling to GithubAvailabilityStrategy (#19978)
* Rough first implememtation of AvailabilityStrategy s

* Basic unit tests for AvailabilityStrategy and ScopedAvailabilityStrategy

* Make availability_strategy a property, separate out tests

* Remove from DeclarativeSource, remove Source parameter from methods, make default no AvailabilityStrategy

* Add skip stream if not available to read()

* Changes to CDK to get source-github working using AvailabilityStrategy, flakecheck

* reorganize cdk class, add HTTPAvailabilityStrategy test

* cleanup, docstrings

* source-github working with availabilitystrategy

* reorganize source-github

* source-github: resolve deprecation warning

* use correct import for HTTPAvailabilitystrategy

* pull out error handling into separate method

* use handle_http_error method

* Pass source and logger to check_connection method

* Add documentation links, handle 403 specifically

* Fix circular import

* Add AvailabilityStrategy to Stream and HTTPStream classes

* Remove AS from abstract_source, add to Stream, HTTPStream, AvailabilityStrategy unit tests passing for per-stream strategies

* Modify MockHttpStream to set no AvailabilityStrategy since source test mocking doesn't support this

* Move AvailabilityStrategy class to sources.streams

* Move HTTPAvailabilityStrategy to http module

* Use pascal case for HttpAvailabilityStrategy

* Remove docs message method :( and default to True availability on unhandled HTTPErrors

* add check_availability method to stream class

* Add optional source parameter

* Add test for connector-specific documentation, small tests refactor

* Add test that performs the read() function for stream with default availability strategy

* Add test for read function behavior when stream is unavailable

* Add 403 info in logger message

* Don't return error for other HTTPErrors

* Split up error handling into methods 'unavailable_error_codes' and 'get_reason_for_error'

* rework overrideable list of status codes to be a dict with reasons, to enforce that users provide reasons for all listed errors

* Fix incorrect typing

* Move HttpAvailability to its own module, fix flake errors

* Fix ScopedAvailabilityStrategy, docstrings and types for streams/availability_strategy.py

* Docstrings and types for core.py and http/availability_strategy.py

* Move _get_stream_slices to a StreamHelper class

* Docstrings + types for stream_helpers.py, cleanup test_availability.py

* Clean up test_source.py

* Move logic of getting the initial record from a stream to StreamHelper class

* Add changelog and bump minor version

* change 'is True' and 'is False' behavior

* use mocker.MagicMock

* Remove ScopedAvailabilityStrategy

* Don't except non-403 errors, check_stream uses availability_strategy if possible

* Move AvailabilityStrategy to stream level, fix tests

* make get_stream_slice public

* Attempt to refactor error code handling into repository-based and organization-based

* split into repository-based availabilitystrategy and organization-based availabilitystrategy

* refactor organization-based availabilitystrategy

* refactor repository-based availabilitystrategy, create separate ones for workflow_runs and projects

* Fix workflow runs availability strategy

* move availability strategies to a different module

* CDK: pass error to reasons_for_error_codes

* make get_stream_slice public

* Revert "make get_stream_slice public"

This reverts commit 9170fe5f59.

* Add tests for raising unhandled errors and retries are handled

* Add tests for CheckStream via AvailabilityStrategy

* Remove moved file

* bump CDK dependency

* Cleanup: Address review comments

* One more fix

* Update changelog and dockerfile version
2022-12-14 16:58:15 -05:00
Serhii Chvaliuk
2e1209e78f Source Github: switch on airbyte-cdk==0.2.0 (#18213)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-20 15:16:58 +03:00
Serhii Chvaliuk
8e6fb793bb Source Github: Use default behaviour, retry on 429 and all 5XX errors (#17852)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-12 11:35:42 +03:00
Serhii Chvaliuk
f4374530e5 Source Github: improve 502 handling for comments stream (#17715)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-07 14:58:31 +03:00
Serhii Chvaliuk
8b4f65de9e Source Github: Skip repository if got HTTP 500 for WorkflowRuns stream (#17555)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-10-04 21:50:20 +03:00
Serhii Chvaliuk
a338df1040 🐛 Source Github: bugfix WorkflowJobs stream (#17287)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-09-30 14:45:25 +03:00
Denys Davydov
57c269d9c6 Source github: improve error logging (#16947)
* #410 oncall source github: improve error logging

* source github: upd changelog

* auto-bump connector version [ci skip]

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
2022-09-21 23:16:40 +03:00
Mainara Cavalcanti
97b9c74718 Source Github: add workflow_jobs stream (#16534)
* feat(workflow-jobs)!: new class 'WorkflowJobs' to get job informations from Github;

* feat(workflow-jobs)!: unit test to new class 'WorkflowJobs';

* feat(test_stream/test_stream_workflow_jobs_read_incremental)!: support to method incremetal in unit test to class 'WorkflowJobs';

* feat(test_workflow): unit test incremental and full refresh to class 'WorkflowJobs';

* docs: updated github docs

* docs: updated github docs

* docs: updated github.md

* docs: updated github.md

* fix: add workflow_jobs in abnormal tests, use 'start date' in source and unit test;

* fix: add workflow_jobs in 'abnormal_state.json' and 'sample_state.json' for integration tests;

Co-authored-by: Paulo Roberto <paulo.rb.beserra@gmail.com>
Co-authored-by: Paulo Roberto <71179739+py-paulo@users.noreply.github.com>
2022-09-21 10:19:35 -04:00
Cameron Yick
9d44dc205b 🐛 Source Github: handle case where user is None (#15420)
* fix(source-github): handle case where user is None

Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
Co-authored-by: Marcos Marx <marcosmarxm@users.noreply.github.com>
Co-authored-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-08-12 10:29:13 +03:00
Serhii Chvaliuk
ae58fa52da 🎉 Source Github: PullRequestCommentReactions - re-implemented using GraphQL (#14795)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-08-02 14:57:27 +03:00
Serhii Chvaliuk
38d8d60f02 🎉 Source Github: add Retry for GraphQL API Resource limitations (#14376)
Signed-off-by: Sergey Chvalyuk <grubberr@gmail.com>
2022-07-05 00:37:39 +03:00