* added description for no repos case error
* added AirbyteTracedException
* added changelog
* removed handling error in validation
* updated github.md
* added unit test test_streams_no_streams_available_error
* auto-bump connector version
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* add pattern descriptors to more fields in github and postgres sources
* update dockerfiles and readmes
* auto-bump connector version
* manually bump github seed source files
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* add grouping and collapsing fields to postgres source
* add auth group to github source connector
* revert postgres field order changes and adjust group of schemas field
* inject group into ssh tunnel spec for postgres only, through overloaded methods
* Automated Change
* bump Dockerfile versions and update changelogs
* bump strict encrypt version as well
* fix postgres acceptance test
* fix acceptance test again
* fix all postgres acceptance tests
* add newline
* undo other changes to postgres readme file
* add security group to tunnel_method in expected_spec.json
* bump version of strict encrypt
* manually bump versions in seed files
---------
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
* Source Github: Mark datefields in spec as dates
* Update changelog
* Update Dockerfile
* auto-bump connector version
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* add grouping and collapsing fields to postgres source
* add auth group to github source connector
* revert postgres field order changes and adjust group of schemas field
* inject group into ssh tunnel spec for postgres only, through overloaded methods
* Automated Change
* bump Dockerfile versions and update changelogs
* bump strict encrypt version as well
* fix postgres acceptance test
* fix acceptance test again
---------
Co-authored-by: lmossman <lmossman@users.noreply.github.com>
* added user friendly messages, added AirbyteTracedException config_error
* added formatting
* fixed 'if' condition
* removed HTTPError words from error examples
* changed http to https for all schemas
* removed http adapter
* updated expected record file - one record for each stream
* updated expected record file
* updated expected record file
* bumped connector version
* auto-bump connector version
---------
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Source Github: handle 502 Bad Gateway with proper log message
* Source Github: bump version
* Source Github: update changelog for new 0.3.12 version
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Source Github: Raise Error if no organizations or repos are available
* Source GitHub: Update docs; bump version
* Source GitHub: Add test
* Source GitHub: Exclude empty streams
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Rough first implememtation of AvailabilityStrategy s
* Basic unit tests for AvailabilityStrategy and ScopedAvailabilityStrategy
* Make availability_strategy a property, separate out tests
* Remove from DeclarativeSource, remove Source parameter from methods, make default no AvailabilityStrategy
* Add skip stream if not available to read()
* Changes to CDK to get source-github working using AvailabilityStrategy, flakecheck
* reorganize cdk class, add HTTPAvailabilityStrategy test
* cleanup, docstrings
* source-github working with availabilitystrategy
* reorganize source-github
* source-github: resolve deprecation warning
* use correct import for HTTPAvailabilitystrategy
* pull out error handling into separate method
* use handle_http_error method
* Pass source and logger to check_connection method
* Add documentation links, handle 403 specifically
* Fix circular import
* Add AvailabilityStrategy to Stream and HTTPStream classes
* Remove AS from abstract_source, add to Stream, HTTPStream, AvailabilityStrategy unit tests passing for per-stream strategies
* Modify MockHttpStream to set no AvailabilityStrategy since source test mocking doesn't support this
* Move AvailabilityStrategy class to sources.streams
* Move HTTPAvailabilityStrategy to http module
* Use pascal case for HttpAvailabilityStrategy
* Remove docs message method :( and default to True availability on unhandled HTTPErrors
* add check_availability method to stream class
* Add optional source parameter
* Add test for connector-specific documentation, small tests refactor
* Add test that performs the read() function for stream with default availability strategy
* Add test for read function behavior when stream is unavailable
* Add 403 info in logger message
* Don't return error for other HTTPErrors
* Split up error handling into methods 'unavailable_error_codes' and 'get_reason_for_error'
* rework overrideable list of status codes to be a dict with reasons, to enforce that users provide reasons for all listed errors
* Fix incorrect typing
* Move HttpAvailability to its own module, fix flake errors
* Fix ScopedAvailabilityStrategy, docstrings and types for streams/availability_strategy.py
* Docstrings and types for core.py and http/availability_strategy.py
* Move _get_stream_slices to a StreamHelper class
* Docstrings + types for stream_helpers.py, cleanup test_availability.py
* Clean up test_source.py
* Move logic of getting the initial record from a stream to StreamHelper class
* Add changelog and bump minor version
* change 'is True' and 'is False' behavior
* use mocker.MagicMock
* Remove ScopedAvailabilityStrategy
* Don't except non-403 errors, check_stream uses availability_strategy if possible
* Move AvailabilityStrategy to stream level, fix tests
* make get_stream_slice public
* Attempt to refactor error code handling into repository-based and organization-based
* split into repository-based availabilitystrategy and organization-based availabilitystrategy
* refactor organization-based availabilitystrategy
* refactor repository-based availabilitystrategy, create separate ones for workflow_runs and projects
* Fix workflow runs availability strategy
* move availability strategies to a different module
* CDK: pass error to reasons_for_error_codes
* make get_stream_slice public
* Revert "make get_stream_slice public"
This reverts commit 9170fe5f59.
* Add tests for raising unhandled errors and retries are handled
* Add tests for CheckStream via AvailabilityStrategy
* Remove moved file
* bump CDK dependency
* Cleanup: Address review comments
* One more fix
* Update changelog and dockerfile version
* Fix events and workflow_runs datetimes in `source-github`
* add PR number
* whitespace
* auto-bump connector version
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* feat(workflow-jobs)!: new class 'WorkflowJobs' to get job informations from Github;
* feat(workflow-jobs)!: unit test to new class 'WorkflowJobs';
* feat(test_stream/test_stream_workflow_jobs_read_incremental)!: support to method incremetal in unit test to class 'WorkflowJobs';
* feat(test_workflow): unit test incremental and full refresh to class 'WorkflowJobs';
* docs: updated github docs
* docs: updated github docs
* docs: updated github.md
* docs: updated github.md
* fix: add workflow_jobs in abnormal tests, use 'start date' in source and unit test;
* fix: add workflow_jobs in 'abnormal_state.json' and 'sample_state.json' for integration tests;
Co-authored-by: Paulo Roberto <paulo.rb.beserra@gmail.com>
Co-authored-by: Paulo Roberto <71179739+py-paulo@users.noreply.github.com>
