* DNC
* Add test models
* Add model test
* Remove underscore from metadata files
* Regenerate models
* Add test to check for key transformation
* Allow additional fields on metadata
* Delete transform
* Add airbyte internal
* Add tests
* First pass
* Set destinations to same levels as sources
* Best guess at missing statuses
* Best guess at _ql
* Add separate enum class
* Fix support level name
* Update templates
* Add one more test
* Move icons to connector folder
* Delete old icons
* Update upload logic
* Add icon url to definitions
* Update registry model
* Populate cdn url
* DNC butcher the pipeline
* Low hanging fruit fixes
* Fix bucket name
* Merge old and new approaches
* Fix metadata upload step
* Format
* Fix test
* rename catalog to registry in metadata service
* rename catalog to registry in metadata files
* Run generate models
* Fix missed renames
* Add github personal access token
* Run black
* Automated Change
---------
Co-authored-by: bnchrch <bnchrch@users.noreply.github.com>
Scripts to
* Run CATs against the local CDK for one connector
* Run CATs against the local CDK for multiple connectors
* Create a connecter image with the local CDK
---------
Co-authored-by: Alexandre Girard <alexandre@airbyte.io>
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
* Expected Records to `.jsonl` format
* fix formatting template
* remove endline
* update templates
* Update docs/connector-development/testing-connectors/source-acceptance-tests-reference.md
Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
Co-authored-by: Pedro S. Lopez <pedroslopez@me.com>
* Source Okta: add resource-sets (inremental supported)
- stop using the deprecated method get_updated_state, use state property and state setter instead
- the payload resource-sets is enveloped, _links.next.href contains the cursor
* fix: change assert statement with flake formatting
* fix unit tests
* clean up get_updated_state overriding
* rename and correct sample config
* correct sample valid and invalid config
* fake the token more
* fix the invalid_config.json
* change the order of stream, hopefully logs is run firstly
* fix: remove log stream from configured catalog
* fix: bump connector version on okta.md and Dockerfile
* auto-bump connector version [ci skip]
Co-authored-by: Sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Added parameter 'start_date' in Okta source
added: parameter 'start_date' to source Okta
changed: unit tests
* changes: fix in the case of ISSUE: https://github.com/airbytehq/airbyte/issues/14196
* Okta documentation in new format
* changes: fix to use super() instead of instance of stream parent
* changes: additional changes into OKTA documentaton
* changes: switch release to beta
* changed: set dockerImageTag -> 0.1.11
* changed: source_specs
* ...
* ...
* Rollback releaseStage
* Refactored start date logic
* Deleted microseconds from state
* Add start date to all streams
* Updated to linter
* Fixed unit tests
* Updated unit tests
* auto-bump connector version [ci skip]
Co-authored-by: Serhii <serglazebny@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Source Okta: add permission stream under a custom role
- it supports full refresh only
- add unit tests
* bump connector version
* bump connector version in Dockerfile
* auto-bump connector version [ci skip]
Co-authored-by: sajarin <sajarindider@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* add statuses to user filtering
* bump version
* upd
* add comment
* auto-bump connector version [ci skip]
* format
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Created UnitTests for Source Okta
added: UnitTests for Source Okta - coverage 100%
changes: minor changes to source
* changes: changes due to recommendation on review
* Add User_Role_Assignments and Group_Role_Assignments stream
- it's all about okta source connector
- these 2 API streams don't support the sync mode incremental, the api doesn's mention cursor or pagination for these 2 APIs
- add $id to shared schema, rename shared schema
* formatted
* chore: added connector to definitions
* auto-bump connector version [ci skip]
Co-authored-by: Harshith Mullapudi <harshithmullapudi@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* add Group_Members stream to okta source
- Group_Members return a list of users, the same schema of Users stream.
- Create a shared schema users, and both group_members and users sechema use it as a reference.
- Add Group_Members stream to source connector
* add tests and fix logs schema
- fix the test error: None is not one of enums though the enum type includes both string and null, it comes from json schema validator
ddb87afad8/jsonschema/_validators.py (L279-L285)
- change grouop_members to use id as the cursor field since `filter` is not supported in the query string
- fix the abnormal state test on logs stream, when since is abnormally large, until has to defined, an equal or a larger value
- remove logs stream from full sync test, because 2 full sync always has a gap -- at least a new log about users or groups api.
* last polish before submit the PR
- bump docker version
- update changelog
- add the right abnormal value for logs stream
- correct the sample catalog
* address comments::
- improve comments for until parameter under the logs stream
- add use_cache on groupMembers
* add use_cache to Group_Members
* change configured_catalog to test
* auto-bump connector version
Co-authored-by: marcosmarxm <marcosmarxm@gmail.com>
Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
* Dockerfile to 3.9
* Python version
* More python updates
* 3.9 on GitHub actions and lint updates
* Test out 3.9.11 on GitHub actions
* install python with an action
* formatting: newline
* Also has python code
* only check first level for changed modules
Previous example (source-google-search-console/credentials)
* Test failure: there is no logger.trace
* fixed bug which crashes okta log incremental sync
* bump connector version
* update acceptance test
* rollback pendulum function
* add comment
* bump version
* run format
Co-authored-by: collin <collinscangarella@gmail.com>
source-acceptance-test framework not longer required json_schema parameters
from catalog file. This parameter is verbose, makes reading config file
complicated and could be misleading when debugging acceptance test issues.
Co-authored-by: Dmytro Rezchykov <dmitry.rezchykov@zazmic.com>
* add AIRBYTE_ENTRYPOINT for kubernetes support
* bump versions
* bump version in seed
* Update generic template
* keep scaffold sources at 0.1.0
* add missing newline
* handle python base versions correctly
* re-bump mysql and postgres sources
* re-bump snowflake destination
* add skip tests option
* switch to running tests
* reverse conditional to make it safer
* fix publish to include the test running
* fix iterable version
* fix file generation
Co-authored-by: Sherif A. Nada <snadalive@gmail.com>
