22f03e31a8
* PyPI (#11) * Init * Update acceptance-test-config.yml * Update * Update acceptance-test-config.yml * Add requested changes and docs * Update acceptance.py * Update acceptance-test-config.yml * Update setup.py * fix EOFL * Update README.md * Update README.md * changes to pass tests * Update source_definitions.yaml * Update source_specs.yaml Co-authored-by: Vincent Koc <koconder@users.noreply.github.com> Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
1.5 KiB
PyPI
This page guides you through the process of setting up the PyPI source connector.
Setup guide
Get package name from PyPI
This is the name given in pip install package_name box. For example, airbyte-cdk is the package name for airbyte-cdk.
Optianlly, provide a version name. If not provided, the release stream, containing data for particular version, cannot be used. The project stream is as same as release stream but contains data for all versions.
Supported streams and sync modes
Performance considerations
Due to the heavy caching and CDN use, there is currently no rate limiting of PyPI APIs at the edge.
In addition, PyPI reserves the right to temporarily or permanently prohibit a consumer based on irresponsible activity.
Try not to make a lot of requests (thousands) in a short amount of time (minutes). Generally PyPI can handle it, but it’s preferred to make requests in serial over a longer amount of time if possible.
Changelog
| Version | Date | Pull Request | Subject |
|---|---|---|---|
| 0.1.0 | 2022-10-29 | 18632 | Initial Release |