Document GitHub Models (#51698)
Co-authored-by: SiaraMist <siaramist@github.com> Co-authored-by: Siara <108543037+SiaraMist@users.noreply.github.com>
This commit is contained in:
Sarah Edwards
11
content/github-models/index.md
Normal file
11
content/github-models/index.md
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
title: GitHub Models
|
||||
intro: 'Find and experiment with AI models for free.'
|
||||
versions:
|
||||
fpt: '*'
|
||||
ghes: '*'
|
||||
ghec: '*'
|
||||
children:
|
||||
- /prototyping-with-ai-models
|
||||
- /responsible-use-of-github-models
|
||||
---
|
||||
153
content/github-models/prototyping-with-ai-models.md
Normal file
153
content/github-models/prototyping-with-ai-models.md
Normal file
@@ -0,0 +1,153 @@
|
||||
---
|
||||
title: Prototyping with AI models
|
||||
intro: 'Find and experiment with AI models for free.'
|
||||
versions:
|
||||
feature: github-models
|
||||
---
|
||||
|
||||
If you want to develop a generative AI application, you can use {% data variables.product.prodname_github_models %} to find and experiment with AI models for free. Once you are ready to bring your application to production, you can switch to a token from a paid Azure account. See the [Azure AI](https://ai.azure.com/github/model/docs) documentation.
|
||||
|
||||
See also "[AUTOTITLE](/github-models/responsible-use-of-github-models)."
|
||||
|
||||
## Finding AI models
|
||||
|
||||
To find AI models, go to [{% data variables.product.prodname_marketplace %}](https://github.com/marketplace/models), then click {% octicon "ai-model" aria-hidden="true" %} **Models** in the sidebar.
|
||||
|
||||
To view details about a model, click on the model's name.
|
||||
|
||||
## Experimenting with AI models in the playground
|
||||
|
||||
>[!NOTE]
|
||||
>
|
||||
> The playground is in limited public beta and subject to change. To request access, [join the waitlist](https://github.com/marketplace/models/waitlist).
|
||||
|
||||
{% data variables.product.prodname_marketplace %} provides a free playground where you can adjust model parameters and submit prompts to see how the model responds.
|
||||
|
||||
To open the playground, go to [{% data variables.product.prodname_marketplace %}](https://github.com/marketplace/models), then click {% octicon "ai-model" aria-hidden="true" %} **Models** in the sidebar. Click on a model's name, then click {% octicon "command-palette" aria-hidden="true" %} **Playground**.
|
||||
|
||||
To adjust parameters for the model, select the **Parameters** tab in the sidebar. To see code that corresponds to the parameters that you selected, switch from the **Chat** tab to the **Code** tab.
|
||||
|
||||
The playground is rate limited. See [Rate limits](#rate-limits) below.
|
||||
|
||||
## Experimenting with AI models using the API
|
||||
|
||||
>[!NOTE]
|
||||
>
|
||||
> The free API usage is in limited public beta and subject to change. To request access, [join the waitlist](https://github.com/marketplace/models/waitlist).
|
||||
|
||||
{% data variables.product.company_short %} provides free API usage so that you can experiment with AI models in your own application.
|
||||
|
||||
To learn how to use a model in your application, go to [{% data variables.product.prodname_marketplace %}](https://github.com/marketplace/models), then click {% octicon "ai-model" aria-hidden="true" %} **Models** in the sidebar. Click on a model's name, then click {% octicon "code" aria-hidden="true" %} **Code**.
|
||||
|
||||
The steps to use each model are similar. In general, you will need to:
|
||||
|
||||
1. Optionally, use the language dropdown to select the programming language.
|
||||
1. Optionally, use the SDK dropdown to select which SDK to use.
|
||||
|
||||
All models can be used with the Azure AI Inference SDK, and some models support additional SDKs. If you want to easily switch between models, you should select "Azure AI Inference SDK". If you selected "REST" as the language, you won't use an SDK. Instead, you will use the API endpoint directly.
|
||||
1. Either open a codespace, or set up your local environment:
|
||||
* To run in a codespace, click {% octicon "codespaces" aria-hidden="true" %} **Run codespace**, then click **Create new codespace**.
|
||||
* To run locally:
|
||||
* Create a {% data variables.product.company_short %} {% data variables.product.pat_generic %}. The token should not have any scopes or permissions. See "[AUTOTITLE](/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)."
|
||||
* Save your token as an environment variable.
|
||||
* Install the dependencies for the SDK, if required.
|
||||
1. Use the example code to make a request to the model.
|
||||
|
||||
The free API usage is rate limited. See [Rate limits](#rate-limits) below.
|
||||
|
||||
## Going to production
|
||||
|
||||
The rate limits for the playground and free API usage are intended to help you experiment with models and develop your AI application. Once you are ready to bring your application to production, you can use a token from a paid Azure account instead of your {% data variables.product.company_short %} {% data variables.product.pat_generic %}. You don't need to change anything else in your code. For more information, see the [Azure AI](https://ai.azure.com/github/model/docs) documentation.
|
||||
|
||||
## Rate limits
|
||||
|
||||
The playground and free API usage are rate limited by requests per minute, requests per day, tokens per request, and concurrent requests. If you get rate limited, you will need to wait for the rate limit that you hit to reset before you can make more requests.
|
||||
|
||||
Low, high, and embedding models have different rate limits. To see which type of model you are using, refer to the model's information in {% data variables.product.prodname_marketplace %}.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th scope="col" style="width:15%"><b>Rate limit tier</b></th>
|
||||
<th scope="col" style="width:25%"><b>Rate limits</b></th>
|
||||
<th scope="col" style="width:20%"><b>Free and Copilot Individual</b></th>
|
||||
<th scope="col" style="width:20%"><b>Copilot Business</b></th>
|
||||
<th scope="col" style="width:20%"><b>Copilot Enterprise</b></th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th rowspan="4" scope="rowgroup"><b>Low</b></th>
|
||||
<th style="padding-left: 0"><b>Requests per minute</b></th>
|
||||
<td>15</td>
|
||||
<td>15</td>
|
||||
<td>20</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Requests per day</b></th>
|
||||
<td>150</td>
|
||||
<td>300</td>
|
||||
<td>450</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Tokens per request</b></th>
|
||||
<td>8000 in, 4000 out</td>
|
||||
<td>8000 in, 4000 out</td>
|
||||
<td>8000 in, 8000 out</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Concurrent requests</b></th>
|
||||
<td>5</td>
|
||||
<td>5</td>
|
||||
<td>8</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th rowspan="4" scope="rowgroup"><b>High</b></th>
|
||||
<th style="padding-left: 0"><b>Requests per minute</b></th>
|
||||
<td>10</td>
|
||||
<td>10</td>
|
||||
<td>15</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Requests per day</b></th>
|
||||
<td>50</td>
|
||||
<td>100</td>
|
||||
<td>150</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Tokens per request</b></th>
|
||||
<td>8000 in, 4000 out</td>
|
||||
<td>8000 in, 4000 out</td>
|
||||
<td>16000 in, 8000 out</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Concurrent requests</b></th>
|
||||
<td>2</td>
|
||||
<td>2</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th rowspan="4" scope="rowgroup" style="box-shadow: none"><b>Embedding</b></th>
|
||||
<th style="padding-left: 0"><b>Requests per minute</b></th>
|
||||
<td>15</td>
|
||||
<td>15</td>
|
||||
<td>20</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Requests per day</b></th>
|
||||
<td>150</td>
|
||||
<td>300</td>
|
||||
<td>450</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Tokens per request</b></th>
|
||||
<td>64000</td>
|
||||
<td>64000</td>
|
||||
<td>64000</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th><b>Concurrent requests</b></th>
|
||||
<td>5</td>
|
||||
<td>5</td>
|
||||
<td>8</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
These limits are subject to change without notice.
|
||||
12
content/github-models/responsible-use-of-github-models.md
Normal file
12
content/github-models/responsible-use-of-github-models.md
Normal file
@@ -0,0 +1,12 @@
|
||||
---
|
||||
title: Responsible use of GitHub Models
|
||||
shortTitle: Responsible use
|
||||
intro: 'Learn how to use {% data variables.product.prodname_github_models %} responsibly by understanding its purposes, capabilities, and limitations.'
|
||||
versions:
|
||||
feature: github-models
|
||||
type: rai
|
||||
---
|
||||
|
||||
With {% data variables.product.prodname_github_models %}, you build your understanding of AI model capabilities by experimenting with model settings and sending prompts through a chat interface. Additionally, you can directly interact with models through an SDK. Refer to a model's "Getting Started" tab for more information about how to use the SDK. Refer to a model’s "README" tab for more information on the model. Remember when interacting with a model you are experimenting with AI, so content mistakes are possible.
|
||||
|
||||
{% data variables.product.prodname_github_models %} is designed to allow for learning, experimentation and proof-of-concept activities. The feature is subject to various limits (including requests per minute, requests per day, tokens per request, and concurrent requests) and is not designed for production use cases. {% data variables.product.prodname_github_models %} employs a number of [content filters](https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety). These filters cannot be turned off as part of the {% data variables.product.prodname_github_models %} experience. If you decide to employ models through [Azure AI](https://ai.azure.com/github/model/docs) or a paid service, please configure your content filters to meet your requirements.
|
||||
Reference in New Issue
Block a user