Update all Copilot usage metrics docs to include organization API availability (#58823)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Isaac Brown <101839405+isaacmbrown@users.noreply.github.com> Co-authored-by: docs-bot <77750099+docs-bot@users.noreply.github.com> Co-authored-by: Alexandra Lato <102535292+alexlato22@users.noreply.github.com>
This commit is contained in:
Sophie
@@ -19,15 +19,21 @@ redirect_from:
|
||||
|
||||
{% data reusables.copilot.usage-metrics-preview %}
|
||||
|
||||
## About {% data variables.product.prodname_copilot_short %} usage metrics
|
||||
|
||||
{% data variables.product.prodname_copilot_short %} usage metrics help key stakeholders and decision-makers understand how their teams are adopting and using {% data variables.product.prodname_copilot_short %}. By tracking usage patterns across the enterprise, you can measure engagement, identify opportunities to increase value, and make data-driven decisions about enablement and rollout.
|
||||
|
||||
Metrics are available through:
|
||||
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics APIs, which provide detailed, user-level data you can export for further analysis.
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics APIs, which provide detailed, exportable data at the enterprise, organization, and user levels.
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics dashboard, which visualizes 28-day usage trends across your enterprise.
|
||||
* The code generation dashboard, which breaks down how code is being generated by users and agents across your enterprise.
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics NDJSON export, which offers raw data for custom BI tools or long-term storage.
|
||||
|
||||
### Organization-level metrics
|
||||
|
||||
Organization-level metrics are **only available through the APIs**. Both dashboards provide enterprise-level reporting.
|
||||
|
||||
For a complete list of available metrics and data fields, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics).
|
||||
|
||||
## Which usage is included?
|
||||
@@ -43,6 +49,22 @@ This means the data **does not include** activity from other {% data variables.p
|
||||
|
||||
License and seat management data are not included in {% data variables.product.prodname_copilot_short %} usage metrics reports. To view or manage license assignments, use the {% data variables.product.prodname_copilot_short %} user management API, which is the source of truth for license and seat information. See [AUTOTITLE](/rest/copilot/copilot-user-management).
|
||||
|
||||
## How are metrics attributed across organizations?
|
||||
|
||||
Organization-level {% data variables.product.prodname_copilot_short %} usage metrics are based on organization membership, not on where individual actions occur. To appear in an enterprise’s metrics, a user must have an active {% data variables.product.prodname_copilot_short %} seat assigned within that enterprise (in any organization that belongs to the enterprise). As a result, a single user’s usage may appear in multiple organization dashboards, while that same user is counted only once in the enterprise-level total. Organization-level analytics are intended for visibility into adoption and usage within an organization and are not designed to be directly compared to enterprise-level totals.
|
||||
|
||||
Organization-level {% data variables.product.prodname_copilot_short %} analytics are available starting December 12, 2025. This is the first date for which organization-level reports are provided.
|
||||
|
||||
|
||||
Once a user has a seat in the enterprise, their usage is attributed to **every organization they belong to**, regardless of where the seat is assigned.
|
||||
|
||||
This means:
|
||||
|
||||
* If licenses are assigned in a dedicated “shell” organization for administrative purposes within the enterprise, users still appear in the metrics for all other organizations in the enterprise they belong to.
|
||||
* If a user also has a {% data variables.product.prodname_copilot_short %} seat in a **separate organization outside the enterprise**, their activity is still included in the enterprise’s organization-level metrics **as long as they have at least one seat within the enterprise**.
|
||||
|
||||
In short: users must be licensed somewhere in the enterprise to appear in its metrics. Once they are, metrics reflect where they work (their organization membership), not which organization provides the {% data variables.product.prodname_copilot_short %} seat or where the activity originated.
|
||||
|
||||
## Supported IDEs
|
||||
|
||||
To be included in the {% data variables.product.prodname_copilot_short %} usage metrics, end users must use one of the following IDEs and {% data variables.copilot.copilot_chat_short %} extension versions.
|
||||
@@ -94,4 +116,4 @@ Now that you understand what each {% data variables.product.prodname_copilot_sho
|
||||
|
||||
* To view adoption and usage trends across your enterprise, see [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/view-usage-and-adoption).
|
||||
* To analyze how code is being generated by users and agents, see [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/view-code-generation).
|
||||
* To access these metrics programmatically, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
* To access enterprise-, organization-, and user-level metrics programmatically, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
|
||||
@@ -24,6 +24,8 @@ The code generation dashboard shows how {% data variables.product.prodname_copil
|
||||
{% data reusables.copilot.access-copilot-metrics-dashboard %}
|
||||
1. In the left sidebar, click **Code generation**.
|
||||
|
||||
{% data reusables.copilot.org-copilot-usage-metrics %}
|
||||
|
||||
## What you can view
|
||||
|
||||
The dashboard shows aggregated code generation activity across your enterprise, including:
|
||||
@@ -39,4 +41,4 @@ For a full list of available metrics and definitions, see [AUTOTITLE](/copilot/r
|
||||
|
||||
* To explore adoption and usage trends across your enterprise, see [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/view-usage-and-adoption).
|
||||
* To understand how Lines of Code (LoC) metrics are measured and what affects their accuracy and coverage, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/lines-of-code-metrics).
|
||||
* To access usage data programmatically, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
* To access enterprise-, organization-, and user-level usage data programmatically, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
|
||||
@@ -24,7 +24,7 @@ After your initial rollout, the {% data variables.product.prodname_copilot_short
|
||||
{% data reusables.copilot.access-copilot-metrics-dashboard %}
|
||||
1. In the left sidebar, click **{% data variables.product.prodname_copilot_short %} usage**.
|
||||
|
||||
Data in the dashboard is based only on IDE telemetry and may appear up to three full UTC days behind the current date. See [AUTOTITLE](/copilot/concepts/copilot-metrics#data-freshness).
|
||||
Data in the dashboard reports data at the enterprise level, is based only on IDE telemetry, and may appear up to three full UTC days behind the current date. See [AUTOTITLE](/copilot/concepts/copilot-metrics).
|
||||
|
||||
## Using {% data variables.copilot.copilot_chat_short %} to analyze exported data
|
||||
|
||||
@@ -32,7 +32,7 @@ For deeper analysis, you can export NDJSON reports from the dashboard and use {%
|
||||
|
||||
```copilot copy prompt
|
||||
* Which users have `user_initiated_interaction_count` > 0 but low `code_acceptance_activity_count`?
|
||||
* Are there specific organizations or teams with lower adoption rates?
|
||||
* Are there specific teams with lower adoption rates?
|
||||
```
|
||||
|
||||
## Next steps
|
||||
|
||||
@@ -17,6 +17,9 @@ topics:
|
||||
|
||||
The {% data variables.product.prodname_copilot_short %} usage metrics dashboard and APIs display and export data using a consistent set of fields. This reference lists all available metrics and describes how to interpret their values in both dashboard visuals and NDJSON or API exports. To retrieve this data programmatically, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics dashboard reports data at the **enterprise** level.
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics APIs support **enterprise-, organization-, and user-level** records.
|
||||
|
||||
For guidance on how to read and interpret these metrics, see [AUTOTITLE](/copilot/concepts/copilot-metrics).
|
||||
|
||||
## {% data variables.product.prodname_copilot_short %} usage dashboard metrics
|
||||
@@ -60,14 +63,15 @@ These metrics appear in the code generation dashboard and provide a breakdown of
|
||||
|
||||
## API and export fields
|
||||
|
||||
These fields appear in the exported NDJSON reports and in the {% data variables.product.prodname_copilot_short %} usage metrics APIs. They provide granular daily records for each user in the enterprise.
|
||||
These fields appear in the exported NDJSON reports and in the {% data variables.product.prodname_copilot_short %} usage metrics APIs. They provide granular daily records for each user included in the requested enterprise or organization scope.
|
||||
|
||||
| Field | Description |
|
||||
|:--|:--|
|
||||
| `agent_edit` | A dedicated bucket in the API and reports. Captures lines added and deleted directly by {% data variables.product.prodname_copilot_short %} Agent and Edit mode.<br>These are not included in suggested metrics, since agent edits don’t follow a simple suggestion to acceptance flow. |
|
||||
| `report_start_day` / `report_end_day` | Start and end dates for the 28-day reporting period. |
|
||||
| `day` | Calendar day this record represents. |
|
||||
| `enterprise_id` | Unique ID of the enterprise or organization. |
|
||||
| `enterprise_id` | Unique ID of the enterprise. |
|
||||
| `organization_id` (API only) | Unique ID of the organization. |
|
||||
| `user_id` / `user_login` | Unique identifier and {% data variables.product.github %} username for the user. |
|
||||
| `user_initiated_interaction_count` | Number of explicit prompts sent to {% data variables.product.prodname_copilot_short %}.<br><br> Only counts messages or prompts actively sent to the model. Does **not** include opening the chat panel, switching modes (for example, Ask, Edit, or Agent), using keyboard shortcuts to open the inline UI, or making configuration changes. |
|
||||
| `code_generation_activity_count` | Number of distinct {% data variables.product.prodname_copilot_short %} output events generated. <br><br> **Includes:** All generated content, including comments and docstrings. <br> **Multiple blocks:** Each distinct code block from a single user prompt counts as a separate generation. <br> **Note:** This metric is not directly comparable to `user_initiated_interaction_count`, since one prompt can produce multiple generations. |
|
||||
|
||||
@@ -16,9 +16,9 @@ redirect_from:
|
||||
|
||||
>[!NOTE] The {% data variables.product.prodname_copilot_short %} usage metrics dashboard is currently in {% data variables.release-phases.public_preview %} and subject to change.
|
||||
|
||||
After you’ve viewed usage data in the {% data variables.product.prodname_copilot_short %} usage metrics dashboard, you can use this article to interpret what each chart means and identify opportunities to increase adoption and engagement across your enterprise.
|
||||
After you’ve viewed the {% data variables.product.prodname_copilot_short %} usage metrics dashboard, you can use this article to interpret each chart and identify opportunities to increase adoption and engagement.
|
||||
|
||||
You can also access these metrics programmatically through the {% data variables.product.prodname_copilot_short %} usage metrics REST API, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
> [!NOTE] The dashboard reports enterprise-level data. Organization-level views are available through the {% data variables.product.prodname_copilot_short %} usage metrics APIs.
|
||||
|
||||
## Reviewing overall usage trends
|
||||
|
||||
@@ -73,7 +73,6 @@ Use trends in usage, feature adoption, and language activity to guide enablement
|
||||
> [!TIP]
|
||||
> Consider combining dashboard trends with feedback from surveys or retrospectives to get a full picture of {% data variables.product.prodname_copilot_short %}’s impact on developer productivity.
|
||||
|
||||
## Further reading
|
||||
## Next steps
|
||||
|
||||
* [AUTOTITLE](/copilot/reference/copilot-usage-metrics/reconciling-usage-metrics)
|
||||
* [AUTOTITLE](/copilot/reference/copilot-usage-metrics/lines-of-code-metrics)
|
||||
* To access metrics programmatically, including enterprise, organization, and user-level records, see [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
|
||||
@@ -22,8 +22,8 @@ Lines of Code (LoC) metrics provide a directional measure of {% data variables.p
|
||||
|
||||
You may encounter LoC metrics in:
|
||||
|
||||
* **Code generation dashboard** — visualizes LoC-based user and agent activity. See [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/view-code-generation).
|
||||
* **Exports and APIs** — expose LoC fields such as `loc_suggested_to_add_sum`, `loc_added_sum`, and `loc_deleted_sum`. See [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
* **Code generation dashboard (enterprise level)** — visualizes LoC-based user and agent activity. See [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/view-code-generation).
|
||||
* **Exports and APIs (enterprise, organization, and user level)** — expose LoC fields such as `loc_suggested_to_add_sum`, `loc_added_sum`, and `loc_deleted_sum`. See [AUTOTITLE](/rest/copilot/copilot-usage-metrics).
|
||||
* **"Data available in {% data variables.product.prodname_copilot_short %} usage metrics" reference** — lists all dashboard and API fields derived from LoC. See [AUTOTITLE](/copilot/reference/copilot-usage-metrics/copilot-usage-metrics).
|
||||
|
||||
This article covers how LoC is measured and what affects its accuracy and coverage. For detailed definitions, refer to the reference article above.
|
||||
|
||||
@@ -18,6 +18,9 @@ redirect_from:
|
||||
|
||||
The {% data variables.product.prodname_copilot_short %} usage metrics dashboard, APIs, and export files all use the same underlying telemetry data, but they aggregate and present it differently. Understanding these differences helps you reconcile numbers across sources and trust your analysis when preparing internal reports.
|
||||
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics dashboard reports data at the **enterprise-level**.
|
||||
* The {% data variables.product.prodname_copilot_short %} usage metrics APIs support **enterprise-, organization-, and user-level** records.
|
||||
|
||||
## Prerequisite
|
||||
|
||||
{% data variables.product.prodname_copilot_short %} usage metrics depend on **telemetry from users’ IDEs**. If a developer has disabled telemetry in their IDE, their {% data variables.product.prodname_copilot_short %} activity will **not** appear in the dashboard, API reports, or exported data.
|
||||
|
||||
@@ -38,6 +38,10 @@ Setting clear goals makes it easier to interpret results and communicate value t
|
||||
|
||||
## Step 2: View adoption and engagement metrics in the dashboard
|
||||
|
||||
> [!NOTE]
|
||||
> * The {% data variables.product.prodname_copilot_short %} usage metrics dashboard reports data at the enterprise level.
|
||||
> * Organization-level metrics are available through the {% data variables.product.prodname_copilot_short %} usage metrics APIs and exports.
|
||||
|
||||
{% data reusables.copilot.access-copilot-metrics-dashboard %}
|
||||
|
||||
The dashboard shows 28 days of aggregated IDE telemetry data for all licensed users in your enterprise. Focus on these **key metrics** during your trial:
|
||||
|
||||
Reference in New Issue
Block a user