[Improvement]: Add more content about Lines of Code metrics (#58252)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Laura Coursen <lecoursen@github.com>
This commit is contained in:
@@ -12,9 +12,7 @@ redirect_from:
|
|||||||
- /early-access/copilot-metrics
|
- /early-access/copilot-metrics
|
||||||
- /early-access/copilot-metrics/apis
|
- /early-access/copilot-metrics/apis
|
||||||
- /early-access/copilot-metrics/dashboards
|
- /early-access/copilot-metrics/dashboards
|
||||||
- /early-access/copilot-metrics/LoC
|
|
||||||
- /early-access/copilot-metrics/apis/about-the-copilot-metrics-apis
|
- /early-access/copilot-metrics/apis/about-the-copilot-metrics-apis
|
||||||
- /early-access/copilot-metrics/LoC/about-the-copilot-metrics-LoC
|
|
||||||
- /early-access/copilot-metrics/dashboards/about-the-copilot-metrics-dashboard
|
- /early-access/copilot-metrics/dashboards/about-the-copilot-metrics-dashboard
|
||||||
- /early-access/copilot-metrics/apis/rest-api-endpoints-for-copilot-enterprise-and-user-usage-metrics
|
- /early-access/copilot-metrics/apis/rest-api-endpoints-for-copilot-enterprise-and-user-usage-metrics
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -48,6 +48,7 @@ They provide granular daily records for each user in the enterprise.
|
|||||||
|
|
||||||
| Field | Description |
|
| Field | Description |
|
||||||
|:--|:--|
|
|:--|:--|
|
||||||
|
| `agent_edit` | A dedicated bucket in the API and reports. Captures lines added and deleted directly by {% data variables.product.prodname_copilot_short %} Agent and Edit mode.<br>These are not included in suggested metrics, since agent edits don’t follow a simple suggestion to acceptance flow. |
|
||||||
| `report_start_day` / `report_end_day` | Start and end dates for the 28-day reporting period. |
|
| `report_start_day` / `report_end_day` | Start and end dates for the 28-day reporting period. |
|
||||||
| `day` | Calendar day this record represents. |
|
| `day` | Calendar day this record represents. |
|
||||||
| `enterprise_id` | Unique ID of the enterprise or organization. |
|
| `enterprise_id` | Unique ID of the enterprise or organization. |
|
||||||
14
content/copilot/reference/copilot-usage-metrics/index.md
Normal file
14
content/copilot/reference/copilot-usage-metrics/index.md
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
---
|
||||||
|
title: GitHub Copilot usage metrics
|
||||||
|
shortTitle: Copilot usage metrics
|
||||||
|
intro: Find information about usage metrics for {% data variables.product.prodname_copilot %}.
|
||||||
|
versions:
|
||||||
|
feature: copilot
|
||||||
|
topics:
|
||||||
|
- Copilot
|
||||||
|
children:
|
||||||
|
- /copilot-usage-metrics
|
||||||
|
- /interpret-copilot-metrics
|
||||||
|
- /reconciling-usage-metrics
|
||||||
|
- /lines-of-code-metrics
|
||||||
|
---
|
||||||
@@ -11,6 +11,7 @@ contentType: reference
|
|||||||
allowTitleToDifferFromFilename: true
|
allowTitleToDifferFromFilename: true
|
||||||
redirect_from:
|
redirect_from:
|
||||||
- /early-access/copilot-metrics/dashboards/interpreting-the-metrics
|
- /early-access/copilot-metrics/dashboards/interpreting-the-metrics
|
||||||
|
- /copilot/reference/interpret-copilot-metrics
|
||||||
---
|
---
|
||||||
|
|
||||||
>[!NOTE] The {% data variables.product.prodname_copilot %} usage metrics dashboard is currently in {% data variables.release-phases.public_preview %} and subject to change.
|
>[!NOTE] The {% data variables.product.prodname_copilot %} usage metrics dashboard is currently in {% data variables.release-phases.public_preview %} and subject to change.
|
||||||
@@ -72,4 +73,5 @@ Use trends in usage, feature adoption, and language activity to guide enablement
|
|||||||
|
|
||||||
## Further reading
|
## Further reading
|
||||||
|
|
||||||
To learn how to access and navigate the {% data variables.product.prodname_copilot_short %} usage metrics dashboard, see [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/view-usage-and-adoption).
|
* [AUTOTITLE](/copilot/reference/copilot-usage-metrics/reconciling-usage-metrics)
|
||||||
|
* [AUTOTITLE](/copilot/reference/copilot-usage-metrics/lines-of-code-metrics)
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
---
|
||||||
|
title: Lines of Code metrics
|
||||||
|
shortTitle: Copilot LoC metrics
|
||||||
|
intro: Understand how Lines of Code metrics measure {% data variables.product.prodname_copilot_short %}’s output and what factors affect their coverage and accuracy.
|
||||||
|
permissions: Enterprise owners and billing managers
|
||||||
|
versions:
|
||||||
|
feature: copilot
|
||||||
|
redirect_from:
|
||||||
|
- /early-access/copilot-metrics/LoC
|
||||||
|
- /early-access/copilot-metrics/LoC/about-the-copilot-metrics-LoC
|
||||||
|
topics:
|
||||||
|
- Copilot
|
||||||
|
contentType: reference
|
||||||
|
allowTitleToDifferFromFilename: true
|
||||||
|
---
|
||||||
|
|
||||||
|
{% data reusables.copilot.usage-metrics-preview %}
|
||||||
|
|
||||||
|
Lines of Code (LoC) metrics offer a directional way to measure {% data variables.product.prodname_copilot_short %}’s tangible output. These metrics quantify how many lines {% data variables.product.prodname_copilot_short %} suggested, added, or deleted in the editor—helping enterprise administrators understand {% data variables.product.prodname_copilot_short %}’s contribution to the codebase over time.
|
||||||
|
|
||||||
|
LoC metrics may vary across IDEs and versions. Keep the following considerations in mind:
|
||||||
|
|
||||||
|
* **Coverage:** Not all IDEs and users will emit LoC telemetry until they upgrade to the required versions. Reports include the `last_known_ide_version` and `last_known_plugin_version` fields so you can monitor coverage across your enterprise.
|
||||||
|
* **Stability:** As {% data variables.product.prodname_copilot_short %} evolves, telemetry definitions may change. Expect small shifts in data as accuracy improves over time.
|
||||||
|
|
||||||
|
## Version requirements
|
||||||
|
|
||||||
|
LoC metrics require specific minimum versions of IDEs and {% data variables.product.prodname_copilot_short %} plugins to function. Users on older versions will not contribute LoC data, which may lead to underreporting until they upgrade.
|
||||||
|
|
||||||
|
| IDE | Feature | Minimum IDE version | Minimum {% data variables.copilot.copilot_chat_short %} extension version |
|
||||||
|
|:--|:--|:--|:--|
|
||||||
|
| Eclipse | <ul><li>`code_completion`</li></ul> | 4.31 | 0.9.3.202507240902 |
|
||||||
|
| | <ul><li>`chat_panel` (Ask, Edit, Agent)</li></ul> | 4.31 | 0.11.0.202508291001 |
|
||||||
|
| | <ul><li>`agent_edit`</li></ul> | 4.31 | 0.10.0.202508110512 |
|
||||||
|
| IntelliJ/JetBrains | <ul><li>`code_completion`</li><li>`chat_panel` (Ask, Edit, Agent)</li><li>`chat_inline`</li><li>`agent_edit`</li></ul> | 2024.2.6 | 1.5.52-241 |
|
||||||
|
| {% data variables.product.prodname_vs %} | <ul><li>`code_completion`</li><li>`chat_panel` (Ask, Edit, Agent)</li></ul> | 17.14.13 | 18.0.471.29466 |
|
||||||
|
| | <ul><li>`chat_inline`</li><li>`agent_edit`</li></ul> | 17.14.14 | 18.0.471.29466 |
|
||||||
|
| {% data variables.product.prodname_vscode_shortname %} | <ul><li>`code_completion`</li></ul> | 1.104.0 | 0.31.0 |
|
||||||
|
| | <ul><li>`chat_panel` (Ask, Edit, Agent)</li></ul> | 1.102.0 | 0.29.0 |
|
||||||
|
| | <ul><li>`chat_inline`</li></ul> | 1.103.2 | 0.30.3 |
|
||||||
|
| | <ul><li>`agent_edit`</li></ul> | 1.103.0 | 0.30.0 |
|
||||||
|
| XCode | <ul><li>`code_completion`</li></ul> | 14.3.1 | 0.40.0 |
|
||||||
|
| | <ul><li>`chat_panel` (Ask, Edit, Agent)</li></ul> | 15.0 | 0.43.0 |
|
||||||
|
| | <ul><li>`agent_edit`</li></ul> | 15.2 | 0.41.0 |
|
||||||
|
|
||||||
|
## Available LoC metrics
|
||||||
|
|
||||||
|
LoC metrics **are** included in:
|
||||||
|
|
||||||
|
* The {% data variables.product.prodname_copilot_short %} user-level report
|
||||||
|
* The {% data variables.product.prodname_copilot_short %} usage metrics API
|
||||||
|
|
||||||
|
They are **not** included in the {% data variables.product.prodname_copilot_short %} usage metrics dashboard.
|
||||||
|
|
||||||
|
LoC metrics measure {% data variables.product.prodname_copilot_short %}’s activity across completions, chat, and agent features. For detailed field descriptions and definitions, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics/copilot-usage-metrics#api-and-export-fields).
|
||||||
|
|
||||||
|
## How agent mode affects LoC metrics
|
||||||
|
|
||||||
|
Unlike completions or chat, {% data variables.copilot.copilot_agent_short %} does not follow a “suggest then accept” flow. Agents plan and execute multi-step tasks, often editing multiple files iteratively without explicit acceptance by the user. Because of this, LoC metrics for agent activity are measured differently.
|
||||||
|
|
||||||
|
| Behavior | How it’s reflected in LoC metrics |
|
||||||
|
|:--|:--|
|
||||||
|
| Agent code suggestions | Counted as `loc_suggested_to_add_sum` when visible code blocks are shown in the chat panel. |
|
||||||
|
| Agent edits in files | Counted as `loc_added_sum` and `loc_deleted_sum` under the `agent_edit` feature bucket. These edits are not included in suggested metrics. |
|
||||||
|
| Multi-file operations | Each file edit contributes to total added and deleted lines, even if triggered by one prompt. |
|
||||||
|
|
||||||
|
Example output for agent-related LoC activity:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"totals_by_language_feature": [
|
||||||
|
{
|
||||||
|
"language": "unknown",
|
||||||
|
"feature": "chat_panel_agent_mode",
|
||||||
|
"code_generation_activity_count": 12,
|
||||||
|
"code_acceptance_activity_count": 0,
|
||||||
|
"loc_suggested_to_add_sum": 86, // code block suggestions in chat panel in agent mode
|
||||||
|
"loc_suggested_to_delete_sum": 0,
|
||||||
|
"loc_added_sum": 5, // only includes code block copy or apply in chat panel, not agent edits
|
||||||
|
"loc_deleted_sum": 0
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"language": "unknown",
|
||||||
|
"feature": "agent_edit",
|
||||||
|
"code_generation_activity_count": 345,
|
||||||
|
"code_acceptance_activity_count": 0,
|
||||||
|
"loc_suggested_to_add_sum": 0,
|
||||||
|
"loc_suggested_to_delete_sum": 0,
|
||||||
|
"loc_added_sum": 2342, // agent and edit mode edits in files
|
||||||
|
"loc_deleted_sum": 947
|
||||||
|
}
|
||||||
|
]
|
||||||
|
```
|
||||||
|
|
||||||
|
For a single agent edit, you may see metrics such as:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"user_initiated_interaction_count": 0,
|
||||||
|
"code_generation_activity_count": 1,
|
||||||
|
"code_acceptance_activity_count": 0
|
||||||
|
```
|
||||||
@@ -11,6 +11,7 @@ contentType: reference
|
|||||||
allowTitleToDifferFromFilename: true
|
allowTitleToDifferFromFilename: true
|
||||||
redirect_from:
|
redirect_from:
|
||||||
- /early-access/copilot-metrics/dashboards/best-practices-for-correlating-usage-metrics
|
- /early-access/copilot-metrics/dashboards/best-practices-for-correlating-usage-metrics
|
||||||
|
- /copilot/reference/reconciling-usage-metrics
|
||||||
---
|
---
|
||||||
|
|
||||||
{% data reusables.copilot.usage-metrics-preview %}
|
{% data reusables.copilot.usage-metrics-preview %}
|
||||||
@@ -18,7 +18,5 @@ children:
|
|||||||
- /agentic-audit-log-events
|
- /agentic-audit-log-events
|
||||||
- /extensions-glossary
|
- /extensions-glossary
|
||||||
- /copilot-usage-metrics
|
- /copilot-usage-metrics
|
||||||
- /interpret-copilot-metrics
|
|
||||||
- /reconciling-usage-metrics
|
|
||||||
contentType: reference
|
contentType: reference
|
||||||
---
|
---
|
||||||
|
|||||||
Reference in New Issue
Block a user