[DO NOT MERGE] November 8–9: GitHub Universe 2023 megabranch (#44245)
Co-authored-by: mchammer01 <42146119+mchammer01@users.noreply.github.com> Co-authored-by: Courtney Claessens <courtneycl@github.com> Co-authored-by: Anne-Marie <102995847+am-stead@users.noreply.github.com> Co-authored-by: Steve Guntrip <stevecat@github.com> Co-authored-by: github-actions <github-actions@github.com> Co-authored-by: Jules <19994093+jules-p@users.noreply.github.com> Co-authored-by: Jules Porter <jules-p@users.noreply.github.com> Co-authored-by: Sarita Iyer <66540150+saritai@users.noreply.github.com> Co-authored-by: hubwriter <hubwriter@github.com> Co-authored-by: Melissa Xie <mxie@users.noreply.github.com> Co-authored-by: Andy Feller <andyfeller@github.com> Co-authored-by: Felicity Chapman <felicitymay@github.com> Co-authored-by: Kelly Arwine <kellyarwine@github.com> Co-authored-by: Ben Ahmady <32935794+subatoi@users.noreply.github.com> Co-authored-by: Max Schaefer <54907921+max-schaefer@users.noreply.github.com> Co-authored-by: Tiferet Gazit <tiferet@github.com> Co-authored-by: Aditya Sharad <6874315+adityasharad@users.noreply.github.com> Co-authored-by: Andrew Eisenberg <aeisenberg@github.com> Co-authored-by: Isaac Brown <101839405+isaacmbrown@users.noreply.github.com> Co-authored-by: Annelisa Stephan <meowius@github.com> Co-authored-by: Vanessa <vgrl@github.com> Co-authored-by: Rachael Rose Renk <91027132+rachaelrenk@users.noreply.github.com> Co-authored-by: Matt Pollard <mattpollard@users.noreply.github.com> Co-authored-by: isaacmbrown <isaacmbrown@github.com> Co-authored-by: Greg Padak <gpadak@github.com>
This commit is contained in:
Sophie
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: About Copilot pull request summaries
|
||||
shortTitle: About PR summaries
|
||||
intro: 'With {% data variables.product.prodname_copilot_for_prs %}, you can create an AI-generated description for a pull request.'
|
||||
versions:
|
||||
feature: copilot
|
||||
topics:
|
||||
- Copilot
|
||||
type: rai
|
||||
product: '{% data reusables.gated-features.copilot-enterprise-beta %}'
|
||||
---
|
||||
|
||||
## About {% data variables.product.prodname_copilot_for_prs %}
|
||||
|
||||
{% data variables.product.prodname_copilot_for_prs %} is an AI-powered feature that allows you to create a summary of the changes that were made in a pull request, which files they impact, and what a reviewer should focus on when they conduct their review.
|
||||
|
||||
When a user requests a summary, {% data variables.product.prodname_copilot_short %} scans through the pull request and provides an overview of the changes made in prose, as well as a bulleted list of changes with the files that they impact.
|
||||
|
||||
{% data variables.product.prodname_copilot_for_prs %} uses a simple-prompt flow leveraging the {% data variables.product.prodname_copilot_short %} API, with no additional trained models. This utilizes the generic large language model.
|
||||
|
||||
### Response generation
|
||||
|
||||
The current process uses GPT 3.5 to initiate the auto-complete process and generate the pull request summary.
|
||||
|
||||
#### Pipeline approach
|
||||
|
||||
When a user requests a summary, a workflow is triggered. The workflow uses the code diffs to build a prompt call, which requests {% data variables.product.prodname_copilot_short %} to generate a summary of the pull request. The summary request initiates a pipeline process which includes raw diffs in a prompt and requests {% data variables.product.prodname_copilot_short %} to generate individual summaries of the various diff hunks. This process creates several individual summaries. {% data variables.product.prodname_copilot_short %} then takes those diff hunk summaries and uses them to create a file-level summary. This file-level summary exists for each file that's summarizable, but excludes binary files and files {% data variables.product.prodname_copilot_short %} deems not appropriate for summarization.
|
||||
|
||||
{% data variables.product.prodname_copilot_short %} then takes the file-level summaries and uses them to request another prompt which then creates an overall summary for the pull request. Every summary request will generate a minimum of N+1 prompts, where N equals the number of summarizable files in the pull request.
|
||||
|
||||
### Output formatting
|
||||
|
||||
A summary generated by {% data variables.product.prodname_copilot_short %} will be in two parts:
|
||||
|
||||
- A three sentence overview, written in prose, to give a user an overview of what the changes in the pull request entail
|
||||
- 3–5 of those changes listed in bulleted form that link out to the respective lines of code that they refer to
|
||||
|
||||
You can initiate this feature when creating a pull request, by editing the pull request description after creation, or in a comment in the pull request thread. This can take a couple of minutes on larger pull requests. You can share feedback directly from the UI.
|
||||
|
||||
## Use case for pull request summaries
|
||||
|
||||
The goal of {% data variables.product.prodname_copilot_for_prs %} is to help optimize an author's ability to quickly provide context when they request a human review that requires sharing context of the changes that were made. It may help increase developer productivity by reducing the time taken to open a pull request.
|
||||
|
||||
For many users, it could provide more helpful context for the changes that were made within a pull request than would normally be available.
|
||||
|
||||
## Improving performance of pull request summaries
|
||||
|
||||
### Use {% data variables.product.prodname_copilot_for_prs %} as a tool, not a replacement
|
||||
|
||||
The feature is intended to supplement rather than replace a human's work to add context, and we encourage you to continue adding useful context and let {% data variables.product.prodname_copilot_short %} do the busy work of parsing the code and linking to specific files. It remains your responsibility to review and assess the accuracy of information in a pull request that you create.
|
||||
|
||||
### Provide feedback
|
||||
|
||||
This feature is currently in beta. If you encounter any issues or limitations with {% data variables.product.prodname_copilot_for_prs %}, we recommend that you provide feedback through the link that appears the UI after a summary is generated. You can provide feedback through the text link which takes you to our survey.
|
||||
|
||||
## Limitations of pull request summaries
|
||||
|
||||
Currently, our team is aware that there are limitations to this feature. Many of them are expected in leveraging our {% data variables.product.prodname_copilot_short %} API; however, there are a few that are specific to {% data variables.product.prodname_copilot_for_prs %} which pertain to limited scope, longer processing times, and inaccurate responses. We also note that users should expect terms used in their PR to appear in the AI-generated summary. This feature has been subject to RAI Red Teaming and we will continue to monitor the efficacy and safety of the feature over time. For more information, see [Microsoft AI Red Team building future of safer AI](https://www.microsoft.com/en-us/security/blog/2023/08/07/microsoft-ai-red-team-building-future-of-safer-ai/) on the Microsoft security blog.
|
||||
|
||||
### Limited scope
|
||||
|
||||
Because of capacity, we know that larger pull requests that reference 30 or more files will require more time to be processed thoroughly. We don't have an exact threshold currently, but have observed the first 30 files being accounted for and then any additional files being omitted from the summarization. We are working to address this current scope limitation.
|
||||
|
||||
### Processing time
|
||||
|
||||
In general, we expect a summary to be returned in 40 seconds or less after a user initiates the action. However, we have heard that this can take up to a minute, and in some cases a couple of minutes. We are working to decrease processing time and we know that users may not want to wait for this to finish before moving on to other parts of the pull request.
|
||||
|
||||
### Inaccurate responses
|
||||
|
||||
The more inputs and context that {% data variables.product.prodname_copilot_short %} can learn from, the better the outputs will become. However, since the feature is quite new, it will take time to reach exact precision with the summaries that are generated. In the meantime, there may be cases where a user's generated summary is less accurate and requires the user to make modifications before saving and publishing their pull request with this description. Reviewing is a requirement, and careful review of the output is highly recommended by our team during the beta.
|
||||
|
||||
### Replication of pull request content
|
||||
|
||||
Because a summary is an outline of the changes that were made in a pull request, if harmful or offensive terms are within the content of the pull request, there is potential for the summary to also include those terms.
|
||||
|
||||
## Further reading
|
||||
|
||||
- [{% data variables.product.prodname_copilot %} Trust Center](https://resources.github.com/copilot-trust-center/)
|
||||
{%- ifversion fpt %}
|
||||
- "[AUTOTITLE](/copilot/github-copilot-enterprise/copilot-pull-request-summaries/creating-a-pull-request-summary-with-github-copilot)" in the {% data variables.product.prodname_ghe_cloud %} documentation.
|
||||
{%- endif %}
|
||||
@@ -0,0 +1,46 @@
|
||||
---
|
||||
title: Creating a pull request summary with GitHub Copilot
|
||||
shortTitle: Create a PR summary
|
||||
intro: 'You can generate a summary in the description of a pull request, or as a comment.'
|
||||
versions:
|
||||
feature: copilot-pr-summaries
|
||||
topics:
|
||||
- Copilot
|
||||
product: '{% data reusables.gated-features.copilot-enterprise-beta %}'
|
||||
---
|
||||
|
||||
## About {% data variables.product.prodname_copilot_for_prs %}
|
||||
|
||||
You can use {% data variables.product.prodname_copilot %} to generate a summary of a pull request on {% data variables.product.prodname_dotcom_the_website %}. You can use the summary to help reviewers understand your changes, or to quickly understand the changes in a pull request you're reviewing.
|
||||
|
||||
{% data variables.product.prodname_copilot %} will scan through the pull request and provide an overview of the changes made in prose, as well as a bulleted list of changes with the files that they impact. You can generate a summary in the following places.
|
||||
|
||||
- In the description of a new pull request you're creating
|
||||
- In the description of an existing pull request, by editing the opening comment
|
||||
- In a comment on the main timeline of a pull request
|
||||
|
||||
To learn more about {% data variables.product.prodname_copilot_for_prs %} and how to use the feature most effectively, see "[AUTOTITLE](/copilot/github-copilot-enterprise/copilot-pull-request-summaries/about-copilot-pull-request-summaries)."
|
||||
|
||||
## Creating a summary for a pull request
|
||||
|
||||
1. On {% data variables.product.prodname_dotcom_the_website %}, create a pull request or navigate to an existing pull request.
|
||||
|
||||
{% note %}
|
||||
|
||||
**Note:** {% data variables.product.prodname_copilot %} does not take into account any existing content in the pull request description, so it is best to start with a blank description.
|
||||
|
||||
{% endnote %}
|
||||
|
||||
1. Navigate to the text field where you want to add the pull request summary.
|
||||
|
||||
- If you're creating a new pull request, use the "Add a description" field.
|
||||
- If you're adding a description to an existing pull request, edit the opening comment.
|
||||
- If you're adding a summary as a comment, navigate to the "Add a comment" section at the bottom of the pull request page.
|
||||
|
||||
1. In the header of the text field, select {% octicon "copilot" aria-label="Copilot actions" %}, then click **Summary**.
|
||||
|
||||
|
||||
|
||||
1. Wait for {% data variables.product.prodname_copilot %} to produce the summary, then check over the results carefully.
|
||||
1. Add any additional context that will help people viewing your pull request.
|
||||
1. When you're happy with the description, click **Create pull request** on a new pull request, or **Update comment** if you're editing an existing description.
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
title: Copilot pull request summaries
|
||||
shortTitle: Copilot pull request summaries
|
||||
intro: 'With {% data variables.product.prodname_copilot_for_prs %}, you can create an AI-generated description for a pull request.'
|
||||
topics:
|
||||
- Copilot
|
||||
versions:
|
||||
feature: copilot
|
||||
children:
|
||||
- /about-copilot-pull-request-summaries
|
||||
- /creating-a-pull-request-summary-with-github-copilot
|
||||
---
|
||||
Reference in New Issue
Block a user