jprdonnelly/docs
1
0
Fork 0
mirror of synced 2025-12-19 09:57:42 -05:00
Code Issues Projects Releases Wiki Activity

Use Copilot to update scripts to add quality query tables (#58621)

Browse Source 
Co-authored-by: Anne-Marie <102995847+am-stead@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: docs-bot <77750099+docs-bot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Óscar San José <oscarsj@github.com>
This commit is contained in:
Felicity Chapman
2025-12-11 16:39:46 +00:00
committed by GitHub
parent bb7e473a53
commit 18acec3172
27 changed files with 1098 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
src/code-scanning/README.md
View File

@@ -1,64 +0,0 @@
# Code scanning query tables
This nascent pipeline creates autogenerated documentation docs.github.com from the query suites included with the [CodeQL bundle](https://github.com/github/codeql-action/releases).
The pipeline is used to generate Markdown tables that are stored in reusable files and used in article pages on the docs.github.com site.
## How does it work
```mermaid
---
title: "Process for generating Code scanning query tables"
---
flowchart TB
accDescr: A flow chart describing how the automation generates documentation for code scanning queries.
start([Start]) --> checkout[Checkout the codeql repository]
start --- download[Download the CodeQL CLI]
checkout --- query-suites[fa:fa-file Query suites *.qls]
query-suites ---> generate[Generate tables:src/code-scanning/scripts/generate-code-scanning-query-lists.ts]
download --- codeql-cli[CodeQL CLI: codeql.exe]
codeql-cli --> generate
generate --- markdown[fa:fa-file Reusable files *.md]
markdown --> pr[Generate a PR overwriting:data/reusables/code-scanning/codeql-query-tables/*.md]
pr --> finish([End])
%% Define styles
classDef start fill:#1AAC9D, color:white
classDef action fill:#6557F6, color:white
classDef finish fill:#F8C324, color:white
classDef file fill:#ddd
%% Assign styles
class start start;
class finish finish;
class checkout,download,generate,pr action;
class markdown,query-suites,codeql-cli file;
```
A [workflow](.github/workflows/generate-code-scanning-query-lists.yml) is used to trigger the automation of the code scanning query tables documentation. The workflow is manually triggered by a member of the GitHub Docs team approximately every two weeks to align to releases of the CodeQL CLI. The workflow takes an input parameter that specifies the branch to pull the source files from in the semmle-code repo. If the branch input is omitted, the workflow will default to the `main` branch.
The workflow runs the `npm run generate-code-scanning-query-list` script, which generates Markdown files under `data/reusables/code-scanning/codeql-query-tables`.
The workflow automatically creates a new pull request with the changes and the label `codeql-query-tables`.
## Local development
To run the pipeline locally, see the comments in the [script](scripts/generate-code-scanning-query-list.ts).
## Content team
The content writers can use the reusables in any content article. They have no need to make any changes to the script unless additional built-in query suites are added.
## How to get help
### For workflow and script problems
Slack: `#docs-engineering`
Repo: `github/docs-engineering`
### For CodeQL repository and CLI problems
Slack: `#code-scanning-internal-dx`
Repo: `github/code-scanning-internal-dx-team`

5
src/code-scanning/scripts/tsconfig.json
View File

@@ -1,5 +0,0 @@
{
"extends": "../../../tsconfig.json",
"include": ["generate-code-scanning-query-list.ts"],
"exclude": ["node_modules"]
}

88
src/codeql-queries/README.md Normal file
View File

@@ -0,0 +1,88 @@
# CodeQL query tables
This pipeline creates autogenerated documentation for docs.github.com from the query suites included with the [CodeQL bundle](https://github.com/github/codeql-action/releases).
The pipeline generates Markdown tables that are stored in reusable files and used in article pages on the docs.github.com site.
There are two types of query table generators:
1. **Security queries** (`generate-code-scanning-query-list.ts`) - generates tables for security-related queries from the `code-scanning` and `security-extended` suites, including CWE information and autofix support.
2. **Code quality queries** (`generate-code-quality-query-list.ts`) - generates tables for code quality queries from the `code-quality` suite, showing query categories (reliability, maintainability).
## How does it work
```mermaid
---
title: "Process for generating CodeQL query tables"
---
flowchart TB
accDescr: A flow chart describing how the automation generates documentation for CodeQL queries.
start([Start]) --> checkout[Checkout the codeql repository]
start --> download[Download the CodeQL CLI]
checkout --> security-suites[fa:fa-file Security query suites *.qls]
checkout --> quality-suites[fa:fa-file Quality query suites *.qls]
security-suites --> generate-security[Generate security tables]
quality-suites --> generate-quality[Generate quality tables]
download --> codeql-cli[CodeQL CLI: codeql.exe]
codeql-cli --> generate-security
codeql-cli --> generate-quality
generate-security --> security-markdown[fa:fa-file Security reusables *.md]
generate-quality --> quality-markdown[fa:fa-file Quality reusables *.md]
security-markdown --> merge[Merge artifacts]
quality-markdown --> merge
merge --> pr[Generate a PR with both:<br/>data/reusables/code-scanning/codeql-query-tables/*.md<br/>data/reusables/code-quality/codeql-query-tables/*.md]
pr --> finish([End])
%% Define styles
classDef start fill:#1AAC9D, color:white
classDef action fill:#6557F6, color:white
classDef finish fill:#F8C324, color:white
classDef file fill:#ddd
%% Assign styles
class start start;
class finish finish;
class checkout,download,generate-security,generate-quality,merge,pr action;
class security-markdown,quality-markdown,security-suites,quality-suites,codeql-cli file;
```
A [workflow](../../.github/workflows/generate-codeql-query-lists.yml) is used to trigger the automation of the CodeQL query tables documentation. The workflow is manually triggered by a member of the GitHub Docs team approximately every two weeks to align to releases of the CodeQL CLI. The workflow takes an input parameter that specifies the branch to pull the source files from in the codeql repo. If the branch input is omitted, the workflow will default to the `main` branch.
The workflow runs two scripts in parallel:
- `npm run generate-code-scanning-query-list` - generates security query tables under `data/reusables/code-scanning/codeql-query-tables/`
- `npm run generate-code-quality-query-list` - generates code quality query tables under `data/reusables/code-quality/codeql-query-tables/`
The workflow automatically creates a new pull request with changes from both scripts and the label `codeql-query-tables`.
## Local development
To run the pipeline locally, see the comments in the scripts:
- Security queries: [generate-code-scanning-query-list.ts](scripts/generate-code-scanning-query-list.ts)
- Code quality queries: [generate-code-quality-query-list.ts](scripts/generate-code-quality-query-list.ts)
## Content team
The content writers can use the reusables in any content article. They have no need to make any changes to the scripts unless additional built-in query suites are added.
For security queries, reusables are stored in `data/reusables/code-scanning/codeql-query-tables/`.
For code quality queries, reusables are stored in `data/reusables/code-quality/codeql-query-tables/`.
## How to get help
### For workflow and script problems
Slack: `#docs-engineering`
Repo: `github/docs-engineering`
### For CodeQL repository and CLI problems
Slack: `#code-scanning-engine-quality`
Repo: `github/code-scanning-engine-quality-team`

291
src/codeql-queries/scripts/generate-code-quality-query-list.ts Normal file
View File

@@ -0,0 +1,291 @@
/**
* This script generates a block of Markdown that can be saved as a reusable.
* The reusable lists all the code quality queries for one programming language, with categories, as a Markdown table.
*
* To be able to execute this script, you need to have the CodeQL CLI installed.
* To do that, you need two things:
*
* 1. The directory where the github/codeql repo is cloned
* 2. The path to the executable `codeql` file.
*
* The directory where the github/codeql repo is cloned is needed because
* that's how it looks up files. You can set it up like this:
*
* cd /tmp
* git clone git@github.com:github/codeql.git
* cd codeql
* pwd
*
* To install the codeql executable, use `gh` like this:
*
* gh extension install github/gh-codeql
* gh codeql set-channel nightly
* gh codeql version
*
* Note that when you run the `gh codeql version` command, it will tell you
* where the executable is installed. For example:
*
* /Users/peterbe/.local/share/gh/extensions/gh-codeql/dist/nightly/codeql-bundle-20231204/codeql
*
* If you've git cloned github/codeql in /tmp/ now you can execute this script.
* For example, to generate the Markdown
* for Python:
*
* npm run generate-code-quality-query-list -- \
* --codeql-path ~/.local/share/gh/extensions/gh-codeql/dist/nightly/codeql-bundle-20231204/codeql \
* --codeql-dir /tmp/codeql python | tee /tmp/python.md
* less /tmp/python.md
*/
import fs from 'fs'
import { execFileSync } from 'child_process'
import chalk from 'chalk'
import { program } from 'commander'
program
.description('Generate a reusable Markdown for code quality queries by language')
.option('--verbose', 'Verbose outputs')
.option('--codeql-path <path>', 'path to the codeql executable', 'codeql')
.option('--codeql-dir <path>', 'path to the codeql executable', '.codeql/')
.option('-o, --output-file <path>', 'output file path (default: stdout)', 'stdout')
.argument('<language>', 'for example java')
.parse(process.argv)
type Options = {
codeqlPath: string
codeqlDir: string
outputFile: string
verbose: boolean
}
type QueryMetadata = {
id?: string
name?: string
tags?: string
severity?: string
problem?: {
severity?: string
}
}
type Query = {
name: string
url: string
categories: string[]
severity: string
}
type QueryExtended = Query & {
primaryCategory: string
}
const opts = program.opts()
main(
{
codeqlPath: opts.codeqlPath,
codeqlDir: opts.codeqlDir,
outputFile: opts.outputFile,
verbose: Boolean(opts.verbose),
},
program.args[0],
)
async function main(options: Options, language: string) {
if (options.verbose && options.outputFile === 'stdout') {
console.warn(chalk.yellow('Verbose mode is on but output is going to stdout'))
}
if (!testCodeQLPath(options)) {
process.exit(1)
}
const queries: {
[id: string]: Query
} = {}
const languagePack = `${language}-code-quality.qls`
if (options.verbose) console.log(chalk.dim(`Searching for queries in ${languagePack}`))
const res = execFileSync(
options.codeqlPath,
['resolve', 'queries', `--search-path=${options.codeqlDir}`, languagePack],
{
encoding: 'utf-8',
},
)
for (const line of res.split('\n')) {
if (line.trim()) {
if (options.verbose) console.log('found', line)
const metadata = getMetadata(options, line)
const { id, name, tags, severity } = metadata
if (id && name) {
const categories = getCategories(tags || '')
const url = getDocsLink(language, id)
// Only include queries that have categories
if (categories.length) {
queries[id] = { url, name, categories, severity: severity || 'N/A' }
} else {
if (options.verbose) {
console.log(chalk.dim(`Skipping ${id} because it has no categories`))
}
}
}
}
}
function decorate(query: Query): QueryExtended {
// Determine primary category for sorting
// Prefer 'maintainability' over 'reliability'
const primaryCategory = query.categories.includes('maintainability')
? 'maintainability'
: query.categories.includes('reliability')
? 'reliability'
: query.categories[0] || ''
return {
...query,
primaryCategory,
}
}
const entries = Object.values(queries).map(decorate)
// Sort by primary category (maintainability first), then alphabetically by name
entries.sort((a, b) => {
if (a.primaryCategory === 'maintainability' && b.primaryCategory !== 'maintainability')
return -1
else if (a.primaryCategory !== 'maintainability' && b.primaryCategory === 'maintainability')
return 1
if (a.primaryCategory === 'reliability' && b.primaryCategory !== 'reliability') return -1
else if (a.primaryCategory !== 'reliability' && b.primaryCategory === 'reliability') return 1
return a.name.localeCompare(b.name)
})
printQueries(options, entries)
}
function printQueries(options: Options, queries: QueryExtended[]) {
const markdown: string[] = []
markdown.push('{% rowheaders %}')
markdown.push('') // blank line
const header = ['Query name', 'Category', 'Severity']
markdown.push(`| ${header.join(' | ')} |`)
markdown.push(`| ${header.map(() => '---').join(' | ')} |`)
for (const query of queries) {
const markdownLink = `[${query.name}](${query.url})`
// Capitalize first letter of category for display
const categoryDisplay = query.categories
.map((cat) => cat.charAt(0).toUpperCase() + cat.slice(1))
.join(', ')
// Capitalize first letter of severity for display
const severityDisplay = query.severity.charAt(0).toUpperCase() + query.severity.slice(1)
const row = [markdownLink, categoryDisplay, severityDisplay]
markdown.push(`| ${row.join(' | ')} |`)
}
markdown.push('') // blank line
markdown.push('{% endrowheaders %}')
markdown.push('') // always end with a blank line
if (options.outputFile === 'stdout') {
console.log(markdown.join('\n'))
} else {
fs.writeFileSync(options.outputFile, markdown.join('\n'), 'utf-8')
}
}
function getMetadata(options: Options, queryFile: string): QueryMetadata {
const metadataJson = execFileSync(options.codeqlPath, ['resolve', 'metadata', queryFile], {
encoding: 'utf-8',
})
const parsed = JSON.parse(metadataJson)
// Extract severity from various possible locations in the metadata
// CodeQL metadata can have @problem.severity in the query file, which may be
// represented in different ways in the JSON output from `codeql resolve metadata`
const severity =
parsed.problem?.severity || // Nested: { problem: { severity: "error" } }
parsed['@problem']?.severity || // Nested with @: { "@problem": { severity: "error" } }
parsed['@problem.severity'] || // Direct key: { "@problem.severity": "error" }
parsed['problem.severity'] || // Direct key without @: { "problem.severity": "error" }
parsed.severity || // Simple: { severity: "error" }
parsed['@severity'] // With @: { "@severity": "error" }
if (options.verbose) {
// On first query only, show all available keys to help debug
if (!getMetadata.shownKeys) {
console.log(chalk.yellow('Available metadata keys:'), Object.keys(parsed))
if (parsed.problem) {
console.log(chalk.yellow('Available problem keys:'), Object.keys(parsed.problem))
}
if (parsed['@problem']) {
console.log(chalk.yellow('Available @problem keys:'), Object.keys(parsed['@problem']))
}
getMetadata.shownKeys = true
}
if (severity) {
console.log(chalk.dim(`Query ${parsed.id} has severity: ${severity}`))
} else {
console.log(chalk.red(`Query ${parsed.id} has NO severity found`))
console.log(chalk.red('Available keys for this query:'), Object.keys(parsed))
}
}
return {
...parsed,
severity,
}
}
// Add a property to track if we've shown keys
getMetadata.shownKeys = false
/**
*
* @param language 'cpp'
* @param queryId 'external-entity-expansion'
* @returns https://codeql.github.com/codeql-query-help/cpp/cpp-external-entity-expansion/
*/
function getDocsLink(language: string, queryId: string) {
return `https://codeql.github.com/codeql-query-help/${language}/${queryId.replaceAll('/', '-')}/`
}
/**
*
* @param tags 'maintainability readability reliability external/cwe/cwe-1078 external/cwe/cwe-670 security'
* @returns ['maintainability', 'reliability']
*/
function getCategories(tags: string) {
const categories: string[] = []
for (const tag of tags.split(/\s+/g)) {
if (tag === 'maintainability' || tag === 'reliability') {
categories.push(tag)
}
}
return categories
}
function testCodeQLPath(options: Options) {
try {
const output = execFileSync(options.codeqlPath, ['--version'], { encoding: 'utf-8' })
if (options.verbose) {
const matched = output.match(/CodeQL command-line toolchain release ([\d.+]+)/)
if (matched) {
console.log('codeql version', chalk.green(matched[0]))
return true
}
}
return true
} catch (error) {
console.error('Could not find codeql executable at', options.codeqlPath)
if (options.verbose) {
throw error
} else {
console.log(chalk.yellow(`${options.codeqlPath} --version`), 'failed')
return false
}
}
}

3
src/code-scanning/scripts/generate-code-scanning-query-list.ts → src/codeql-queries/scripts/generate-code-scanning-query-list.ts
View File

@@ -53,7 +53,8 @@ import chalk from 'chalk'
import { program } from 'commander'
// We don't want to introduce a global dependency on @github/cocofix, so we install it by hand
// as described above and suppress the import warning.
import { getSupportedQueries } from '@github/cocofix/dist/querySuites' /* eslint-disable-line import/no-extraneous-dependencies, import/no-unresolved */
// eslint-disable-next-line import/no-unresolved -- @github/cocofix is installed manually
import { getSupportedQueries } from '@github/cocofix/dist/querySuites'
import type { Language } from 'codeql-ts'
program

5
src/codeql-queries/scripts/tsconfig.json Normal file
View File

@@ -0,0 +1,5 @@
{
"extends": "../../../tsconfig.json",
"include": ["generate-code-scanning-query-list.ts", "generate-code-quality-query-list.ts"],
"exclude": ["node_modules"]
}