--- title: Migrating from the CodeQL runner to CodeQL CLI shortTitle: Migrating from the CodeQL runner intro: 'You can use the {% data variables.product.prodname_codeql_cli %} to complete the same tasks as with the {% data variables.product.prodname_codeql_runner %}.' product: '{% data reusables.gated-features.code-scanning %}' versions: fpt: '*' ghes: '>=3.1' ghae: 'next' ghec: '*' topics: - Advanced Security - Code scanning - CodeQL --- # Migrating from the {% data variables.product.prodname_codeql_runner %} to the {% data variables.product.prodname_codeql_cli %} The {% data variables.product.prodname_codeql_runner %} is being deprecated. You can use the {% data variables.product.prodname_codeql_cli %} version 2.6.2 and greater instead. This document describes how to migrate common workflows from the {% data variables.product.prodname_codeql_runner %} to the {% data variables.product.prodname_codeql_cli %}. ## Installation Download the **{% data variables.product.prodname_codeql %} bundle** from the [`github/codeql-action` repository](https://github.com/github/codeql-action/releases). This bundle contains the {% data variables.product.prodname_codeql_cli %} and the standard {% data variables.product.prodname_codeql %} queries and libraries. For more information on setting up the {% data variables.product.prodname_codeql_cli %}, see "[Installing {% data variables.product.prodname_codeql_cli %} in your CI system](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system)." ## Overview of workflow changes A typical workflow that uses the {% data variables.product.prodname_codeql_runner %} to analyze a codebase has the following steps. - `codeql-runner- init` to start creating {% data variables.product.prodname_codeql %} databases and read the configuration. - For compiled languages: set environment variables produced by the `init` step. - For compiled languages: run autobuild or manual build steps. - `codeql-runner- analyze` to finish creating {% data variables.product.prodname_codeql %} databases, run queries to analyze each {% data variables.product.prodname_codeql %} database, summarize the results in a SARIF file, and upload the results to {% data variables.product.prodname_dotcom %}. A typical workflow that uses the {% data variables.product.prodname_codeql_cli %} to analyze a codebase has the following steps. - `codeql database create` to create {% data variables.product.prodname_codeql %} databases. - For compiled languages: Optionally provide a build command. - `codeql database analyze` to run queries to analyze each {% data variables.product.prodname_codeql %} database and summarize the results in a SARIF file. This command must be run once for each language or database. - `codeql github upload-results` to upload the resulting SARIF files to {% data variables.product.prodname_dotcom %}, to be displayed as code scanning alerts. This command must be run once for each language or SARIF file. For more information, see "[Configuring {% data variables.product.prodname_codeql_cli %} in your CI system](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system)." ## Examples of common uses for the {% data variables.product.prodname_codeql_cli %} ### About the examples These examples assume that the source code has been checked out to the current working directory. If you use a different directory, change the `--source-root` argument and the build steps accordingly. These examples also assume that the {% data variables.product.prodname_codeql_cli %} is placed on the current PATH. In these examples, a {% data variables.product.prodname_dotcom %} token with suitable scopes is stored in the `$TOKEN` environment variable and passed to the example commands via stdin, or is stored in the `$GITHUB_TOKEN` environment variable. The ref name and commit SHA being checked out and analyzed in these examples are known during the workflow. For a branch, use `refs/heads/BRANCH-NAME` as the ref. For the head commit of a pull request, use `refs/pulls/NUMBER/head`. For a {% data variables.product.prodname_dotcom %}-generated merge commit of a pull request, use `refs/pulls/NUMBER/merge`. The examples below all use `refs/heads/main`. If you use a different branch name, you must modify the sample code. ### Single non-compiled language (JavaScript) Runner: ```bash echo "$TOKEN" | codeql-runner-linux init --repository my-org/example-repo \ --languages javascript \ --github-url https://github.com --github-auth-stdin echo "$TOKEN" | codeql-runner-linux analyze --repository my-org/example-repo --github-url https://github.com --github-auth-stdin --commit deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 --ref refs/heads/main ``` CLI: ```bash codeql database create /codeql-dbs/example-repo --language=javascript \ --source-root=. # The default query suite is called `-code-scanning.qls`. codeql database analyze /codeql-dbs/example-repo \ javascript-code-scanning.qls --sarif-category=javascript \ --format=sarif-latest --output=/temp/example-repo-js.sarif echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif=/temp/example-repo-js.sarif --github-auth-stdin ``` ### Single non-compiled language (JavaScript) using a different query suite (security-and-quality) A similar approach can be taken for compiled languages, or multiple languages. Runner: ```bash echo "$TOKEN" | codeql-runner-linux init --repository my-org/example-repo \ --languages javascript \ --github-url https://github.com --github-auth-stdin echo "$TOKEN" | codeql-runner-linux analyze --repository my-org/example-repo \ --queries security-and-quality \ --github-url https://github.com --github-auth-stdin \ --commit deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 --ref refs/heads/main ``` CLI: ```bash codeql database create /codeql-dbs/example-repo --language=javascript \ --source-root=. # Use `-.qls` codeql database analyze /codeql-dbs/example-repo \ javascript-security-and-quality.qls --sarif-category=javascript --format=sarif-latest --output=/temp/example-repo-js.sarif echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif=/temp/example-repo-js.sarif --github-auth-stdin ``` ### Single non-compiled language (JavaScript) using a custom configuration file A similar approach can be taken for compiled languages, or multiple languages. Runner: ```bash echo "$TOKEN" | codeql-runner-linux init --repository my-org/example-repo \ --languages javascript \ --config-file .github/codeql/codeql-config.yml \ --github-url https://github.com --github-auth-stdin echo "$TOKEN" | codeql-runner-linux analyze --repository my-org/example-repo \ --github-url https://github.com --github-auth-stdin \ --commit deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 --ref refs/heads/main ``` CLI: ```bash # Use `--codescanning-config` with the path to the YAML configuration file. codeql database create /codeql-dbs/example-repo --language=javascript \ --codescanning-config=.github/codeql/codeql-config.yml \ --source-root=. codeql database analyze /codeql-dbs/example-repo \ --sarif-category=javascript --format=sarif-latest --output=/temp/example-repo-js.sarif echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif=/temp/example-repo-js.sarif --github-auth-stdin ``` ### Single compiled language using autobuild (Java) Runner: ```bash echo "$TOKEN" | codeql-runner-linux init --repository my-org/example-repo \ --languages java \ --github-url https://github.com --github-auth-stdin # Source the script generated by the init step to set up the environment to monitor the build. . codeql-runner/codeql-env.sh # Run the autobuilder for the given language. codeql-runner-linux autobuild --language java echo "$TOKEN" | codeql-runner-linux analyze --repository my-org/example-repo --github-url https://github.com --github-auth-stdin --commit deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 --ref refs/heads/main ``` CLI: ```bash # Run `codeql database create` without `--command`. # This will run the autobuilder for the given language. codeql database create /codeql-dbs/example-repo --language=java \ --source-root=. codeql database analyze /codeql-dbs/example-repo \ javascript-code-scanning.qls --sarif-category=java --format=sarif-latest --output=/temp/example-repo-java.sarif echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif=/temp/example-repo-java.sarif --github-auth-stdin ``` ### Single compiled language using a custom build command (Java) Runner: ```bash echo "$TOKEN" | codeql-runner-linux init --repository my-org/example-repo \ --languages java \ --github-url https://github.com --github-auth-stdin # Source the script generated by the init step to set up the environment to monitor the build. . codeql-runner/codeql-env.sh # Run a custom build command. mvn compile -DskipTests echo "$TOKEN" | codeql-runner-linux analyze --repository my-org/example-repo --github-url https://github.com --github-auth-stdin --commit deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 --ref refs/heads/main ``` CLI: ```bash # Provide an explicit build command using `--command`. codeql database create /codeql-dbs/example-repo --language=java \ --command="mvn compile -DskipTests" --source-root=. codeql database analyze /codeql-dbs/example-repo \ java-code-scanning.qls --sarif-category=java --format=sarif-latest --output=/temp/example-repo-java.sarif echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif=/temp/example-repo-java.sarif --github-auth-stdin ``` ### Single compiled language using indirect build tracing (C# on Windows within Azure DevOps) Indirect build tracing for a compiled language enables {% data variables.product.prodname_codeql %} to detect all build steps between the `init` and `analyze` steps, when the code cannot be built using the autobuilder or an explicit build command line. This is useful when using preconfigured build steps from your CI system, such as the `VSBuild` and `MSBuild` tasks in Azure DevOps. Runner: ```yaml - task: CmdLine@1 displayName: CodeQL Initialization inputs: script: "%CodeQLRunner%\\codeql-runner-win.exe init --repository my-org/example-repo --languages csharp --github-url https://github.com --github-auth $(Token)" # Set the generated environment variables so they are available for subsequent commands, in the format required by Azure Pipelines. - task: PowerShell@1 displayName: Set CodeQL Environment Variables inputs: targetType: inline script: > $json = Get-Content $(System.DefaultWorkingDirectory)/codeql-runner/codeql-env.json | ConvertFrom-Json $json.PSObject.Properties | ForEach-Object { $template = "##vso[task.setvariable variable=" $template += $_.Name $template += "]" $template += $_.Value echo "$template" } # Execute a clean build using the VSBuild task. - task: VSBuild@1 inputs: solution: '**/*.sln' msbuildArgs: '/p:OutDir=$(Build.ArtifactStagingDirectory) /p:UseSharedCompilation=false' platform: Any CPU configuration: Release clean: True displayName: Visual Studio Build # Analyze the database created as part of the build, by running the selected queries against it, and upload results to GitHub. - task: CmdLine@2 displayName: CodeQL Analyze inputs: script: '%CodeQLRunner%\codeql-runner-win.exe analyze --repository my-org/example-repo --commit $(Build.SourceVersion) --ref $(Build.SourceBranch) --github-url https://github.com --github-auth $(Token)' ``` CLI: ```yaml # Run any pre-build tasks, for example, restore NuGet dependencies... # Initialize the CodeQL database using `codeql database init --begin tracing`. - task: CmdLine@1 displayName: Initialize CodeQL database inputs: # Assumes the source code is checked out to the current working directory. # Creates a database at `/codeql-dbs/example-repo`. # Running on Windows, so specifies a trace process level. script: "codeql database init --language csharp --trace-process-name Agent.Worker.exe --source-root . --begin-tracing /codeql-dbs/example-repo" # For CodeQL to trace future build steps without knowing the explicit build commands, # it requires certain environment variables to be set during the build. # Read these generated environment variables and values, and set them so they are available for subsequent commands # in the build pipeline. This is done in PowerShell in this example. - task: PowerShell@1 displayName: Set CodeQL environment variables inputs: targetType: inline script: > $json = Get-Content /codeql-dbs/example-repo/temp/tracingEnvironment/start-tracing.json | ConvertFrom-Json $json.PSObject.Properties | ForEach-Object { $template = "##vso[task.setvariable variable=" $template += $_.Name $template += "]" $template += $_.Value echo "$template" } # Execute the pre-defined build step. Note the `msbuildArgs` variable. - task: VSBuild@1 inputs: solution: '**/*.sln' # Disable MSBuild shared compilation for C# builds. msbuildArgs: /p:OutDir=$(Build.ArtifactStagingDirectory) /p:UseSharedCompilation=false platform: Any CPU configuration: Release # Execute a clean build, in order to remove any existing build artifacts prior to the build. clean: True displayName: Visual Studio Build # Read and set the generated environment variables to end build tracing. This is done in PowerShell in this example. - task: PowerShell@1 displayName: Clear CodeQL environment variables inputs: targetType: inline script: > $json = Get-Content $(System.DefaultWorkingDirectory)/db/temp/tracingEnvironment/end-tracing.json | ConvertFrom-Json $json.PSObject.Properties | ForEach-Object { $template = "##vso[task.setvariable variable=" $template += $_.Name $template += "]" $template += $_.Value echo "$template" } # Use `codeql database finalize` to complete database creation after the build is done. - task: CmdLine@2 displayName: Finalize CodeQL database inputs: script: 'codeql database finalize /codeql-dbs/example-repo' # Analyze the database and upload the results. - task: CmdLine@2 displayName: Analyze CodeQL database inputs: script: 'codeql database analyze /codeql-dbs/example-repo csharp-code-scanning.qls --sarif-category=csharp --format=sarif-latest --output=/temp/example-repo-csharp.sarif' - task: CmdLine@2 displayName: Upload CodeQL results inputs: script: 'echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif=/temp/example-repo-csharp.sarif --github-auth-stdin' ``` ### Multiple languages using autobuild (C++, Python) This example is not strictly possible with the {% data variables.product.prodname_codeql_runner %}. Only one language (the compiled language with the most files) will be analyzed. Runner: ```bash echo "$TOKEN" | codeql-runner-linux init --repository my-org/example-repo \ --languages cpp,python \ --github-url https://github.com --github-auth-stdin # Source the script generated by the init step to set up the environment to monitor the build. . codeql-runner/codeql-env.sh # Run the autobuilder for the language with the most files. codeql-runner-linux autobuild echo "$TOKEN" | codeql-runner-linux analyze --repository my-org/example-repo --github-url https://github.com --github-auth-stdin --commit deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 --ref refs/heads/main ``` CLI: ```bash # Create multiple databases using `--db-cluster`. # Run autobuild by omitting `--command`. codeql database create /codeql-dbs/example-repo-multi \ --db-cluster --language cpp,python \ --no-run-unnecessary-builds \ --source-root . # Analyze each database in turn and upload the results. for language in cpp python; do codeql database analyze "/codeql-dbs/example-repo-multi/$language" \ "$language-code-scanning.qls" --sarif-category="$language" --format=sarif-latest --output="/temp/example-repo-$language.sarif" echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif="/temp/example-repo-$language.sarif" --github-auth-stdin done ``` ### Multiple languages using a custom build command (C++, Python) Runner: ```bash echo "$TOKEN" | codeql-runner-linux init --repository my-org/example-repo \ --languages cpp,python \ --github-url https://github.com --github-auth-stdin # Source the script generated by the init step to set up the environment to monitor the build. . codeql-runner/codeql-env.sh # Run a custom build command. make echo "$TOKEN" | codeql-runner-linux analyze --repository my-org/example-repo --github-url https://github.com --github-auth-stdin --commit deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 --ref refs/heads/main ``` CLI: ```bash # Create multiple databases using `--db-cluster`. codeql database create /codeql-dbs/example-repo-multi \ --db-cluster --language cpp,python \ --command make --no-run-unnecessary-builds \ --source-root . # Analyze each database in turn and upload the results. for language in cpp python; do codeql database analyze "/codeql-dbs/example-repo-multi/$language" \ "$language-code-scanning.qls" --sarif-category="$language" --format=sarif-latest --output="/temp/example-repo-$language.sarif" echo "$TOKEN" | codeql github upload-results --repository=my-org/example-repo \ --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ --sarif="/temp/example-repo-$language.sarif" --github-auth-stdin done ```