2024-03-26: Updates for the release of CodeQL analysis of Java without building it (ship delayed) (#49452)
Co-authored-by: Henry Mercer <henrymercer@github.com> Co-authored-by: Ben Ahmady <32935794+subatoi@users.noreply.github.com> Co-authored-by: Florin Coada <coadaflorin@github.com>
This commit is contained in:
Felicity Chapman
@@ -1,9 +1,9 @@
|
||||
---
|
||||
title: CodeQL code scanning for compiled languages
|
||||
shortTitle: CodeQL for compiled languages
|
||||
intro: 'Understand the autobuild method {% data variables.product.prodname_codeql %} analysis uses to build code for compiled languages and learn how you can customize the build command if you need to.'
|
||||
intro: 'Understand how {% data variables.product.prodname_codeql %} analyzes compiled languages, the build options available, and learn how you can customize the database generation process if you need to.'
|
||||
product: '{% data reusables.gated-features.code-scanning %}'
|
||||
permissions: 'If you have write permissions to a repository, you can configure {% data variables.product.prodname_code_scanning %} for that repository.'
|
||||
permissions: 'People with write permissions to a repository can configure {% data variables.product.prodname_code_scanning %} for that repository by editing a workflow{% ifversion code-scanning-without-workflow %}, when advanced setup is enabled (admin permission is required to change setup){% endif %}.'
|
||||
redirect_from:
|
||||
- /github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning-for-compiled-languages
|
||||
- /github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-the-codeql-action-for-compiled-languages
|
||||
@@ -34,68 +34,276 @@ topics:
|
||||
|
||||
## About the {% data variables.code-scanning.codeql_workflow %} and compiled languages
|
||||
|
||||
{% data variables.product.prodname_code_scanning_caps %} works by running queries against one or more databases. Each database contains a representation of all of the code in a single language in your repository. For the compiled languages {% data variables.code-scanning.compiled_languages %}, the process of populating this database involves building the code and extracting data.
|
||||
{% data variables.product.prodname_code_scanning_caps %} works by running queries against one or more {% data variables.product.prodname_codeql %} databases. Each database contains a representation of the code in a single language in your repository. For the compiled languages {% data variables.code-scanning.compiled_languages %}, the process of populating this database {% ifversion codeql-no-build %}often{% endif %} involves building the code and extracting data.
|
||||
|
||||
{% data reusables.code-scanning.autobuild-compiled-languages %}
|
||||
{% ifversion codeql-no-build %}
|
||||
|
||||
{% ifversion code-scanning-without-workflow-310 %}
|
||||
When you enable {% data variables.product.prodname_code_scanning %}, both default and advanced setup generate a {% data variables.product.prodname_codeql %} database for analysis using the simplest method available. For {% data variables.code-scanning.no_build_support %}, the {% data variables.product.prodname_codeql %} database is generated directly from the codebase without requiring a build (`none` build mode). For other compiled languages, {% data variables.product.prodname_codeql %} builds the codebase using the `autobuild` build mode. Alternatively, you can use the `manual` build mode to specify explicit build commands to analyze only the files that are built by these custom commands.
|
||||
|
||||
For {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %}, you can use default setup, which analyzes your code and automatically configures your {% data variables.product.prodname_code_scanning %}, or advanced setup, which generates a workflow file you can edit. {% ifversion codeql-swift-advanced-setup %}Default setup can analyze all compiled languages supported by {% data variables.product.prodname_codeql %}.{% endif %} For more information about advanced setup, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning#configuring-advanced-setup-for-code-scanning-with-codeql)."
|
||||
{% elsif ghes > 3.9 %}
|
||||
|
||||
If you enable default setup, the `autobuild` action will be used to build your code, as part of your automatically configured {% data variables.code-scanning.codeql_workflow %}. If you enable advanced setup, the basic {% data variables.code-scanning.codeql_workflow %} uses `autobuild`. Alternatively, you can disable `autobuild` and instead specify explicit build commands to analyze only the files that are built by these custom commands.
|
||||
|
||||
{% else %}
|
||||
|
||||
The basic {% data variables.code-scanning.codeql_workflow %} uses the `autobuild` action to build your code. Alternatively, you can disable `autobuild` and instead specify explicit build commands to analyze only the files that are built by these custom commands.
|
||||
|
||||
{% ifversion code-scanning-default-setup-self-hosted-310 or default-setup-self-hosted-runners-GHEC %}
|
||||
You can use default setup with self-hosted runners for all {% data variables.product.prodname_codeql %}-supported languages{% ifversion codeql-swift-advanced-setup %} except Swift{% endif %}. Default setup will always run the `autobuild` action, so you should configure your self-hosted runners to make sure they can run all necessary commands for C/C++, C#, and Java analysis. Analysis of JavaScript/TypeScript, Go, Ruby, Python, and Kotlin code does not currently require special configuration.
|
||||
{% endif %}
|
||||
|
||||
{% elsif code-scanning-without-workflow %}
|
||||
{% ifversion ghes < 3.10 and ghes > 3.8 %}
|
||||
|
||||
In {% data variables.product.product_name %} {{ allVersions[currentVersion].currentRelease }}, default setup does not support any compiled languages, so you must use advanced setup. Advanced setup generates a workflow file you can edit. The starter workflow files use `autobuild` to analyze compiled languages. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning#configuring-advanced-setup-for-code-scanning-with-codeql)."
|
||||
|
||||
{% else %}
|
||||
{% elsif ghes < 3.9 %}
|
||||
|
||||
You set up {% data variables.product.prodname_dotcom %} to run {% data variables.product.prodname_code_scanning %} for your repository by adding a {% data variables.product.prodname_actions %} workflow to the repository. For {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %}, you add the {% data variables.code-scanning.codeql_workflow %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning#configuring-code-scanning-using-the-codeql-action)."
|
||||
|
||||
{% endif %}
|
||||
|
||||
{% ifversion codeql-no-build %}
|
||||
|
||||
## {% data variables.product.prodname_codeql %} build modes
|
||||
|
||||
{% data reusables.code-scanning.beta-no-build %}
|
||||
|
||||
The {% data variables.product.prodname_codeql %} action supports three different build modes for compiled languages:
|
||||
|
||||
- `none` - the {% data variables.product.prodname_codeql %} database is created directly from the codebase without building the codebase (supported for all interpreted languages, and additionally supported in beta for {% data variables.code-scanning.no_build_support %}).
|
||||
- `autobuild` - {% data variables.product.prodname_codeql %} detects the most likely build method and uses this to attempt to build the codebase and create a database for analysis (supported for all compiled languages).
|
||||
- `manual` - you define the build steps to use for the codebase in the workflow (supported for all compiled languages).
|
||||
|
||||
### Comparison of the build modes
|
||||
|
||||
{% rowheaders %}
|
||||
|
||||
| Build mode characteristic | None | Autobuild | Manual |
|
||||
|---------------------------|-------------|-----------|--------|
|
||||
| Used by default setup and for organization-level enablement | Yes ({% data variables.code-scanning.no_build_support %}) | Yes, where `none` is not supported | No |
|
||||
| Analysis succeeds without user configuration | Yes | Variable | No |
|
||||
| Completeness of analysis | Generated code not analyzed | Variable | User controlled |
|
||||
| Accuracy of analysis | Good | Good | Best |
|
||||
|
||||
{% endrowheaders %}
|
||||
|
||||
### Recommendations
|
||||
|
||||
When you are setting up {% data variables.product.prodname_code_scanning %} for the first time, or across multiple repositories, it's best to use default setup. Default setup uses the simplest method available to generate a {% data variables.product.prodname_codeql %} database and analyze your code, so that you can start fixing alerts as soon as possible. Once you have resolved the initial alerts, you may want to switch to advanced setup with a manual build process for high risk repositories.
|
||||
|
||||
### Using multiple build modes in a multi-language repository
|
||||
|
||||
For repositories with multiple compiled languages, you can use different build modes for different languages. For example, if your repository contains C/C++, C# and Java, you might want to provide manual build steps for one language (here C/C++). This workflow specifies a different build mode for each language.
|
||||
|
||||
```yaml
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
# Analyzes C and C++ code using the commands in `Build C and C++ code`
|
||||
- language: c-cpp
|
||||
build-mode: manual
|
||||
# Analyzes C# code by automatically detecting a build
|
||||
- language: csharp
|
||||
build-mode: autobuild
|
||||
# Analyzes Java code directly from the codebase without a build
|
||||
- language: java-kotlin
|
||||
build-mode: none # analyzes Java only
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: {% data reusables.actions.action-checkout %}
|
||||
|
||||
# Initializes CodeQL tools and creates a codebase for analysis.
|
||||
- name: Initialize CodeQL
|
||||
uses: {% data reusables.actions.action-codeql-action-init %}
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
- if: ${{ matrix.build-mode == 'manual' }}
|
||||
name: Build C and C++ code
|
||||
run: |
|
||||
echo 'If you are using a "manual" build mode for one or more of the' \
|
||||
'languages you are analyzing, replace this with the commands to build' \
|
||||
'your code, for example:'
|
||||
echo ' make bootstrap'
|
||||
echo ' make release'
|
||||
exit 1
|
||||
```
|
||||
|
||||
{% endif %}
|
||||
|
||||
For information about the languages, libraries, and frameworks that are supported in the latest version of {% data variables.product.prodname_codeql %}, see "[Supported languages and frameworks](https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks)" in the {% data variables.product.prodname_codeql %} documentation. For information about the system requirements for running the latest version of {% data variables.product.prodname_codeql %}, see "[System requirements](https://codeql.github.com/docs/codeql-overview/system-requirements/#additional-software-requirements)" in the {% data variables.product.prodname_codeql %} documentation.
|
||||
|
||||
If your workflow uses a `language` matrix, `autobuild` attempts to build each of the compiled languages listed in the matrix. Without a matrix `autobuild` attempts to build the supported compiled language that has the most source files in the repository. With the exception of Go, analysis of other compiled languages in your repository will fail unless you supply explicit build commands.
|
||||
## About Autobuild for {% data variables.product.prodname_codeql %}
|
||||
|
||||
## About `autobuild` for {% data variables.product.prodname_codeql %}
|
||||
{% ifversion codeql-no-build or code-scanning-without-workflow-310 %}
|
||||
|
||||
{% data reusables.code-scanning.autobuild-compiled-languages %}
|
||||
The {% data variables.product.prodname_codeql %} action uses `autobuild` to analyze compiled languages in the following cases.
|
||||
|
||||
- [`autobuild` for C/C++](#autobuild-for-cc)
|
||||
- [`autobuild` for C#](#autobuild-for-c){% ifversion codeql-go-autobuild %}
|
||||
- [`autobuild` for Go](#autobuild-for-go){% endif %}{% ifversion codeql-kotlin-beta %}
|
||||
- [`autobuild` for Java and Kotlin](#autobuild-for-java--and-kotlin){% else %}
|
||||
- [`autobuild` for Java](#autobuild-for-java){% endif %}{% ifversion codeql-swift-beta %}
|
||||
- [`autobuild` for Swift](#autobuild-for-swift){% endif %}
|
||||
- Default setup is enabled{% ifversion codeql-no-build %} and the language does not support `none` build (beta release supported for {% data variables.code-scanning.no_build_support %}).
|
||||
- Advanced setup is enabled and the workflow specifies `build-mode: autobuild`{% endif %}.
|
||||
- Advanced setup is enabled and the workflow has an Autobuild step for the language using the `autobuild` action (`{% data reusables.actions.action-codeql-action-autobuild %}`).
|
||||
|
||||
{% endif %}
|
||||
|
||||
{% ifversion codeql-no-build %}
|
||||
|
||||
### Example using the `build-mode` option
|
||||
|
||||
```yaml
|
||||
# Initializes the CodeQL tools for scanning.
|
||||
name: Analyze
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
# Analyze C and C++ code
|
||||
- language: c-cpp
|
||||
build-mode: autobuild
|
||||
# Analyze Go code
|
||||
- language: go
|
||||
build-mode: autobuild
|
||||
|
||||
steps:
|
||||
- uses: {% data reusables.actions.action-codeql-action-init %}
|
||||
with:
|
||||
languages: {% raw %}${{ matrix.language }}{% endraw %}
|
||||
build-mode: {% raw %}${{ matrix.build-mode }}{% endraw %}
|
||||
```
|
||||
|
||||
### Example using the Autobuild step
|
||||
|
||||
{% elsif ghes and code-scanning-without-workflow-310 %}
|
||||
|
||||
<!--Nothing to display-->
|
||||
|
||||
{% elsif ghes %}
|
||||
|
||||
The basic {% data variables.code-scanning.codeql_workflow %} uses the `autobuild` action to build your code.
|
||||
|
||||
{% endif %}
|
||||
|
||||
```yaml
|
||||
# Initializes the CodeQL tools for scanning.
|
||||
- name: Initialize CodeQL
|
||||
uses: {% data reusables.actions.action-codeql-action-init %}
|
||||
with:
|
||||
languages: {% raw %}${{ matrix.language }}{% endraw %}
|
||||
|
||||
- name: Autobuild
|
||||
uses: {% data reusables.actions.action-codeql-action-autobuild %}
|
||||
```
|
||||
|
||||
## About specifying build steps manually
|
||||
|
||||
{% ifversion codeql-no-build or code-scanning-without-workflow-310 %}
|
||||
You can only specify manual build steps if you have enabled advanced setup, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning#configuring-advanced-setup-for-a-repository)."{% endif %}
|
||||
|
||||
{% data reusables.code-scanning.autobuild-add-build-steps %} For information on how to edit the workflow file, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#editing-a-code-scanning-workflow)."
|
||||
|
||||
{% ifversion codeql-no-build %}
|
||||
Update your workflow to define the `build-mode` as `manual`.
|
||||
|
||||
```yaml
|
||||
# Initializes the CodeQL tools for scanning.
|
||||
- name: Initialize CodeQL
|
||||
- uses: {% data reusables.actions.action-codeql-action-init %}
|
||||
with:
|
||||
languages: {% raw %}${{ matrix.language }}{% endraw %}
|
||||
build-mode: manual
|
||||
- uses: {% data reusables.actions.action-codeql-action-analyze %}
|
||||
with:
|
||||
category: {% raw %}"/language:${{ matrix.language }}"{% endraw %}
|
||||
```
|
||||
|
||||
Alternatively, update your workflow to comment out the "Autobuild" step.
|
||||
|
||||
{% endif %}
|
||||
|
||||
```yaml
|
||||
# Autobuild attempts to build any compiled languages.
|
||||
# - name: Autobuild
|
||||
# uses: {% data reusables.actions.action-codeql-action-autobuild %}
|
||||
```
|
||||
|
||||
### Specifying build commands
|
||||
|
||||
When manual building is enabled, uncomment the `run` step in the workflow and add build commands that are suitable for your repository. The `run` step runs command-line programs using the operating system's shell. You can modify these commands and add more commands to customize the build process.
|
||||
|
||||
``` yaml
|
||||
- run: |
|
||||
make bootstrap
|
||||
make release
|
||||
```
|
||||
|
||||
For more information about the `run` keyword, see "[AUTOTITLE](/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun)."
|
||||
|
||||
{% ifversion codeql-no-build %}<!-- For "no-build" this is covered earlier in the article under "About CodeQL build modes". -->
|
||||
{% elsif ghes %}
|
||||
|
||||
### Specifying build commands for multiple languages
|
||||
|
||||
For repositories with multiple compiled languages, you can specify language-specific build commands. For example, if your repository contains C/C++, C# and Java, you might want to provide manual build steps for one language (here Java). This specifies build steps for Java while still using `autobuild` for C/C++ and C#.
|
||||
|
||||
```yaml
|
||||
- if: matrix.language == {% ifversion codeql-language-identifiers-311 %}'c-cpp'{% else %}'cpp'{% endif %} || matrix.language == 'csharp'
|
||||
name: Autobuild
|
||||
uses: {% data reusables.actions.action-codeql-action-autobuild %}
|
||||
- if: matrix.language == {% ifversion codeql-language-identifiers-311 %}'java-kotlin'{% else %}'java'{% endif %}
|
||||
name: Build Java
|
||||
run: |
|
||||
make bootstrap
|
||||
make release
|
||||
```
|
||||
|
||||
For more information about the `if` conditional, see "[AUTOTITLE](/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsif)."
|
||||
{% endif %}
|
||||
|
||||
If you added manual build steps for compiled languages and {% data variables.product.prodname_code_scanning %} is still not working on your repository, contact {% data variables.contact.contact_support %}.
|
||||
|
||||
## Autobuild steps for compiled languages
|
||||
|
||||
{% ifversion fpt or ghec %}{% data variables.product.prodname_dotcom %}-hosted runners are always run with the software required by `autobuild`.{% endif %} If you use self-hosted runners for {% data variables.product.prodname_actions %}, you may need to install additional software to use the `autobuild` process. Additionally, if your repository requires a specific version of a build tool, you may need to install it manually. {% ifversion code-scanning-default-setup-self-hosted-310 or default-setup-self-hosted-runners-GHEC %} For self-hosted runners, you should install dependencies directly in the runners themselves. We provide examples of common dependencies for C/C++, C#, and Java in each of the `autobuild` sections of this article for those languages. For more information, see "[AUTOTITLE](/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners)."{% endif %}
|
||||
|
||||
- [Building C/C++](#building-cc)
|
||||
- [Building C#](#building-c){% ifversion codeql-go-autobuild %}
|
||||
- [Building Go](#building-go){% endif %}{% ifversion codeql-kotlin-beta %}
|
||||
- [Building Java and Kotlin](#building-java--and-kotlin){% else %}
|
||||
- [Building Java](#building-java){% endif %}{% ifversion codeql-swift-beta %}
|
||||
- [Building Swift](#building-swift){% endif %}
|
||||
|
||||
{% note %}
|
||||
|
||||
**Note**: If you use self-hosted runners for {% data variables.product.prodname_actions %}, you may need to install additional software to use the `autobuild` process. Additionally, if your repository requires a specific version of a build tool, you may need to install it manually. {% ifversion code-scanning-default-setup-self-hosted-310 or default-setup-self-hosted-runners-GHEC %} For self-hosted runners, you should install dependencies directly in the runners themselves. We provide examples of common dependencies for C/C++, C#, and Java in each of the `autobuild` sections of this article for those languages. For more information, see "[AUTOTITLE](/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners)."{% endif %}{% ifversion fpt or ghec %}{% data variables.product.prodname_dotcom %}-hosted runners are always run with the software required by `autobuild`.{% endif %}
|
||||
**Note**: If your workflow uses a `language` matrix, `autobuild` attempts to build each of the compiled languages listed in the matrix. Without a matrix `autobuild` attempts to build the supported compiled language that has the most source files in the repository. With the exception of Go, analysis of other compiled languages in your repository will fail unless you supply explicit build commands.
|
||||
|
||||
{% endnote %}
|
||||
|
||||
### `autobuild` for C/C++
|
||||
## Building C/C++
|
||||
|
||||
{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for C/C++ code.
|
||||
|
||||
### Autobuild summary{% endif %}
|
||||
|
||||
| Supported system type | System name |
|
||||
|----|----|
|
||||
| Operating system | Windows, macOS, and Linux |
|
||||
| Build system | Windows: MSbuild and build scripts<br/>Linux and macOS: Autoconf, Make, CMake, qmake, Meson, Waf, SCons, Linux Kbuild, and build scripts |
|
||||
|
||||
The behavior of the `autobuild` step varies according to the operating system that the extraction runs on. On Windows, the `autobuild` step attempts to autodetect a suitable build method for C/C++ using the following approach:
|
||||
The behavior of the `autobuild` step varies according to the operating system that the extraction runs on.
|
||||
|
||||
### Windows autodetection
|
||||
|
||||
On Windows, the `autobuild` step attempts to autodetect a suitable build method for C/C++ using the following approach:
|
||||
|
||||
1. Invoke `MSBuild.exe` on the solution (`.sln`) or project (`.vcxproj`) file closest to the root.
|
||||
If `autobuild` detects multiple solution or project files at the same (shortest) depth from the top level directory, it will attempt to build all of them.
|
||||
1. Invoke a script that looks like a build script—_build.bat_, _build.cmd_, _and build.exe_ (in that order).
|
||||
|
||||
### Linux and macOS autodetection
|
||||
|
||||
On Linux and macOS, the `autobuild` step reviews the files present in the repository to determine the build system used:
|
||||
|
||||
1. Look for a build system in the root directory.
|
||||
1. If none are found, search subdirectories for a unique directory with a build system for C/C++.
|
||||
1. Run an appropriate command to configure the system.
|
||||
|
||||
#### Runner requirements
|
||||
|
||||
{% ifversion codeql-cpp-autoinstall-dependencies %}
|
||||
On Ubuntu Linux runners, `autobuild` may try to automatically install dependencies required by the detected configuration and build steps. By default, this behavior is enabled on {% data variables.product.prodname_dotcom %}-hosted runners and disabled on self-hosted runners. You can enable or disable this feature explicitly by setting `CODEQL_EXTRACTOR_CPP_AUTOINSTALL_DEPENDENCIES` to `true` or `false` in the environment. For more information about defining environment variables, see "[AUTOTITLE](/actions/learn-github-actions/variables#defining-environment-variables-for-a-single-workflow)."
|
||||
{% endif %}
|
||||
@@ -105,33 +313,60 @@ For self-hosted runners{% ifversion codeql-cpp-autoinstall-dependencies %}, unle
|
||||
If you enable automatic installation of dependencies, you must ensure that the runner is using Ubuntu and that it can run `sudo apt-get` without requiring a password.
|
||||
{%- endif %}
|
||||
|
||||
### `autobuild` for C#
|
||||
## Building C#
|
||||
|
||||
{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for C# code.
|
||||
|
||||
### Autobuild summary{% endif %}
|
||||
|
||||
| Supported system type | System name |
|
||||
|----|----|
|
||||
| Operating system | Windows, macOS, and Linux |
|
||||
| Build system | .NET and MSbuild, as well as build scripts |
|
||||
|
||||
### Windows autodetection
|
||||
|
||||
The `autobuild` process attempts to autodetect a suitable build method for C# using the following approach:
|
||||
|
||||
1. Invoke `dotnet build` on the solution (`.sln`) or project (`.csproj`) file closest to the root.
|
||||
1. Invoke `MSbuild` (Linux) or `MSBuild.exe` (Windows) on the solution or project file closest to the root.
|
||||
1. Invoke `MSBuild.exe` on the solution or project file closest to the root.
|
||||
If `autobuild` detects multiple solution or project files at the same (shortest) depth from the top level directory, it will attempt to build all of them.
|
||||
1. Invoke a script that looks like a build script—_build_ and _build.sh_ (in that order, for Linux) or _build.bat_, _build.cmd_, _and build.exe_ (in that order, for Windows).
|
||||
1. Invoke a script that looks like a build script—`build.bat`, `build.cmd`, and `build.exe` (in that order).
|
||||
|
||||
#### Runner requirements
|
||||
|
||||
For .NET Core application development on self-hosted runners, the .NET SDK is required (for `dotnet`).
|
||||
|
||||
For .NET Framework application development, on Windows, you will need Microsoft Build Tools (for `msbuild`) and Nuget CLI (for `nuget`). On Linux and macOS, you will require Mono Runtime (to run `mono`, `msbuild`, or `nuget`).
|
||||
For .NET Framework application development, you will need Microsoft Build Tools (for `msbuild`) and Nuget CLI (for `nuget`).
|
||||
|
||||
### Linux and macOS autodetection
|
||||
|
||||
1. Invoke `dotnet build` on the solution (`.sln`) or project (`.csproj`) file closest to the root.
|
||||
1. Invoke `MSbuild` on the solution or project file closest to the root.
|
||||
If `autobuild` detects multiple solution or project files at the same (shortest) depth from the top level directory, it will attempt to build all of them.
|
||||
1. Invoke a script that looks like a build script—`build` and `build.sh` (in that order).
|
||||
|
||||
#### Runner requirements
|
||||
|
||||
For .NET Core application development on self-hosted runners, the .NET SDK is required (for `dotnet`).
|
||||
|
||||
For .NET Framework application development, you will require Mono Runtime (to run `mono`, `msbuild`, or `nuget`).
|
||||
|
||||
{% ifversion codeql-go-autobuild %}
|
||||
|
||||
### `autobuild` for Go
|
||||
## Building Go
|
||||
|
||||
{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for Go code.
|
||||
|
||||
### Autobuild summary{% endif %}
|
||||
|
||||
| Supported system type | System name |
|
||||
|----|----|
|
||||
| Operating system | Windows, macOS, and Linux |
|
||||
| Build system | Go modules, `dep` and Glide, as well as build scripts including Makefiles and Ninja scripts |
|
||||
|
||||
### Autodetection
|
||||
|
||||
The `autobuild` process attempts to autodetect a suitable way to install the dependencies needed by a Go repository before extracting all `.go` files:
|
||||
|
||||
1. Invoke `make`, `ninja`, `./build` or `./build.sh` (in that order) until one of these commands succeeds and a subsequent `go list ./...` also succeeds, indicating that the needed dependencies have been installed.
|
||||
@@ -150,19 +385,34 @@ The `autobuild` process attempts to autodetect a suitable way to install the dep
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
|
||||
### `autobuild` for Java {% ifversion codeql-kotlin-beta %} and Kotlin {% endif %}
|
||||
## Building Java {% ifversion codeql-kotlin-beta %} and Kotlin {% endif %}
|
||||
|
||||
{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports the following build modes.
|
||||
|
||||
- Java: `none`, `autobuild`, or `manual`
|
||||
- Kotlin: `autobuild` or `manual`
|
||||
|
||||
When you first enable default setup for a repository, if only Java code is detected then the build mode is set to `none`. If Kotlin or a combination of Java and Kotlin code is detected, then the build mode is set to `autobuild`.
|
||||
|
||||
If you later add Kotlin code to a repository that uses the `none` build mode, {% data variables.product.prodname_codeql %} analysis reports a warning message explaining that Kotlin is not supported. You will need to disable default setup and re-enable it. When you re-enable default setup, the build mode will change to `autobuild` so that both languages can be analyzed. Alternatively, you can change to an advanced setup. For more information, see "[AUTOTITLE](/code-security/code-scanning/troubleshooting-code-scanning/kotlin-detected-in-no-build)."
|
||||
|
||||
### Autobuild summary{% endif %}
|
||||
|
||||
| Supported system type | System name |
|
||||
|----|----|
|
||||
| Operating system | Windows, macOS, and Linux (no restriction) |
|
||||
| Build system | Gradle, Maven and Ant |
|
||||
|
||||
### Autodetection
|
||||
|
||||
The `autobuild` process tries to determine the build system for Java codebases by applying this strategy:
|
||||
|
||||
1. Search for a build file in the root directory. Check for Gradle then Maven then Ant build files.
|
||||
1. Run the first build file found. If both Gradle and Maven files are present, the Gradle file is used.
|
||||
1. Otherwise, search for build files in direct subdirectories of the root directory. If only one subdirectory contains build files, run the first file identified in that subdirectory (using the same preference as for 1). If more than one subdirectory contains build files, report an error.
|
||||
|
||||
### Runner requirements
|
||||
|
||||
If you're using self-hosted runners, the required version(s) of Java should be present:
|
||||
|
||||
- If the runner will be used for analyzing repositories that need a single version of Java, then the appropriate JDK version needs to be installed, and needs to be present in the PATH variable (so that `java` and `javac` can be found).
|
||||
@@ -179,7 +429,11 @@ You will also need to install the build system (for example `make`, `cmake`, `ba
|
||||
|
||||
{% ifversion codeql-swift-beta %}
|
||||
|
||||
### `autobuild` for Swift
|
||||
## Building Swift
|
||||
|
||||
{% ifversion codeql-no-build %}{% data variables.product.prodname_codeql %} supports build modes `autobuild` or `manual` for Swift code.
|
||||
|
||||
### Autobuild summary{% endif %}
|
||||
|
||||
| Supported system type | System name |
|
||||
|----|----|
|
||||
@@ -198,7 +452,7 @@ Code scanning of Swift code uses macOS runners by default. {% ifversion fpt or g
|
||||
|
||||
{% data reusables.code-scanning.default-setup-swift-self-hosted-runners %}
|
||||
|
||||
#### Customizing Swift compilation in a {% data variables.code-scanning.codeql_workflow %}
|
||||
### Customizing Swift compilation in a {% data variables.code-scanning.codeql_workflow %}
|
||||
|
||||
`xcodebuild` and `swift build` are both supported for Swift builds. We recommend only targeting one architecture during the build. For example, `ARCH=arm64` for `xcodebuild`, or `--arch arm64` for `swift build`.
|
||||
|
||||
@@ -207,36 +461,3 @@ You can pass the `archive` and `test` options to `xcodebuild`. However, the stan
|
||||
For Swift analysis, you must always explicitly install dependencies managed via CocoaPods or Carthage before generating the {% data variables.product.prodname_codeql %} database.
|
||||
|
||||
{% endif %}
|
||||
|
||||
## Adding build steps for a compiled language
|
||||
|
||||
{% data reusables.code-scanning.autobuild-add-build-steps %} For information on how to edit the workflow file, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#editing-a-code-scanning-workflow)."
|
||||
|
||||
After removing the `autobuild` step, uncomment the `run` step and add build commands that are suitable for your repository. The workflow `run` step runs command-line programs using the operating system's shell. You can modify these commands and add more commands to customize the build process.
|
||||
|
||||
``` yaml
|
||||
- run: |
|
||||
make bootstrap
|
||||
make release
|
||||
```
|
||||
|
||||
For more information about the `run` keyword, see "[AUTOTITLE](/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun)."
|
||||
|
||||
If your repository contains multiple compiled languages, you can specify language-specific build commands. For example, if your repository contains C/C++, C# and Java, and `autobuild` correctly builds C/C++ and C# but fails to build Java, you could use the following configuration in your workflow, after the `init` step. This specifies build steps for Java while still using `autobuild` for C/C++ and C#:
|
||||
|
||||
```yaml
|
||||
- if: matrix.language == {% ifversion codeql-language-identifiers-311 %}'c-cpp'{% else %}'cpp'{% endif %} || matrix.language == 'csharp'
|
||||
name: Autobuild
|
||||
uses: {% data reusables.actions.action-codeql-action-autobuild %}
|
||||
- if: matrix.language == {% ifversion codeql-language-identifiers-311 %}'java-kotlin'{% else %}'java'{% endif %}
|
||||
name: Build Java
|
||||
run: |
|
||||
make bootstrap
|
||||
make release
|
||||
```
|
||||
|
||||
For more information about the `if` conditional, see "[AUTOTITLE](/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsif)."
|
||||
|
||||
For more tips and tricks about why `autobuild` won't build your code, see "[AUTOTITLE](/code-security/code-scanning/troubleshooting-code-scanning/automatic-build-failed-for-a-compiled-language)."
|
||||
|
||||
If you added manual build steps for compiled languages and {% data variables.product.prodname_code_scanning %} is still not working on your repository, contact {% data variables.contact.contact_support %}.
|
||||
|
||||
@@ -3,7 +3,7 @@ title: Configuring {% ifversion code-scanning-without-workflow %}advanced setup
|
||||
shortTitle: Configure {% ifversion code-scanning-without-workflow %}advanced setup{% else %}{% data variables.product.prodname_code_scanning %}{% endif %}
|
||||
intro: 'You can configure {% ifversion code-scanning-without-workflow %}advanced setup{% else %}{% data variables.product.prodname_code_scanning %}{% endif %} for a repository to find security vulnerabilities in your code{% ifversion code-scanning-without-workflow %} using a highly customizable {% data variables.product.prodname_code_scanning %} configuration{% endif %}.'
|
||||
product: '{% data reusables.gated-features.code-scanning %}'
|
||||
permissions: 'People with admin permissions to a repository, or the security manager role for the repository, can configure {% data variables.product.prodname_code_scanning %} for that repository. People with write permissions to a repository can also configure {% data variables.product.prodname_code_scanning %}, but only by creating a workflow file or manually uploading a SARIF file.'
|
||||
permissions: '{% ifversion code-scanning-without-workflow %}People with admin permissions to a repository, or the security manager role for the repository, can enable {% data variables.product.prodname_code_scanning %} for that repository.{% else %}People with write permissions to a repository can enable {% data variables.product.prodname_code_scanning %} for that repository.{% endif %}'
|
||||
redirect_from:
|
||||
- /code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-advanced-setup-for-code-scanning
|
||||
versions:
|
||||
@@ -24,20 +24,17 @@ allowTitleToDifferFromFilename: true
|
||||
## About {% ifversion code-scanning-without-workflow %}advanced setup for {% endif %}{% data variables.product.prodname_code_scanning %}
|
||||
|
||||
{% ifversion code-scanning-without-workflow %}
|
||||
Advanced setup for {% data variables.product.prodname_code_scanning %} is helpful when you need more granular control over your {% data variables.product.prodname_code_scanning %} configuration. By creating and editing a {% data variables.product.prodname_codeql %} workflow file, you can change the scan schedule, scan any {% data variables.product.prodname_codeql %}-supported language, use a matrix build, and more.
|
||||
Advanced setup for {% data variables.product.prodname_code_scanning %} is helpful when you need to customize your {% data variables.product.prodname_code_scanning %}. By creating and editing a workflow file, you can define how to build compiled languages, choose which queries to run, select the languages to scan, use a matrix build, and more. You also have access to all the options for controlling workflows, for example: changing the scan schedule, defining workflow triggers, specifying specialist runners to use. For more information about {% data variables.product.prodname_actions %} workflows, see "[AUTOTITLE](/actions/using-workflows/about-workflows)."
|
||||
|
||||
{% else %}
|
||||
{% data variables.product.prodname_code_scanning_caps %} helps you catch vulnerabilities in the code in your repository. With {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %}, you can select custom or built-in query suites for use in your analysis, set a specific scan schedule, choose which events trigger a scan, and more.
|
||||
{% endif %}
|
||||
|
||||
{% ifversion fpt or ghec %}
|
||||
|
||||
You can also configure {% data variables.product.prodname_code_scanning %} with third-party tools. For more information, see "[Configuring {% data variables.product.prodname_code_scanning %} using third-party actions](#configuring-code-scanning-using-third-party-actions)."
|
||||
|
||||
{% else %}
|
||||
|
||||
Your site administrator can also make third-party actions available to users for {% data variables.product.prodname_code_scanning %}, by setting up {% data variables.product.prodname_github_connect %}. For more information, see "[AUTOTITLE](/admin/code-security/managing-github-advanced-security-for-your-enterprise/configuring-code-scanning-for-your-appliance#configuring-github-connect-to-sync-github-actions)."
|
||||
|
||||
{% endif %}
|
||||
|
||||
{% data reusables.code-scanning.about-multiple-configurations-link %}
|
||||
@@ -49,11 +46,11 @@ If you do not need a highly customizable {% data variables.product.prodname_code
|
||||
|
||||
### Prerequisites
|
||||
|
||||
Your repository is eligible for {% ifversion code-scanning-without-workflow %}advanced setup{% else %}{% data variables.product.prodname_code_scanning %}{% endif %} if:
|
||||
- it uses {% data variables.product.prodname_codeql %}-supported languages or you plan to generate code scanning results with a third-party tool.
|
||||
Your repository is eligible for {% ifversion code-scanning-without-workflow %}advanced setup{% else %}{% data variables.product.prodname_code_scanning %}{% endif %} if it meets these requirements.
|
||||
- It uses {% data variables.product.prodname_codeql %}-supported languages or you plan to generate code scanning results with a third-party tool.
|
||||
- {% data variables.product.prodname_actions %} are enabled.{% ifversion fpt %}
|
||||
- it is publicly visible.{%- elsif ghec %}
|
||||
- it is publicly visible, or {% data variables.product.prodname_GH_advanced_security %} is enabled.{%- elsif ghes %}
|
||||
- It is publicly visible.{%- elsif ghec %}
|
||||
- It is publicly visible, or {% data variables.product.prodname_GH_advanced_security %} is enabled.{%- elsif ghes %}
|
||||
- {% data variables.product.prodname_GH_advanced_security %} is enabled.{% endif %}
|
||||
|
||||
{% ifversion ghes %}
|
||||
@@ -62,13 +59,9 @@ If the server on which you are running {% data variables.product.prodname_ghe_se
|
||||
|
||||
{% ifversion code-scanning-without-workflow %}
|
||||
|
||||
## Configuring advanced setup for a repository
|
||||
## Configuring advanced setup for {% data variables.product.prodname_code_scanning %} with {% data variables.product.prodname_codeql %}
|
||||
|
||||
Advanced setup for {% data variables.product.prodname_code_scanning %} is helpful when you need to customize your {% data variables.product.prodname_code_scanning %}. By creating and editing a workflow file, you can choose which queries to run, change the scan schedule, select the languages to scan, use a matrix build, and more.
|
||||
|
||||
### Configuring advanced setup for {% data variables.product.prodname_code_scanning %} with {% data variables.product.prodname_codeql %}
|
||||
|
||||
You can customize your {% data variables.product.prodname_code_scanning %} by creating and editing a workflow file. Selecting advanced setup generates a basic workflow file for you to customize.
|
||||
You can customize your {% data variables.product.prodname_codeql %} analysis by creating and editing a workflow file. Selecting advanced setup generates a basic workflow file for you to customize using standard workflow syntax and specifying options for the {% data variables.product.prodname_codeql %} action. See "[AUTOTITLE](/actions/using-workflows/about-workflows)" and "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning)."
|
||||
|
||||
{% data reusables.code-scanning.billing %}
|
||||
|
||||
@@ -109,7 +102,7 @@ You can customize your {% data variables.product.prodname_code_scanning %} by cr
|
||||
|
||||
In the suggested {% data variables.code-scanning.codeql_workflow %}, {% data variables.product.prodname_code_scanning %} is configured to analyze your code each time you either push a change to the default branch or any protected branches, or raise a pull request against the default branch. As a result, {% data variables.product.prodname_code_scanning %} will now commence.
|
||||
|
||||
The `on:pull_request` and `on:push` triggers for code scanning are each useful for different purposes. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#configuring-frequency)."
|
||||
The `on:pull_request` and `on:push` triggers for code scanning are each useful for different purposes. See "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#configuring-frequency)" and [AUTOTITLE](/actions/using-workflows/triggering-a-workflow)."
|
||||
|
||||
{% else %}
|
||||
|
||||
@@ -137,7 +130,7 @@ The `on:pull_request` and `on:push` triggers for code scanning are each useful f
|
||||
|
||||
In the suggested {% data variables.code-scanning.codeql_workflow %}, {% data variables.product.prodname_code_scanning %} is configured to analyze your code each time you either push a change to the default branch or any protected branches, or raise a pull request against the default branch. As a result, {% data variables.product.prodname_code_scanning %} will now commence.
|
||||
|
||||
The `on:pull_request` and `on:push` triggers for code scanning are each useful for different purposes. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#configuring-frequency)."
|
||||
The `on:pull_request` and `on:push` triggers for code scanning are each useful for different purposes. See "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#configuring-frequency)" and [AUTOTITLE](/actions/using-workflows/triggering-a-workflow)."
|
||||
|
||||
{% endif %}
|
||||
|
||||
@@ -145,11 +138,9 @@ For information on bulk enablement, see "[AUTOTITLE](/code-security/code-scannin
|
||||
|
||||
{% ifversion fpt or ghec %}
|
||||
|
||||
### Configuring {% data variables.product.prodname_code_scanning %} using third-party actions
|
||||
## Configuring {% data variables.product.prodname_code_scanning %} using third-party actions
|
||||
|
||||
{% data reusables.advanced-security.starter-workflows-beta %}
|
||||
|
||||
{% data reusables.advanced-security.starter-workflow-overview %}
|
||||
{% data variables.product.product_name %} includes starter workflows for third-party actions, as well as the {% data variables.product.prodname_codeql %} action. Using a starter workflow is much easier than writing a workflow unaided.
|
||||
|
||||
{% data reusables.code-scanning.billing %}
|
||||
|
||||
@@ -173,12 +164,19 @@ For information on bulk enablement, see "[AUTOTITLE](/code-security/code-scannin
|
||||
|
||||
## Next steps
|
||||
|
||||
After configuring {% data variables.product.prodname_code_scanning %}, and allowing its actions to complete, you can:
|
||||
After your workflow runs successfully at least once, you are ready to start examining and resolving {% data variables.product.prodname_code_scanning %} alerts. For more information on {% data variables.product.prodname_code_scanning %} alerts, see "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/about-code-scanning-alerts)" and "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/managing-code-scanning-alerts-for-your-repository)."
|
||||
|
||||
- View all of the {% data variables.product.prodname_code_scanning %} alerts generated for this repository. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/managing-code-scanning-alerts-for-your-repository)."
|
||||
- View any alerts generated for a pull request submitted after you configure {% data variables.product.prodname_code_scanning %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/triaging-code-scanning-alerts-in-pull-requests)."
|
||||
- Configure notifications for completed runs. For more information, see "[AUTOTITLE](/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/configuring-notifications#github-actions-notification-options)."
|
||||
- Learn about {% data variables.product.prodname_code_scanning %} checks on pull requests. For more information, "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/triaging-code-scanning-alerts-in-pull-requests#about-code-scanning-as-a-pull-request-check)."
|
||||
- View the logs generated by the {% data variables.product.prodname_code_scanning %} analysis. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs)."
|
||||
- Investigate any problems that occur with the initial configuration of {% data variables.product.prodname_code_scanning %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/troubleshooting-code-scanning)."
|
||||
- Customize how {% data variables.product.prodname_code_scanning %} scans the code in your repository. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning)."
|
||||
Learn how {% data variables.product.prodname_code_scanning %} runs behave as checks on pull requests, see "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/triaging-code-scanning-alerts-in-pull-requests#about-code-scanning-as-a-pull-request-check)."
|
||||
|
||||
{% ifversion code-scanning-tool-status-page %}
|
||||
|
||||
You can find detailed information about your {% data variables.product.prodname_code_scanning %} configuration, including timestamps for each scan and the percentage of files scanned, on the tool status page. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page)."
|
||||
|
||||
{% endif %}
|
||||
|
||||
### Further reading
|
||||
|
||||
- "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/triaging-code-scanning-alerts-in-pull-requests)."
|
||||
- "[AUTOTITLE](/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/configuring-notifications#github-actions-notification-options)."
|
||||
- "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning)."
|
||||
- "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs)."
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: Customizing {% ifversion code-scanning-without-workflow %}your advanced setup for {% endif %}code scanning
|
||||
intro: 'You can customize how {% ifversion code-scanning-without-workflow %}your advanced setup {% else %}{% data variables.product.prodname_dotcom %} {% endif %}scans the code in your project for vulnerabilities and errors.'
|
||||
product: '{% data reusables.gated-features.code-scanning %}'
|
||||
permissions: 'People with write permissions to a repository can customize {% data variables.product.prodname_code_scanning %} for the repository.'
|
||||
permissions: 'People with write permissions to a repository can customize {% ifversion code-scanning-without-workflow %}advanced setup for {% endif %}{% data variables.product.prodname_code_scanning %}.'
|
||||
redirect_from:
|
||||
- /github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning
|
||||
- /code-security/secure-coding/configuring-code-scanning
|
||||
@@ -567,7 +567,7 @@ For more information about using `exclude` and `include` filters in your custom
|
||||
|
||||
### Specifying directories to scan
|
||||
|
||||
For the interpreted languages that {% data variables.product.prodname_codeql %} supports (Python, Ruby, and JavaScript/TypeScript), you can restrict {% data variables.product.prodname_code_scanning %} to files in specific directories by adding a `paths` array to the configuration file. You can exclude the files in specific directories from analysis by adding a `paths-ignore` array.
|
||||
When codebases are analyzed without building the code, you can restrict {% data variables.product.prodname_code_scanning %} to files in specific directories by adding a `paths` array to the configuration file. You can also exclude the files in specific directories from analysis by adding a `paths-ignore` array. You can use this option when you run the {% data variables.product.prodname_codeql %} actions on an interpreted language (Python, Ruby, and JavaScript/TypeScript){% ifversion codeql-no-build %} or when you analyze a compiled language without building the code (currently supported for {% data variables.code-scanning.no_build_support %}){% endif %}.
|
||||
|
||||
``` yaml copy
|
||||
paths:
|
||||
@@ -587,7 +587,7 @@ paths-ignore:
|
||||
|
||||
{% endnote %}
|
||||
|
||||
For compiled languages, if you want to limit {% data variables.product.prodname_code_scanning %} to specific directories in your project, you must specify appropriate build steps in the workflow. The commands you need to use to exclude a directory from the build will depend on your build system. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
||||
For analysis where code is built, if you want to limit {% data variables.product.prodname_code_scanning %} to specific directories in your project, you must specify appropriate build steps in the workflow. The commands you need to use to exclude a directory from the build will depend on your build system. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
||||
|
||||
You can quickly analyze small portions of a monorepo when you modify code in specific directories. You'll need to both exclude directories in your build steps and use the `paths-ignore` and `paths` keywords for [`on.<push|pull_request>`](/actions/using-workflows/workflow-syntax-for-github-actions#onpushpull_requestpull_request_targetpathspaths-ignore) in your workflow.
|
||||
|
||||
@@ -640,9 +640,15 @@ In the following example, `vars.CODEQL_CONF` is a {% data variables.product.prod
|
||||
|
||||
## Configuring {% data variables.product.prodname_code_scanning %} for compiled languages
|
||||
|
||||
{% data reusables.code-scanning.autobuild-compiled-languages %}
|
||||
{% ifversion codeql-no-build %}
|
||||
|
||||
{% data reusables.code-scanning.autobuild-add-build-steps %} For more information about how to configure {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} for compiled languages, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages)."
|
||||
For compiled languages, you can decide how the {% data variables.product.prodname_codeql %} action creates a {% data variables.product.prodname_codeql %} database for analysis. For information about the build options available, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages)."
|
||||
|
||||
{% else %}
|
||||
|
||||
For compiled languages, the {% data variables.product.prodname_codeql %} action builds the codebase to create a {% data variables.product.prodname_codeql %} database for analysis. By default, {% data variables.product.prodname_codeql %} uses `autobuild` steps to identify the most likely build method for the codebase. {% data reusables.code-scanning.autobuild-add-build-steps %} For more information about how to configure {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} for compiled languages, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages)."
|
||||
|
||||
{% endif %}
|
||||
|
||||
## Uploading {% data variables.product.prodname_code_scanning %} data to {% data variables.product.prodname_dotcom %}
|
||||
|
||||
|
||||
@@ -27,9 +27,9 @@ versions:
|
||||
## About default setup
|
||||
|
||||
Default setup for {% data variables.product.prodname_code_scanning %} is the quickest, easiest, most low-maintenance way to enable {% data variables.product.prodname_code_scanning %} for your repository. Based on the code in your repository, default setup will automatically create a custom {% data variables.product.prodname_code_scanning %} configuration. After enabling default setup, the code written in {% data variables.product.prodname_codeql %}-supported languages in your repository will be scanned:
|
||||
- on each push to the repository's default branch, or any protected branch. For more information on protected branches, see "[AUTOTITLE](/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches)."
|
||||
- when creating or committing to a pull request based against the repository's default branch, or any protected branch.{% ifversion default-setup-scan-on-schedule %}
|
||||
- on a weekly schedule.
|
||||
- On each push to the repository's default branch, or any protected branch. For more information on protected branches, see "[AUTOTITLE](/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches)."
|
||||
- When creating or committing to a pull request based against the repository's default branch, or any protected branch.{% ifversion default-setup-scan-on-schedule %}
|
||||
- On a weekly schedule.
|
||||
|
||||
{% note %}
|
||||
|
||||
@@ -45,11 +45,11 @@ If you need more granular control over your {% data variables.product.prodname_c
|
||||
### Requirements for using default setup
|
||||
|
||||
Your repository is eligible for default setup for {% data variables.product.prodname_code_scanning %} if:{% ifversion default-setup-pre-enablement %}{% else %}{% ifversion code-scanning-without-workflow-310 %}
|
||||
- it includes at least one {% data variables.product.prodname_codeql %}-supported language{% ifversion codeql-swift-advanced-setup %} aside from Swift{% endif %}.{% else %}
|
||||
- it includes only the following {% data variables.product.prodname_codeql %}-supported languages: {% ifversion code-scanning-default-setup-go %}Go, {% endif %}JavaScript/TypeScript, Python, or Ruby.{% endif %}{% endif %}
|
||||
- It includes at least one {% data variables.product.prodname_codeql %}-supported language{% ifversion codeql-swift-advanced-setup %} aside from Swift{% endif %}.{% else %}
|
||||
- It includes only the following {% data variables.product.prodname_codeql %}-supported languages: {% ifversion code-scanning-default-setup-go %}Go, {% endif %}JavaScript/TypeScript, Python, or Ruby.{% endif %}{% endif %}
|
||||
- {% data variables.product.prodname_actions %} are enabled.{% ifversion fpt %}
|
||||
- it is publicly visible.{%- elsif ghec %}
|
||||
- it is publicly visible, or {% data variables.product.prodname_GH_advanced_security %} is enabled.{%- elsif ghes %}
|
||||
- It is publicly visible.{%- elsif ghec %}
|
||||
- It is publicly visible, or {% data variables.product.prodname_GH_advanced_security %} is enabled.{%- elsif ghes %}
|
||||
- {% data variables.product.prodname_GH_advanced_security %} is enabled.{% endif %}
|
||||
|
||||
{% ifversion default-setup-pre-enablement %}
|
||||
@@ -58,6 +58,10 @@ Your repository is eligible for default setup for {% data variables.product.prod
|
||||
If your repository includes at least one {% data variables.product.prodname_codeql %}-supported language, you can use default setup even if your repository also includes languages that aren't supported by {% data variables.product.prodname_codeql %}, such as R. Unsupported languages will not be scanned by default setup. For more information on {% data variables.product.prodname_codeql %}-supported languages, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql#about-codeql)."
|
||||
{% endif %}
|
||||
|
||||
{% ifversion code-scanning-default-setup-self-hosted-310 or default-setup-self-hosted-runners-GHEC %}
|
||||
You can use default setup with self-hosted runners for all {% data variables.product.prodname_codeql %}-supported languages{% ifversion codeql-swift-advanced-setup %} except Swift{% endif %}. {% ifversion codeql-no-build %}Default setup uses the `none` build mode for {% data variables.code-scanning.no_build_support %} and uses the `autobuild` build mode for other compiled languages. You should configure your self-hosted runners to make sure they can run all the necessary commands for C/C++, C#, and Swift analysis. Analysis of JavaScript/TypeScript, Go, Ruby, Python, and Kotlin code does not currently require special configuration.{% else %}Default setup runs the `autobuild` action, so you should configure your self-hosted runners to make sure they can run all the necessary commands for {% data variables.code-scanning.compiled_languages %} analysis. Analysis of JavaScript/TypeScript, Go, Ruby, Python, and Kotlin code does not currently require special configuration.{% endif %}
|
||||
{% endif %}
|
||||
|
||||
### Customizing default setup
|
||||
|
||||
We recommend that you start using {% data variables.product.prodname_code_scanning %} with default setup. After you've initially configured default setup, you can evaluate {% data variables.product.prodname_code_scanning %} to see how it's working for you. If you find that something isn't working as you expect, you can customize default setup to better meet your code security needs. For more information, see "[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/evaluating-default-setup-for-code-scanning)."
|
||||
|
||||
@@ -64,7 +64,13 @@ For information about {% data variables.product.prodname_code_scanning %} alerts
|
||||
|
||||
## About {% data variables.product.prodname_codeql %} queries
|
||||
|
||||
{% data variables.product.company_short %} experts, security researchers, and community contributors write and maintain the default {% data variables.product.prodname_codeql %} queries used for {% data variables.product.prodname_code_scanning %}. The queries are regularly updated to improve analysis and reduce any false positive results. The queries are open source, so you can view and contribute to the queries in the [`github/codeql`](https://github.com/github/codeql) repository. For more information, see [{% data variables.product.prodname_codeql %}](https://codeql.github.com/) on the {% data variables.product.prodname_codeql %} website. You can also write your own queries. For more information, see "[About {% data variables.product.prodname_codeql %} queries](https://codeql.github.com/docs/writing-codeql-queries/about-codeql-queries/)" in the {% data variables.product.prodname_codeql %} documentation.
|
||||
{% data variables.product.company_short %} experts, security researchers, and community contributors write and maintain the default {% data variables.product.prodname_codeql %} queries used for {% data variables.product.prodname_code_scanning %}. The queries are regularly updated to improve analysis and reduce any false positive results.{% ifversion ghes > 3.8 %} For details of the queries available in the default and extended packs, see "[Queries included in the default and security-extended query suites](/code-security/code-scanning/managing-your-code-scanning-configuration/codeql-query-suites#queries-included-in-the-default-and-security-extended-query-suites)."{% endif %}
|
||||
|
||||
### Writing your own queries
|
||||
|
||||
The queries are open source, so you can view and contribute to the queries in the [github/codeql](https://github.com/github/codeql) repository. For more information, see "[About {% data variables.product.prodname_codeql %} queries](https://codeql.github.com/docs/writing-codeql-queries/about-codeql-queries/)" in the {% data variables.product.prodname_codeql %} documentation.
|
||||
|
||||
### Running additional queries
|
||||
|
||||
If you are scanning your code with advanced setup or an external CI system, you can run additional queries as part of your analysis.
|
||||
|
||||
|
||||
@@ -68,7 +68,7 @@ To see the full set of alerts for the analyzed branch, click **View all branch a
|
||||
|
||||
If the {% data variables.product.prodname_code_scanning %} results check finds any problems with a severity of `error`, `critical`, or `high`, the check fails and the error is reported in the check results. If all the results found by {% data variables.product.prodname_code_scanning %} have lower severities, the alerts are treated as warnings or notes and the check succeeds.
|
||||
|
||||
|
||||
|
||||
|
||||
You can override the default behavior in your repository settings, by specifying the level of severities and security severities that will cause a pull request check failure. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#defining-the-severities-causing-pull-request-check-failure)".
|
||||
|
||||
@@ -76,7 +76,7 @@ You can override the default behavior in your repository settings, by specifying
|
||||
|
||||
Depending on your configuration, you may see additional checks running on pull requests with {% data variables.product.prodname_code_scanning %} configured. These are usually workflows that analyze the code or that upload {% data variables.product.prodname_code_scanning %} results. These checks are useful for troubleshooting when there are problems with the analysis.
|
||||
|
||||
For example, if the repository uses the {% data variables.code-scanning.codeql_workflow %} a **{% data variables.product.prodname_codeql %} / Analyze (LANGUAGE)** check is run for each language before the results check runs. The analysis check may fail if there are configuration problems, or if the pull request breaks the build for a language that the analysis needs to compile (for example, C/C++, C#, or Java).
|
||||
For example, if the repository uses the {% data variables.code-scanning.codeql_workflow %} a **{% data variables.product.prodname_codeql %} / Analyze (LANGUAGE)** check is run for each language before the results check runs. The analysis check may fail if there are configuration problems, or if the pull request breaks the build for a language that the analysis compiles (for example, {% data variables.code-scanning.compiled_languages %}).
|
||||
|
||||
As with other pull request checks, you can see full details of the check failure on the **Checks** tab. For more information about configuring and troubleshooting, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning)" or "[AUTOTITLE](/code-security/code-scanning/troubleshooting-code-scanning)."
|
||||
|
||||
@@ -85,7 +85,7 @@ As with other pull request checks, you can see full details of the check failure
|
||||
{% ifversion code-scanning-pr-conversations-tab %}
|
||||
You can see any {% data variables.product.prodname_code_scanning %} alerts {% ifversion code-scanning-alerts-in-pr-diff %}that are inside the diff of the changes{% endif %} introduced in a pull request by viewing the **Conversation** tab. {% data variables.product.prodname_code_scanning_caps %} posts a pull request review that shows each alert as an annotation on the lines of code that triggered the alert. You can comment on the alerts, dismiss the alerts, and view paths for the alerts, directly from the annotations. You can view the full details of an alert by clicking the "Show more details" link, which will take you to the alert details page.
|
||||
|
||||
|
||||
|
||||
|
||||
You can also view all {% data variables.product.prodname_code_scanning %} alerts {% ifversion code-scanning-alerts-in-pr-diff %}that are inside the diff of the changes introduced in the pull request in the **Files changed** tab{% else %}in the **Files changed** tab of the pull request. Existing {% data variables.product.prodname_code_scanning %} alerts on a file that are outside the diff of the changes introduced in the pull request will only appear in the **Files changed** tab{% endif %}.
|
||||
|
||||
@@ -128,7 +128,7 @@ Anyone with push access to a pull request can fix a {% data variables.product.pr
|
||||
|
||||
Autofix, powered by {% data variables.product.prodname_copilot %}, is an expansion of {% data variables.product.prodname_code_scanning %} that provides you with targeted recommendations to help you fix {% data variables.product.prodname_code_scanning %} alerts in pull requests. The potential fixes are generated automatically by large language models (LLMs) using data from the codebase, the pull request, and from {% data variables.product.prodname_codeql %} analysis.
|
||||
|
||||
|
||||
|
||||
|
||||
### Generating autofix suggestions and publishing to a pull request
|
||||
|
||||
@@ -142,7 +142,7 @@ When autofix is enabled for a repository, alerts are displayed in pull requests
|
||||
|
||||
Usually, when you suggest changes to a pull request, your comment contains changes for a single file that is changed in the pull request. The following screenshot shows an autofix comment that suggests changes to the `index.js` file where the alert is displayed. Since the potential fix requires a new dependency on `escape-html`, the comment also suggests adding this dependency to the `package.json` file, even though the original pull request makes no changes to this file.
|
||||
|
||||
|
||||
|
||||
|
||||
### Assessing and committing an autofix suggestion
|
||||
|
||||
@@ -165,7 +165,7 @@ If you decide to reject an autofix suggestion, click **Dismiss suggestion** in t
|
||||
|
||||
An alternative way of closing an alert is to dismiss it. You can dismiss an alert if you don't think it needs to be fixed. {% data reusables.code-scanning.close-alert-examples %} If you have write permission for the repository, a **Dismiss alert** button is available in code annotations and in the alerts summary. When you click **Dismiss alert** you will be prompted to choose a reason for closing the alert.
|
||||
{% ifversion comment-dismissed-code-scanning-alert %}
|
||||
|
||||
|
||||
{% else %}
|
||||
|
||||
{% endif %}
|
||||
|
||||
@@ -13,4 +13,10 @@ redirect_from:
|
||||
|
||||
{% data reusables.code-scanning.codeql-action-version-ghes %}
|
||||
|
||||
If an automatic build of code for a compiled language within your project fails, you can try removing the `autobuild` step from your {% data variables.product.prodname_code_scanning %} workflow and adding specific build steps. If you're not already using advanced setup, you'll need to enable it first to create a workflow you can edit. For more information about advanced setup, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning)." For more information about specifically replacing the `autobuild` step, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
||||
If an automatic build of code for a compiled language within your project fails, you can try {% ifversion codeql-no-build %}changing to the `manual` build mode or {% endif %}removing the `autobuild` step from your {% data variables.product.prodname_code_scanning %} workflow and adding specific build steps. If you're not already using advanced setup, you'll need to enable it first to create a workflow you can edit.
|
||||
|
||||
## Further reading
|
||||
|
||||
- "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning)."{% ifversion codeql-no-build %}
|
||||
- "[CodeQL build modes](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes)"{% elsif ghes %}
|
||||
- "[Adding build steps for a compiled language](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."{% endif %}
|
||||
|
||||
@@ -13,18 +13,18 @@ redirect_from:
|
||||
|
||||
## About analysis of compiled languages
|
||||
|
||||
For compiled languages like C/C++, C#, Go, and Java, {% data variables.product.prodname_codeql %} only scans files that are built during the analysis. Therefore the number of lines of code scanned will be lower than expected if some of the source code isn't compiled correctly. This can happen for several reasons:
|
||||
{% ifversion codeql-no-build %}When compiled languages are analyzed using the `autobuild` or `manual` build mode,{% elsif ghes %}For compiled languages like {% data variables.code-scanning.compiled_languages %},{% endif %} {% data variables.product.prodname_codeql %} only scans files that are built during the analysis. Therefore the number of lines of code scanned will be lower than expected if some of the source code isn't compiled correctly. This can happen for several reasons:
|
||||
|
||||
1. The {% data variables.product.prodname_codeql %} `autobuild` feature uses heuristics to build the code in a repository. However, sometimes this approach results in an incomplete analysis of a repository. For example, when multiple `build.sh` commands exist in a single repository, the analysis may not be complete since the `autobuild` step will only execute one of the commands, and therefore some source files may not be compiled.
|
||||
|
||||
1. Some compilers do not work with {% data variables.product.prodname_codeql %} and can cause issues while analyzing the code. For example, most vendor-specific C compilers will not be recognized by {% data variables.product.prodname_codeql %}. C code will need to be compiled with a recognized compiler (for example GCC, Clang or MSVC) in order to be analyzed.
|
||||
|
||||
If your {% data variables.product.prodname_codeql %} analysis scans fewer lines of code than expected, you can try replacing the `autobuild step`, or inspecting the copy of the source files in the {% data variables.product.prodname_codeql %} database.
|
||||
If your {% data variables.product.prodname_codeql %} analysis scans fewer lines of code than expected, you can try {% ifversion codeql-no-build %}changing the build mode to `manual` and specifying build commands if your workflow specifies a build mode, {% endif %} replacing the `autobuild` step with build commands if your workflow contains an `autobuild` step, or inspecting the copy of the source files in the {% data variables.product.prodname_codeql %} database.
|
||||
|
||||
## Replace the `autobuild` step
|
||||
## {% ifversion codeql-no-build %}Change to a `manual` build process{% elsif ghes %}Replace the `autobuild` step{% endif %}
|
||||
|
||||
Replace the `autobuild` step with the same build commands you would use in production. This makes sure that {% data variables.product.prodname_codeql %} knows exactly how to compile all of the source files you want to scan.
|
||||
For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
||||
Replace the `autobuild` process with the same build commands you would use in production. This makes sure that {% data variables.product.prodname_codeql %} knows exactly how to compile all of the source files you want to scan.
|
||||
For more information about defining build steps, see {% ifversion codeql-no-build %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#using-build-mode-manual-and-specifying-build-steps){% elsif ghes %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language){% endif %}."
|
||||
|
||||
## Inspect the copy of the source files in the {% data variables.product.prodname_codeql %} database
|
||||
|
||||
|
||||
@@ -35,4 +35,5 @@ children:
|
||||
- /two-codeql-workflows
|
||||
- /unclear-what-triggered-a-workflow
|
||||
- /unnecessary-step-found
|
||||
- /kotlin-detected-in-no-build
|
||||
---
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
title: 'Warning: Detected X Kotlin files in your project that could not be processed without a build'
|
||||
shortTitle: Kotlin detected in no build
|
||||
allowTitleToDifferFromFilename: true
|
||||
intro: '{% data variables.product.prodname_codeql %} databases can be created for {% data variables.code-scanning.no_build_support %} without building the code, but Kotlin files are excluded unless the code is built.'
|
||||
versions:
|
||||
feature: codeql-no-build
|
||||
---
|
||||
|
||||
{% data reusables.code-scanning.beta-no-build %}
|
||||
|
||||
## About this warning
|
||||
|
||||
```text
|
||||
Warning: Detected X Kotlin files in your project that could not be processed without a build. To process these files...
|
||||
```
|
||||
|
||||
This warning is reported when Kotlin files are detected in a repository that ran {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} for Java using the build mode of `none` (default setup), or if you run the {% data variables.product.prodname_codeql_cli %} using `--build-mode none` for a repository containing Java and Kotlin files.
|
||||
|
||||
## Confirming the cause of the warning
|
||||
|
||||
This warning is only displayed when the build mode of `none` is used for a repository with both Java and Kotlin files.
|
||||
|
||||
The {% data variables.product.prodname_codeql %} action and {% data variables.product.prodname_codeql_cli %} support a build mode of `none` for {% data variables.code-scanning.no_build_support %}. This provides an easy way to enable analysis for Java code without building the codebase. However, Kotlin files are not included in the resulting {% data variables.product.prodname_codeql %} database.
|
||||
|
||||
You can verify the presence of Kotlin files by looking at the repository or pull request that triggered the warning. The `none` build mode is used only in the following circumstances:
|
||||
|
||||
- {% data variables.product.prodname_code_scanning_caps %} was enabled for the repository before Kotlin code was added and after the new mode was introduced (previously it would have used the `autobuild` mode).
|
||||
- The {% data variables.product.prodname_codeql %} workflow specifies a build mode of `none` for the repository (check for `build-mode: none`).
|
||||
- The {% data variables.product.prodname_codeql_cli %} is called without a `--command` and with `--build-mode none`.
|
||||
|
||||
## Fixing the problem
|
||||
|
||||
You may not want to analyze the Kotlin files, in which case you can ignore the warning message.
|
||||
|
||||
If you want to update the analysis to also include Kotlin files, then {% data variables.product.prodname_codeql %} will need to build the Java and Kotlin code.
|
||||
|
||||
### {% data variables.product.prodname_code_scanning_caps %} default setup
|
||||
|
||||
1. Wait until the Kotlin code is merged into the default branch for the repository.
|
||||
1. Disable and then re-enable default setup on the "Settings" page for your repository.
|
||||
|
||||
This will trigger a new analysis using automatic build detection. See "[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)" and "[Building Java and Kotlin](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#building-java--and-kotlin)."
|
||||
|
||||
If the automatic build detection fails, you will need to use advanced setup with the correct build commands for the project to analyze both languages.
|
||||
|
||||
### {% data variables.product.prodname_code_scanning_caps %} advanced setup
|
||||
|
||||
If you already use advanced setup, you can edit the {% data variables.product.prodname_codeql %} workflow and change the build mode for `java-kotlin` from `none` to either `autobuild` to automatically build your project, or `manual` to specify your own build steps. "[Building Java and Kotlin](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#building-java--and-kotlin)."
|
||||
|
||||
If you need to convert from default setup to advanced setup, you need enable advanced setup on the on the "Settings" page for your repository and create a {% data variables.product.prodname_codeql %} workflow. Then you can define a `manual` build mode for `java-kotlin` and define the build commands for the project.
|
||||
|
||||
### Running the {% data variables.product.prodname_codeql_cli %} directly
|
||||
|
||||
Update your calls to run the {% data variables.product.prodname_codeql_cli %} for the repository and pull requests to replace `--build-mode none` by `--build-mode autobuild` to try the automatic build detection. If automatic build detection is unsuccessful, remove the `--build-mode` option and include one or more `--command` options detailing the build script or steps required to build the project.
|
||||
|
||||
## Further reading
|
||||
|
||||
- "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning)"
|
||||
- "[Building Java and Kotlin](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#building-java--and-kotlin){% ifversion codeql-no-build %}
|
||||
- "[CodeQL build modes](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes)"{% elsif ghes %}
|
||||
- "[Adding build steps for a compiled language](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)"{% endif %}
|
||||
@@ -13,9 +13,9 @@ versions:
|
||||
|
||||
If your workflow fails with `Error: "No source code was seen during the build"` or `The process '/opt/hostedtoolcache/CodeQL/0.0.0-20200630/x64/codeql/codeql' failed with exit code 32`, this indicates that {% data variables.product.prodname_codeql %} was unable to monitor your code. There are six possible reasons for this:
|
||||
|
||||
1. The repository may not contain source code that is written in languages supported by {% data variables.product.prodname_codeql %}. Check the list of supported languages and, if this is the case, remove the {% data variables.product.prodname_codeql %} workflow. For more information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql#about-codeql)."
|
||||
1. _No supported languages:_ The repository may not contain source code that is written in languages supported by {% data variables.product.prodname_codeql %}. Check the list of supported languages and, if this is the case, remove the {% data variables.product.prodname_codeql %} workflow. For more information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql#about-codeql)."
|
||||
|
||||
1. Automatic language detection identified a supported language, but there is no analyzable code of that language in the repository. A typical example is when our language detection service finds a file associated with a particular programming language like a `.h`, or `.gyp` file, but no corresponding executable code is present in the repository. To solve the problem, you can manually define the languages you want to analyze by updating the list of languages in the `language` matrix. For example, the following configuration will analyze only Go, and JavaScript.
|
||||
1. _No analyzable code of the detected languages:_ Automatic language detection identified a supported language, but there is no analyzable code of that language in the repository. A typical example is when our language detection service finds a file associated with a particular programming language like a `.h`, or `.gyp` file, but no corresponding executable code is present in the repository. To solve the problem, you can manually define the languages you want to analyze by updating the list of languages in the `language` matrix. For example, the following configuration will analyze only Go, and JavaScript.
|
||||
|
||||
```yaml
|
||||
strategy:
|
||||
@@ -28,13 +28,13 @@ If your workflow fails with `Error: "No source code was seen during the build"`
|
||||
|
||||
For more information, see the workflow extract in "[AUTOTITLE](/code-security/code-scanning/troubleshooting-code-scanning/some-languages-were-not-analyzed)".
|
||||
|
||||
1. Your {% data variables.product.prodname_code_scanning %} workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but the code was not compiled. By default, the {% data variables.product.prodname_codeql %} analysis workflow contains an `autobuild` step, however, this step represents a best effort process, and may not succeed in building your code, depending on your specific build environment. Compilation may also fail if you have removed the `autobuild` step and did not include build steps manually. For more information about specifying build steps, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
|
||||
1. _Compilation of a compiled language failed:_ Your {% data variables.product.prodname_code_scanning %} workflow tries to compile a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but the code was not compiled. {% ifversion codeql-no-build %}When a workflow specifies `build-mode: autobuild` for a language or contains an `autobuild` step,{% elsif ghes %}By default, the {% data variables.product.prodname_codeql %} analysis workflow contains an `autobuild` step and{% endif %} {% data variables.product.prodname_codeql %} makes a best effort to detect a suitable build method and build your code. The `autobuild` process may not succeed in building your code, depending on your specific build environment. Compilation may also fail if you have removed the `autobuild` step and did not include build steps manually. For more information about defining build steps, see {% ifversion codeql-no-build %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#using-build-mode-manual-and-specifying-build-steps){% elsif ghes %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language){% endif %}."
|
||||
|
||||
1. Your workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but portions of your build are cached to improve performance (most likely to occur with build systems like Gradle or Bazel). Since {% data variables.product.prodname_codeql %} observes the activity of the compiler to understand the data flows in a repository, {% data variables.product.prodname_codeql %} requires a complete build to take place in order to perform analysis.
|
||||
1. _Cached components not detected:_ Your workflow builds a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java) to create a {% data variables.product.prodname_codeql %} database for analysis, but portions of your build are cached to improve performance (most likely to occur with build systems like Gradle or Bazel). Since {% data variables.product.prodname_codeql %} observes the activity of the compiler to understand the data flows in a repository, {% data variables.product.prodname_codeql %} requires a complete build to take place in order to perform analysis.
|
||||
|
||||
1. Your workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but compilation does not occur between the `init` and `analyze` steps in the workflow. {% data variables.product.prodname_codeql %} requires that your build happens in between these two steps in order to observe the activity of the compiler and perform analysis.
|
||||
1. _Compilation outside `init` and `analyze` steps:_ Your workflow builds a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but compilation does not occur between the `init` and `analyze` steps in the workflow. {% data variables.product.prodname_codeql %} requires that your build happens in between these two steps in order to observe the activity of the compiler and perform analysis.
|
||||
|
||||
1. Your compiled code (in C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java) was compiled successfully, but {% data variables.product.prodname_codeql %} was unable to detect the compiler invocations. The most common causes are:
|
||||
1. _Compilation not detected by {% data variables.product.prodname_codeql %}:_ Your compiled code (in C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java) was compiled successfully, but {% data variables.product.prodname_codeql %} was unable to detect the compiler invocations. The most common causes are:
|
||||
|
||||
- Running your build process in a separate container to {% data variables.product.prodname_codeql %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/running-codeql-code-scanning-in-a-container)."
|
||||
- Building using a distributed build system external to GitHub Actions, using a daemon process.
|
||||
|
||||
@@ -41,7 +41,7 @@ You can use the {% data variables.product.prodname_codeql_cli %} to:
|
||||
The {% data variables.product.prodname_codeql_cli %} can analyze:
|
||||
|
||||
- Dynamic languages, for example, JavaScript and Python.
|
||||
- Compiled languages, for example, C/C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} and Java.
|
||||
- Compiled languages, for example, {% data variables.code-scanning.compiled_languages %}
|
||||
- Codebases written in a mixture of languages.
|
||||
|
||||
For information about setting up the {% data variables.product.prodname_codeql_cli %}, see
|
||||
|
||||
@@ -36,10 +36,10 @@ Before you generate a {% data variables.product.prodname_codeql %} database, you
|
||||
1. Check out the code that you want to analyze:
|
||||
- For a branch, check out the head of the branch that you want to analyze.
|
||||
- For a pull request, check out either the head commit of the pull request, or check out a {% data variables.product.prodname_dotcom %}-generated merge commit of the pull request.
|
||||
1. Set up the environment for the codebase, making sure that any dependencies are available. For more information, see "[Creating databases for non-compiled languages](#creating-databases-for-non-compiled-languages)" and "[Creating databases for compiled languages](#creating-databases-for-compiled-languages)."
|
||||
1. Find the build command, if any, for the codebase. Typically this is available in a configuration file in the CI system.
|
||||
1. Set up the environment for the codebase, making sure that any dependencies are available.
|
||||
1. For the best results with compiled languages, find the build command, if any, for the codebase. Typically this is available in a configuration file in the CI system.
|
||||
|
||||
Once the codebase is ready, you can run `codeql database create` to create the database.
|
||||
Once the codebase is ready, you can run `codeql database create` to create the database. For more information, see "[Creating databases for non-compiled languages](#creating-databases-for-non-compiled-languages)" and "[Creating databases for compiled languages](#creating-databases-for-compiled-languages)."
|
||||
|
||||
## Running `codeql database create`
|
||||
|
||||
@@ -54,8 +54,9 @@ You must specify:
|
||||
- `<database>`: a path to the new database to be created. This directory will be created when you execute the command—you cannot specify an existing directory.
|
||||
- `--language`: the identifier for the language to create a database for. When used with `--db-cluster`, the option accepts a comma-separated list, or can be specified more than once. {% data variables.product.prodname_codeql %} supports creating databases for the following languages:
|
||||
|
||||
{% data reusables.code-scanning.codeql-language-identifiers-table %}
|
||||
{% data reusables.code-scanning.beta-kotlin-or-swift-support %}
|
||||
{% data reusables.code-scanning.codeql-language-identifiers-table %}
|
||||
|
||||
{% data reusables.code-scanning.beta-kotlin-or-swift-support %}
|
||||
|
||||
If your codebase has a build command or script that invokes the build process, we recommend that you specify it as well:
|
||||
|
||||
@@ -64,6 +65,8 @@ If your codebase has a build command or script that invokes the build process, w
|
||||
--language=<language-identifier>
|
||||
```
|
||||
|
||||
### Options for creating databases
|
||||
|
||||
You can specify additional options depending on the location of your source file, if the code needs to be compiled, and if you want to create {% data variables.product.prodname_codeql %} databases for more than one language.
|
||||
|
||||
| Option | Required | Usage |
|
||||
@@ -71,7 +74,8 @@ You can specify additional options depending on the location of your source file
|
||||
| `<database>` | {% octicon "check" aria-label="Required" %} | Specify the name and location of a directory to create for the {% data variables.product.prodname_codeql %} database. The command will fail if you try to overwrite an existing directory. If you also specify `--db-cluster`, this is the parent directory and a subdirectory is created for each language analyzed. | {% ifversion codeql-language-identifiers-311 %}
|
||||
| <code><span style="white-space: nowrap;">--language</span></code> | {% octicon "check" aria-label="Required" %} | Specify the identifier for the language to create a database for, one of: {% data reusables.code-scanning.codeql-languages-keywords %}. When used with <code><span style="white-space: nowrap;">--db-cluster</span></code>, the option accepts a comma-separated list, or can be specified more than once. | {% else %}
|
||||
| <code><span style="white-space: nowrap;">--language</span></code> | {% octicon "check" aria-label="Required" %} | Specify the identifier for the language to create a database for, one of: {% data reusables.code-scanning.codeql-languages-keywords %} (use `javascript` to analyze TypeScript code {% ifversion codeql-kotlin-beta %} and `java` to analyze Kotlin code{% endif %}). When used with <code><span style="white-space: nowrap;">--db-cluster</span></code>, the option accepts a comma-separated list, or can be specified more than once. | {% endif %}
|
||||
| <code><span style="white-space: nowrap;">--command</span></code> | {% octicon "x" aria-label="Optional" %} | **Recommended.** Use to specify the build command or script that invokes the build process for the codebase. Commands are run from the current folder or, where it is defined, from <code><span style="white-space: nowrap;">--source-root</span></code>. Not needed for Python and JavaScript/TypeScript analysis. |
|
||||
| <code><span style="white-space: nowrap;">--command</span></code> | {% octicon "x" aria-label="Optional" %} | **Recommended.** Use to specify the build command or script that invokes the build process for the codebase. Commands are run from the current folder or, where it is defined, from <code><span style="white-space: nowrap;">--source-root</span></code>. Not needed for Python and JavaScript/TypeScript analysis. |{% ifversion codeql-no-build %}
|
||||
| <code><span style="white-space: nowrap;">--build-mode</span></code> | {% octicon "x" aria-label="Optional" %} | **Beta.** Use for {% data variables.code-scanning.no_build_support %} when not providing a `--command` to specify whether to create a CodeQL database without a build (`none`) or by attempting to automatically detect a build command (`autobuild`). By default, autobuild detection is used. For a comparison of build modes, see "[CodeQL build modes](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes)." |{% endif %}
|
||||
| <code><span style="white-space: nowrap;">--db-cluster</span></code> | {% octicon "x" aria-label="Optional" %} | Use in multi-language codebases to generate one database for each language specified by <code><span style="white-space: nowrap;">--language</span></code>. |
|
||||
| <code><span style="white-space: nowrap;">--no-run-unnecessary-builds</span></code> | {% octicon "x" aria-label="Optional" %} | **Recommended.** Use to suppress the build command for languages where the {% data variables.product.prodname_codeql_cli %} does not need to monitor the build (for example, Python and JavaScript/TypeScript). |
|
||||
| <code><span style="white-space: nowrap;">--source-root</span></code> | {% octicon "x" aria-label="Optional" %} | Use if you run the CLI outside the checkout root of the repository. By default, the `database create` command assumes that the current directory is the root directory for the source files, use this option to specify a different location. |
|
||||
@@ -133,7 +137,7 @@ $
|
||||
|
||||
## Progress and results
|
||||
|
||||
Errors are reported if there are any problems with the options you have specified. For interpreted languages, the extraction progress is displayed in the console. For each source file, the console shows if extraction was successful or if it failed. For compiled languages, the console will display the output of the build system.
|
||||
Errors are reported if there are any problems with the options you have specified. For interpreted languages and when you specify `--build-mode none` for {% data variables.code-scanning.no_build_support %}, the extraction progress is displayed in the console. For each source file, the console shows if extraction was successful or if it failed. When a compiled language is built, the console will display the output of the build system.
|
||||
|
||||
When the database is successfully created, you’ll find a new directory at the path specified in the command. If you used the `--db-cluster` option to create more than one database, a subdirectory is created for each language. Each {% data variables.product.prodname_codeql %} database directory contains a number of subdirectories, including the relational data (required for analysis) and a source archive—a copy of the source files made at the time the database was created—which is used for displaying analysis results.
|
||||
|
||||
@@ -188,17 +192,26 @@ Here, we have specified a `--source-root` path, which is the location where data
|
||||
|
||||
## Creating databases for compiled languages
|
||||
|
||||
For compiled languages, {% data variables.product.prodname_codeql %} needs to invoke the required build system to generate a database, therefore the build method must be available to the CLI.
|
||||
{% data reusables.code-scanning.beta-no-build-cli %}
|
||||
|
||||
### Detecting the build system
|
||||
For {% ifversion codeql-no-build %}most{% endif %} compiled languages, {% data variables.product.prodname_codeql %} needs to invoke the required build system to generate a database, therefore the build method must be available to the CLI. This approach creates databases that include generated code. {% data variables.product.prodname_codeql %} has two methods for building codebases:
|
||||
|
||||
The {% data variables.product.prodname_codeql_cli %} includes autobuilders for {% data variables.code-scanning.compiled_languages %} code. {% data variables.product.prodname_codeql %} autobuilders allow you to build projects for compiled languages without specifying any build commands. When an autobuilder is invoked, {% data variables.product.prodname_codeql %} examines the source for evidence of a build system and attempts to run the optimal set of commands required to extract a database. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#about-autobuild-for-codeql)."
|
||||
- "[Automatic build detection (autobuild)](#automatically-detecting-the-build-system)"
|
||||
- "[User-specified build commands](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis#specifying-build-commands)
|
||||
|
||||
An autobuilder is invoked automatically when you execute `codeql database create` for a compiled `--language` if you don’t include a
|
||||
`--command` option. For example, for a Java codebase, you would simply run:
|
||||
{% ifversion codeql-no-build %}
|
||||
In addition, for {% data variables.code-scanning.no_build_support %}, there is an option to generate a database without building the code. This is particularly useful when you want to enable {% data variables.product.prodname_code_scanning %} for many repositories. For more information, see "[CodeQL build modes](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes)."
|
||||
{% endif %}
|
||||
|
||||
### Automatically detecting the build system
|
||||
|
||||
The {% data variables.product.prodname_codeql_cli %} includes autobuilders for {% data variables.code-scanning.compiled_languages %} code. {% data variables.product.prodname_codeql %} autobuilders allow you to build projects for compiled languages without specifying any build commands. When an autobuilder is invoked, {% data variables.product.prodname_codeql %} examines the source for evidence of a build system and attempts to run the optimal set of commands required to extract a database. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#about-autobuild)."
|
||||
|
||||
An autobuilder is invoked automatically when you execute `codeql database create` for a compiled language if you don’t include a
|
||||
`--command` option{% ifversion codeql-no-build %} or set `--build-mode none`{% endif %}. For example, for a C# codebase, you could simply run:
|
||||
|
||||
```shell
|
||||
codeql database create --language={% ifversion codeql-language-identifiers-311 %}java-kotlin{% else %}java{% endif %} <output-folder>/java-database
|
||||
codeql database create --language=csharp <output-folder>/csharp-database
|
||||
```
|
||||
|
||||
If a codebase uses a standard build system, relying on an autobuilder is often the simplest way to create a database. For sources that require non-standard build steps, you may need to explicitly define each step in the command line.
|
||||
|
||||
6
data/features/codeql-no-build.yml
Normal file
6
data/features/codeql-no-build.yml
Normal file
@@ -0,0 +1,6 @@
|
||||
# Reference: #12924 (Java)
|
||||
|
||||
versions:
|
||||
fpt: '*'
|
||||
ghec: '*'
|
||||
ghes: '>=3.13'
|
||||
@@ -1 +0,0 @@
|
||||
{% data variables.product.product_name %} provides starter workflows for security features such as {% data variables.product.prodname_code_scanning %}. You can use these suggested workflows to construct your {% data variables.product.prodname_code_scanning %} workflows, instead of starting from scratch.
|
||||
@@ -1 +1,6 @@
|
||||
If `autobuild` fails, or you want to analyze a different set of source files from those built by the `autobuild` process, you'll need to remove the `autobuild` step from the workflow, and manually add build steps. For C/C++, C#, Go, {% ifversion codeql-kotlin-beta %} Kotlin, {% endif %}{% ifversion codeql-swift-beta %}Java, and Swift projects, {% else %} and Java projects,{% endif %} {% data variables.product.prodname_codeql %} will analyze whatever source code is built by your specified build steps.
|
||||
If `autobuild` fails, or you want to analyze a different set of source files from those built by the `autobuild` process, you'll need to {% ifversion codeql-no-build %}do the following:
|
||||
|
||||
- If your workflow specifies a build mode for the language, change the build mode to `manual`.
|
||||
- If your workflow contains an `autobuild` step, remove or comment out the `autobuild` step in the workflow.
|
||||
|
||||
{% else %}remove or comment out the autobuild step in the workflow.{% endif %} Then uncomment the `run` step and manually specify the build process to use. For {% data variables.code-scanning.compiled_languages %}, {% data variables.product.prodname_codeql %} will analyze whatever source code is built by your specified build steps.
|
||||
|
||||
@@ -1,8 +1,6 @@
|
||||
{% data variables.product.prodname_codeql %} analyzes the {% data variables.code-scanning.compiled_languages %} source files in your repository that are built.
|
||||
|
||||
{% ifversion code-scanning-without-workflow-310 %}
|
||||
|
||||
If you enable default setup, the `autobuild` action will be used to build your code, as part of your automatically configured {% data variables.code-scanning.codeql_workflow %}. If you enable advanced setup, the basic {% data variables.code-scanning.codeql_workflow %} uses `autobuild`. Alternatively, you can disable `autobuild` and instead specify explicit build commands to analyze only the files that are built by these custom commands.
|
||||
If you enable default setup, the `autobuild` build mode will be used to build compiled languages, as part of your automatically configured {% data variables.code-scanning.codeql_workflow %}. If you enable advanced setup, the basic {% data variables.code-scanning.codeql_workflow %} uses the `autobuild` build mode. Alternatively, you can change the build mode for your compiled languages to `manual` and instead specify explicit build commands to analyze only the files that are built by these custom commands.
|
||||
|
||||
{% else %}
|
||||
|
||||
|
||||
9
data/reusables/code-scanning/beta-no-build-cli.md
Normal file
9
data/reusables/code-scanning/beta-no-build-cli.md
Normal file
@@ -0,0 +1,9 @@
|
||||
{% ifversion codeql-no-build %}
|
||||
|
||||
{% note %}
|
||||
|
||||
**Note:** The option to create a {% data variables.product.prodname_codeql %} database for a compiled language without building it is currently in beta and subject to change. During the beta, this option is supported only for {% data variables.code-scanning.no_build_support %} codebases.
|
||||
|
||||
{% endnote %}
|
||||
|
||||
{% endif %}
|
||||
9
data/reusables/code-scanning/beta-no-build.md
Normal file
9
data/reusables/code-scanning/beta-no-build.md
Normal file
@@ -0,0 +1,9 @@
|
||||
{% ifversion codeql-no-build %}
|
||||
|
||||
{% note %}
|
||||
|
||||
**Note:** The option to analyze a compiled language without building it using {% data variables.product.prodname_codeql %} is currently in beta and subject to change. During the beta, this option is supported only for {% data variables.code-scanning.no_build_support %} codebases.
|
||||
|
||||
{% endnote %}
|
||||
|
||||
{% endif %}
|
||||
@@ -6,6 +6,9 @@ codeql_workflow: 'CodeQL analysis workflow'
|
||||
# The tool status page
|
||||
tool_status_page: 'tool status page'
|
||||
|
||||
# List of compiled languages supported for `no-build` extraction
|
||||
no_build_support: 'Java'
|
||||
|
||||
# List of compiled languages
|
||||
compiled_languages: 'C/C++, C#, {% ifversion codeql-go-autobuild %} Go,{% endif %} {% ifversion codeql-swift-beta %} Java, and Swift{% else %} and Java{% endif %}'
|
||||
|
||||
|
||||
Reference in New Issue
Block a user