2024-03-26: Updates for the release of CodeQL analysis of Java without building it (ship delayed) (#49452)

Co-authored-by: Henry Mercer <henrymercer@github.com>
Co-authored-by: Ben Ahmady <32935794+subatoi@users.noreply.github.com>
Co-authored-by: Florin Coada <coadaflorin@github.com>

content/code-security/code-scanning/troubleshooting-code-scanning/automatic-build-failed.md

@@ -13,4 +13,10 @@ redirect_from:
 
 {% data reusables.code-scanning.codeql-action-version-ghes %}
 
-If an automatic build of code for a compiled language within your project fails, you can try removing the `autobuild` step from your {% data variables.product.prodname_code_scanning %} workflow and adding specific build steps. If you're not already using advanced setup, you'll need to enable it first to create a workflow you can edit. For more information about advanced setup, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning)." For more information about specifically replacing the `autobuild` step, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
+If an automatic build of code for a compiled language within your project fails, you can try {% ifversion codeql-no-build %}changing to the `manual` build mode or {% endif %}removing the `autobuild` step from your {% data variables.product.prodname_code_scanning %} workflow and adding specific build steps. If you're not already using advanced setup, you'll need to enable it first to create a workflow you can edit.
+
+## Further reading
+
+- "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning)."{% ifversion codeql-no-build %}
+- "[CodeQL build modes](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes)"{% elsif ghes %}
+- "[Adding build steps for a compiled language](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."{% endif %}

content/code-security/code-scanning/troubleshooting-code-scanning/fewer-lines-scanned-than-expected.md

@@ -13,18 +13,18 @@ redirect_from:
 
 ## About analysis of compiled languages
 
-For compiled languages like C/C++, C#, Go, and Java, {% data variables.product.prodname_codeql %} only scans files that are built during the analysis. Therefore the number of lines of code scanned will be lower than expected if some of the source code isn't compiled correctly. This can happen for several reasons:
+{% ifversion codeql-no-build %}When compiled languages are analyzed using the `autobuild` or `manual` build mode,{% elsif ghes %}For compiled languages like {% data variables.code-scanning.compiled_languages %},{% endif %} {% data variables.product.prodname_codeql %} only scans files that are built during the analysis. Therefore the number of lines of code scanned will be lower than expected if some of the source code isn't compiled correctly. This can happen for several reasons:
 
 1. The {% data variables.product.prodname_codeql %} `autobuild` feature uses heuristics to build the code in a repository. However, sometimes this approach results in an incomplete analysis of a repository. For example, when multiple `build.sh` commands exist in a single repository, the analysis may not be complete since the `autobuild` step will only execute one of the commands, and therefore some source files may not be compiled.
 
 1. Some compilers do not work with {% data variables.product.prodname_codeql %} and can cause issues while analyzing the code. For example, most vendor-specific C compilers will not be recognized by {% data variables.product.prodname_codeql %}. C code will need to be compiled with a recognized compiler (for example GCC, Clang or MSVC) in order to be analyzed.
 
-If your {% data variables.product.prodname_codeql %} analysis scans fewer lines of code than expected, you can try replacing the `autobuild step`, or inspecting the copy of the source files in the {% data variables.product.prodname_codeql %} database.
+If your {% data variables.product.prodname_codeql %} analysis scans fewer lines of code than expected, you can try {% ifversion codeql-no-build %}changing the build mode to `manual` and specifying build commands if your workflow specifies a build mode, {% endif %} replacing the `autobuild` step with build commands if your workflow contains an `autobuild` step, or inspecting the copy of the source files in the {% data variables.product.prodname_codeql %} database.
 
-## Replace the `autobuild` step
+## {% ifversion codeql-no-build %}Change to a `manual` build process{% elsif ghes %}Replace the `autobuild` step{% endif %}
 
-Replace the `autobuild` step with the same build commands you would use in production. This makes sure that {% data variables.product.prodname_codeql %} knows exactly how to compile all of the source files you want to scan.
-For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
+Replace the `autobuild` process with the same build commands you would use in production. This makes sure that {% data variables.product.prodname_codeql %} knows exactly how to compile all of the source files you want to scan.
+For more information about defining build steps, see {% ifversion codeql-no-build %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#using-build-mode-manual-and-specifying-build-steps){% elsif ghes %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language){% endif %}."
 
 ## Inspect the copy of the source files in the {% data variables.product.prodname_codeql %} database
 

content/code-security/code-scanning/troubleshooting-code-scanning/index.md

@@ -35,4 +35,5 @@ children:
   - /two-codeql-workflows
   - /unclear-what-triggered-a-workflow
   - /unnecessary-step-found
+  - /kotlin-detected-in-no-build
 ---

content/code-security/code-scanning/troubleshooting-code-scanning/kotlin-detected-in-no-build.md

@@ -0,0 +1,62 @@
+---
+title: 'Warning: Detected X Kotlin files in your project that could not be processed without a build'
+shortTitle: Kotlin detected in no build
+allowTitleToDifferFromFilename: true
+intro: '{% data variables.product.prodname_codeql %} databases can be created for {% data variables.code-scanning.no_build_support %} without building the code, but Kotlin files are excluded unless the code is built.'
+versions:
+  feature: codeql-no-build
+---
+
+{% data reusables.code-scanning.beta-no-build %}
+
+## About this warning
+
+```text
+Warning: Detected X Kotlin files in your project that could not be processed without a build. To process these files...
+```
+
+This warning is reported when Kotlin files are detected in a repository that ran {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} for Java using the build mode of `none` (default setup), or if you run the {% data variables.product.prodname_codeql_cli %} using `--build-mode none` for a repository containing Java and Kotlin files.
+
+## Confirming the cause of the warning
+
+This warning is only displayed when the build mode of `none` is used for a repository with both Java and Kotlin files.
+
+The {% data variables.product.prodname_codeql %} action and {% data variables.product.prodname_codeql_cli %} support a build mode of `none` for {% data variables.code-scanning.no_build_support %}. This provides an easy way to enable analysis for Java code without building the codebase. However, Kotlin files are not included in the resulting {% data variables.product.prodname_codeql %} database.
+
+You can verify the presence of Kotlin files by looking at the repository or pull request that triggered the warning. The `none` build mode is used only in the following circumstances:
+
+- {% data variables.product.prodname_code_scanning_caps %} was enabled for the repository before Kotlin code was added and after the new mode was introduced (previously it would have used the `autobuild` mode).
+- The {% data variables.product.prodname_codeql %} workflow specifies a build mode of `none` for the repository (check for `build-mode: none`).
+- The {% data variables.product.prodname_codeql_cli %} is called without a `--command` and with `--build-mode none`.
+
+## Fixing the problem
+
+You may not want to analyze the Kotlin files, in which case you can ignore the warning message.
+
+If you want to update the analysis to also include Kotlin files, then {% data variables.product.prodname_codeql %} will need to build the Java and Kotlin code.
+
+### {% data variables.product.prodname_code_scanning_caps %} default setup
+
+1. Wait until the Kotlin code is merged into the default branch for the repository.
+1. Disable and then re-enable default setup on the "Settings" page for your repository.
+
+This will trigger a new analysis using automatic build detection. See "[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)" and "[Building Java and Kotlin](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#building-java--and-kotlin)."
+
+If the automatic build detection fails, you will need to use advanced setup with the correct build commands for the project to analyze both languages.
+
+### {% data variables.product.prodname_code_scanning_caps %} advanced setup
+
+If you already use advanced setup, you can edit the {% data variables.product.prodname_codeql %} workflow and change the build mode for `java-kotlin` from `none` to either `autobuild` to automatically build your project, or `manual` to specify your own build steps. "[Building Java and Kotlin](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#building-java--and-kotlin)."
+
+If you need to convert from default setup to advanced setup, you need enable advanced setup on the on the "Settings" page for your repository and create a {% data variables.product.prodname_codeql %} workflow. Then you can define a `manual` build mode for `java-kotlin` and define the build commands for the project.
+
+### Running the {% data variables.product.prodname_codeql_cli %} directly
+
+Update your calls to run the {% data variables.product.prodname_codeql_cli %} for the repository and pull requests to replace `--build-mode none` by `--build-mode autobuild` to try the automatic build detection. If automatic build detection is unsuccessful, remove the `--build-mode` option and include one or more `--command` options detailing the build script or steps required to build the project.
+
+## Further reading
+
+- "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning)"
+- "[Building Java and Kotlin](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#building-java--and-kotlin){% ifversion codeql-no-build %}
+- "[CodeQL build modes](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#codeql-build-modes)"{% elsif ghes %}
+- "[Adding build steps for a compiled language](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)"{% endif %}

content/code-security/code-scanning/troubleshooting-code-scanning/no-source-code-seen-during-build.md

@@ -13,9 +13,9 @@ versions:
 
 If your workflow fails with `Error: "No source code was seen during the build"` or `The process '/opt/hostedtoolcache/CodeQL/0.0.0-20200630/x64/codeql/codeql' failed with exit code 32`, this indicates that {% data variables.product.prodname_codeql %} was unable to monitor your code. There are six possible reasons for this:
 
-1. The repository may not contain source code that is written in languages supported by {% data variables.product.prodname_codeql %}. Check the list of supported languages and, if this is the case, remove the {% data variables.product.prodname_codeql %} workflow. For more information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql#about-codeql)."
+1. _No supported languages:_ The repository may not contain source code that is written in languages supported by {% data variables.product.prodname_codeql %}. Check the list of supported languages and, if this is the case, remove the {% data variables.product.prodname_codeql %} workflow. For more information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql#about-codeql)."
 
-1. Automatic language detection identified a supported language, but there is no analyzable code of that language in the repository. A typical example is when our language detection service finds a file associated with a particular programming language like a `.h`, or `.gyp` file, but no corresponding executable code is present in the repository. To solve the problem, you can manually define the languages you want to analyze by updating the list of languages in the `language` matrix. For example, the following configuration will analyze only Go, and JavaScript.
+1. _No analyzable code of the detected languages:_ Automatic language detection identified a supported language, but there is no analyzable code of that language in the repository. A typical example is when our language detection service finds a file associated with a particular programming language like a `.h`, or `.gyp` file, but no corresponding executable code is present in the repository. To solve the problem, you can manually define the languages you want to analyze by updating the list of languages in the `language` matrix. For example, the following configuration will analyze only Go, and JavaScript.
 
    ```yaml
    strategy:
@@ -28,13 +28,13 @@ If your workflow fails with `Error: "No source code was seen during the build"`
 
    For more information, see the workflow extract in "[AUTOTITLE](/code-security/code-scanning/troubleshooting-code-scanning/some-languages-were-not-analyzed)".
 
-1. Your {% data variables.product.prodname_code_scanning %} workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but the code was not compiled. By default, the {% data variables.product.prodname_codeql %} analysis workflow contains an `autobuild` step, however, this step represents a best effort process, and may not succeed in building your code, depending on your specific build environment. Compilation may also fail if you have removed the `autobuild` step and did not include build steps manually.  For more information about specifying build steps, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language)."
+1. _Compilation of a compiled language failed:_ Your {% data variables.product.prodname_code_scanning %} workflow tries to compile a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but the code was not compiled. {% ifversion codeql-no-build %}When a workflow specifies `build-mode: autobuild` for a language or contains an `autobuild` step,{% elsif ghes %}By default, the {% data variables.product.prodname_codeql %} analysis workflow contains an `autobuild` step and{% endif %} {% data variables.product.prodname_codeql %} makes a best effort to detect a suitable build method and build your code. The `autobuild` process may not succeed in building your code, depending on your specific build environment. Compilation may also fail if you have removed the `autobuild` step and did not include build steps manually.  For more information about defining build steps, see {% ifversion codeql-no-build %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#using-build-mode-manual-and-specifying-build-steps){% elsif ghes %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#adding-build-steps-for-a-compiled-language){% endif %}."
 
-1. Your workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but portions of your build are cached to improve performance (most likely to occur with build systems like Gradle or Bazel). Since {% data variables.product.prodname_codeql %} observes the activity of the compiler to understand the data flows in a repository, {% data variables.product.prodname_codeql %} requires a complete build to take place in order to perform analysis.
+1. _Cached components not detected:_ Your workflow builds a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java) to create a {% data variables.product.prodname_codeql %} database for analysis, but portions of your build are cached to improve performance (most likely to occur with build systems like Gradle or Bazel). Since {% data variables.product.prodname_codeql %} observes the activity of the compiler to understand the data flows in a repository, {% data variables.product.prodname_codeql %} requires a complete build to take place in order to perform analysis.
 
-1. Your workflow is analyzing a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but compilation does not occur between the `init` and `analyze` steps in the workflow. {% data variables.product.prodname_codeql %} requires that your build happens in between these two steps in order to observe the activity of the compiler and perform analysis.
+1. _Compilation outside `init` and `analyze` steps:_ Your workflow builds a compiled language (C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java), but compilation does not occur between the `init` and `analyze` steps in the workflow. {% data variables.product.prodname_codeql %} requires that your build happens in between these two steps in order to observe the activity of the compiler and perform analysis.
 
-1. Your compiled code (in C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java) was compiled successfully, but {% data variables.product.prodname_codeql %} was unable to detect the compiler invocations. The most common causes are:
+1. _Compilation not detected by {% data variables.product.prodname_codeql %}:_ Your compiled code (in C, C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} or Java) was compiled successfully, but {% data variables.product.prodname_codeql %} was unable to detect the compiler invocations. The most common causes are:
 
    - Running your build process in a separate container to {% data variables.product.prodname_codeql %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/running-codeql-code-scanning-in-a-container)."
    - Building using a distributed build system external to GitHub Actions, using a daemon process.