diff --git a/content/admin/code-security/managing-github-advanced-security-for-your-enterprise/configuring-code-scanning-for-your-appliance.md b/content/admin/code-security/managing-github-advanced-security-for-your-enterprise/configuring-code-scanning-for-your-appliance.md index 0b39df9d97..979347c502 100644 --- a/content/admin/code-security/managing-github-advanced-security-for-your-enterprise/configuring-code-scanning-for-your-appliance.md +++ b/content/admin/code-security/managing-github-advanced-security-for-your-enterprise/configuring-code-scanning-for-your-appliance.md @@ -96,4 +96,4 @@ If you configure the {% data variables.product.prodname_codeql %} action sync to If you don't want to use {% data variables.product.prodname_actions %}, you should run {% data variables.product.prodname_code_scanning %} using the {% data variables.product.prodname_codeql_cli %}. -The {% data variables.product.prodname_codeql_cli %} is a command-line tool that you use to analyze codebases on any machine, including a third-party CI/CD system. For more information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system)." +The {% data variables.product.prodname_codeql_cli %} is a command-line tool that you use to analyze codebases on any machine, including a third-party CI/CD system. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)." diff --git a/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.md b/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.md index dfb2ca60f9..7deac6feaf 100644 --- a/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.md +++ b/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.md @@ -37,7 +37,7 @@ make sure that you also update the MS short link: https://aka.ms/code-scanning-d ## About {% data variables.product.prodname_code_scanning %} configuration -You can run {% data variables.product.prodname_code_scanning %} on {% data variables.product.product_name %}, using {% data variables.product.prodname_actions %}, or from your continuous integration (CI) system. For more information, see "[AUTOTITLE](/actions/learn-github-actions)" or "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system)." +You can run {% data variables.product.prodname_code_scanning %} on {% data variables.product.product_name %}, using {% data variables.product.prodname_actions %}, or from your continuous integration (CI) system. For more information, see "[AUTOTITLE](/actions/learn-github-actions)" or "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)." {% ifversion code-scanning-without-workflow %}With advanced setup for {% data variables.product.prodname_code_scanning %}, you can customize a {% data variables.product.prodname_code_scanning %} workflow for granular control over your configuration. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning)."{% else %}This article is about running {% data variables.product.prodname_code_scanning %} on {% data variables.product.product_name %} using actions.{% endif %} diff --git a/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/running-codeql-code-scanning-in-a-container.md b/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/running-codeql-code-scanning-in-a-container.md index 18dec8f97a..51c4e87319 100644 --- a/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/running-codeql-code-scanning-in-a-container.md +++ b/content/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/running-codeql-code-scanning-in-a-container.md @@ -32,7 +32,7 @@ topics: If you're configuring {% data variables.product.prodname_code_scanning %} for a compiled language, and you're building the code in a containerized environment, the analysis may fail with the error message "No source code was seen during the build." This indicates that {% data variables.product.prodname_codeql %} was unable to monitor your code as it was compiled. -You must run {% data variables.product.prodname_codeql %} inside the container in which you build your code. This applies whether you are using the {% data variables.product.prodname_codeql_cli %} or {% data variables.product.prodname_actions %}. For the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system)" for more information. If you're using {% data variables.product.prodname_actions %}, configure your workflow to run all the actions in the same container. For more information, see "[Example workflow](#example-workflow)." +You must run {% data variables.product.prodname_codeql %} inside the container in which you build your code. This applies whether you are using the {% data variables.product.prodname_codeql_cli %} or {% data variables.product.prodname_actions %}. For the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)" for more information. If you're using {% data variables.product.prodname_actions %}, configure your workflow to run all the actions in the same container. For more information, see "[Example workflow](#example-workflow)." {% note %} diff --git a/content/code-security/code-scanning/index.md b/content/code-security/code-scanning/index.md index 2aef6ed41a..3c6a8c085a 100644 --- a/content/code-security/code-scanning/index.md +++ b/content/code-security/code-scanning/index.md @@ -24,7 +24,6 @@ children: - /managing-code-scanning-alerts - /managing-your-code-scanning-configuration - /integrating-with-code-scanning - - /using-codeql-code-scanning-with-your-existing-ci-system - /troubleshooting-code-scanning - /troubleshooting-sarif-uploads --- diff --git a/content/code-security/code-scanning/integrating-with-code-scanning/about-integration-with-code-scanning.md b/content/code-security/code-scanning/integrating-with-code-scanning/about-integration-with-code-scanning.md index a259c51b18..3b1ee023b8 100644 --- a/content/code-security/code-scanning/integrating-with-code-scanning/about-integration-with-code-scanning.md +++ b/content/code-security/code-scanning/integrating-with-code-scanning/about-integration-with-code-scanning.md @@ -21,13 +21,12 @@ topics: - Integration --- +## About integration with code scanning {% data reusables.code-scanning.beta %} {% data reusables.code-scanning.enterprise-enable-code-scanning %} -As an alternative to running {% data variables.product.prodname_code_scanning %} within {% data variables.product.prodname_dotcom %}, you can perform analysis elsewhere and then upload the results. Alerts for {% data variables.product.prodname_code_scanning %} that you run externally are displayed in the same way as those for {% data variables.product.prodname_code_scanning %} that you run within {% data variables.product.prodname_dotcom %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/managing-code-scanning-alerts-for-your-repository)." - -If you use a third-party static analysis tool that can produce results as Static Analysis Results Interchange Format (SARIF) 2.1.0 data, you can upload this to {% data variables.product.prodname_dotcom %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github)." +As an alternative to running {% data variables.product.prodname_code_scanning %} within {% data variables.product.prodname_dotcom %}, you can perform analysis elsewhere, using the CodeQL CLI or another static analysis tool, and then upload the results. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)." {% data reusables.code-scanning.about-multiple-configurations-link %} @@ -38,5 +37,5 @@ You can use {% data variables.product.prodname_code_scanning %} webhooks to buil ## Further reading - "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning)" -- "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system)" +- "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)" - "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning)" diff --git a/content/code-security/code-scanning/integrating-with-code-scanning/index.md b/content/code-security/code-scanning/integrating-with-code-scanning/index.md index d283a0732c..5bb0debf1a 100644 --- a/content/code-security/code-scanning/integrating-with-code-scanning/index.md +++ b/content/code-security/code-scanning/integrating-with-code-scanning/index.md @@ -18,6 +18,7 @@ topics: - Integration children: - /about-integration-with-code-scanning + - /using-code-scanning-with-your-existing-ci-system - /uploading-a-sarif-file-to-github - /sarif-support-for-code-scanning --- diff --git a/content/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning.md b/content/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning.md index b159ffbb89..1f37109c3d 100644 --- a/content/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning.md +++ b/content/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning.md @@ -31,7 +31,7 @@ SARIF (Static Analysis Results Interchange Format) is an [OASIS Standard](https: To upload a SARIF file from a third-party static code analysis engine, you'll need to ensure that uploaded files use the SARIF 2.1.0 version. {% data variables.product.prodname_dotcom %} will parse the SARIF file and show alerts using the results in your repository as a part of the {% data variables.product.prodname_code_scanning %} experience. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github)." For more information about the SARIF 2.1.0 JSON schema, see [`sarif-schema-2.1.0.json`](https://github.com/oasis-tcs/sarif-spec/blob/123e95847b13fbdd4cbe2120fa5e33355d4a042b/Schemata/sarif-schema-2.1.0.json). -If you're using {% data variables.product.prodname_actions %} with the {% data variables.code-scanning.codeql_workflow %} or using the {% data variables.product.prodname_codeql_cli %}, then the {% data variables.product.prodname_code_scanning %} results will automatically use the supported subset of SARIF 2.1.0. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning)" or "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system)." +If you're using {% data variables.product.prodname_actions %} with the {% data variables.code-scanning.codeql_workflow %} or using the {% data variables.product.prodname_codeql_cli %}, then the {% data variables.product.prodname_code_scanning %} results will automatically use the supported subset of SARIF 2.1.0. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning)" or "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)." {% data variables.product.prodname_dotcom %} uses properties in the SARIF file to display alerts. For example, the `shortDescription` and `fullDescription` appear at the top of a {% data variables.product.prodname_code_scanning %} alert. The `location` allows {% data variables.product.prodname_dotcom %} to show annotations in your code file. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/managing-code-scanning-alerts-for-your-repository)." @@ -103,7 +103,7 @@ For each gzip-compressed SARIF file, SARIF upload supports a maximum size of 10 ## Uploading more than one SARIF file for a commit You can upload multiple SARIF files for the same commit, and display the data from each file as {% data variables.product.prodname_code_scanning %} results. When you upload multiple SARIF files for a commit, you must indicate a "category" for each analysis. The way to specify a category varies according to the analysis method: -- Using the {% data variables.product.prodname_codeql_cli %} directly, pass the `--sarif-category` argument to the `codeql database analyze` command when you generate SARIF files. For more information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#about-generating-code-scanning-results-with-codeql-cli)." +- Using the {% data variables.product.prodname_codeql_cli %} directly, pass the `--sarif-category` argument to the `codeql database analyze` command when you generate SARIF files. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/about-the-codeql-cli#about-generating-code-scanning-results-with-the-codeql-cli)." - Using {% data variables.product.prodname_actions %} with `codeql-action/analyze`, the category is set automatically from the workflow name and any matrix variables (typically, `language`). You can override this by specifying a `category` input for the action, which is useful when you analyze different sections of a monorepo in a single workflow. - Using {% data variables.product.prodname_actions %} to upload results from other static analysis tools, then you must specify a `category` input if you upload more than one file of results for the same tool in one workflow. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github#uploading-a-code-scanning-analysis-with-github-actions)." - If you are not using either of these approaches, you must specify a unique `runAutomationDetails.id` in each SARIF file to upload. For more information about this property, see "[`runAutomationDetails` object](#runautomationdetails-object)." diff --git a/content/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github.md b/content/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github.md index adca9f93cb..9704ca643f 100644 --- a/content/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github.md +++ b/content/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github.md @@ -40,7 +40,7 @@ You can upload the results using {% data variables.product.prodname_actions %}, - {% data variables.product.prodname_actions %} to run the {% data variables.product.prodname_codeql %} action, there is no further action required. The {% data variables.product.prodname_codeql %} action uploads the SARIF file automatically when it completes analysis. - {% data variables.product.prodname_actions %} to run a SARIF-compatible analysis tool, you could update the workflow to include a final step that uploads the results (see below). -- The {% data variables.product.prodname_codeql_cli %} to run {% data variables.product.prodname_code_scanning %} in your CI system, you can use the CLI to upload results to {% data variables.product.prodname_dotcom %} (for more information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system)"). +- The {% data variables.product.prodname_codeql_cli %} to run {% data variables.product.prodname_code_scanning %} in your CI system, you can use the CLI to upload results to {% data variables.product.prodname_dotcom %} (for more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)"). - A tool that generates results as an artifact outside of your repository, you can use the {% data variables.product.prodname_code_scanning %} API to upload the file (for more information, see "[AUTOTITLE](/rest/code-scanning#upload-an-analysis-as-sarif-data)"). {% ifversion fpt or ghec %} @@ -158,5 +158,5 @@ jobs: {% ifversion code-scanning-tool-status-page %}- "[AUTOTITLE](/code-security/code-scanning/troubleshooting-sarif-uploads)"{% endif %} - "[AUTOTITLE](/actions/using-workflows/workflow-syntax-for-github-actions)" - "[AUTOTITLE](/actions/monitoring-and-troubleshooting-workflows/viewing-workflow-run-history)" -- "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system)" +- "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)" - "[AUTOTITLE](/rest/code-scanning#upload-an-analysis-as-sarif-data)" diff --git a/content/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system.md b/content/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system.md new file mode 100644 index 0000000000..9ffa93326c --- /dev/null +++ b/content/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system.md @@ -0,0 +1,69 @@ +--- +title: Using code scanning with your existing CI system +intro: 'You can analyze your code with the {% data variables.product.prodname_codeql_cli %} or another tool in a third-party continuous integration system and upload the results to {% data variables.location.product_location %}. The resulting {% data variables.product.prodname_code_scanning %} alerts are shown alongside any alerts generated within {% data variables.product.product_name %}.' +product: '{% data reusables.gated-features.code-scanning %}' +redirect_from: + - /github/finding-security-vulnerabilities-and-errors-in-your-code/using-codeql-code-scanning-with-your-existing-ci-system + - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system + - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system +versions: + fpt: '*' + ghes: '*' + ghae: '*' + ghec: '*' +type: overview +topics: + - Advanced Security + - Code scanning + - CodeQL + - Repositories + - Pull requests + - Integration + - CI + - SARIF +--- + +{% data reusables.code-scanning.beta %} +{% data reusables.code-scanning.enterprise-enable-code-scanning %} + +## About using {% data variables.product.prodname_code_scanning %} with your existing CI system + +As an alternative to running {% data variables.product.prodname_code_scanning %} within {% data variables.product.prodname_dotcom %} using {% data variables.product.prodname_actions %}, you can analyze code in an external continuous integration or continuous delivery/deployment (CI/CD) system, then upload the results to {% data variables.product.product_name %}. + +You can add the {% data variables.product.prodname_codeql_cli %} to your third-party system, or use another third-party static analysis tool that can produce results as Static Analysis Results Interchange Format (SARIF) 2.1.0 data. For more information about the supported SARIF format, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning)." + +The {% data variables.product.prodname_codeql_cli %} is a standalone, command-line tool that you can use to analyze code. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/about-the-codeql-cli)." + +Alerts for {% data variables.product.prodname_code_scanning %} that you generate externally are displayed in the same way as those for {% data variables.product.prodname_code_scanning %} that you generate within {% data variables.product.prodname_dotcom %}. {% data reusables.code-scanning.about-multiple-configurations-link %} + +{% data reusables.code-scanning.upload-sarif-ghas %} + +## Setting up your analysis tool + +You will first need to download your analysis tool of choice and set it up with your CI system. + +If you are using the {% data variables.product.prodname_codeql_cli %}, you need to make the full contents of the {% data variables.product.prodname_codeql_cli %} bundle available to every CI server that you want to run {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} analysis on. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli)." + +Once you've made your analysis tool available to servers in your CI system, you're ready to generate data. + +## Analyzing code + +To analyze code with the {% data variables.product.prodname_codeql_cli %} or another analysis tool, you will want to check out the code you want to analyze and set up the codebase environment, making sure that any dependencies are available. You may also want to find the build command for the codebase, typically available in your CI system's configuration file. + +You can then complete the steps to analyze your codebase and produce results, which will differ based on the static analysis tool you are using. + +If you are using the {% data variables.product.prodname_codeql_cli %}, you will first need to create a {% data variables.product.prodname_codeql %} database from your code, then analyze the database to produce SARIF results. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis)" and "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries)." + +## Generating a token for authentication with {% data variables.product.product_name %} + +Each CI server needs a {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} to use to upload results to {% data variables.product.product_name %}, whether you are using the {% data variables.product.prodname_codeql_cli %}, the REST API, or another method. You must use an access token or a {% data variables.product.prodname_github_app %} with the `security_events` write permission. If CI servers already use a token with this scope to checkout repositories from {% data variables.product.product_name %}, you could potentially use the same token. Otherwise, you should create a new token with the `security_events` write permission and add this to the CI system's secret store. For information, see "[AUTOTITLE](/apps/creating-github-apps)" and "[AUTOTITLE](/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)." + +For more information on the different methods for uploading results to {% data variables.product.product_name %}, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github)." + +## Uploading your results to {% data variables.product.product_name %} + +Once you have analyzed your code, produced SARIF results, and ensured you can authenticate with {% data variables.product.product_name %}, you can upload the results to {% data variables.product.product_name %}. For more information on the different methods you can use to upload your results, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github)." + +For specific details on uploading your results to {% data variables.product.product_name %} using the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github)." + +By default, {% data variables.product.prodname_code_scanning %} expects one SARIF results file per analysis for a repository. Consequently, when you upload a second SARIF results file for a commit, it is treated as a replacement for the original set of data. You may want to upload two different SARIF files for one analysis if, for example, your analysis tool generates a different SARIF file for each language it analyzes or each set of rules it uses. If you want to upload more than one set of results for a commit in a repository, you must identify each set of results as a unique set. The way to specify a category for a SARIF upload varies according to the analysis method. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning#uploading-more-than-one-sarif-file-for-a-commit)." diff --git a/content/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql.md b/content/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql.md index 25b9740c79..e04618bc1d 100644 --- a/content/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql.md +++ b/content/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql.md @@ -40,7 +40,7 @@ There are two main ways to use {% data variables.product.prodname_codeql %} anal {% endif -%} -- Run the {% data variables.product.prodname_codeql_cli %} directly in an external CI system and upload the results to {% data variables.product.prodname_dotcom %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system)." +- Run the {% data variables.product.prodname_codeql_cli %} directly in an external CI system and upload the results to {% data variables.product.prodname_dotcom %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)." {% ifversion ghes or ghae %} diff --git a/content/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs.md b/content/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs.md index acc37ec721..8a773edf21 100644 --- a/content/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs.md +++ b/content/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs.md @@ -35,7 +35,7 @@ Diagnostic information is displayed in the Action workflow logs and consists of If you're using the {% data variables.product.prodname_codeql_cli %} outside {% data variables.product.prodname_dotcom %}, you'll see diagnostic information in the output generated during database analysis. This information is also included in the SARIF results file you upload to {% data variables.product.prodname_dotcom %} with the {% data variables.product.prodname_code_scanning %} results. -For information about the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#viewing-log-and-diagnostic-information)." +For information about the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries#viewing-log-and-diagnostic-information)." ### About summary metrics diff --git a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system.md b/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system.md deleted file mode 100644 index 84356a598f..0000000000 --- a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -title: About CodeQL code scanning in your CI system -shortTitle: Code scanning in your CI -intro: 'You can analyze your code with {% data variables.product.prodname_codeql %} in a third-party continuous integration system and upload the results to {% data variables.location.product_location %}. The resulting {% data variables.product.prodname_code_scanning %} alerts are shown alongside any alerts generated within {% data variables.product.product_name %}.' -product: '{% data reusables.gated-features.code-scanning %}' -versions: - fpt: '*' - ghes: '*' - ghae: '*' - ghec: '*' -topics: - - Advanced Security - - Code scanning - - CodeQL - - Repositories - - Pull requests - - Integration - - CI - - SARIF -redirect_from: - - /code-security/secure-coding/about-codeql-code-scanning-in-your-ci-system - - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system - # Add redirects from CodeQL runner article - - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/running-codeql-runner-in-your-ci-system - - /github/finding-security-vulnerabilities-and-errors-in-your-code/using-codeql-code-scanning-with-your-existing-ci-system/running-codeql-code-scanning-in-your-ci-system - - /github/finding-security-vulnerabilities-and-errors-in-your-code/running-codeql-code-scanning-in-your-ci-system - - /github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-codeql-code-scanning-in-your-ci-system - - /github/finding-security-vulnerabilities-and-errors-in-your-code/using-codeql-code-scanning-with-your-existing-ci-system/troubleshooting-codeql-code-scanning-in-your-ci-system - - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/migrating-from-the-codeql-runner-to-codeql-cli - # End redirects from CodeQL runner article ---- - - -{% data reusables.code-scanning.beta %} -{% data reusables.code-scanning.enterprise-enable-code-scanning %} - -## About {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} in your CI system - -{% data reusables.code-scanning.about-code-scanning %} For information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql)." - -{% data reusables.code-scanning.codeql-context-for-actions-and-third-party-tools %} - -You add the {% data variables.product.prodname_codeql_cli %} to your third-party system, then call the tool to analyze code and upload the SARIF results to {% data variables.product.product_name %}. The resulting {% data variables.product.prodname_code_scanning %} alerts are shown alongside any alerts generated within {% data variables.product.product_name %}. - -{% data reusables.code-scanning.about-multiple-configurations-link %} - -{% data reusables.code-scanning.upload-sarif-ghas %} - -## About the {% data variables.product.prodname_codeql_cli %} - -{% data reusables.code-scanning.what-is-codeql-cli %} - -Use the {% data variables.product.prodname_codeql_cli %} to analyze: - -- Dynamic languages, for example, JavaScript and Python. -- Compiled languages, for example, C/C++, C#,{% ifversion codeql-go-autobuild %} Go,{% endif %} and Java. -- Codebases written in a mixture of languages. - -For more information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system)." - -{% data reusables.code-scanning.licensing-note %} diff --git a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system.md b/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system.md deleted file mode 100644 index 4ac34d16b2..0000000000 --- a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system.md +++ /dev/null @@ -1,449 +0,0 @@ ---- -title: Configuring CodeQL CLI in your CI system -shortTitle: Configure CodeQL CLI -intro: 'You can configure your continuous integration system to run the {% data variables.product.prodname_codeql_cli %}, perform {% data variables.product.prodname_codeql %} analysis, and upload the results to {% data variables.product.product_name %} for display as {% data variables.product.prodname_code_scanning %} alerts.' -product: '{% data reusables.gated-features.code-scanning %}' -redirect_from: - - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system - - /github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-codeql-code-scanning-in-your-ci-system - - /github/finding-security-vulnerabilities-and-errors-in-your-code/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-code-scanning-in-your-ci-system -versions: - fpt: '*' - ghes: '*' - ghae: '*' - ghec: '*' -type: how_to -topics: - - Advanced Security - - Code scanning - - CodeQL - - Repositories - - Pull requests - - Integration - - CI - - SARIF ---- -{% data reusables.code-scanning.enterprise-enable-code-scanning %} - -{% data reusables.code-scanning.codeql-cli-version-ghes %} - -## About generating code scanning results with {% data variables.product.prodname_codeql_cli %} - -Once you've made the {% data variables.product.prodname_codeql_cli %} available to servers in your CI system, and ensured that they can authenticate with {% data variables.product.product_name %}, you're ready to generate data. - -You use three different commands to generate results and upload them to {% data variables.product.product_name %}: - - -1. `database create` to create a {% data variables.product.prodname_codeql %} database to represent the hierarchical structure of each supported programming language in the repository. -1. `database analyze` to run queries to analyze each {% data variables.product.prodname_codeql %} database and summarize the results in a SARIF file. -1. `github upload-results` to upload the resulting SARIF files to {% data variables.product.product_name %} where the results are matched to a branch or pull request and displayed as {% data variables.product.prodname_code_scanning %} alerts. - -You can display the command-line help for any command using the --help option. - -{% data reusables.code-scanning.upload-sarif-ghas %} - -## Creating {% data variables.product.prodname_codeql %} databases to analyze - -1. Check out the code that you want to analyze: - - For a branch, check out the head of the branch that you want to analyze. - - For a pull request, check out either the head commit of the pull request, or check out a {% data variables.product.prodname_dotcom %}-generated merge commit of the pull request. -1. Set up the environment for the codebase, making sure that any dependencies are available. For more information, see "[Creating databases for non-compiled languages](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis#creating-databases-for-non-compiled-languages)" and "[Creating databases for compiled languages](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis#creating-databases-for-compiled-languages)" in "Preparing your code for {% data variables.product.prodname_codeql %} analysis". -1. Find the build command, if any, for the codebase. Typically this is available in a configuration file in the CI system. -1. Run `codeql database create` from the checkout root of your repository and build the codebase. - - ```shell - # Single supported language - create one CodeQL database - codeql database create --command \ - --language= - - # Multiple supported languages - create one CodeQL database per language - codeql database create --command \ - --db-cluster --language=, - ``` - - {% note %} - - **Note:** If you use a containerized build, you need to run the {% data variables.product.prodname_codeql_cli %} inside the container where your build task takes place. - - {% endnote %} - -| Option | Required | Usage | -|--------|:--------:|-----| -| `` | {% octicon "check" aria-label="Required" %} | Specify the name and location of a directory to create for the {% data variables.product.prodname_codeql %} database. The command will fail if you try to overwrite an existing directory. If you also specify `--db-cluster`, this is the parent directory and a subdirectory is created for each language analyzed. | {% ifversion codeql-language-identifiers-311 %} -| --language | {% octicon "check" aria-label="Required" %} | Specify the identifier for the language to create a database for, one of: {% data reusables.code-scanning.codeql-languages-keywords %}. When used with --db-cluster, the option accepts a comma-separated list, or can be specified more than once. | {% else %} -| --language | {% octicon "check" aria-label="Required" %} | Specify the identifier for the language to create a database for, one of: {% data reusables.code-scanning.codeql-languages-keywords %} (use `javascript` to analyze TypeScript code {% ifversion codeql-kotlin-beta %} and `java` to analyze Kotlin code{% endif %}). When used with --db-cluster, the option accepts a comma-separated list, or can be specified more than once. | {% endif %} -| --command | {% octicon "x" aria-label="Optional" %} | (Recommended) Use to specify the build command or script that invokes the build process for the codebase. Commands are run from the current folder or, where it is defined, from --source-root. Not needed for Python and JavaScript/TypeScript analysis. | -| --db-cluster | {% octicon "x" aria-label="Optional" %} | Use in multi-language codebases to generate one database for each language specified by --language. | -| --no-run-unnecessary-builds | {% octicon "x" aria-label="Optional" %} | (Recommended) Use to suppress the build command for languages where the {% data variables.product.prodname_codeql_cli %} does not need to monitor the build (for example, Python and JavaScript/TypeScript). | -| --source-root | {% octicon "x" aria-label="Optional" %} | Use if you run the CLI outside the checkout root of the repository. By default, the `database create` command assumes that the current directory is the root directory for the source files, use this option to specify a different location. | -| --codescanning-config | {% octicon "x" aria-label="Optional" %} | Advanced. Use if you have a configuration file that specifies how to create the {% data variables.product.prodname_codeql %} databases and what queries to run in later steps. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#using-a-custom-configuration-file)" and "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/database-create#--codescanning-configfile)." | - -For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis)." - -### Single language example - -This example creates a {% data variables.product.prodname_codeql %} database for the repository checked out at `/checkouts/example-repo`. It uses the JavaScript extractor to create a hierarchical representation of the JavaScript and TypeScript code in the repository. The resulting database is stored in `/codeql-dbs/example-repo`. - -```shell -$ codeql database create /codeql-dbs/example-repo --language={% ifversion codeql-language-identifiers-311 %}javascript-typescript{% else %}javascript{% endif %} \ - --source-root /checkouts/example-repo - -> Initializing database at /codeql-dbs/example-repo. -> Running command [/codeql-home/codeql/javascript/tools/autobuild.cmd] - in /checkouts/example-repo. -> [build-stdout] Single-threaded extraction. -> [build-stdout] Extracting -... -> Finalizing database at /codeql-dbs/example-repo. -> Successfully created database at /codeql-dbs/example-repo. -``` - -### Multiple language example - -This example creates two {% data variables.product.prodname_codeql %} databases for the repository checked out at `/checkouts/example-repo-multi`. It uses: - -- `--db-cluster` to request analysis of more than one language. -- `--language` to specify which languages to create databases for. -- `--command` to tell the tool the build command for the codebase, here `make`. -- `--no-run-unnecessary-builds` to tell the tool to skip the build command for languages where it is not needed (like Python). - -The resulting databases are stored in `python` and `cpp` subdirectories of `/codeql-dbs/example-repo-multi`. - -```shell -$ codeql database create /codeql-dbs/example-repo-multi \ - --db-cluster --language python,{% ifversion codeql-language-identifiers-311 %}c-cpp{% else %}cpp{% endif %} \ - --command make --no-run-unnecessary-builds \ - --source-root /checkouts/example-repo-multi -Initializing databases at /codeql-dbs/example-repo-multi. -Running build command: [make] -[build-stdout] Calling python3 /codeql-bundle/codeql/python/tools/get_venv_lib.py -[build-stdout] Calling python3 -S /codeql-bundle/codeql/python/tools/python_tracer.py -v -z all -c /codeql-dbs/example-repo-multi/python/working/trap_cache -p ERROR: 'pip' not installed. -[build-stdout] /usr/local/lib/python3.6/dist-packages -R /checkouts/example-repo-multi -[build-stdout] [INFO] Python version 3.6.9 -[build-stdout] [INFO] Python extractor version 5.16 -[build-stdout] [INFO] [2] Extracted file /checkouts/example-repo-multi/hello.py in 5ms -[build-stdout] [INFO] Processed 1 modules in 0.15s -[build-stdout] -Finalizing databases at /codeql-dbs/example-repo-multi. -Successfully created databases at /codeql-dbs/example-repo-multi. -$ -``` - -## Analyzing a {% data variables.product.prodname_codeql %} database - -1. Create a {% data variables.product.prodname_codeql %} database (see above). -1. Run `codeql database analyze` on the database and specify which {% ifversion codeql-packs %}packs and/or {% endif %}queries to use. - - ```shell - codeql database analyze --format= \ - --output= {% ifversion codeql-packs %}--download {% else %}{% endif %} - ``` - -{% note %} - -**Note:** If you analyze more than one {% data variables.product.prodname_codeql %} database for a single commit, you must specify a SARIF category for each set of results generated by this command. When you upload the results to {% data variables.product.product_name %}, {% data variables.product.prodname_code_scanning %} uses this category to store the results for each language separately. If you forget to do this, each upload overwrites the previous results. - -```shell -codeql database analyze --format= \ - --sarif-category= --output= \ - {% ifversion codeql-packs %}{% else %}{% endif %} -``` - -{% endnote %} - -| Option | Required | Usage | -|--------|:--------:|-----| -| `` | {% octicon "check" aria-label="Required" %} | Specify the path for the directory that contains the {% data variables.product.prodname_codeql %} database to analyze. | -| `` | {% octicon "x" aria-label="Optional" %} | Specify {% data variables.product.prodname_codeql %} packs or queries to run. To run the standard queries used for {% data variables.product.prodname_code_scanning %}, omit this parameter. To see the other query suites included in the {% data variables.product.prodname_codeql_cli %} bundle, look in `//qlpacks/codeql/-queries/codeql-suites`. For information about creating your own query suite, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-codeql-query-suites)" in the documentation for the {% data variables.product.prodname_codeql_cli %}. -| --format | {% octicon "check" aria-label="Required" %} | Specify the format for the results file generated by the command. For upload to {% data variables.product.company_short %} this should be: {% ifversion fpt or ghae or ghec %}`sarif-latest`{% else %}`sarifv2.1.0`{% endif %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning)." -| --output | {% octicon "check" aria-label="Required" %} | Specify where to save the SARIF results file. -| --sarif-category | {% octicon "question" aria-label="Required with multiple results sets" %} | Optional for single database analysis. Required to define the language when you analyze multiple databases for a single commit in a repository.

Specify a category to include in the SARIF results file for this analysis. A category is used to distinguish multiple analyses for the same tool and commit, but performed on different languages or different parts of the code.|{% ifversion code-scanning-tool-status-page %} -| --sarif-add-baseline-file-info | {% octicon "x" aria-label="Optional" %} | (Recommended) Use to submit file coverage information to the tool status page. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page#how-codeql-defines-scanned-files)." | {% endif %} -| --sarif-add-query-help | {% octicon "x" aria-label="Optional" %} | Use if you want to include any available markdown-rendered query help for custom queries used in your analysis. Any query help for custom queries included in the SARIF output will be displayed in the code scanning UI if the relevant query generates an alert. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries#including-query-help-for-custom-codeql-queries-in-sarif-files)."{% ifversion codeql-packs %} -| `` | {% octicon "x" aria-label="Optional" %} | Use if you want to include {% data variables.product.prodname_codeql %} query packs in your analysis. For more information, see "[Downloading and using {% data variables.product.prodname_codeql %} packs](#downloading-and-using-codeql-query-packs)." -| --download | {% octicon "x" aria-label="Optional" %} | Use if some of your {% data variables.product.prodname_codeql %} query packs are not yet on disk and need to be downloaded before running queries.{% endif %} -| --threads | {% octicon "x" aria-label="Optional" %} | Use if you want to use more than one thread to run queries. The default value is `1`. You can specify more threads to speed up query execution. To set the number of threads to the number of logical processors, specify `0`. -| --verbose | {% octicon "x" aria-label="Optional" %} | Use to get more detailed information about the analysis process and diagnostic data from the database creation process. - -For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries)." - -### Basic example of analyzing a {% data variables.product.prodname_codeql %} database - -This example analyzes a {% data variables.product.prodname_codeql %} database stored at `/codeql-dbs/example-repo` and saves the results as a SARIF file: `/temp/example-repo-js.sarif`. It uses `--sarif-category` to include extra information in the SARIF file that identifies the results as JavaScript. This is essential when you have more than one {% data variables.product.prodname_codeql %} database to analyze for a single commit in a repository. - -```shell -$ codeql database analyze /codeql-dbs/example-repo \ - javascript-code-scanning.qls --sarif-category={% ifversion codeql-language-identifiers-311 %}javascript-typescript{% else %}javascript{% endif %} \ - --format={% ifversion fpt or ghae or ghec %}sarif-latest{% else %}sarifv2.1.0{% endif %} --output=/temp/example-repo-js.sarif - -> Running queries. -> Compiling query plan for /codeql-home/codeql/qlpacks/codeql-javascript/AngularJS/DisablingSce.ql. -... -> Shutting down query evaluator. -> Interpreting results. -``` - -{% ifversion code-scanning-tool-status-page %} - -### Adding file coverage information to your results for monitoring - -You can optionally submit file coverage information to {% data variables.product.product_name %} for display on the tool status page for {% data variables.product.prodname_code_scanning %}. For more information about file coverage information, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page#how-codeql-defines-scanned-files)." - -To include file coverage information with your {% data variables.product.prodname_code_scanning %} results, add the `--sarif-add-baseline-file-info` flag to the `codeql database analyze` invocation in your CI system, for example: - -```shell -$ codeql database analyze /codeql-dbs/example-repo \ - javascript-code-scanning.qls --sarif-category={% ifversion codeql-language-identifiers-311 %}javascript-typescript{% else %}javascript{% endif %} \ - --sarif-add-baseline-file-info \ --format={% ifversion fpt or ghae or ghec %}sarif-latest{% else %}sarifv2.1.0{% endif %} \ - --output=/temp/example-repo-js.sarif -``` - -{% endif %} - -## Uploading results to {% data variables.product.product_name %} - -{% data reusables.code-scanning.upload-sarif-alert-limit %} - -Before you can upload results to {% data variables.product.product_name %}, you must determine the best way to pass the {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} you created earlier to the {% data variables.product.prodname_codeql_cli %} (see [Installing {% data variables.product.prodname_codeql_cli %} in your CI system](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system#generating-a-token-for-authentication-with-github)). We recommend that you review your CI system's guidance on the secure use of a secret store. The {% data variables.product.prodname_codeql_cli %} supports: - -- Interfacing with a secret store using the `--github-auth-stdin` option (recommended). -- Saving the secret in the environment variable `GITHUB_TOKEN` and running the CLI without including the `--github-auth-stdin` option. -- For testing purposes you can pass the `--github-auth-stdin` command-line option and supply a temporary token via standard input. - -When you have decided on the most secure and reliable method for your CI server, run `codeql github upload-results` on each SARIF results file and include `--github-auth-stdin` unless the token is available in the environment variable `GITHUB_TOKEN`. - -```shell -# {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} available from a secret store - | codeql github upload-results \ - --repository= \ - --ref= --commit= \ - --sarif= {% ifversion ghes or ghae %}--github-url= \ - {% endif %}--github-auth-stdin - -# {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} available in GITHUB_TOKEN -codeql github upload-results \ - --repository= \ - --ref= --commit= \ - --sarif= {% ifversion ghes or ghae %}--github-url= \ - {% endif %} -``` - -| Option | Required | Usage | -|--------|:--------:|-----| -| --repository | {% octicon "check" aria-label="Required" %} | Specify the _OWNER/NAME_ of the repository to upload data to. The owner must be an organization within an enterprise that has a license for {% data variables.product.prodname_GH_advanced_security %} and {% data variables.product.prodname_GH_advanced_security %} must be enabled for the repository{% ifversion fpt or ghec %}, unless the repository is public{% endif %}. For more information, see "[AUTOTITLE](/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-security-and-analysis-settings-for-your-repository)." -| --ref | {% octicon "check" aria-label="Required" %} | Specify the name of the `ref` you checked out and analyzed so that the results can be matched to the correct code. For a branch use: `refs/heads/BRANCH-NAME`, for the head commit of a pull request use `refs/pull/NUMBER/head`, or for the {% data variables.product.prodname_dotcom %}-generated merge commit of a pull request use `refs/pull/NUMBER/merge`. -| --commit | {% octicon "check" aria-label="Required" %} | Specify the full SHA of the commit you analyzed. -| --sarif | {% octicon "check" aria-label="Required" %} | Specify the SARIF file to load.{% ifversion ghes or ghae %} -| --github-url | {% octicon "check" aria-label="Required" %} | Specify the URL for {% data variables.product.product_name %}.{% endif %} -| --github-auth-stdin | {% octicon "x" aria-label="Optional" %} | Pass the CLI the {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} created for authentication with {% data variables.product.company_short %}'s REST API from your secret store via standard input. This is not needed if the command has access to a `GITHUB_TOKEN` environment variable set with this token. - -For more information, see "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/github-upload-results)." - -### Basic example of uploading results to {% data variables.product.product_name %} - -The following example uploads results from the SARIF file `temp/example-repo-js.sarif` to the repository `my-org/example-repo`. It tells the {% data variables.product.prodname_code_scanning %} API that the results are for the commit `deb275d2d5fe9a522a0b7bd8b6b6a1c939552718` on the `main` branch. The example assumes that the {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} created for authentication with {% data variables.product.company_short %}'s REST API uses the `GITHUB_TOKEN` environment variable. - -```shell -codeql github upload-results \ - --repository=my-org/example-repo \ - --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ - --sarif=/temp/example-repo-js.sarif {% ifversion ghes or ghae %}--github-url={% data variables.command_line.git_url_example %} \ - {% endif %} -``` - -There is no output from this command unless the upload was unsuccessful. The command prompt returns when the upload is complete and data processing has begun. On smaller codebases, you should be able to explore the {% data variables.product.prodname_code_scanning %} alerts in {% data variables.product.product_name %} shortly afterward. You can see alerts directly in the pull request or on the **Security** tab for branches, depending on the code you checked out. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/triaging-code-scanning-alerts-in-pull-requests)" and "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/managing-code-scanning-alerts-for-your-repository)." - -{% ifversion code-scanning-tool-status-page %} - -## Uploading diagnostic information to {% data variables.product.product_name %} if the analysis fails - -When {% data variables.product.prodname_codeql_cli %} finishes analyzing a database successfully, it gathers diagnostic information such as file coverage, warnings, and errors, and includes it in the SARIF file with the results. When you upload the SARIF file to {% data variables.product.company_short %} the diagnostic information is displayed on the {% data variables.product.prodname_code_scanning %} tool status page for the repository to make it easy to see how well {% data variables.product.prodname_codeql %} is working and debug any problems. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page)." - -However, if `codeql database analyze` fails for any reason there is no SARIF file to upload to {% data variables.product.company_short %} and no diagnostic information to show on the {% data variables.product.prodname_code_scanning %} tool status page for the repository. This makes it difficult for users to troubleshoot analysis unless they have access to log files in your CI system. - -We recommend that you configure your CI workflow to export and upload diagnostic information to {% data variables.product.product_name %} when an analysis fails. You can do this using the following simple commands to export diagnostic information and upload it to {% data variables.product.company_short %}. - -### Exporting diagnostic information if the analysis fails - -You can create a SARIF file for the failed analysis using "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/database-export-diagnostics)", for example: - -```bash -$ codeql database export-diagnostics codeql-dbs/example-repo \ - --sarif-category={% ifversion codeql-language-identifiers-311 %}javascript-typescript{% else %}javascript{% endif %} --format={% ifversion fpt or ghae or ghec %}sarif-latest{% else %}sarifv2.1.0{% endif %} \ - --output=/temp/example-repo-js.sarif -``` - -This SARIF file will contain diagnostic information for the failed analysis, including any file coverage information, warnings, and errors generated during the analysis. - -### Uploading diagnostic information if the analysis fails - -You can make this diagnostic information available on the tool status page by uploading the SARIF file to {% data variables.product.product_name %} using "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/github-upload-results)", for example: - -```bash -codeql github upload-results \ - --repository=my-org/example-repo \ - --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ - --sarif=/temp/example-repo-js.sarif {% ifversion ghes or ghae %}--github-url={% data variables.command_line.git_url_example %} \ - {% endif %} -``` - -This is the same as the process for uploading SARIF files from successful analyses. -{% endif %} - -{% ifversion codeql-packs %} - -## Downloading and using {% data variables.product.prodname_codeql %} query packs - -{% data reusables.code-scanning.beta-codeql-packs-cli %} - -The {% data variables.product.prodname_codeql_cli %} bundle includes queries that are maintained by {% data variables.product.company_short %} experts, security researchers, and community contributors. If you want to run queries developed by other organizations, {% data variables.product.prodname_codeql %} query packs provide an efficient and reliable way to download and run queries. For more information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql#about-codeql-queries)." - -Before you can use a {% data variables.product.prodname_codeql %} pack to analyze a database, you must download any packages you require from the {% data variables.product.company_short %} {% data variables.product.prodname_container_registry %}. This can be done either by using the `--download` flag as part of the `codeql database analyze` command, or running `codeql pack download`. If a package is not publicly available, you will need to use a {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} to authenticate. For more information and an example, see "[Uploading results to {% data variables.product.product_name %}](#uploading-results-to-github)". - -| Option | Required | Usage | -|--------|:--------:|-----| -| | {% octicon "check" aria-label="Required" %} | Specify the scope and name of one or more {% data variables.product.prodname_codeql %} query packs to download using a comma-separated list. Optionally, include the version to download and unzip. By default the latest version of this pack is downloaded. Optionally, include a path to a query, directory, or query suite to run. If no path is included, then run the default queries of this pack. | -| --github-auth-stdin | {% octicon "x" aria-label="Optional" %} | Pass the CLI the {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} created for authentication with {% data variables.product.company_short %}'s REST API from your secret store via standard input. This is not needed if the command has access to a `GITHUB_TOKEN` environment variable set with this token. - -{% ifversion query-pack-compatibility %} -{% note %} - -**Note:** If you specify a particular version of a query pack to use, be aware that the version you specify may eventually become too old for the latest version of {% data variables.product.prodname_codeql %} to make efficient use of. To ensure optimal performance, if you need to specify exact query pack versions, you should reevaluate which versions you pin to whenever you upgrade the {% data variables.product.prodname_codeql_cli %} you're using. - -For more information about pack compatibility, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs#about-codeql-pack-compatibility)." - -{% endnote %} -{% endif %} - -### Basic example of downloading and using query packs - -This example runs the `codeql database analyze` command with the `--download` option to: - -1. Download the latest version of the `octo-org/security-queries` pack. -1. Download a version of the `octo-org/optional-security-queries` pack that is _compatible_ with version 1.0.1 (in this case, it is version 1.0.2). For more information on semver compatibility, see [npm's semantic version range documentation](https://github.com/npm/node-semver#ranges). -1. Run all the default queries in `octo-org/security-queries`. -1. Run only the query `queries/csrf.ql` from `octo-org/optional-security-queries` - -```shell -$ echo $OCTO-ORG_ACCESS_TOKEN | codeql database analyze --download /codeql-dbs/example-repo \ - octo-org/security-queries \ - octo-org/optional-security-queries@~1.0.1:queries/csrf.ql \ - --format=sarif-latest --output=/temp/example-repo-js.sarif - -> Download location: /Users/mona/.codeql/packages -> Installed fresh octo-org/security-queries@1.0.0 -> Installed fresh octo-org/optional-security-queries@1.0.2 -> Running queries. -> Compiling query plan for /Users/mona/.codeql/packages/octo-org/security-queries/1.0.0/potential-sql-injection.ql. -> [1/2] Found in cache: /Users/mona/.codeql/packages/octo-org/security-queries/1.0.0/potential-sql-injection.ql. -> Starting evaluation of octo-org/security-queries/query1.ql. -> Compiling query plan for /Users/mona/.codeql/packages/octo-org/optional-security-queries/1.0.2/queries/csrf.ql. -> [2/2] Found in cache: /Users/mona/.codeql/packages/octo-org/optional-security-queries/1.0.2/queries/csrf.ql. -> Starting evaluation of octo-org/optional-security-queries/queries/csrf.ql. -> [2/2 eval 694ms] Evaluation done; writing results to octo-org/security-queries/query1.bqrs. -> Shutting down query evaluator. -> Interpreting results. -``` - -### Direct download of {% data variables.product.prodname_codeql %} packs - -If you want to download a {% data variables.product.prodname_codeql %} pack without running it immediately, then you can use the `codeql pack download` command. This is useful if you want to avoid accessing the internet when running {% data variables.product.prodname_codeql %} queries. When you run the {% data variables.product.prodname_codeql %} analysis, you can specify packs, versions, and paths in the same way as in the previous example: - -```shell -echo $OCTO-ORG_ACCESS_TOKEN | codeql pack download ... -``` - -### Downloading {% data variables.product.prodname_codeql %} packs from multiple {% data variables.product.company_short %} container registries - -If your {% data variables.product.prodname_codeql %} packs reside on multiple container registries, then you must instruct the {% data variables.product.prodname_codeql_cli %} where to find each pack. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#downloading-codeql-packs-from-github-enterprise-server)." -{% endif %} - -## Example CI configuration for {% data variables.product.prodname_codeql %} analysis - -This is an example of the series of commands that you might use to analyze a codebase with two supported languages and then upload the results to {% data variables.product.product_name %}. - -```shell -# Create CodeQL databases for Java and Python in the 'codeql-dbs' directory -# Call the normal build script for the codebase: 'myBuildScript' - -codeql database create codeql-dbs --source-root=src \ - --db-cluster --language={% ifversion codeql-language-identifiers-311 %}java-kotlin{% else %}java{% endif %},python --command=./myBuildScript - -# Analyze the CodeQL database for Java, 'codeql-dbs/java' -# Tag the data as 'java' results and store in: 'java-results.sarif' - -codeql database analyze codeql-dbs/java java-code-scanning.qls \ - --format=sarif-latest --sarif-category={% ifversion codeql-language-identifiers-311 %}java-kotlin{% else %}java{% endif %} --output=java-results.sarif - -# Analyze the CodeQL database for Python, 'codeql-dbs/python' -# Tag the data as 'python' results and store in: 'python-results.sarif' - -codeql database analyze codeql-dbs/python python-code-scanning.qls \ - --format=sarif-latest --sarif-category=python --output=python-results.sarif - -# Upload the SARIF file with the Java results: 'java-results.sarif' -# The {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} created for authentication -# with {% data variables.product.company_short %}'s REST API is available in the `GITHUB_TOKEN` environment variable. - -codeql github upload-results \ - --repository=my-org/example-repo \ - --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ - --sarif=java-results.sarif - -# Upload the SARIF file with the Python results: 'python-results.sarif' - -codeql github upload-results \ - --repository=my-org/example-repo \ - --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ - --sarif=python-results.sarif -``` - -{% ifversion codeql-model-packs-java %} - -## Using model packs to analyze calls to custom dependencies - -You can include published model packs in a {% data variables.product.prodname_code_scanning %} analysis with the `--model-packs` option. For example: - -```shell -$ codeql database analyze /codeql-dbs/my-company --format=sarif-latest \ - --model-packs my-repo/my-java-model-pack \ - --output=/temp/my-company.sarif codeql/java-queries -``` - -In this example, the relevant queries in the standard query pack `codeql/java-queries` will use the dependency information from the model pack, `my-repo/my-java-model-pack`, to check for vulnerabilities in code that calls those dependencies. - -You can specify multiple published model packs in an analysis. - -For more information about writing your own model packs, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#creating-a-model-pack). - -{% endif %} - -## Troubleshooting the {% data variables.product.prodname_codeql_cli %} in your CI system - -### Viewing log and diagnostic information - -When you analyze a {% data variables.product.prodname_codeql %} database using a {% data variables.product.prodname_code_scanning %} query suite, in addition to generating detailed information about alerts, the CLI reports diagnostic data from the database generation step and summary metrics. For repositories with few alerts, you may find this information useful for determining if there are genuinely few problems in the code, or if there were errors generating the {% data variables.product.prodname_codeql %} database. For more detailed output from `codeql database analyze`, use the `--verbose` option. - -For more information about the type of diagnostic information available, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs#about-analysis-and-diagnostic-information)". - -### {% data variables.product.prodname_code_scanning_caps %} only shows analysis results from one of the analyzed languages - -By default, {% data variables.product.prodname_code_scanning %} expects one SARIF results file per analysis for a repository. Consequently, when you upload a second SARIF results file for a commit, it is treated as a replacement for the original set of data. - -If you want to upload more than one set of results to the {% data variables.product.prodname_code_scanning %} API for a commit in a repository, you must identify each set of results as a unique set. For repositories where you create more than one {% data variables.product.prodname_codeql %} database to analyze for each commit, use the `--sarif-category` option to specify a language or other unique category for each SARIF file that you generate for that repository. - -{% ifversion fpt or ghec or ghes > 3.7 or ghae > 3.7 %} - -### Issues with Python extraction - -We are deprecating Python 2 support for the {% data variables.product.prodname_codeql_cli %}, more specifically for the {% data variables.product.prodname_codeql %} database generation phase (code extraction). - -If you use the {% data variables.product.prodname_codeql_cli %} to run {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} on code written in Python, you must make sure that your CI system has Python 3 installed. - -{% endif %} - -## Further reading - -- "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis)." -- "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries)."{% ifversion codeql-packs %} -- [AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs){% endif %} diff --git a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/index.md b/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/index.md deleted file mode 100644 index 387c795dd6..0000000000 --- a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/index.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: Using CodeQL code scanning with your existing CI system -shortTitle: Use CodeQL in CI system -intro: 'You can run {% data variables.product.prodname_codeql %} analysis in your existing CI system and upload the results to {% data variables.product.product_name %} for display as {% data variables.product.prodname_code_scanning %} alerts.' -product: '{% data reusables.gated-features.code-scanning %}' -redirect_from: - - /github/finding-security-vulnerabilities-and-errors-in-your-code/using-codeql-code-scanning-with-your-existing-ci-system - - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system -versions: - fpt: '*' - ghes: '*' - ghae: '*' - ghec: '*' -type: how_to -topics: - - Advanced Security - - Code scanning - - Integration - - CI - - CodeQL -children: - - /about-codeql-code-scanning-in-your-ci-system - - /installing-codeql-cli-in-your-ci-system - - /configuring-codeql-cli-in-your-ci-system ---- diff --git a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system.md b/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system.md deleted file mode 100644 index 7a25d1d7dc..0000000000 --- a/content/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -title: Installing CodeQL CLI in your CI system -shortTitle: Install CodeQL CLI -intro: 'You can install the {% data variables.product.prodname_codeql_cli %} and use it to perform {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} in a third-party continuous integration system.' -product: '{% data reusables.gated-features.code-scanning %}' -versions: - fpt: '*' - ghes: '*' - ghae: '*' - ghec: '*' -type: how_to -topics: - - Advanced Security - - Code scanning - - CodeQL - - Repositories - - Pull requests - - Integration - - CI - - SARIF -redirect_from: - - /code-security/secure-coding/running-codeql-cli-in-your-ci-system - - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/running-codeql-cli-in-your-ci-system - - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system ---- -{% data reusables.code-scanning.enterprise-enable-code-scanning %} - -## About using the {% data variables.product.prodname_codeql_cli %} for {% data variables.product.prodname_code_scanning %} - -You can use the {% data variables.product.prodname_codeql_cli %} to run {% data variables.product.prodname_code_scanning %} on code that you're processing in a third-party continuous integration (CI) system. {% data reusables.code-scanning.about-code-scanning %} For information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql)." For recommended specifications (RAM, CPU cores, and disk) for running {% data variables.product.prodname_codeql %} analysis, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/recommended-hardware-resources-for-running-codeql)." - -{% data reusables.code-scanning.what-is-codeql-cli %} - -Alternatively, you can use {% data variables.product.prodname_actions %} to run {% data variables.product.prodname_code_scanning %} within {% data variables.product.product_name %}. For information about {% data variables.product.prodname_code_scanning %} using actions, see {% ifversion code-scanning-without-workflow %}"[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)" and {% endif %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning)." For an overview of the options for CI systems, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system)". - -{% data reusables.code-scanning.licensing-note %} - -## Downloading the {% data variables.product.prodname_codeql_cli %} - -You should download the {% data variables.product.prodname_codeql %} bundle from https://github.com/github/codeql-action/releases. The bundle contains: - -- {% data variables.product.prodname_codeql_cli %} product -- A compatible version of the queries and libraries from https://github.com/github/codeql -- Precompiled versions of all the queries included in the bundle - -{% ifversion ghes or ghae %} - -{% note %} -For {% data variables.product.product_name %}{% ifversion ghes %} {{ allVersions[currentVersion].currentRelease }}{% endif %}, we recommend {% data variables.product.prodname_codeql_cli %} version {% data variables.product.codeql_cli_ghes_recommended_version %}. -{% endnote %} - -{% endif %} - -You should always use the {% data variables.product.prodname_codeql %} bundle as this ensures compatibility and also gives much better performance than a separate download of the {% data variables.product.prodname_codeql_cli %} and checkout of the {% data variables.product.prodname_codeql %} queries. If you will only be running the CLI on one specific platform, download the appropriate `codeql-bundle-PLATFORM.tar.gz` file. Alternatively, you can download `codeql-bundle.tar.gz`, which contains the CLI for all supported platforms. - -{% data reusables.code-scanning.beta-codeql-packs-cli %} - -## Setting up the {% data variables.product.prodname_codeql_cli %} in your CI system - -You need to make the full contents of the {% data variables.product.prodname_codeql_cli %} bundle available to every CI server that you want to run {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} analysis on. For example, you might configure each server to copy the bundle from a central, internal location and extract it. Alternatively, you could use the REST API to get the bundle directly from {% data variables.product.prodname_dotcom %}, ensuring that you benefit from the latest improvements to queries. Updates to the {% data variables.product.prodname_codeql_cli %} are released every 2-3 weeks. - -First you'll need to extract the {% data variables.product.prodname_codeql_cli %} bundle. You can then run the `codeql` executable on the server in one of two ways: - -- By executing `//codeql/codeql`, where `` is the folder where you extracted the {% data variables.product.prodname_codeql_cli %} bundle. -- By adding `//codeql` to your `PATH`, so that you can run the executable as just `codeql`. - -{% ifversion fpt or ghec or ghes > 3.7 or ghae > 3.7 %} -{% note %} - -If you use the {% data variables.product.prodname_codeql_cli %} to analyze code written in Python, you must make sure that your CI system has Python 3 installed. - -{% endnote %} -{% endif %} - -## Testing the {% data variables.product.prodname_codeql_cli %} configuration - -After you extract the {% data variables.product.prodname_codeql_cli %} bundle, you can run the following command to verify that the CLI is correctly configured to create and analyze databases. - -- `codeql resolve qlpacks` if `//codeql` is on the `PATH`. -- `//codeql/codeql resolve qlpacks` otherwise. - -The following is an extract of successful output: - -```shell -codeql/cpp-all (//qlpacks/codeql/cpp-all/) -codeql/cpp-examples (//qlpacks/codeql/cpp-examples/) -codeql/cpp-queries (//qlpacks/codeql/cpp-queries/) -codeql/csharp-all (//qlpacks/codeql/charp-all/) -codeql/csharp-examples (//qlpacks/codeql/charp-examples/) -codeql/csharp-queries (//qlpacks/codeql/charp-queries/) -codeql/java-all (//qlpacks/codeql/java-all/) -codeql/java-examples (//qlpacks/codeql/java-examples/) -codeql/java-queries (//qlpacks/codeql/java-queries/) -codeql/javascript-all (//qlpacks/codeql/javascript-all/) -codeql/javascript-examples (//qlpacks/codeql/javascript-examples/) -codeql/javascript-queries (//qlpacks/codeql/javascript-queries/) -codeql/python-all (//qlpacks/codeql/python-all/) -codeql/python-examples (//qlpacks/codeql/python-examples/) -codeql/python-queries (//qlpacks/codeql/python-queries/) -codeql/ruby-all (//qlpacks/codeql/ruby-all/) -codeql/ruby-examples (//qlpacks/codeql/ruby-examples/) -codeql/ruby-queries (//qlpacks/codeql/ruby-queries/) -... -``` - -You should check that the output contains the expected languages and also that the directory location for the qlpack files is correct. The location should be within the extracted {% data variables.product.prodname_codeql_cli %} bundle, shown above as ``, unless you are using a checkout of `github/codeql`. If the {% data variables.product.prodname_codeql_cli %} is unable to locate the qlpacks for the expected languages, check that you downloaded the {% data variables.product.prodname_codeql %} bundle and not a standalone copy of the {% data variables.product.prodname_codeql_cli %}. - -## Generating a token for authentication with {% data variables.product.product_name %} - -Each CI server needs a {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} for the {% data variables.product.prodname_codeql_cli %} to use to upload results to {% data variables.product.product_name %}. You must use an access token or a {% data variables.product.prodname_github_app %} with the `security_events` write permission. If CI servers already use a token with this scope to checkout repositories from {% data variables.product.product_name %}, you could potentially allow the {% data variables.product.prodname_codeql_cli %} to use the same token. Otherwise, you should create a new token with the `security_events` write permission and add this to the CI system's secret store. For information, see "[AUTOTITLE](/apps/creating-github-apps)" and "[AUTOTITLE](/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)." - -## Next steps - -You're now ready to configure the CI system to run {% data variables.product.prodname_codeql %} analysis, generate results, and upload them to {% data variables.product.product_name %} where the results will be matched to a branch or pull request and displayed as {% data variables.product.prodname_code_scanning %} alerts. For detailed information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system)." diff --git a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/about-the-codeql-cli.md b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/about-the-codeql-cli.md index 2a60076c3a..ce2ce92b49 100644 --- a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/about-the-codeql-cli.md +++ b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/about-the-codeql-cli.md @@ -14,6 +14,15 @@ topics: redirect_from: - /code-security/codeql-cli/about-the-codeql-cli - /code-security/codeql-cli/using-the-codeql-cli/about-the-codeql-cli + - /code-security/secure-coding/about-codeql-code-scanning-in-your-ci-system + - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system + - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/running-codeql-runner-in-your-ci-system + - /github/finding-security-vulnerabilities-and-errors-in-your-code/using-codeql-code-scanning-with-your-existing-ci-system/running-codeql-code-scanning-in-your-ci-system + - /github/finding-security-vulnerabilities-and-errors-in-your-code/running-codeql-code-scanning-in-your-ci-system + - /github/finding-security-vulnerabilities-and-errors-in-your-code/troubleshooting-codeql-code-scanning-in-your-ci-system + - /github/finding-security-vulnerabilities-and-errors-in-your-code/using-codeql-code-scanning-with-your-existing-ci-system/troubleshooting-codeql-code-scanning-in-your-ci-system + - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/migrating-from-the-codeql-runner-to-codeql-cli + - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system --- ## About the {% data variables.product.prodname_codeql_cli %} @@ -39,11 +48,9 @@ The {% data variables.product.prodname_codeql_cli %} can analyze: For information about setting up the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli)." -For information about using the {% data variables.product.prodname_codeql_cli %} in a third-party CI system to create results to display in {% data variables.product.prodname_dotcom %} as code scanning alerts, see [Configuring {% data variables.product.prodname_codeql_cli %} in your CI system](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system). For information about enabling {% data variables.product.prodname_codeql %} code scanning using {% data variables.product.prodname_actions %}, see {% ifversion code-scanning-without-workflow %}"[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)" and {% endif %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning)." - ## About using the {% data variables.product.prodname_codeql_cli %} for {% data variables.product.prodname_code_scanning %} -You can use the {% data variables.product.prodname_codeql_cli %} to run {% data variables.product.prodname_code_scanning %} on code that you're processing in a third-party continuous integration (CI) system. {% data reusables.code-scanning.about-code-scanning %} For an overview of the options for CI systems, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system)." For recommended specifications (RAM, CPU cores, and disk) for running {% data variables.product.prodname_codeql %} analysis, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/recommended-hardware-resources-for-running-codeql)." +You can use the {% data variables.product.prodname_codeql_cli %} to run {% data variables.product.prodname_code_scanning %} on code that you're processing in a third-party continuous integration (CI) system. {% data reusables.code-scanning.about-code-scanning %} For an overview of using code scanning with external CI systems, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)." For recommended specifications (RAM, CPU cores, and disk) for running {% data variables.product.prodname_codeql %} analysis, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/recommended-hardware-resources-for-running-codeql)." Alternatively, you can use {% data variables.product.prodname_actions %} or Azure DevOps pipelines to scan code using the {% data variables.product.prodname_codeql_cli %}. For more information, see {% ifversion code-scanning-without-workflow %}"[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)"{% else %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning)"{% endif %} or [Configure {% data variables.product.prodname_ghas_azdo %}](https://learn.microsoft.com/en-us/azure/devops/repos/security/configure-github-advanced-security-features) in Microsoft Learn. @@ -51,9 +58,9 @@ For an overview of all the options for using {% data variables.product.prodname_ {% data reusables.code-scanning.licensing-note %} -## About generating code scanning results with {% data variables.product.prodname_codeql_cli %} +## About generating code scanning results with the {% data variables.product.prodname_codeql_cli %} -If you choose to run the {% data variables.product.prodname_codeql_cli %} directly, you first have to install the {% data variables.product.prodname_codeql_cli %} locally. If you are planning to use the {% data variables.product.prodname_codeql_cli %} with an external CI system, you need to make the {% data variables.product.prodname_codeql_cli %} available to servers in your CI system, and ensure that they can authenticate with {% data variables.product.product_name %}. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli)." +If you choose to run the {% data variables.product.prodname_codeql_cli %} directly, you first have to install the {% data variables.product.prodname_codeql_cli %} locally. If you are planning to use the {% data variables.product.prodname_codeql_cli %} with an external CI system, you need to make the {% data variables.product.prodname_codeql_cli %} available to servers in your CI system. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli)." Once the {% data variables.product.prodname_codeql_cli %} is set up, you can use three different commands to generate results and upload them to {% data variables.product.product_name %}: @@ -61,6 +68,48 @@ Once the {% data variables.product.prodname_codeql_cli %} is set up, you can use 1. `database analyze` to run queries to analyze each {% data variables.product.prodname_codeql %} database and summarize the results in a SARIF file. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries)." 1. `github upload-results` to upload the resulting SARIF files to {% data variables.product.product_name %} where the results are matched to a branch or pull request and displayed as {% data variables.product.prodname_code_scanning %} alerts. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github)." +{% data reusables.code-scanning.upload-sarif-ghas %} + +### Example CI configuration for {% data variables.product.prodname_codeql %} analysis + +This is an example of the full series of commands for the {% data variables.product.prodname_codeql_cli %} that you might use to analyze a codebase with two supported languages and then upload the results to {% data variables.product.product_name %}. + +```shell +# Create CodeQL databases for Java and Python in the 'codeql-dbs' directory +# Call the normal build script for the codebase: 'myBuildScript' + +codeql database create codeql-dbs --source-root=src \ + --db-cluster --language=java,python --command=./myBuildScript + +# Analyze the CodeQL database for Java, 'codeql-dbs/java' +# Tag the data as 'java' results and store in: 'java-results.sarif' + +codeql database analyze codeql-dbs/java java-code-scanning.qls \ + --format=sarif-latest --sarif-category=java --output=java-results.sarif + +# Analyze the CodeQL database for Python, 'codeql-dbs/python' +# Tag the data as 'python' results and store in: 'python-results.sarif' + +codeql database analyze codeql-dbs/python python-code-scanning.qls \ + --format=sarif-latest --sarif-category=python --output=python-results.sarif + +# Upload the SARIF file with the Java results: 'java-results.sarif' +# The {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} created for authentication +# with {% data variables.product.company_short %}'s REST API is available in the `GITHUB_TOKEN` environment variable. + +codeql github upload-results \ + --repository=my-org/example-repo \ + --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ + --sarif=java-results.sarif + +# Upload the SARIF file with the Python results: 'python-results.sarif' + +codeql github upload-results \ + --repository=my-org/example-repo \ + --ref=refs/heads/main --commit=deb275d2d5fe9a522a0b7bd8b6b6a1c939552718 \ + --sarif=python-results.sarif +``` + ## About the {% data variables.product.prodname_dotcom %} {% data variables.product.prodname_codeql %} license **License notice:** If you don’t have a {% data variables.product.prodname_enterprise %} license then, by installing this product, you are agreeing to the [{% data variables.product.prodname_dotcom %} {% data variables.product.prodname_codeql %} Terms and Conditions](https://securitylab.github.com/tools/codeql/license). diff --git a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries.md b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries.md index e8474713b8..98cee11b21 100644 --- a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries.md +++ b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries.md @@ -21,28 +21,7 @@ redirect_from: {% data reusables.code-scanning.codeql-cli-version-ghes %} -To analyze a codebase, you run queries against a {% data variables.product.prodname_codeql %} database extracted from the code. - -{% data variables.product.prodname_codeql %} analyses produce [interpreted results](https://codeql.github.com/docs/codeql-overview/about-codeql/#interpret-query-results) that can be displayed as alerts or paths in source code. -For information about writing queries to run with `database analyze`, see "[Using custom queries with the {% data variables.product.prodname_codeql_cli %}](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/using-custom-queries-with-the-codeql-cli)." - -### Other query-running commands - -Queries run with `database analyze` have strict [metadata requirements](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/using-custom-queries-with-the-codeql-cli#including-query-metadata). You can also execute queries using the following -plumbing-level subcommands: - -- [AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/database-run-queries), which - outputs non-interpreted results in an intermediate binary format called [BQRS](https://codeql.github.com/docs/codeql-overview/codeql-glossary/#bqrs-file) - -- [AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/query-run), which will output BQRS files, or print - results tables directly to the command line. Viewing results directly in - the command line may be useful for iterative query development using the CLI. - -Queries run with these commands don't have the same metadata requirements. -However, to save human-readable data you have to process each BQRS results -file using the [AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/bqrs-decode) plumbing -subcommand. Therefore, for most use cases it's easiest to use `database -analyze` to directly generate interpreted results. +To analyze a codebase, you run queries against a {% data variables.product.prodname_codeql %} database extracted from the code. {% data variables.product.prodname_codeql %} analyses produce results that can be uploaded to {% data variables.product.product_name %} to generate code scanning alerts. ## Prerequisites @@ -51,9 +30,7 @@ Before starting an analysis you must: - [Set up the {% data variables.product.prodname_codeql_cli %}](/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli) to run commands locally. - [Create a {% data variables.product.prodname_codeql %} database](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis) for the source code you want to analyze. -The simplest way to run `codeql database analyze` is using {% data variables.product.prodname_codeql %} packs. You can also -run the command using queries from a local checkout of the {% data variables.product.prodname_codeql %} repository, -which you may want to do if you want to customize the {% data variables.product.prodname_codeql %} core queries. +The simplest way to run `codeql database analyze` is using the standard queries included in the {% data variables.product.prodname_codeql_cli %} bundle. ## Running `codeql database analyze` @@ -90,10 +67,10 @@ You must specify ``, `--format`, and `--output`. You can specify addit | `` | {% octicon "check" aria-label="Required" %} | Specify the path for the directory that contains the {% data variables.product.prodname_codeql %} database to analyze. | | `` | {% octicon "x" aria-label="Optional" %} | Specify {% data variables.product.prodname_codeql %} packs or queries to run. To run the standard queries used for {% data variables.product.prodname_code_scanning %}, omit this parameter. To see the other query suites included in the {% data variables.product.prodname_codeql_cli %} bundle, look in `//qlpacks/codeql/-queries/codeql-suites`. For information about creating your own query suite, see [AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-codeql-query-suites) in the documentation for the {% data variables.product.prodname_codeql_cli %}. | --format | {% octicon "check" aria-label="Required" %} | Specify the format for the results file generated during analysis. A number of different formats are supported, including CSV, [SARIF](https://codeql.github.com/docs/codeql-overview/codeql-glossary/#sarif-file), and graph formats. For upload to {% data variables.product.company_short %} this should be: {% ifversion fpt or ghae or ghec %}`sarif-latest`{% else %}`sarifv2.1.0`{% endif %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning)." -| --output | {% octicon "check" aria-label="Required" %} | Specify where to save the SARIF results file. +| --output | {% octicon "check" aria-label="Required" %} | Specify the location where you want to save the SARIF results file, including the desired filename with the `.sarif` extension. | --sarif-category | {% octicon "question" aria-label="Required with multiple results sets" %} | Optional for single database analysis. Required to define the language when you analyze multiple databases for a single commit in a repository.

Specify a category to include in the SARIF results file for this analysis. A category is used to distinguish multiple analyses for the same tool and commit, but performed on different languages or different parts of the code.|{% ifversion code-scanning-tool-status-page %} | --sarif-add-baseline-file-info | {% octicon "x" aria-label="Optional" %} | **Recommended.** Use to submit file coverage information to the {% data variables.code-scanning.tool_status_page %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/about-the-tool-status-page#how-codeql-defines-scanned-files)." | {% endif %} -| --sarif-add-query-help | {% octicon "x" aria-label="Optional" %} | Use if you want to include any available markdown-rendered query help for custom queries used in your analysis. Any query help for custom queries included in the SARIF output will be displayed in the code scanning UI if the relevant query generates an alert. For more information, see "[Including query help for custom {% data variables.product.prodname_codeql %} queries in SARIF files](#including-query-help-for-custom-codeql-queries-in-sarif-files)."{% ifversion codeql-packs %} +| --sarif-add-query-help | {% octicon "x" aria-label="Optional" %} | Use if you want to include any available markdown-rendered query help for custom queries used in your analysis. Any query help for custom queries included in the SARIF output will be displayed in the code scanning UI if the relevant query generates an alert. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/using-custom-queries-with-the-codeql-cli#including-query-help-for-custom-codeql-queries-in-sarif-files)."{% ifversion codeql-packs %} | `` | {% octicon "x" aria-label="Optional" %} | Use if you want to include {% data variables.product.prodname_codeql %} query packs in your analysis. For more information, see "[Downloading and using {% data variables.product.prodname_codeql %} packs](/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs#downloading-and-using-codeql-query-packs)." | --download | {% octicon "x" aria-label="Optional" %} | Use if some of your {% data variables.product.prodname_codeql %} query packs are not yet on disk and need to be downloaded before running queries.{% endif %} | --threads | {% octicon "x" aria-label="Optional" %} | Use if you want to use more than one thread to run queries. The default value is `1`. You can specify more threads to speed up query execution. To set the number of threads to the number of logical processors, specify `0`. @@ -308,7 +285,7 @@ codeql database analyze codeql/cpp-queries:codeql-suites/cpp-code ``` This command downloads the `codeql/cpp-queries` {% data variables.product.prodname_codeql %} query pack, runs the analysis, and generates a file in the SARIF version 2.1.0 format that is supported by all versions of {% data variables.product.prodname_dotcom %}. This file can be uploaded to {% data variables.product.prodname_dotcom %} by executing `codeql github upload-results` or the code scanning API. -For more information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#analyzing-a-codeql-database)" +For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github)" or "[AUTOTITLE](/rest/code-scanning)". {% data variables.product.prodname_codeql %} query suites are `.qls` files that use directives to select queries to run @@ -319,65 +296,24 @@ For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advan For information about creating custom query suites, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-codeql-query-suites)." -#### Diagnostic and summary information - -When you create a {% data variables.product.prodname_codeql %} database, the extractor stores diagnostic data in the database. The code scanning query suites include additional queries to report on this diagnostic data and calculate summary metrics. When the `database analyze` command completes, the CLI generates the results file and reports any diagnostic and summary data to standard output. If you choose to generate SARIF output, the additional data is also included in the SARIF file. - -If the analysis found fewer results for standard queries than you expected, review the results of the diagnostic and summary queries to check whether the {% data variables.product.prodname_codeql %} database is likely to be a good representation of the codebase that you want to analyze. - -### Integrating a {% data variables.product.prodname_codeql %} pack into a code scanning workflow in {% data variables.product.prodname_dotcom %} - -You can use {% data variables.product.prodname_codeql %} query packs in your code scanning setup. This allows you to select query packs published by various sources and use them to analyze your code. -For more information, see "[Using {% data variables.product.prodname_codeql %} query packs in the {% data variables.product.prodname_codeql %} action](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#using-query-packs/)" or "[Downloading and using {% data variables.product.prodname_codeql %} query packs in your CI system](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#downloading-and-using-codeql-query-packs)." - -### Including query help for custom {% data variables.product.prodname_codeql %} queries in SARIF files - -If you use the {% data variables.product.prodname_codeql_cli %} to run code scanning analyses on third party CI/CD systems, -you can include the query help for your custom queries in SARIF files generated during an analysis. -After uploading the SARIF file to {% data variables.product.prodname_dotcom %}, the query help is shown in the code scanning UI for any -alerts generated by the custom queries. - -From {% data variables.product.prodname_codeql_cli %} v2.7.1 onwards, you can include markdown-rendered query help in SARIF files -by providing the `--sarif-add-query-help` option when running -`codeql database analyze`. -For more information, see [AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system#analyzing-a-codeql-database). - -You can write query help for custom queries directly in a markdown file and save it alongside the -corresponding query. Alternatively, for consistency with the standard {% data variables.product.prodname_codeql %} queries, -you can write query help in the `.qhelp` format. Query help written in `.qhelp` -files can’t be included in SARIF files, and they can’t be processed by code -scanning so must be converted to markdown before running -the analysis. For more information, see "[Query help files](https://codeql.github.com/docs/writing-codeql-queries/query-help-files/#query-help-files)" -and "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-query-help-files)." - ## Results -You can save analysis results in a number of different formats, including SARIF -and CSV. +You can save analysis results in a number of different formats, including SARIF and CSV. -The SARIF format is designed to represent the output of a broad range of static -analysis tools. For more information, see [AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/sarif-output). +The SARIF format is designed to represent the output of a broad range of static analysis tools. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/sarif-output)." -If you choose to generate results in CSV format, then each line in the output file -corresponds to an alert. Each line is a comma-separated list with the following information. +For more information about what the results look like in CSV format, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/csv-output)." -**Property**|**Description**|**Example** ------|-----|----- -Name | Name of the query that identified the result. | `Inefficient regular expression` -Description | Description of the query.| `A regular expression that requires exponential time to match certain inputs can be a performance bottleneck, and may be vulnerable to denial-of-service attacks.` -Severity | Severity of the query.| `error` -Message | Alert message.| `This part of the regular expression may cause exponential backtracking on strings containing many repetitions of '\\\\'.` -Path | Path of the file containing the alert. | `/vendor/codemirror/markdown.js` -Start line | Line of the file where the code that triggered the alert begins. | `617` -Start column | Column of the start line that marks the start of the alert code. Not included when equal to 1. | `32` -End line | Line of the file where the code that triggered the alert ends. Not included when the same value as the start line. | `64` -End column | Where available, the column of the end line that marks the end of the alert code. Otherwise the end line is repeated. | `617` +Results files can be integrated into your own code-review or debugging infrastructure. For example, SARIF file output can be used to highlight alerts in the correct location in your source code using a SARIF viewer plugin for your IDE. -Results files can be integrated into your own code-review or debugging -infrastructure. For example, SARIF file output can be used to highlight alerts -in the correct location in your source code using a SARIF viewer plugin for your -IDE. +## Viewing log and diagnostic information -## Further reading +When you analyze a {% data variables.product.prodname_codeql %} database using a {% data variables.product.prodname_code_scanning %} query suite, in addition to generating detailed information about alerts, the CLI reports diagnostic data from the database generation step and summary metrics. If you choose to generate SARIF output, the additional data is also included in the SARIF file. For repositories with few alerts, you may find this information useful for determining if there are genuinely few problems in the code, or if there were errors generating the {% data variables.product.prodname_codeql %} database. For more detailed output from `codeql database analyze`, use the `--verbose` option. -- "[Analyzing your projects in {% data variables.product.prodname_codeql %} for VS Code](https://codeql.github.com/docs/codeql-for-visual-studio-code/analyzing-your-projects/#analyzing-your-projects)" +For more information about the type of diagnostic information available, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/viewing-code-scanning-logs#about-analysis-and-diagnostic-information)". + +You can choose to export and upload diagnostic information to {% data variables.product.product_name %} even if a {% data variables.product.prodname_codeql %} analysis fails. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github#uploading-diagnostic-information-to-github-if-the-analysis-fails)." + +## Next steps + +- To learn how to upload your {% data variables.product.prodname_codeql %} analysis results to {% data variables.product.product_name %}, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github)." diff --git a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs.md b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs.md index 49e94cc087..772e6809e1 100644 --- a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs.md +++ b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs.md @@ -20,41 +20,37 @@ redirect_from: {% data reusables.code-scanning.codeql-cli-version-ghes %} -{% data variables.product.prodname_codeql %} packs are used to create, share, depend on, and run {% data variables.product.prodname_codeql %} queries and libraries. You can publish your own {% data variables.product.prodname_codeql %} packs and download packs created by others. {% data variables.product.prodname_codeql %} packs contain queries, library files, query suites, and metadata. + {% data variables.product.prodname_codeql %} packs are used to create, share, depend on, and run {% data variables.product.prodname_codeql %} queries and libraries. {% data variables.product.prodname_codeql %} packs contain queries, library files, query suites, and metadata. You can customize your {% data variables.product.prodname_codeql %} analysis by downloading packs created by others and running them on your codebase. There are{% ifversion codeql-model-packs-java %} three{% else %} two{% endif %} types of {% data variables.product.prodname_codeql %} packs: {% ifversion codeql-model-packs-java %}query packs, library packs, and model packs{% else %} query packs and library packs{% endif %}. -- Query packs contain a set of pre-compiled queries that can be evaluated on a {% data variables.product.prodname_codeql %} database. Query parks are designed to be run. When a query pack is published, the bundle includes all the transitive dependencies and {% ifversion query-pack-compatibility %}pre-compiled representations of each query, in addition to the query sources{% else %}a compilation cache{% endif %}. This ensures consistent and efficient execution of the queries in the pack. +- Query packs contain a set of pre-compiled queries that can be evaluated on a {% data variables.product.prodname_codeql %} database. Query packs are designed to be run. When a query pack is published, the bundle includes all the transitive dependencies and {% ifversion query-pack-compatibility %}pre-compiled representations of each query, in addition to the query sources{% else %}a compilation cache{% endif %}. This ensures consistent and efficient execution of the queries in the pack. - Library packs are designed to be used by query packs (or other library packs) and do not contain queries themselves. The libraries are not compiled {% ifversion query-pack-compatibility %}separately{% else %}and there is no compilation cache included when the pack is published{% endif %}.{% ifversion codeql-model-packs-java %} -- Model packs can be used to expand {% data variables.product.prodname_code_scanning %} analysis to recognize libraries and frameworks that are not supported by default. Model packs are currently in beta and subject to change. During the beta, model packs are available for Java analysis at the repository level. For more information about creating your own model packs, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#creating-a-model-pack)." +- Model packs can be used to expand {% data variables.product.prodname_code_scanning %} analysis to recognize libraries and frameworks that are not supported by default. Model packs are currently in beta and subject to change. During the beta, model packs are available for Java analysis at the repository level. For more information about creating your own model packs, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#creating-a-codeql-model-pack)." {% endif %} -You can use the package management commands in the {% data variables.product.prodname_codeql_cli %} to create {% data variables.product.prodname_codeql %} packs, add dependencies to packs, and install or update dependencies. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#creating-and-working-with-codeql-packs)." You can also publish and download {% data variables.product.prodname_codeql %} packs using the {% data variables.product.prodname_codeql_cli %}. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs)." +The standard {% data variables.product.prodname_codeql %} packs for all supported languages are published in the [{% data variables.product.prodname_container_registry %}](https://github.com/orgs/codeql/packages). If you installed the {% data variables.product.prodname_codeql_cli %} in the standard way, using the {% data variables.product.prodname_codeql_cli %} bundle, the core query packs are already downloaded and available to you. They are: -The standard {% data variables.product.prodname_codeql %} packages for all supported languages are published in the [{% data variables.product.prodname_container_registry %}](https://github.com/orgs/codeql/packages). -The [{% data variables.product.prodname_codeql %} repository](https://github.com/github/codeql) contains source files for the standard {% data variables.product.prodname_codeql %} packs for all supported languages. + - `codeql/cpp-queries` + - `codeql/csharp-queries` + - `codeql/go-queries` + - `codeql/java-queries` + - `codeql/javascript-queries` + - `codeql/python-queries` + - `codeql/ruby-queries` -## {% data variables.product.prodname_codeql %} pack structure +You can also use the {% data variables.product.prodname_codeql_cli %} to create your own {% data variables.product.prodname_codeql %} packs, add dependencies to packs, and install or update dependencies. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#creating-and-working-with-codeql-packs)." -A {% data variables.product.prodname_codeql %} pack must contain a file called `qlpack.yml` in its root directory. In the `qlpack.yml` file, the `name:` field must have a value that follows the format of `/`, where `` is the {% data variables.product.prodname_dotcom %} organization or user account that the pack will be published to and `` is the name of the pack. Additionally, query packs and library packs with {% data variables.product.prodname_codeql %} tests contain a `codeql-pack.lock.yml` file that contains the resolved dependencies of the pack. This file is generated during a call to the `codeql pack install` command, is not meant to be edited by hand, and should be added to your version control system. - -The other files and directories within the pack should be logically organized. For example, typically: - -- Queries are organized into directories for specific categories. - -- Queries for specific products, libraries, and frameworks are organized into -their own top-level directories. - -{% ifversion codeql-packs %} +You can publish {% data variables.product.prodname_codeql %} packs that you have created, using the {% data variables.product.prodname_codeql_cli %}. For more information on publishing and downloading {% data variables.product.prodname_codeql %} packs, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs)." ## Downloading and using {% data variables.product.prodname_codeql %} query packs The {% data variables.product.prodname_codeql_cli %} bundle includes queries that are maintained by {% data variables.product.company_short %} experts, security researchers, and community contributors. If you want to run queries developed by other organizations, {% data variables.product.prodname_codeql %} query packs provide an efficient and reliable way to download and run queries{% ifversion codeql-model-packs-java %}, while model packs (beta) can be used to expand {% data variables.product.prodname_code_scanning %} analysis to recognize libraries and frameworks that are not supported by default{% endif %}. For more information about query packs, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql#about-codeql-queries)." {% ifversion codeql-model-packs-java %} For information about writing your own model packs, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#creating-a-model-pack)."{% endif %} -Before you can use a {% data variables.product.prodname_codeql %} query pack to analyze a database, you must download any packages you require from the {% data variables.product.company_short %} {% data variables.product.prodname_container_registry %}. This can be done either by using the `--download` flag as part of the `codeql database analyze` command, or running `codeql pack download`. If a package is not publicly available, you will need to use a {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} to authenticate. For more information and an example, see "[Uploading results to {% data variables.product.product_name %}](#uploading-results-to-github)". +Before you can use a {% data variables.product.prodname_codeql %} query pack to analyze a database, you must download any packages you require from the {% data variables.product.company_short %} {% data variables.product.prodname_container_registry %}. This can be done either by using the `--download` flag as part of the `codeql database analyze` command, or running `codeql pack download`. If a package is not publicly available, you will need to use a {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} to authenticate. For more information and an example, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github#uploading-results-to-github)." | Option | Required | Usage | |--------|:--------:|-----| @@ -112,7 +108,6 @@ echo $OCTO-ORG_ACCESS_TOKEN | codeql pack download @@ -33,13 +37,11 @@ Before you generate a {% data variables.product.prodname_codeql %} database, you 1. Check out the code that you want to analyze: - For a branch, check out the head of the branch that you want to analyze. - For a pull request, check out either the head commit of the pull request, or check out a {% data variables.product.prodname_dotcom %}-generated merge commit of the pull request. -1. Set up the environment for the codebase, making sure that any dependencies are available. For more information, see "[Creating databases for non-compiled languages](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis#creating-databases-for-non-compiled-languages)" and "[Creating databases for compiled languages](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis#creating-databases-for-compiled-languages)" in "Preparing your code for {% data variables.product.prodname_codeql %} analysis". +1. Set up the environment for the codebase, making sure that any dependencies are available. For more information, see "[Creating databases for non-compiled languages](#creating-databases-for-non-compiled-languages)" and "[Creating databases for compiled languages](#creating-databases-for-compiled-languages)." 1. Find the build command, if any, for the codebase. Typically this is available in a configuration file in the CI system. Once the codebase is ready, you can run `codeql database create` to create the database. -For information about using the {% data variables.product.prodname_codeql_cli %} in a third-party CI system to create results to display in {% data variables.product.prodname_dotcom %} as code scanning alerts, see [Configuring {% data variables.product.prodname_codeql_cli %} in your CI system](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system). For information about enabling {% data variables.product.prodname_codeql %} code scanning using {% data variables.product.prodname_actions %}, see {% ifversion code-scanning-without-workflow %}"[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)" and {% endif %}"[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning)." - ## Running `codeql database create` {% data variables.product.prodname_codeql %} databases are created by running the following command from the checkout root of your project: @@ -57,6 +59,13 @@ You must specify: {% data reusables.code-scanning.beta-kotlin-or-swift-support %} {% data reusables.code-scanning.beta-ruby-support %} +If your codebase has a build command or script that invokes the build process, we recommend that you specify it as well: + +```shell + codeql database create --command \ + --language= + ``` + You can specify additional options depending on the location of your source file, if the code needs to be compiled, and if you want to create {% data variables.product.prodname_codeql %} databases for more than one language. | Option | Required | Usage | @@ -77,7 +86,7 @@ For full details of all the options you can use when creating databases, see "[A ### Single language example -This example creates a {% data variables.product.prodname_codeql %} database for the repository checked out at `/checkouts/example-repo`. It uses the JavaScript extractor to create a hierarchical representation of the JavaScript and TypeScript code in the repository. The resulting database is stored in `/codeql-dbs/example-repo`. +This example creates a single {% data variables.product.prodname_codeql %} database for the repository checked out at `/checkouts/example-repo`. It uses the JavaScript extractor to create a hierarchical representation of the JavaScript and TypeScript code in the repository. The resulting database is stored in `/codeql-dbs/example-repo`. ```shell $ codeql database create /codeql-dbs/example-repo --language={% ifversion codeql-language-identifiers-311 %}javascript-typescript{% else %}javascript{% endif %} \ @@ -185,11 +194,9 @@ For compiled languages, {% data variables.product.prodname_codeql %} needs to in ### Detecting the build system -{% data reusables.code-scanning.beta-kotlin-or-swift-support %} +The {% data variables.product.prodname_codeql_cli %} includes autobuilders for {% data variables.code-scanning.compiled_languages %} code. {% data variables.product.prodname_codeql %} autobuilders allow you to build projects for compiled languages without specifying any build commands. When an autobuilder is invoked, {% data variables.product.prodname_codeql %} examines the source for evidence of a build system and attempts to run the optimal set of commands required to extract a database. For more information, see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages#about-autobuild-for-codeql)." -The {% data variables.product.prodname_codeql_cli %} includes autobuilders for {% data variables.code-scanning.compiled_languages %} code. {% data variables.product.prodname_codeql %} autobuilders allow you to build projects for compiled languages without specifying any build commands. When an autobuilder is invoked, {% data variables.product.prodname_codeql %} examines the source for evidence of a build system and attempts to run the optimal set of commands required to extract a database. - -An autobuilder is invoked automatically when you execute `codeql database create` for a compiled `--language` if don’t include a +An autobuilder is invoked automatically when you execute `codeql database create` for a compiled `--language` if you don’t include a `--command` option. For example, for a Java codebase, you would simply run: ```shell @@ -450,28 +457,6 @@ steps: # then `codeql github upload-results` ... ``` -## Downloading databases from {% data variables.product.prodname_dotcom_the_website %} +## Next steps -{% data variables.product.prodname_dotcom %} stores {% data variables.product.prodname_codeql %} databases for over 200,000 repos on {% data variables.product.prodname_dotcom_the_website %}, which you can download using the REST API. The list of repos is constantly growing and evolving to make sure that it includes the most interesting codebases for security research. - -You can check if a repository has any {% data variables.product.prodname_codeql %} databases available for download using the `/repos///code-scanning/codeql/databases` endpoint. For example, to check for {% data variables.product.prodname_codeql %} databases using the [{% data variables.product.prodname_cli %}](https://cli.github.com/manual/gh_api) you would run: - -```shell -gh api /repos///code-scanning/codeql/databases -``` - -This command returns information about any {% data variables.product.prodname_codeql %} databases that are available for a repository, including the language the database represents, and when the database was last updated. If no {% data variables.product.prodname_codeql %} databases are available, the response is empty. - -When you have confirmed that a {% data variables.product.prodname_codeql %} database exists for the language you are interested in, you can download it using the following command: - -```shell -gh api /repos///code-scanning/codeql/databases/ -H 'Accept: application/zip' > path/to/local/database.zip -``` - -For more information, see the documentation for the [Get {% data variables.product.prodname_codeql %} database endpoint](/rest/code-scanning?apiVersion=2022-11-28#get-a-codeql-database-for-a-repository). - -Before running an analysis with the {% data variables.product.prodname_codeql_cli %}, you must unzip the databases. - -## Further reading - -- "[Analyzing your projects in {% data variables.product.prodname_codeql %} for VS Code](https://codeql.github.com/docs/codeql-for-visual-studio-code/analyzing-your-projects/#analyzing-your-projects)" +- To learn how to use the {% data variables.product.prodname_codeql_cli %} to analyze the database you created from your code, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries)." diff --git a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli.md b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli.md index 59c8443d80..ee85d619c8 100644 --- a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli.md +++ b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli.md @@ -13,6 +13,10 @@ topics: - CodeQL redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/getting-started-with-the-codeql-cli + - /code-security/secure-coding/running-codeql-cli-in-your-ci-system + - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/running-codeql-cli-in-your-ci-system + - /code-security/secure-coding/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system + - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system --- @@ -23,13 +27,13 @@ redirect_from: To run {% data variables.product.prodname_codeql %} commands, you need to set up the CLI so that it can access the tools, queries, and libraries required to create and analyze databases. -The {% data variables.product.prodname_codeql_cli %} can be set up to support many different use cases and directory -structures. To get started quickly, we recommend adopting a relatively simple -setup, as outlined in the steps below. +The {% data variables.product.prodname_codeql_cli %} can be set up to support many different use cases and directory structures. To get started quickly, we recommend adopting a relatively simple setup, as outlined in the steps below. -If you use Linux, Windows, or macOS version 10.14 ("Mojave") or earlier, simply -follow the steps below. For macOS version 10.15 ("Catalina") or newer, there are additional notes for some of the steps. If you are using macOS -on Apple Silicon (for example, Apple M1), ensure that the [Xcode command-line developer +If you plan to use the {% data variables.product.prodname_codeql_cli %} for security research or to test or contribute queries, you may want a more advanced setup of {% data variables.product.prodname_codeql_cli %}. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/advanced-setup-of-the-codeql-cli)." + +If you are setting up the {% data variables.product.prodname_codeql_cli %} in your CI system, you need to make the full contents of the {% data variables.product.prodname_codeql_cli %} bundle available to every CI server that you want to run {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %} analysis on. For example, you might configure each server to copy the bundle from a central, internal location and extract it. Alternatively, you could use the REST API to get the bundle directly from {% data variables.product.prodname_dotcom %}, ensuring that you benefit from the latest improvements to queries. For more information, see "[AUTOTITLE](/rest/releases)" in the REST API documentation. + +If you are using macOS on Apple Silicon (for example, Apple M1), ensure that the [Xcode command-line developer tools](https://developer.apple.com/downloads/index.action) and [Rosetta 2](https://support.apple.com/en-us/HT211861) are installed. {% note %} @@ -38,28 +42,9 @@ tools](https://developer.apple.com/downloads/index.action) and [Rosetta 2](https {% endnote %} -For information about installing the {% data variables.product.prodname_codeql_cli %} in a CI system to create results to display in {% data variables.product.prodname_dotcom %} as code scanning alerts, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system)." - ### 1. Download the {% data variables.product.prodname_codeql_cli %} zip package -The {% data variables.product.prodname_codeql_cli %} download package is a zip archive containing tools, scripts, and -various {% data variables.product.prodname_codeql %}-specific files. If you don’t have a {% data variables.product.prodname_enterprise %} license then, by -downloading this archive, you are agreeing to the [{% data variables.product.prodname_dotcom %} {% data variables.product.prodname_codeql %} Terms and -Conditions](https://securitylab.github.com/tools/codeql/license). - -You should download the {% data variables.product.prodname_codeql %} bundle from https://github.com/github/codeql-action/releases. The bundle contains: - -- {% data variables.product.prodname_codeql_cli %} product -- A compatible version of the queries and libraries from https://github.com/github/codeql -- Precompiled versions of all the queries included in the bundle - -{% ifversion ghes or ghae %} - -{% note %} -For {% data variables.product.product_name %}{% ifversion ghes %} {{ allVersions[currentVersion].currentRelease }}{% endif %}, we recommend {% data variables.product.prodname_codeql_cli %} version {% data variables.product.codeql_cli_ghes_recommended_version %}. -{% endnote %} - -{% endif %} +{% data reusables.codeql-cli.download-codeql-cli-zip %} You should always use the {% data variables.product.prodname_codeql %} bundle as this ensures compatibility and also gives much better performance than a separate download of the {% data variables.product.prodname_codeql_cli %} and checkout of the {% data variables.product.prodname_codeql %} queries. If you will only be running the CLI on one specific platform, download the appropriate `codeql-bundle-PLATFORM.tar.gz` file. Alternatively, you can download `codeql-bundle.tar.gz`, which contains the CLI for all supported platforms. @@ -67,12 +52,7 @@ You should always use the {% data variables.product.prodname_codeql %} bundle as #### Download information for macOS "Catalina" (or newer) users -From macOS version 10.15 ("Catalina") onwards you need to ensure that your web browser does not automatically extract zip files. If you use Safari, complete the following steps before downloading the {% data variables.product.prodname_codeql_cli %} zip archive: - -1. Open Safari. -1. From the Safari menu, select **Preferences...** or **Settings...** (version 13 "Ventura" onwards). -1. Click the **General** Tab. -1. Ensure the check-box labeled **Open "safe" files after downloading** is unchecked. +{% data reusables.codeql-cli.download-info-macos-catalina-or-newer %} ### 2. Extract the zip archive @@ -80,23 +60,11 @@ For Linux, Windows, and macOS users (version 10.14 "Mojave", and earlier) simply #### Extraction information for macOS "Catalina" (or newer) users -macOS "Catalina", "Big Sur", "Monterey", or "Ventura" users should run the following commands in the Terminal, where `${extraction-root}` is the path to the directory where you will extract the {% data variables.product.prodname_codeql_cli %} zip archive: - -1. `mv ~/Downloads/codeql\*.zip ${extraction-root}` -1. `cd ${extraction-root}` -1. `/usr/bin/xattr -c codeql\*.zip` -1. `unzip codeql\*.zip` +{% data reusables.codeql-cli.extraction-info-macos-catalina-or-newer %} ### 3. Launch `codeql` -Once extracted, you can run {% data variables.product.prodname_codeql %} processes by running the `codeql` executable in a couple of ways: - -- By executing `/codeql/codeql`, where `` is the folder where you extracted the {% data variables.product.prodname_codeql_cli %} -package. -- By adding `/codeql` to your `PATH`, so that you -can run the executable as just `codeql`. - -At this point, you can execute {% data variables.product.prodname_codeql %} commands. For a full list of the {% data variables.product.prodname_codeql_cli %} commands, see "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual)." +{% data reusables.codeql-cli.launch-codeql %} {% note %} @@ -112,7 +80,7 @@ After you extract the {% data variables.product.prodname_codeql_cli %} bundle, y - `codeql resolve qlpacks` if `//codeql` is on the `PATH`. - `//codeql/codeql resolve qlpacks` otherwise. -Extract from successful output: +You should see output similar to the extract below, if successful: ```shell codeql/cpp-all (//qlpacks/codeql/cpp-all/) @@ -136,113 +104,15 @@ codeql/ruby-queries (//qlpacks/codeql/ruby-queries/) ... ``` -You should check that the output contains the expected languages and also that the directory location for the qlpack files is correct. The location should be within the extracted {% data variables.product.prodname_codeql_cli %} bundle, shown in the earlier example as ``, unless you are using a checkout of `github/codeql`. If the {% data variables.product.prodname_codeql_cli %} is unable to locate the qlpacks for the expected languages, check that you downloaded the {% data variables.product.prodname_codeql %} bundle and not a standalone copy of the {% data variables.product.prodname_codeql_cli %}. +You should check that the output contains the expected languages and also that the directory location for the qlpack files is correct. The location should be within the extracted {% data variables.product.prodname_codeql_cli %} bundle, shown in the earlier example as ``. If the {% data variables.product.prodname_codeql_cli %} is unable to locate the qlpacks for the expected languages, check that you downloaded the {% data variables.product.prodname_codeql %} bundle and not a standalone copy of the {% data variables.product.prodname_codeql_cli %}. You can also run `codeql resolve languages` to show which languages are available for database creation. This will list the languages supported by default in your {% data variables.product.prodname_codeql_cli %} package. {% ifversion codeql-packs %} -(Optional) You can download some "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs)" containing pre-compiled queries you would like to run. To do this, run `codeql pack download [...pack-name]`, where `pack-name` is the name of the pack you want to download. The core query packs are a good place to start. They are: - - - `codeql/cpp-queries` - - `codeql/csharp-queries` - - `codeql/go-queries` - - `codeql/java-queries` - - `codeql/javascript-queries` - - `codeql/python-queries` - - `codeql/ruby-queries` - -Alternatively, you can download query packs during the analysis by using the `--download` flag of the `codeql database analyze` command. +Optionally, you can download some CodeQL packs containing pre-compiled queries you would like to run. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs)." {% endif %} -## Generating a token for authentication with {% data variables.product.product_name %} +## Next steps -If you eventually want to upload your results to {% data variables.product.product_name %} to display as code scanning alerts, you will need to generate a {% data variables.product.pat_generic %} with the `security_events` write permission. For more information, see "[AUTOTITLE](/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)." - -If you have installed the {% data variables.product.prodname_codeql_cli %} in a third-party CI system to create results to display in {% data variables.product.prodname_dotcom %} as code scanning alerts, you can use a {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} to upload results to {% data variables.product.product_name %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system#generating-a-token-for-authentication-with-github)." - -## Checking out the {% data variables.product.prodname_codeql %} source code directly - -Some users prefer working with {% data variables.product.prodname_codeql %} query sources directly in order to work on or contribute to the Open Source shared queries. In order to do this, the following steps are recommended. Note that the following instructions are a slightly more complicated alternative to working with {% data variables.product.prodname_codeql %} packages as explained above. - -### 1. Download the {% data variables.product.prodname_codeql_cli %} zip - -Follow [step 1 from the previous section](#1-download-the-codeql-cli-zip-package). - -### 2. Create a new {% data variables.product.prodname_codeql %} directory - -Create a new directory where you can place the CLI and any queries and libraries -you want to use. For example, `$HOME/codeql-home`. - -The CLI’s built-in search operations automatically look in all of its sibling -directories for the files used in database creation and analysis. Keeping these -components in their own directory prevents the CLI searching unrelated sibling -directories while ensuring all files are available without specifying any -further options on the command line. - -### 3. Obtain a local copy of the {% data variables.product.prodname_codeql %} queries - -The [{% data variables.product.prodname_codeql %} repository](https://github.com/github/codeql) contains -the queries and libraries required for {% data variables.product.prodname_codeql %} analysis of all supported languages. -Clone a copy of this repository into `codeql-home`. - -By default, the root of the cloned repository will be called `codeql`. -Rename this folder `codeql-repo` to avoid conflicting with the {% data variables.product.prodname_codeql_cli %} that you will extract in step 1. If you use git on the command line, you can -clone and rename the repository in a single step by running -`git clone git@github.com:github/codeql.git codeql-repo` in the `codeql-home` folder. - -{% ifversion ghae < 3.6 %} -{% note %} - -**Note:** The {% data variables.product.prodname_codeql %} libraries and queries for Go analysis used to live in a separate [{% data variables.product.prodname_codeql %} for Go repository](https://github.com/github/codeql-go/). These have been moved to the `github/codeql` repository. It is no longer necessary to clone the `github/codeql-go` into a separate `codeql-home/codeql-go` folder. - -For more information, see the [Relocation announcement](https://github.com/github/codeql-go/issues/741). - -{% endnote %} -{% endif %} - -{% ifversion codeql-packs %} - -Within this repository, the queries and libraries are organized into {% data variables.product.prodname_codeql %} -packs. Along with the queries themselves, {% data variables.product.prodname_codeql %} packs contain important metadata -that tells the {% data variables.product.prodname_codeql_cli %} how to process the query files. For more information, -see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/customizing-analysis-with-codeql-packs)." -{% endif %} - -{% note %} - -**Note:** There are different versions of the {% data variables.product.prodname_codeql %} queries available for different users. Check out the correct version for your use case: - -- For the queries that are intended to be used with the latest {% data variables.product.prodname_codeql_cli %} release, check out the branch tagged `codeql-cli/latest`. You should use this branch for databases you’ve built using the {% data variables.product.prodname_codeql_cli %}, fetched from code scanning on {% data variables.product.prodname_dotcom %}, or recently downloaded from {% data variables.product.prodname_dotcom_the_website %}. -- For the most up to date {% data variables.product.prodname_codeql %} queries, check out the `main` branch. This branch represents the very latest version of {% data variables.product.prodname_codeql %}’s analysis. - -{% endnote %} - -### 4. Extract the zip archive - -For Linux, Windows, and macOS users (version 10.14 "Mojave", and earlier) simply extract the zip archive into the directory you created in step 2. - -For example, if the path to your copy of the {% data variables.product.prodname_codeql %} repository is `$HOME/codeql-home/codeql-repo`, then extract the CLI into -`$HOME/codeql-home/`. - -### 5. Launch `codeql` - -See [step 3 from the previous section](#3-launch-codeql). - -### 6. Verify your {% data variables.product.prodname_codeql_cli %} setup - -{% data variables.product.prodname_codeql_cli %} has subcommands you can execute to verify that you are correctly set up to create and analyze databases: - -- Run `codeql resolve languages` to show which languages are available for database creation. This will list the languages supported by default in your {% data variables.product.prodname_codeql_cli %} package. -- Run `codeql resolve qlpacks` to show which {% data variables.product.prodname_codeql %} packs the CLI can find. This will display the names of all the {% data variables.product.prodname_codeql %} packs directly available to the {% data variables.product.prodname_codeql_cli %}. This should include: -- Query packs for each supported language, for example, `codeql/{language}-queries`. These packs contain the standard queries that will be run for each analysis. -- Library packs for each supported language, for example, `codeql/{language}-all`. These packs contain query libraries, such as control flow and data flow libraries, that may be useful to query writers. -- Example packs for each supported language, for example, `codeql/{language}-examples`. These packs contain useful snippets of {% data variables.product.prodname_codeql %} that query writers may find useful. -- Legacy packs that ensure custom queries and libraries created using older products are compatible with your version of {% data variables.product.prodname_codeql %}. - -## Using two versions of the {% data variables.product.prodname_codeql_cli %} - -If you want to use the latest {% data variables.product.prodname_codeql %} features to execute queries or {% data variables.product.prodname_codeql %} tests, but also want to prepare databases that are compatible with a specific version of {% data variables.product.prodname_codeql %} code scanning on {% data variables.product.prodname_ghe_server %}, you may need to install two versions of the CLI. The recommended directory setup depends on which versions you want to install: - -- If both versions are 2.0.2 (or newer), you can unpack both CLI archives in the same parent directory. -- If at least one of the versions is 2.0.1 (or older), the unpacked CLI archives cannot be in the same parent directory, but they can share the same grandparent directory. For example, if you unpack version 2.0.2 into `$HOME/codeql-home/codeql-cli`, the older version should be unpacked into `$HOME/codeql-older-version/old-codeql-cli`. Here, the common grandparent is the `$HOME` directory. +To learn how to prepare your code to be analyzed by the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/preparing-your-code-for-codeql-analysis)." diff --git a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github.md b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github.md index ee73fa7573..8b7e41fe5e 100644 --- a/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github.md +++ b/content/code-security/codeql-cli/getting-started-with-the-codeql-cli/uploading-codeql-analysis-results-to-github.md @@ -22,11 +22,19 @@ After analyzing a {% data variables.product.prodname_codeql %} database using th If you used a method other than the {% data variables.product.prodname_codeql_cli %} to generate results, you can use other upload methods. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github)." +{% data reusables.code-scanning.upload-sarif-ghas %} + +## Generating a token for authentication with {% data variables.product.product_name %} + +Before you can upload your results to {% data variables.product.product_name %}, you will first need to generate a {% data variables.product.pat_generic %} with the `security_events` write permission. For more information, see "[AUTOTITLE](/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)." + +If you have installed the {% data variables.product.prodname_codeql_cli %} in a third-party CI system to create results to display in {% data variables.product.prodname_dotcom %} as code scanning alerts, you can use a {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} to upload results to {% data variables.product.product_name %}. For more information, see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system#generating-a-token-for-authentication-with-github)." + ## Uploading results to {% data variables.product.product_name %} {% data reusables.code-scanning.upload-sarif-alert-limit %} -Before you can upload results to {% data variables.product.product_name %}, you must determine the best way to pass the {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} you created earlier to the {% data variables.product.prodname_codeql_cli %} (see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system#generating-a-token-for-authentication-with-github)"). We recommend that you review your CI system's guidance on the secure use of a secret store. The {% data variables.product.prodname_codeql_cli %} supports: +Before you can upload results to {% data variables.product.product_name %}, you must determine the best way to pass the {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} you created in the previous section to the {% data variables.product.prodname_codeql_cli %}. We recommend that you review your CI system's guidance on the secure use of a secret store. The {% data variables.product.prodname_codeql_cli %} supports: - Interfacing with a secret store using the `--github-auth-stdin` option (recommended). - Saving the secret in the environment variable `GITHUB_TOKEN` and running the CLI without including the `--github-auth-stdin` option. @@ -61,6 +69,12 @@ codeql github upload-results \ For more information, see "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/github-upload-results)." +{% note %} + +**Note:** If you analyzed more than one {% data variables.product.prodname_codeql %} database for a single commit, you must have specified a SARIF category for each set of results generated by this command. When you upload the results to {% data variables.product.product_name %}, {% data variables.product.prodname_code_scanning %} uses this category to store the results for each language separately. If you forget to do this, each upload overwrites the previous results. For more information, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/analyzing-your-code-with-codeql-queries#running-codeql-database-analyze)." + +{% endnote %} + ### Basic example of uploading results to {% data variables.product.product_name %} The following example uploads results from the SARIF file `temp/example-repo-js.sarif` to the repository `my-org/example-repo`. It tells the {% data variables.product.prodname_code_scanning %} API that the results are for the commit `deb275d2d5fe9a522a0b7bd8b6b6a1c939552718` on the `main` branch. The example assumes that the {% data variables.product.prodname_github_app %} or {% data variables.product.pat_generic %} created for authentication with {% data variables.product.company_short %}'s REST API uses the `GITHUB_TOKEN` environment variable. diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/about-codeql-workspaces.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/about-codeql-workspaces.md index a1696cd3d2..7283c8842d 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/about-codeql-workspaces.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/about-codeql-workspaces.md @@ -13,8 +13,6 @@ redirect_from: - /code-security/codeql-cli/codeql-cli-reference/about-codeql-workspaces --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About {% data variables.product.prodname_codeql %} workspaces {% data reusables.code-scanning.codeql-action-version-ghes %} diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/advanced-setup-of-the-codeql-cli.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/advanced-setup-of-the-codeql-cli.md new file mode 100644 index 0000000000..01d3460d8e --- /dev/null +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/advanced-setup-of-the-codeql-cli.md @@ -0,0 +1,139 @@ +--- +title: Advanced setup of the CodeQL CLI +intro: 'You can modify your {% data variables.product.prodname_codeql_cli %} setup to use a local checkout of the {% data variables.product.prodname_codeql %} repository for analysis, set up multiple versions of the {% data variables.product.prodname_codeql_cli %}, and analyze databases you have downloaded from {% data variables.product.prodname_dotcom_the_website %}.' +product: '{% data reusables.gated-features.codeql %}' +versions: + fpt: '*' + ghes: '*' + ghae: '*' + ghec: '*' +topics: + - Advanced Security + - Code scanning + - CodeQL +--- + +## About advanced setup of the {% data variables.product.prodname_codeql_cli %} + +If you plan to use {% data variables.product.prodname_codeql %} for more than just code scanning, you may prefer an advanced setup of the {% data variables.product.prodname_codeql_cli %}. + +- If you want to contribute to open source shared {% data variables.product.prodname_codeql %} queries, you may prefer working with the {% data variables.product.prodname_codeql %} source code directly. +- If you want to use the latest {% data variables.product.prodname_codeql %} features to generate code scanning alerts for a codebase, but also want to analyze another codebase that is only compatible with a specific version of the {% data variables.product.prodname_codeql_cli %}, you may want to install multiple versions of the {% data variables.product.prodname_codeql_cli %}. +- If you are researching or developing queries, you may want to download interesting or unique databases from {% data variables.product.prodname_dotcom_the_website %}. + +For information on the most simple setup of the {% data variables.product.prodname_codeql_cli %}, see "[AUTOTITLE](/code-security/codeql-cli/getting-started-with-the-codeql-cli/setting-up-the-codeql-cli)." + +## Checking out the {% data variables.product.prodname_codeql %} source code directly + +Some users prefer working with {% data variables.product.prodname_codeql %} query sources directly in order to work on or contribute to the Open Source shared queries. In order to do this, the following steps are recommended. + +### 1. Download the {% data variables.product.prodname_codeql_cli %} zip + +{% data reusables.codeql-cli.download-codeql-cli-zip %} + +#### Download information for macOS "Catalina" (or newer) users + +{% data reusables.codeql-cli.download-info-macos-catalina-or-newer %} + +### 2. Create a new {% data variables.product.prodname_codeql %} directory + +Create a new directory where you can place the CLI and any queries and libraries +you want to use. For example, `$HOME/codeql-home`. + +The CLI’s built-in search operations automatically look in all of its sibling +directories for the files used in database creation and analysis. Keeping these +components in their own directory prevents the CLI searching unrelated sibling +directories while ensuring all files are available without specifying any +further options on the command line. + +### 3. Obtain a local copy of the {% data variables.product.prodname_codeql %} queries + +The [{% data variables.product.prodname_codeql %} repository](https://github.com/github/codeql) contains +the queries and libraries required for {% data variables.product.prodname_codeql %} analysis of all supported languages. +Clone a copy of this repository into `codeql-home`. + +By default, the root of the cloned repository will be called `codeql`. +Rename this folder `codeql-repo` to avoid conflicting with the {% data variables.product.prodname_codeql_cli %} that you will extract in step 1. If you use git on the command line, you can +clone and rename the repository in a single step by running +`git clone git@github.com:github/codeql.git codeql-repo` in the `codeql-home` folder. + +{% ifversion ghae < 3.6 %} +{% note %} + +**Note:** The {% data variables.product.prodname_codeql %} libraries and queries for Go analysis used to live in a separate [{% data variables.product.prodname_codeql %} for Go repository](https://github.com/github/codeql-go/). These have been moved to the `github/codeql` repository. It is no longer necessary to clone the `github/codeql-go` into a separate `codeql-home/codeql-go` folder. + +For more information, see the [Relocation announcement](https://github.com/github/codeql-go/issues/741). + +{% endnote %} +{% endif %} + +{% ifversion codeql-packs %} + +Within this repository, the queries and libraries are organized into {% data variables.product.prodname_codeql %} +packs. Along with the queries themselves, {% data variables.product.prodname_codeql %} packs contain important metadata +that tells the {% data variables.product.prodname_codeql_cli %} how to process the query files. For more information, +see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs)." +{% endif %} + +{% note %} + +**Note:** There are different versions of the {% data variables.product.prodname_codeql %} queries available for different users. Check out the correct version for your use case: + +- For the queries that are intended to be used with the latest {% data variables.product.prodname_codeql_cli %} release, check out the branch tagged `codeql-cli/latest`. You should use this branch for databases you’ve built using the {% data variables.product.prodname_codeql_cli %} or recently downloaded from {% data variables.product.prodname_dotcom_the_website %}. +- For the most up to date {% data variables.product.prodname_codeql %} queries, check out the `main` branch. This branch represents the very latest version of {% data variables.product.prodname_codeql %}’s analysis. + +{% endnote %} + +### 4. Extract the zip archive + +For Linux, Windows, and macOS users (version 10.14 "Mojave", and earlier) simply extract the zip archive into the directory you created in step 2. + +For example, if the path to your copy of the {% data variables.product.prodname_codeql %} repository is `$HOME/codeql-home/codeql-repo`, then extract the CLI into +`$HOME/codeql-home/`. + +#### Extraction information for macOS "Catalina" (or newer) users + +{% data reusables.codeql-cli.extraction-info-macos-catalina-or-newer %} + +### 5. Launch `codeql` + +{% data reusables.codeql-cli.launch-codeql %} + +### 6. Verify your {% data variables.product.prodname_codeql_cli %} setup + +{% data variables.product.prodname_codeql_cli %} has subcommands you can execute to verify that you are correctly set up to create and analyze databases: + +- Run `codeql resolve languages` to show which languages are available for database creation. This will list the languages supported by default in your {% data variables.product.prodname_codeql_cli %} package. +- Run `codeql resolve qlpacks` to show which {% data variables.product.prodname_codeql %} packs the CLI can find. This will display the names of all the {% data variables.product.prodname_codeql %} packs directly available to the {% data variables.product.prodname_codeql_cli %}. This should include: +- Query packs for each supported language, for example, `codeql/{language}-queries`. These packs contain the standard queries that will be run for each analysis. +- Library packs for each supported language, for example, `codeql/{language}-all`. These packs contain query libraries, such as control flow and data flow libraries, that may be useful to query writers. +- Example packs for each supported language, for example, `codeql/{language}-examples`. These packs contain useful snippets of {% data variables.product.prodname_codeql %} that query writers may find useful. +- Legacy packs that ensure custom queries and libraries created using older products are compatible with your version of {% data variables.product.prodname_codeql %}. + +## Using two versions of the {% data variables.product.prodname_codeql_cli %} + +If you want to use the latest {% data variables.product.prodname_codeql %} features to execute queries or {% data variables.product.prodname_codeql %} tests, but also want to prepare databases that are compatible with a specific version of {% data variables.product.prodname_codeql %} code scanning on {% data variables.product.prodname_ghe_server %}, you may need to install two versions of the CLI. You can download the versions of the {% data variables.product.prodname_codeql_cli %} that you want, and unpack both CLI archives in the same parent directory. + +## Downloading databases from {% data variables.product.prodname_dotcom_the_website %} + +{% data variables.product.prodname_dotcom %} stores {% data variables.product.prodname_codeql %} databases for over 200,000 repos on {% data variables.product.prodname_dotcom_the_website %}, which you can download using the REST API. The list of repos is constantly growing and evolving to make sure that it includes the most interesting codebases for security research. + +You can also analyze databases from {% data variables.product.prodname_dotcom_the_website %} using the {% data variables.product.prodname_codeql %} for VS Code extension. For more information, see "[Analyzing your projects](https://codeql.github.com/docs/codeql-for-visual-studio-code/analyzing-your-projects)." + +You can check if a repository has any {% data variables.product.prodname_codeql %} databases available for download using the `/repos///code-scanning/codeql/databases` endpoint. For example, to check for {% data variables.product.prodname_codeql %} databases using the [{% data variables.product.prodname_cli %}](https://cli.github.com/manual/gh_api) you would run: + +```shell +gh api /repos///code-scanning/codeql/databases +``` + +This command returns information about any {% data variables.product.prodname_codeql %} databases that are available for a repository, including the language the database represents, and when the database was last updated. If no {% data variables.product.prodname_codeql %} databases are available, the response is empty. + +When you have confirmed that a {% data variables.product.prodname_codeql %} database exists for the language you are interested in, you can download it using the following command: + +```shell +gh api /repos///code-scanning/codeql/databases/ -H 'Accept: application/zip' > path/to/local/database.zip +``` + +For more information, see the documentation for the [Get {% data variables.product.prodname_codeql %} database endpoint](/rest/code-scanning?apiVersion=2022-11-28#get-a-codeql-database-for-a-repository). + +Before running an analysis with the {% data variables.product.prodname_codeql_cli %}, you must unzip the databases. diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs.md index e4468f860c..66d6459ded 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs.md @@ -13,15 +13,13 @@ redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/creating-and-working-with-codeql-packs --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - {% data reusables.codeql-cli.beta-note-package-management %} ## About {% data variables.product.prodname_codeql %} packs and the {% data variables.product.prodname_codeql_cli %} {% data reusables.code-scanning.codeql-cli-version-ghes %} -With {% data variables.product.prodname_codeql %} packs and the package management commands in the {% data variables.product.prodname_codeql_cli %}, you can publish your custom queries and integrate them into your codebase analysis. +{% data variables.product.prodname_codeql %} packs are used to create, share, depend on, and run {% data variables.product.prodname_codeql %} queries and libraries. {% data variables.product.prodname_codeql %} packs contain queries, library files, query suites, and metadata. With {% data variables.product.prodname_codeql %} packs and the package management commands in the {% data variables.product.prodname_codeql_cli %}, you can publish your custom queries and integrate them into your codebase analysis. There are{% ifversion codeql-model-packs-java %} three{% else %} two{% endif %} types of {% data variables.product.prodname_codeql %} packs: {% ifversion codeql-model-packs-java %}query packs, library packs, and model packs{% else %} query packs and library packs{% endif %}. @@ -29,7 +27,7 @@ There are{% ifversion codeql-model-packs-java %} three{% else %} two{% endif %} - Library packs are designed to be used by query packs (or other library packs) and do not contain queries themselves. The libraries are not compiled {% ifversion query-pack-compatibility %}separately{% else %}and there is no compilation cache included when the pack is published{% endif %}.{% ifversion codeql-model-packs-java %} -- Model packs can be used to expand {% data variables.product.prodname_code_scanning %} analysis to include dependencies that are not supported by default. Model packs are currently in beta and subject to change. During the beta, model packs are available for Java analysis at the repository level. For more information about creating your own model packs, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#creating-a-model-pack)."{% endif %} +- Model packs can be used to expand {% data variables.product.prodname_code_scanning %} analysis to include dependencies that are not supported by default. Model packs are currently in beta and subject to change. During the beta, model packs are available for Java analysis at the repository level. For more information about creating your own model packs, see "[{% data variables.product.prodname_codeql %}](#creating-a-codeql-model-pack)."{% endif %} You can use the `pack` command in the {% data variables.product.prodname_codeql_cli %} to create {% data variables.product.prodname_codeql %} packs, add dependencies to packs, and install or update dependencies. You can also publish and download {% data variables.product.prodname_codeql %} packs using the `pack` command. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs)." @@ -37,6 +35,27 @@ You can use the `pack` command in the {% data variables.product.prodname_codeql_ For more information about compatibility between published query packs and different {% data variables.product.prodname_codeql %} releases, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs#about-codeql-pack-compatibility)." {% endif %} +The standard {% data variables.product.prodname_codeql %} packages for all supported languages are published in the [{% data variables.product.prodname_container_registry %}](https://github.com/orgs/codeql/packages). The [{% data variables.product.prodname_codeql %} repository](https://github.com/github/codeql) contains source files for the standard {% data variables.product.prodname_codeql %} packs for all supported languages. The core query packs, which are included in the CodeQL CLI bundle, but you can otherwise download, are: + + - `codeql/cpp-queries` + - `codeql/csharp-queries` + - `codeql/go-queries` + - `codeql/java-queries` + - `codeql/javascript-queries` + - `codeql/python-queries` + - `codeql/ruby-queries` + +## {% data variables.product.prodname_codeql %} pack structure + +A {% data variables.product.prodname_codeql %} pack must contain a file called `qlpack.yml` in its root directory. In the `qlpack.yml` file, the `name:` field must have a value that follows the format of `/`, where `` is the {% data variables.product.prodname_dotcom %} organization or user account that the pack will be published to and `` is the name of the pack. Additionally, query packs and library packs with {% data variables.product.prodname_codeql %} tests contain a `codeql-pack.lock.yml` file that contains the resolved dependencies of the pack. This file is generated during a call to the `codeql pack install` command, is not meant to be edited by hand, and should be added to your version control system. + +The other files and directories within the pack should be logically organized. For example, typically: + +- Queries are organized into directories for specific categories. + +- Queries for specific products, libraries, and frameworks are organized into +their own top-level directories. + ## Creating a {% data variables.product.prodname_codeql %} pack You can create a {% data variables.product.prodname_codeql %} pack by running the following command from the checkout root of your project: diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-codeql-query-suites.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-codeql-query-suites.md index 3562bd8a2b..f3e224f2d2 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-codeql-query-suites.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-codeql-query-suites.md @@ -16,8 +16,6 @@ redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/creating-codeql-query-suites --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About creating {% data variables.product.prodname_codeql %} query suites {% data reusables.code-scanning.codeql-cli-version-ghes %} diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/csv-output.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/csv-output.md new file mode 100644 index 0000000000..9a9632ded3 --- /dev/null +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/csv-output.md @@ -0,0 +1,35 @@ +--- +title: CodeQL CLI CSV output +intro: 'You can output results from the {% data variables.product.prodname_codeql_cli %} in CSV format to share with other systems.' +product: '{% data reusables.gated-features.codeql %}' +allowTitleToDifferFromFilename: true +versions: + fpt: '*' + ghes: '*' + ghae: '*' + ghec: '*' +topics: + - Advanced Security + - Code scanning + - CodeQL +--- + +## About CSV output + +You can save analysis results from the {% data variables.product.prodname_codeql_cli %} in a number of different formats, including SARIF and CSV. We do generally recommend SARIF because it is a standard output for static analysis tools and easier to parse. You can also upload SARIF files to {% data variables.product.product_name %}. However, CSV format may be useful if you need to further process the analysis results using your own tools. For more information on selecting a file format for your analysis results, see "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual/database-analyze)." + +For more information about the SARIF format, see [AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/sarif-output). + +If you choose to generate results in CSV format, then each line in the output file corresponds to an alert. Each line is a comma-separated list with the following information. + +**Property**|**Description**|**Example** +-----|-----|----- +Name | Name of the query that identified the result. | `Inefficient regular expression` +Description | Description of the query.| `A regular expression that requires exponential time to match certain inputs can be a performance bottleneck, and may be vulnerable to denial-of-service attacks.` +Severity | Severity of the query.| `error` +Message | Alert message.| `This part of the regular expression may cause exponential backtracking on strings containing many repetitions of '\\\\'.` +Path | Path of the file containing the alert. | `/vendor/codemirror/markdown.js` +Start line | Line of the file where the code that triggered the alert begins. | `617` +Start column | Column of the start line that marks the start of the alert code. Not included when equal to 1. | `32` +End line | Line of the file where the code that triggered the alert ends. Not included when the same value as the start line. | `64` +End column | Where available, the column of the end line that marks the end of the alert code. Otherwise the end line is repeated. | `617` diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/exit-codes.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/exit-codes.md index 817c2167cd..4570eab193 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/exit-codes.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/exit-codes.md @@ -16,8 +16,6 @@ redirect_from: - /code-security/codeql-cli/codeql-cli-reference/exit-codes --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About exit codes The {% data variables.product.prodname_codeql_cli %} reports the status of each command it runs as an exit code. diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/extractor-options.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/extractor-options.md index c0ce624c2a..130c881929 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/extractor-options.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/extractor-options.md @@ -18,8 +18,6 @@ redirect_from: -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About extractors The {% data variables.product.prodname_codeql_cli %} uses special programs, called extractors, to extract information from the source code of a software system into a database that can be queried. You can customize the behavior of extractors by setting extractor configuration options through the {% data variables.product.prodname_codeql_cli %}. diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/index.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/index.md index 4448c65796..db4690830d 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/index.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/index.md @@ -13,6 +13,7 @@ topics: - Code scanning - CodeQL children: + - /advanced-setup-of-the-codeql-cli - /about-codeql-workspaces - /using-custom-queries-with-the-codeql-cli - /creating-codeql-query-suites @@ -23,6 +24,7 @@ children: - /specifying-command-options-in-a-codeql-configuration-file - /query-reference-files - /sarif-output + - /csv-output - /extractor-options - /exit-codes redirect_from: diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs.md index bbdc67e899..0b5b9b69bc 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/publishing-and-using-codeql-packs.md @@ -13,8 +13,6 @@ redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/publishing-and-using-codeql-packs --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - {% data reusables.codeql-cli.beta-note-package-management %} ## Configuring the `qlpack.yml` file before publishing @@ -417,7 +415,7 @@ In most cases, the `codeql-pack.lock.yml` file is only relevant for query packs ## Examples of custom {% data variables.product.prodname_codeql %} packs -When you write custom queries or tests, you should save them in custom {% data variables.product.prodname_codeql %} packs. For simplicity, try to organize each pack logically. For more information, see "[{% data variables.product.prodname_codeql %} pack structure](#codeql-pack-structure)." Save files for queries and tests in separate packs and, where possible, organize custom packs into specific folders for each target language. This is particularly useful if you intend to publish your {% data variables.product.prodname_codeql %} packs so they can be shared with others or used in code scanning. For more information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql)." +When you write custom queries or tests, you should save them in custom {% data variables.product.prodname_codeql %} packs. For simplicity, try to organize each pack logically. For more information, see "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/creating-and-working-with-codeql-packs#codeql-pack-structure)." Save files for queries and tests in separate packs and, where possible, organize custom packs into specific folders for each target language. This is particularly useful if you intend to publish your {% data variables.product.prodname_codeql %} packs so they can be shared with others or used in code scanning. For more information, see "[AUTOTITLE](/code-security/code-scanning/introduction-to-code-scanning/about-code-scanning-with-codeql)." ### {% data variables.product.prodname_codeql %} packs for custom libraries diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/query-reference-files.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/query-reference-files.md index d42f16c155..f00f9fca1f 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/query-reference-files.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/query-reference-files.md @@ -16,8 +16,6 @@ redirect_from: - /code-security/codeql-cli/codeql-cli-reference/query-reference-files --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About query reference files A query reference file is text file that defines the location of one query to test. diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/sarif-output.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/sarif-output.md index ff194fabe3..ab04a7f1c4 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/sarif-output.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/sarif-output.md @@ -17,8 +17,6 @@ redirect_from: - /code-security/codeql-cli/codeql-cli-reference/sarif-output --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About SARIF output SARIF is designed to represent the output of a broad range of static analysis tools, and there are many features in the SARIF specification that are diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/specifying-command-options-in-a-codeql-configuration-file.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/specifying-command-options-in-a-codeql-configuration-file.md index 3336a13db6..234302dba6 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/specifying-command-options-in-a-codeql-configuration-file.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/specifying-command-options-in-a-codeql-configuration-file.md @@ -16,8 +16,6 @@ redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/specifying-command-options-in-a-codeql-configuration-file --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About specifying command options in a {% data variables.product.prodname_codeql %} configuration file You can save default or frequently used options for your commands in a per-user configuration file. diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-custom-queries.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-custom-queries.md index 4dafc1d376..49ff20dbf3 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-custom-queries.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-custom-queries.md @@ -13,8 +13,6 @@ redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/testing-custom-queries --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About testing custom queries {% data variables.product.prodname_codeql %} provides a simple test framework for automated regression testing diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-query-help-files.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-query-help-files.md index a679e937f4..0e13feb187 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-query-help-files.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-query-help-files.md @@ -16,8 +16,6 @@ redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/testing-query-help-files --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About testing query help files Test query help files by rendering them as Markdown to ensure they are valid before uploading them to the {% data variables.product.prodname_codeql %} repository or using them in code scanning. diff --git a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/using-custom-queries-with-the-codeql-cli.md b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/using-custom-queries-with-the-codeql-cli.md index ef539152e7..fb3f10cb59 100644 --- a/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/using-custom-queries-with-the-codeql-cli.md +++ b/content/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/using-custom-queries-with-the-codeql-cli.md @@ -16,8 +16,6 @@ redirect_from: - /code-security/codeql-cli/using-the-codeql-cli/using-custom-queries-with-the-codeql-cli --- -{% data reusables.codeql-cli.codeql-site-migration-note %} - ## About custom queries and the {% data variables.product.prodname_codeql_cli %} @@ -70,6 +68,25 @@ For each {% data variables.product.prodname_codeql %} pack, the `qlpack.yml` fil {% endif %} +## Including query help for custom {% data variables.product.prodname_codeql %} queries in SARIF files + +If you use the {% data variables.product.prodname_codeql_cli %} to run code scanning analyses on third party CI/CD systems, +you can include the query help for your custom queries in SARIF files generated during an analysis. +After uploading the SARIF file to {% data variables.product.prodname_dotcom %}, the query help is shown in the code scanning UI for any +alerts generated by the custom queries. + +From {% data variables.product.prodname_codeql_cli %} v2.7.1 onwards, you can include markdown-rendered query help in SARIF files +by providing the `--sarif-add-query-help` option when running +`codeql database analyze`. + +You can write query help for custom queries directly in a markdown file and save it alongside the +corresponding query. Alternatively, for consistency with the standard {% data variables.product.prodname_codeql %} queries, +you can write query help in the `.qhelp` format. Query help written in `.qhelp` +files can’t be included in SARIF files, and they can’t be processed by code +scanning so must be converted to markdown before running +the analysis. For more information, see "[Query help files](https://codeql.github.com/docs/writing-codeql-queries/query-help-files/#query-help-files)" +and "[AUTOTITLE](/code-security/codeql-cli/using-the-advanced-functionality-of-the-codeql-cli/testing-query-help-files)." + ## Contributing to the {% data variables.product.prodname_codeql %} repository If you would like to share your query with other {% data variables.product.prodname_codeql %} users, you can open a pull request in the [{% data variables.product.prodname_codeql %} repository](https://github.com/github/codeql). For more information, see [Contributing to {% data variables.product.prodname_codeql %}](https://github.com/github/codeql/blob/main/CONTRIBUTING.md). diff --git a/content/code-security/index.md b/content/code-security/index.md index dc65fa4281..e4f727c218 100644 --- a/content/code-security/index.md +++ b/content/code-security/index.md @@ -20,7 +20,7 @@ featuredLinks: - '{% ifversion ghes < 3.9 or ghae < 3.9 %}/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning{% endif %}' - '{% ifversion ghae %}/code-security/secret-scanning/configuring-secret-scanning-for-your-repositories{% endif %}' - '{% ifversion ghae %}/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github{% endif %}' - - '{% ifversion ghae %}/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system{% endif %}' + - '{% ifversion ghae %}/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system{% endif %}' - /code-security/supply-chain-security/end-to-end-supply-chain/end-to-end-supply-chain-overview popular: - '{% ifversion ghes %}/admin/release-notes{% endif %}' diff --git a/data/learning-tracks/code-security.yml b/data/learning-tracks/code-security.yml index 149ad441e2..e0a7e7e30b 100644 --- a/data/learning-tracks/code-security.yml +++ b/data/learning-tracks/code-security.yml @@ -207,18 +207,6 @@ code_security_integration: - >- /code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning - /rest/code-scanning -code_security_ci: - title: Run CodeQL code scanning in your CI - description: >- - Set up CodeQL within your existing CI and upload results to GitHub code - scanning. - guides: - - >- - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system - - >- - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/installing-codeql-cli-in-your-ci-system - - >- - /code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/configuring-codeql-cli-in-your-ci-system end_to_end_supply_chain: title: End-to-end supply chain description: >- diff --git a/data/reusables/code-scanning/enabling-options.md b/data/reusables/code-scanning/enabling-options.md index eb010bc5c6..4dba0f99b9 100644 --- a/data/reusables/code-scanning/enabling-options.md +++ b/data/reusables/code-scanning/enabling-options.md @@ -1,6 +1,6 @@ {% ifversion code-scanning-without-workflow %} -- **{% data variables.product.prodname_codeql %}**: Uses {% data variables.product.prodname_actions %} with either default setup (see "[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)") or advanced setup (see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning#configuring-advanced-setup-for-code-scanning-with-codeql)"), or runs {% data variables.product.prodname_codeql %} analysis in a third-party continuous integration (CI) system (see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system)"). +- **{% data variables.product.prodname_codeql %}**: Uses {% data variables.product.prodname_actions %} with either default setup (see "[AUTOTITLE](/code-security/code-scanning/enabling-code-scanning/configuring-default-setup-for-code-scanning)") or advanced setup (see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning#configuring-advanced-setup-for-code-scanning-with-codeql)"), or runs {% data variables.product.prodname_codeql %} analysis in a third-party continuous integration (CI) system (see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)"). {% else %} -- **{% data variables.product.prodname_codeql %}**: Uses {% data variables.product.prodname_actions %} (see below) or runs {% data variables.product.prodname_codeql %} analysis in a third-party continuous integration (CI) system (see "[AUTOTITLE](/code-security/code-scanning/using-codeql-code-scanning-with-your-existing-ci-system/about-codeql-code-scanning-in-your-ci-system)"). +- **{% data variables.product.prodname_codeql %}**: Uses {% data variables.product.prodname_actions %} (see below) or runs {% data variables.product.prodname_codeql %} analysis in a third-party continuous integration (CI) system (see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/using-code-scanning-with-your-existing-ci-system)"). {% endif %} - **Third‑party**: Uses {% data variables.product.prodname_actions %} {% ifversion fpt or ghec %}(see "[AUTOTITLE](/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/configuring-advanced-setup-for-code-scanning#configuring-code-scanning-using-third-party-workflows)") {% endif %}or third-party tools and uploads results to {% data variables.product.product_name %} (see "[AUTOTITLE](/code-security/code-scanning/integrating-with-code-scanning/uploading-a-sarif-file-to-github)"). diff --git a/data/reusables/codeql-cli/codeql-site-migration-note.md b/data/reusables/codeql-cli/codeql-site-migration-note.md deleted file mode 100644 index 5267d1f319..0000000000 --- a/data/reusables/codeql-cli/codeql-site-migration-note.md +++ /dev/null @@ -1,5 +0,0 @@ -{% note %} - -**Note:** This article was migrated from the [CodeQL documentation website](https://codeql.github.com/docs/codeql-cli/) in January 2023. - -{% endnote %} diff --git a/data/reusables/codeql-cli/download-codeql-cli-zip.md b/data/reusables/codeql-cli/download-codeql-cli-zip.md new file mode 100644 index 0000000000..2ba9445c16 --- /dev/null +++ b/data/reusables/codeql-cli/download-codeql-cli-zip.md @@ -0,0 +1,18 @@ +The {% data variables.product.prodname_codeql_cli %} download package is a zip archive containing tools, scripts, and +various {% data variables.product.prodname_codeql %}-specific files. If you don’t have a {% data variables.product.prodname_enterprise %} license then, by +downloading this archive, you are agreeing to the [{% data variables.product.prodname_dotcom %} {% data variables.product.prodname_codeql %} Terms and +Conditions](https://securitylab.github.com/tools/codeql/license). + +You should download the {% data variables.product.prodname_codeql %} bundle from https://github.com/github/codeql-action/releases. The bundle contains: + +- {% data variables.product.prodname_codeql_cli %} product +- A compatible version of the queries and libraries from https://github.com/github/codeql +- Precompiled versions of all the queries included in the bundle + +{% ifversion ghes or ghae %} + +{% note %} +For {% data variables.product.product_name %}{% ifversion ghes %} {{ allVersions[currentVersion].currentRelease }}{% endif %}, we recommend {% data variables.product.prodname_codeql_cli %} version {% data variables.product.codeql_cli_ghes_recommended_version %}. +{% endnote %} + +{% endif %} diff --git a/data/reusables/codeql-cli/download-info-macos-catalina-or-newer.md b/data/reusables/codeql-cli/download-info-macos-catalina-or-newer.md new file mode 100644 index 0000000000..d6c446cdfb --- /dev/null +++ b/data/reusables/codeql-cli/download-info-macos-catalina-or-newer.md @@ -0,0 +1,6 @@ +From macOS version 10.15 ("Catalina") onwards you need to ensure that your web browser does not automatically extract zip files. If you use Safari, complete the following steps before downloading the {% data variables.product.prodname_codeql_cli %} zip archive: + +1. Open Safari. +1. From the Safari menu, select **Preferences...** or **Settings...** (version 13 "Ventura" onwards). +1. Click the **General** Tab. +1. Ensure the check-box labeled **Open "safe" files after downloading** is unchecked. diff --git a/data/reusables/codeql-cli/extraction-info-macos-catalina-or-newer.md b/data/reusables/codeql-cli/extraction-info-macos-catalina-or-newer.md new file mode 100644 index 0000000000..6b589e2c01 --- /dev/null +++ b/data/reusables/codeql-cli/extraction-info-macos-catalina-or-newer.md @@ -0,0 +1,6 @@ +macOS "Catalina", "Big Sur", "Monterey", or "Ventura" users should run the following commands in the Terminal, where `${extraction-root}` is the path to the directory where you will extract the {% data variables.product.prodname_codeql_cli %} zip archive: + +1. `mv ~/Downloads/codeql\*.zip ${extraction-root}` +1. `cd ${extraction-root}` +1. `/usr/bin/xattr -c codeql\*.zip` +1. `unzip codeql\*.zip` diff --git a/data/reusables/codeql-cli/launch-codeql.md b/data/reusables/codeql-cli/launch-codeql.md new file mode 100644 index 0000000000..0957064659 --- /dev/null +++ b/data/reusables/codeql-cli/launch-codeql.md @@ -0,0 +1,8 @@ +Once extracted, you can run {% data variables.product.prodname_codeql %} processes by running the `codeql` executable in a couple of ways: + +- By executing `/codeql/codeql`, where `` is the folder where you extracted the {% data variables.product.prodname_codeql_cli %} +package. +- By adding `/codeql` to your `PATH`, so that you +can run the executable as just `codeql`. + +At this point, you can execute {% data variables.product.prodname_codeql %} commands. For a full list of the {% data variables.product.prodname_codeql_cli %} commands, see "[AUTOTITLE](/code-security/codeql-cli/codeql-cli-manual)."