1
0
mirror of synced 2025-12-19 18:10:59 -05:00
Files
docs/content/code-security/secret-scanning/using-advanced-secret-scanning-and-push-protection-features/custom-patterns/defining-custom-patterns-for-secret-scanning.md
2024-07-05 10:00:57 +02:00

18 KiB

title, shortTitle, intro, product, redirect_from, versions, type, topics
title shortTitle intro product redirect_from versions type topics
Defining custom patterns for secret scanning Define custom patterns You can define your own custom patterns to extend the capabilities of {% data variables.product.prodname_secret_scanning %} by generating one or more regular expressions. {% data reusables.gated-features.secret-scanning %}
/code-security/secret-security/defining-custom-patterns-for-secret-scanning
/code-security/secret-scanning/defining-custom-patterns-for-secret-scanning
ghes ghec
* *
how_to
Advanced Security
Secret scanning

About custom patterns for {% data variables.product.prodname_secret_scanning %}

You can define custom patterns to identify secrets that are not detected by the default patterns supported by {% data variables.product.prodname_secret_scanning %}. For example, you might have a secret pattern that is internal to your organization. For details of the supported secrets and service providers, see "AUTOTITLE."

You can define custom patterns for your enterprise, organization, or repository. {% data variables.product.prodname_secret_scanning_caps %} supports up to 500 custom patterns for each organization or enterprise account, and up to 100 custom patterns per repository.

{% ifversion secret-scanning-push-protection-custom-patterns %}You can also enable push protection for custom patterns. For more information about push protection, see "AUTOTITLE."{% endif %}

About using regular expressions for custom patterns

You can specify custom patterns for {% data variables.product.prodname_secret_scanning %} as one or more regular expressions.

{% data variables.product.prodname_secret_scanning_caps %} uses the Hyperscan library and only supports Hyperscan regex constructs, which are a subset of PCRE syntax. Hyperscan option modifiers are not supported. For more information on Hyperscan pattern constructs, see "Pattern support" in the Hyperscan documentation.

{% ifversion secret-scanning-custom-pattern-ai-generated %}Regular expressions can be entered manually or generated using the regular expression generator.

Regular expression syntax for manually defining custom patterns {% endif %}

The More options {% octicon "chevron-down" aria-label="down" %} section in the UI helps you write regular expressions manually.

  • Secret format: an expression that describes the format of the secret itself.
  • Before secret: an expression that describes the characters that come before the secret. By default, this is set to \A|[^0-9A-Za-z] which means that the secret must be at the start of a line or be preceded by a non-alphanumeric character.
  • After secret: an expression that describes the characters that come after the secret. By default, this is set to \z|[^0-9A-Za-z] which means that the secret must be followed by a new line or a non-alphanumeric character.
  • Additional match requirements: one or more optional expressions that the secret itself must or must not match.

For simple tokens you will usually only need to specify a secret format. The other fields provide flexibility so that you can specify more complex secrets without creating complex regular expressions. For an example of a custom pattern, see "Example of a custom pattern specified using additional requirements" below.

{% ifversion secret-scanning-custom-pattern-ai-generated %}

Using the regular expression generator

{% data reusables.secret-scanning.regular-expression-generator-overview %} For more information, see "AUTOTITLE" and "AUTOTITLE."

{% endif %}

Defining a custom pattern for a repository

Before defining a custom pattern, you must ensure that {% data variables.product.prodname_secret_scanning %} is enabled on your repository. For more information, see "AUTOTITLE."

{% data reusables.repositories.navigate-to-repo %} {% data reusables.repositories.sidebar-settings %} {% data reusables.repositories.navigate-to-code-security-and-analysis %} {% data reusables.repositories.navigate-to-ghas-settings %} {% data reusables.advanced-security.secret-scanning-new-custom-pattern %} {% data reusables.advanced-security.secret-scanning-add-custom-pattern-details %}{% ifversion custom-pattern-dry-run-ga %}

  1. When you're ready to test your new custom pattern, to identify matches in the repository without creating alerts, click Save and dry run. {% data reusables.advanced-security.secret-scanning-dry-run-results %} {% endif %} {% data reusables.advanced-security.secret-scanning-create-custom-pattern %}{% ifversion secret-scanning-push-protection-custom-patterns %}

  2. Optionally, to enable push protection for your custom pattern, click Enable. {% note %}

    Note: The "Enable" button isn't available until after the dry run succeeds and you publish the pattern.

    {% endnote %}

    For more information about push protection, see "AUTOTITLE."

{% endif %}

After your pattern is created, {% data reusables.secret-scanning.secret-scanning-process %} For more information on viewing {% data variables.secret-scanning.alerts %}, see "AUTOTITLE."

Example of a custom pattern specified using additional requirements

A company has an internal token with five characteristics. They use the different fields to specify how to identify tokens as follows:

Characteristic Field and regular expression
Length between 5 and 10 characters Secret format: [$#%@AA-Za-z0-9]{5,10}
Does not end in a . After secret: [^\.]
Contains numbers and uppercase letters Additional requirements: secret must match [A-Z] and [0-9]
Does not include more than one lowercase letter in a row Additional requirements: secret must not match [a-z]{2,}
Contains one of $%@! Additional requirements: secret must match [$%@!]

These tokens would match the custom pattern described above:

a9@AAfT!         # Secret string match: a9@AAfT
ee95GG@ZA942@aa  # Secret string match: @ZA942@a
a9@AA!ee9        # Secret string match: a9@AA

These strings would not match the custom pattern described above:

a9@AA.!
a@AAAAA
aa9@AA!ee9
aAAAe9

Defining a custom pattern for an organization

Before defining a custom pattern, you must ensure that you enable {% data variables.product.prodname_secret_scanning %} for the repositories that you want to scan in your organization. To enable {% data variables.product.prodname_secret_scanning %} on all repositories in your organization, see "AUTOTITLE."

{% data reusables.profile.access_org %} {% data reusables.profile.org_settings %} {% data reusables.organizations.security-and-analysis %}

{% ifversion security-configurations %} {% data reusables.security-configurations.changed-org-settings-global-settings-callout %} For next steps on defining a custom pattern for your organization with {% data variables.product.prodname_global_settings %}, see "AUTOTITLE." {% endif %}

{% data reusables.repositories.navigate-to-ghas-settings %} {% data reusables.advanced-security.secret-scanning-new-custom-pattern %} {% data reusables.advanced-security.secret-scanning-add-custom-pattern-details %} {%- ifversion custom-pattern-dry-run-ga %}

  1. When you're ready to test your new custom pattern, to identify matches in select repositories without creating alerts, click Save and dry run. {% data reusables.advanced-security.secret-scanning-dry-run-select-repos %} {% data reusables.advanced-security.secret-scanning-dry-run-results %} {%- endif %} {% data reusables.advanced-security.secret-scanning-create-custom-pattern %}{% ifversion secret-scanning-push-protection-custom-patterns %}
  2. Optionally, to enable push protection for your custom pattern, click Enable. For more information, see "AUTOTITLE."

{% indented_data_reference reusables.secret-scanning.push-protection-org-notes spaces=3 %}{% endif %}

After your pattern is created, {% data variables.product.prodname_secret_scanning %} scans for any secrets in repositories in your organization, including their entire Git history on all branches. Organization owners and repository administrators will be alerted to any secrets found and can review the alert in the repository where the secret is found. For more information on viewing {% data variables.secret-scanning.alerts %}, see "AUTOTITLE."

Defining a custom pattern for an enterprise account

Before defining a custom pattern, you must ensure that you enable secret scanning for your enterprise account. For more information, see "[Enabling {% data variables.product.prodname_GH_advanced_security %} for your enterprise]({% ifversion fpt or ghec %}/enterprise-server@latest/{% endif %}/admin/advanced-security/enabling-github-advanced-security-for-your-enterprise)."

{% note %}

{% ifversion custom-pattern-dry-run-ga %} Notes:

  • At the enterprise level, only the creator of a custom pattern can edit the pattern, and use it in a dry run.
  • {% data reusables.secret-scanning.dry-runs-enterprise-permissions %} {% else %} Note: As there is no dry-run functionality, we recommend that you test your custom patterns in a repository before defining them for your entire enterprise. That way, you can avoid creating excess false-positive {% data variables.secret-scanning.alerts %}.

{% endif %}

{% endnote %}

{% data reusables.enterprise-accounts.access-enterprise %} {% data reusables.enterprise-accounts.policies-tab %}{% ifversion security-feature-enablement-policies %} {% data reusables.enterprise-accounts.code-security-and-analysis-policies %}

  1. Under "Code security and analysis", click Security features.{% else %} {% data reusables.enterprise-accounts.advanced-security-policies %} {% data reusables.enterprise-accounts.advanced-security-security-features %}{% endif %}
  2. Under "Secret scanning custom patterns", click New pattern. {% data reusables.advanced-security.secret-scanning-add-custom-pattern-details %} {%- ifversion custom-pattern-dry-run-ga %}
  3. When you're ready to test your new custom pattern, to identify matches in the enterprise without creating alerts, click Save and dry run. {% data reusables.advanced-security.secret-scanning-dry-run-select-enterprise-repos %} {% data reusables.advanced-security.secret-scanning-dry-run-results %} {%- endif %} {% data reusables.advanced-security.secret-scanning-create-custom-pattern %}{% ifversion secret-scanning-push-protection-custom-patterns %}
  4. Optionally, to enable push protection for your custom pattern, click Enable. For more information, see "AUTOTITLE." {% indented_data_reference reusables.secret-scanning.push-protection-enterprise-note spaces=3 %}{% endif %}

After your pattern is created, {% data variables.product.prodname_secret_scanning %} scans for any secrets in repositories within your enterprise's organizations with {% data variables.product.prodname_GH_advanced_security %} enabled, including their entire Git history on all branches. Organization owners and repository administrators will be alerted to any secrets found, and can review the alert in the repository where the secret is found. For more information on viewing {% data variables.secret-scanning.alerts %}, see "AUTOTITLE."

{% ifversion secret-scanning-push-protection-custom-patterns %}

Enabling push protection for a custom pattern

You can enable {% data variables.product.prodname_secret_scanning %} as a push protection for custom patterns stored at {% ifversion ghec or ghes %}the enterprise, organization, or repository level{% else%} the organization or repository level{% endif %}.

{% ifversion ghec or ghes %}

Enabling push protection for a custom pattern stored in an enterprise

{% data reusables.secret-scanning.push-protection-enterprise-note %}

Before enabling push protection for a custom pattern at enterprise level, you must also{% ifversion custom-pattern-dry-run-ga %} test your custom patterns using dry runs. {% data reusables.secret-scanning.dry-runs-enterprise-permissions %}{% else %} test your custom patterns in a repository before defining them for your entire enterprise, as there is no dry-run functionality. That way, you can avoid creating excess false-positive {% data variables.secret-scanning.alerts %}.{% endif %}

{% data reusables.enterprise-accounts.access-enterprise %} {% data reusables.enterprise-accounts.policies-tab %}{% ifversion security-feature-enablement-policies %} {% data reusables.enterprise-accounts.code-security-and-analysis-policies %}

  1. Under "Code security and analysis", click Security features.{% else %} {% data reusables.enterprise-accounts.advanced-security-policies %} {% data reusables.enterprise-accounts.advanced-security-security-features %}{% endif %} {% data reusables.advanced-security.secret-scanning-edit-custom-pattern %}

    {% ifversion custom-pattern-dry-run-ga %}

    [!NOTE] At the enterprise level, you can only edit and enable push protection for custom patterns that you created. {%- endif %}

  2. To enable push protection for your custom pattern, scroll down to "Push Protection", and click Enable.

    {% data reusables.secret-scanning.custom-pattern-push-protection-enable-button %}

    Screenshot of the custom pattern page with the button to enable push protection highlighted with a dark orange outline.

{% endif %}

Enabling {% data variables.product.prodname_secret_scanning %} as a push protection in an organization for a custom pattern

Before enabling push protection for a custom pattern at organization level, you must ensure that you enable {% data variables.product.prodname_secret_scanning %} for the repositories that you want to scan in your organization. To enable {% data variables.product.prodname_secret_scanning %} on all repositories in your organization, see "AUTOTITLE."

{% data reusables.profile.access_org %} {% data reusables.profile.org_settings %} {% data reusables.organizations.security-and-analysis %}

{% ifversion security-configurations %} {% data reusables.security-configurations.changed-org-settings-global-settings-callout %} For next steps on managing custom patterns for your organization with {% data variables.product.prodname_global_settings %}, see "AUTOTITLE." For information on enabling push protection for specific custom patterns, reference the following steps. {% endif %}

{% data reusables.repositories.navigate-to-ghas-settings %} {% data reusables.advanced-security.secret-scanning-edit-custom-pattern %}

  1. To enable push protection for your custom pattern, scroll down to "Push Protection", and click Enable. {% indented_data_reference reusables.secret-scanning.push-protection-org-notes spaces=3 %}

    Screenshot of the "Push protection" section of the custom pattern page. A button, labeled "Enable", is outlined in dark orange.

Enabling {% data variables.product.prodname_secret_scanning %} as a push protection in a repository for a custom pattern

Before enabling push protection for a custom pattern at repository level, you must define the custom pattern for the repository, and test it in the repository. For more information, see "AUTOTITLE."

{% data reusables.repositories.navigate-to-repo %} {% data reusables.repositories.sidebar-settings %} {% data reusables.repositories.navigate-to-code-security-and-analysis %} {% data reusables.repositories.navigate-to-ghas-settings %} {% data reusables.advanced-security.secret-scanning-edit-custom-pattern %}

  1. To enable push protection for your custom pattern, scroll down to "Push Protection", and click Enable.

    {% data reusables.secret-scanning.custom-pattern-push-protection-enable-button %}

    Screenshot of the "Push protection" section of the custom pattern page. A button, labeled "Enable", is outlined in dark orange.

{% endif %}