|
|
|
|
@@ -1,7 +1,7 @@
|
|
|
|
|
---
|
|
|
|
|
title: Caching dependencies to speed up workflows
|
|
|
|
|
shortTitle: Cache dependencies
|
|
|
|
|
intro: 'To make your workflows faster and more efficient, you can create and use caches for dependencies and other commonly reused files.'
|
|
|
|
|
title: Dependency caching reference
|
|
|
|
|
shortTitle: Dependency caching reference
|
|
|
|
|
intro: 'Find information on the functionality of dependency caching in workflows.'
|
|
|
|
|
redirect_from:
|
|
|
|
|
- /github/automating-your-workflow-with-github-actions/caching-dependencies-to-speed-up-workflows
|
|
|
|
|
- /actions/automating-your-workflow-with-github-actions/caching-dependencies-to-speed-up-workflows
|
|
|
|
|
@@ -10,75 +10,21 @@ redirect_from:
|
|
|
|
|
- /actions/advanced-guides/caching-dependencies-to-speed-up-workflows
|
|
|
|
|
- /actions/using-workflows/caching-dependencies-to-speed-up-workflows
|
|
|
|
|
- /actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows
|
|
|
|
|
- /actions/how-tos/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows
|
|
|
|
|
versions:
|
|
|
|
|
fpt: '*'
|
|
|
|
|
ghec: '*'
|
|
|
|
|
ghes: '*'
|
|
|
|
|
type: tutorial
|
|
|
|
|
topics:
|
|
|
|
|
- Workflows
|
|
|
|
|
ghec: '*'
|
|
|
|
|
type: overview
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## About caching workflow dependencies
|
|
|
|
|
|
|
|
|
|
Workflow runs often reuse the same outputs or downloaded dependencies from one run to another. For example, package and dependency management tools such as Maven, Gradle, npm, and Yarn keep a local cache of downloaded dependencies.
|
|
|
|
|
|
|
|
|
|
{% ifversion fpt or ghec %} Jobs on {% data variables.product.prodname_dotcom %}-hosted runners start in a clean runner image and must download dependencies each time, causing increased network utilization, longer runtime, and increased cost. {% endif %}To help speed up the time it takes to recreate files like dependencies, {% data variables.product.prodname_dotcom %} can cache files you frequently use in workflows.
|
|
|
|
|
|
|
|
|
|
To cache dependencies for a job, you can use {% data variables.product.prodname_dotcom %}'s [`cache` action](https://github.com/actions/cache). The action creates and restores a cache identified by a unique key. Alternatively, if you are caching the package managers listed below, using their respective setup-* actions requires minimal configuration and will create and restore dependency caches for you.
|
|
|
|
|
|
|
|
|
|
| Package managers | setup-* action for caching |
|
|
|
|
|
|---|---|
|
|
|
|
|
| npm, Yarn, pnpm | [setup-node](https://github.com/actions/setup-node#caching-global-packages-data) |
|
|
|
|
|
| pip, pipenv, Poetry | [setup-python](https://github.com/actions/setup-python#caching-packages-dependencies) |
|
|
|
|
|
| Gradle, Maven | [setup-java](https://github.com/actions/setup-java#caching-packages-dependencies) |
|
|
|
|
|
| RubyGems | [setup-ruby](https://github.com/ruby/setup-ruby#caching-bundle-install-automatically) |
|
|
|
|
|
| Go `go.sum` | [setup-go](https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs) |
|
|
|
|
|
| .NET NuGet | [setup-dotnet](https://github.com/actions/setup-dotnet?tab=readme-ov-file#caching-nuget-packages) |
|
|
|
|
|
|
|
|
|
|
> [!WARNING]
|
|
|
|
|
> {% ifversion fpt or ghec %}Be mindful of the following when using caching with {% data variables.product.prodname_actions %}:
|
|
|
|
|
>
|
|
|
|
|
> * {% endif %}We recommend that you don't store any sensitive information in the cache. For example, sensitive information can include access tokens or login credentials stored in a file in the cache path. Also, command line interface (CLI) programs like `docker login` can save access credentials in a configuration file. Anyone with read access can create a pull request on a repository and access the contents of a cache. Forks of a repository can also create pull requests on the base branch and access caches on the base branch.
|
|
|
|
|
{%- ifversion fpt or ghec %}
|
|
|
|
|
> * When using self-hosted runners, caches from workflow runs are stored on {% data variables.product.company_short %}-owned cloud storage. A customer-owned storage solution is only available with {% data variables.product.prodname_ghe_server %}.
|
|
|
|
|
{%- endif %}
|
|
|
|
|
|
|
|
|
|
{% data reusables.actions.comparing-artifacts-caching %}
|
|
|
|
|
|
|
|
|
|
For more information on workflow run artifacts, see [AUTOTITLE](/actions/using-workflows/storing-workflow-data-as-artifacts).
|
|
|
|
|
|
|
|
|
|
## Restrictions for accessing a cache
|
|
|
|
|
|
|
|
|
|
Access restrictions provide cache isolation and security by creating a logical boundary between different branches or tags.
|
|
|
|
|
Workflow runs can restore caches created in either the current branch or the default branch (usually `main`). If a workflow run is triggered for a pull request, it can also restore caches created in the base branch, including base branches of forked repositories. For example, if the branch `feature-b` has the base branch `feature-a`, a workflow run triggered on a pull request would have access to caches created in the default `main` branch, the base `feature-a` branch, and the current `feature-b` branch.
|
|
|
|
|
|
|
|
|
|
Workflow runs cannot restore caches created for child branches or sibling branches. For example, a cache created for the child `feature-b` branch would not be accessible to a workflow run triggered on the parent `main` branch. Similarly, a cache created for the `feature-a` branch with the base `main` would not be accessible to its sibling `feature-c` branch with the base `main`. Workflow runs also cannot restore caches created for different tag names. For example, a cache created for the tag `release-a` with the base `main` would not be accessible to a workflow run triggered for the tag `release-b` with the base `main`.
|
|
|
|
|
|
|
|
|
|
When a cache is created by a workflow run triggered on a pull request, the cache is created for the merge ref (`refs/pull/.../merge`). Because of this, the cache will have a limited scope and can only be restored by re-runs of the pull request. It cannot be restored by the base branch or other pull requests targeting that base branch.
|
|
|
|
|
|
|
|
|
|
Multiple workflow runs in a repository can share caches. A cache created for a branch in a workflow run can be accessed and restored from another workflow run for the same repository and branch.
|
|
|
|
|
|
|
|
|
|
{% ifversion ghes %}
|
|
|
|
|
|
|
|
|
|
> [!NOTE]
|
|
|
|
|
> As objects are retrieved from or put into the cache directly from runners, Actions runners must have direct connectivity to the Actions object storage configured in {% data variables.product.prodname_ghe_server %}, such as AWS S3 or Azure Blob Storage. Self-hosted runners authenticate with the blob storage provider using an access URL provided by the {% data variables.product.prodname_ghe_server %} instance. This URL supplies the blob storage provider with valid temporary authentication credentials. This process is initiated by the instance itself, which mediates all requests to the object storage.
|
|
|
|
|
>
|
|
|
|
|
> This means that `actions/cache` requires a HTTPS connection to the blob storage in order to work correctly.
|
|
|
|
|
>
|
|
|
|
|
> All the metadata are managed by the artifact cache service, which is a microservice within {% data variables.product.prodname_actions %}.
|
|
|
|
|
>
|
|
|
|
|
> For more information on cache storage, see [External storage requirements](/admin/github-actions/getting-started-with-github-actions-for-your-enterprise/getting-started-with-github-actions-for-github-enterprise-server#external-storage-requirements).
|
|
|
|
|
|
|
|
|
|
{% endif %}
|
|
|
|
|
|
|
|
|
|
## Using the `cache` action
|
|
|
|
|
## `cache` action usage
|
|
|
|
|
|
|
|
|
|
The [`cache` action](https://github.com/actions/cache) will attempt the following sequence when restoring a cache:
|
|
|
|
|
|
|
|
|
|
1. First, it searches for an exact match to your provided `key`.
|
|
|
|
|
1. If no exact match is found, it will search for partial matches of the `key`.
|
|
|
|
|
1. If there is still no match found, and you've provided `restore-keys`, these keys will be checked sequentially for partial matches. For more information, see [Matching a cache key](#matching-a-cache-key).
|
|
|
|
|
1. If there is still no match found, and you've provided `restore-keys`, these keys will be checked sequentially for partial matches. For more information, see [Cache key matching](#cache-key-matching).
|
|
|
|
|
|
|
|
|
|
If there is an exact match to the provided `key`, this is considered a cache hit. If no cache exactly matches the provided `key`, this is considered a cache miss. On a cache miss, the action automatically creates a new cache if the job completes successfully. The new cache will use the `key` you provided and contains the files you specify in `path`. For more information about how this is handled, see [Cache hits and misses](#cache-hits-and-misses).
|
|
|
|
|
|
|
|
|
|
@@ -116,6 +62,9 @@ You cannot change the contents of an existing cache. Instead, you can create a n
|
|
|
|
|
|
|
|
|
|
* `enableCrossOsArchive`: **Optional** A boolean value that when enabled, allows Windows runners to save or restore caches independent of the operating system the cache was created on. If this parameter is not set, it defaults to `false`. For more information, see [Cross OS cache](https://github.com/actions/cache/blob/main/tips-and-workarounds.md#cross-os-cache) in the Actions Cache documentation.
|
|
|
|
|
|
|
|
|
|
> [!NOTE]
|
|
|
|
|
> We recommend that you don't store any sensitive information, such as access tokens or login credentials, in files in the cache path. Anyone with read access can create a pull request on a repository and access the contents of a cache. Additionally, forks of a repository can create pull requests on the base branch and access caches on the base branch.
|
|
|
|
|
|
|
|
|
|
### Output parameters for the `cache` action
|
|
|
|
|
|
|
|
|
|
* `cache-hit`: A boolean value to indicate an exact match was found for the key.
|
|
|
|
|
@@ -134,7 +83,7 @@ When a cache miss occurs, the action also searches your specified `restore-keys`
|
|
|
|
|
1. The `cache` action completes and the next step in the job runs.
|
|
|
|
|
1. If the job completes successfully, the action automatically creates a new cache with the contents of the `path` directory.
|
|
|
|
|
|
|
|
|
|
For a more detailed explanation of the cache matching process, see [Matching a cache key](#matching-a-cache-key).
|
|
|
|
|
For a more detailed explanation of the cache matching process, see [Cache key matching](#cache-key-matching).
|
|
|
|
|
|
|
|
|
|
### Example using the `cache` action
|
|
|
|
|
|
|
|
|
|
@@ -213,7 +162,7 @@ In the example workflow above, there is a step that lists the state of the Node
|
|
|
|
|
run: npm list
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Matching a cache key
|
|
|
|
|
## Cache key matching
|
|
|
|
|
|
|
|
|
|
The `cache` action first searches for cache hits for `key` and the cache _version_ in the branch containing the workflow run. If there is no hit, it searches for prefix-matches for `key`, and if there is still no hit, it searches for `restore-keys` and the _version_. If there are still no hits in the current branch, the `cache` action retries the same steps on the default branch. Please note that the scope restrictions apply during the search. For more information, see [Restrictions for accessing a cache](#restrictions-for-accessing-a-cache).
|
|
|
|
|
|
|
|
|
|
@@ -272,97 +221,51 @@ For example, if a pull request contains a `feature` branch and targets the defau
|
|
|
|
|
1. Key `npm-feature-` in the `main` branch
|
|
|
|
|
1. Key `npm-` in the `main` branch
|
|
|
|
|
|
|
|
|
|
## `setup-*` actions for specific package managers
|
|
|
|
|
|
|
|
|
|
If you are caching the package managers listed below, using their respective setup-* actions requires minimal configuration and will create and restore dependency caches for you.
|
|
|
|
|
|
|
|
|
|
| Package managers | setup-* action for caching |
|
|
|
|
|
|---|---|
|
|
|
|
|
| npm, Yarn, pnpm | [setup-node](https://github.com/actions/setup-node#caching-global-packages-data) |
|
|
|
|
|
| pip, pipenv, Poetry | [setup-python](https://github.com/actions/setup-python#caching-packages-dependencies) |
|
|
|
|
|
| Gradle, Maven | [setup-java](https://github.com/actions/setup-java#caching-packages-dependencies) |
|
|
|
|
|
| RubyGems | [setup-ruby](https://github.com/ruby/setup-ruby#caching-bundle-install-automatically) |
|
|
|
|
|
| Go `go.sum` | [setup-go](https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs) |
|
|
|
|
|
| .NET NuGet | [setup-dotnet](https://github.com/actions/setup-dotnet?tab=readme-ov-file#caching-nuget-packages) |
|
|
|
|
|
|
|
|
|
|
## Restrictions for accessing a cache
|
|
|
|
|
|
|
|
|
|
Access restrictions provide cache isolation and security by creating a logical boundary between different branches or tags.
|
|
|
|
|
Workflow runs can restore caches created in either the current branch or the default branch (usually `main`). If a workflow run is triggered for a pull request, it can also restore caches created in the base branch, including base branches of forked repositories. For example, if the branch `feature-b` has the base branch `feature-a`, a workflow run triggered on a pull request would have access to caches created in the default `main` branch, the base `feature-a` branch, and the current `feature-b` branch.
|
|
|
|
|
|
|
|
|
|
Workflow runs cannot restore caches created for child branches or sibling branches. For example, a cache created for the child `feature-b` branch would not be accessible to a workflow run triggered on the parent `main` branch. Similarly, a cache created for the `feature-a` branch with the base `main` would not be accessible to its sibling `feature-c` branch with the base `main`. Workflow runs also cannot restore caches created for different tag names. For example, a cache created for the tag `release-a` with the base `main` would not be accessible to a workflow run triggered for the tag `release-b` with the base `main`.
|
|
|
|
|
|
|
|
|
|
When a cache is created by a workflow run triggered on a pull request, the cache is created for the merge ref (`refs/pull/.../merge`). Because of this, the cache will have a limited scope and can only be restored by re-runs of the pull request. It cannot be restored by the base branch or other pull requests targeting that base branch.
|
|
|
|
|
|
|
|
|
|
Multiple workflow runs in a repository can share caches. A cache created for a branch in a workflow run can be accessed and restored from another workflow run for the same repository and branch.
|
|
|
|
|
|
|
|
|
|
{% ifversion ghes %}
|
|
|
|
|
|
|
|
|
|
> [!NOTE]
|
|
|
|
|
> As objects are retrieved from or put into the cache directly from runners, Actions runners must have direct connectivity to the Actions object storage configured in {% data variables.product.prodname_ghe_server %}, such as AWS S3 or Azure Blob Storage. Self-hosted runners authenticate with the blob storage provider using an access URL provided by the {% data variables.product.prodname_ghe_server %} instance. This URL supplies the blob storage provider with valid temporary authentication credentials. This process is initiated by the instance itself, which mediates all requests to the object storage.
|
|
|
|
|
>
|
|
|
|
|
> This means that `actions/cache` requires a HTTPS connection to the blob storage in order to work correctly.
|
|
|
|
|
>
|
|
|
|
|
> All the metadata are managed by the artifact cache service, which is a microservice within {% data variables.product.prodname_actions %}.
|
|
|
|
|
>
|
|
|
|
|
> For more information on cache storage, see [External storage requirements](/admin/github-actions/getting-started-with-github-actions-for-your-enterprise/getting-started-with-github-actions-for-github-enterprise-server#external-storage-requirements).
|
|
|
|
|
|
|
|
|
|
{% endif %}
|
|
|
|
|
|
|
|
|
|
## Usage limits and eviction policy
|
|
|
|
|
|
|
|
|
|
{% data variables.product.prodname_dotcom %} will remove any cache entries that have not been accessed in over 7 days. There is no limit on the number of caches you can store, but the total size of all caches in a repository is limited{% ifversion ghes %}. By default, the limit is 10 GB per repository, but this limit might be different depending on policies set by your enterprise owners or repository administrators.{% else %} to 10 GB.{% endif %} {% data reusables.actions.cache-eviction-policy %}
|
|
|
|
|
|
|
|
|
|
{% data reusables.actions.cache-eviction-process %} The cache eviction process may cause cache thrashing, where caches are created and deleted at a high frequency. To reduce this, you can review the caches for a repository and take corrective steps, such as removing caching from specific workflows. For more information, see [Managing caches](#managing-caches).{% ifversion ghes %} You can also increase the cache size limit for a repository. For more information, see [AUTOTITLE](/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#configuring-cache-storage-for-a-repository).
|
|
|
|
|
{% data reusables.actions.cache-eviction-process %} The cache eviction process may cause cache thrashing, where caches are created and deleted at a high frequency. To reduce this, you can review the caches for a repository and take corrective steps, such as removing caching from specific workflows. See [AUTOTITLE](/actions/how-tos/managing-workflow-runs-and-deployments/managing-workflow-runs/manage-caches).{% ifversion ghes %} You can also increase the cache size limit for a repository. For more information, see [AUTOTITLE](/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#configuring-cache-storage-for-a-repository).
|
|
|
|
|
|
|
|
|
|
{% endif %}
|
|
|
|
|
|
|
|
|
|
## Managing caches
|
|
|
|
|
## Next steps
|
|
|
|
|
|
|
|
|
|
To manage caches created from your workflows, you can:
|
|
|
|
|
|
|
|
|
|
* View a list of all cache entries for a repository.
|
|
|
|
|
* Filter and sort the list of caches using specific metadata such as cache size, creation time, or last accessed time.
|
|
|
|
|
* Delete cache entries from a repository.
|
|
|
|
|
* Monitor aggregate cache usage for repositories and organizations.
|
|
|
|
|
|
|
|
|
|
There are multiple ways to manage caches for your repositories:
|
|
|
|
|
|
|
|
|
|
* Using the {% data variables.product.prodname_dotcom %} web interface, as shown below.
|
|
|
|
|
* Using the REST API. For more information, see [AUTOTITLE](/rest/actions/cache).
|
|
|
|
|
* Installing the `gh cache` subcommand to manage your caches from the command line. For more information, see the [GitHub CLI documentation](https://cli.github.com/manual/gh_cache).
|
|
|
|
|
|
|
|
|
|
> [!NOTE]
|
|
|
|
|
> If you are doing this manually, ensure you have version 2.32.0 or higher of the CLI installed.
|
|
|
|
|
|
|
|
|
|
### Viewing cache entries
|
|
|
|
|
|
|
|
|
|
You can use the web interface to view a list of cache entries for a repository. In the cache list, you can see how much disk space each cache is using, when the cache was created, and when the cache was last used.
|
|
|
|
|
|
|
|
|
|
{% data reusables.repositories.navigate-to-repo %}
|
|
|
|
|
{% data reusables.repositories.actions-tab %}
|
|
|
|
|
{% data reusables.repositories.actions-cache-list %}
|
|
|
|
|
1. Review the list of cache entries for the repository.
|
|
|
|
|
|
|
|
|
|
* To search for cache entries used for a specific branch, click the **Branch** dropdown menu and select a branch. The cache list will display all of the caches used for the selected branch.
|
|
|
|
|
* To search for cache entries with a specific cache key, use the syntax `key: key-name` in the **Filter caches** field. The cache list will display caches from all branches where the key was used.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Deleting cache entries
|
|
|
|
|
|
|
|
|
|
Users with `write` access to a repository can use the {% data variables.product.prodname_dotcom %} web interface to delete cache entries.
|
|
|
|
|
|
|
|
|
|
{% data reusables.repositories.navigate-to-repo %}
|
|
|
|
|
{% data reusables.repositories.actions-tab %}
|
|
|
|
|
{% data reusables.repositories.actions-cache-list %}
|
|
|
|
|
1. To the right of the cache entry you want to delete, click {% octicon "trash" aria-label="Delete cache" %}.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Force deleting cache entries
|
|
|
|
|
|
|
|
|
|
Caches have branch scope restrictions in place, which means some caches have limited usage options. For more information on cache scope restrictions, see [Restrictions for accessing a cache](#restrictions-for-accessing-a-cache), earlier in this article. If caches limited to a specific branch are using a lot of storage quota, it may cause caches from the `default` branch to be created and deleted at a high frequency.
|
|
|
|
|
|
|
|
|
|
For example, a repository could have many new pull requests opened, each with their own caches that are restricted to that branch. These caches could take up the majority of the cache storage for that repository. {% data reusables.actions.cache-eviction-policy %} In order to prevent cache thrashing when this happens, you can set up workflows to delete caches on a faster cadence than the cache eviction policy will. You can use the {% data variables.product.prodname_cli %} to delete caches for specific branches.
|
|
|
|
|
|
|
|
|
|
The following example workflow uses `gh cache` to delete up to 100 caches created by a branch once a pull request is closed.
|
|
|
|
|
|
|
|
|
|
To run the following example on cross-repository pull requests or pull requests from forks, you can trigger the workflow with the `pull_request_target` event. If you do use `pull_request_target` to trigger the workflow, there are security considerations to keep in mind. For more information, see [AUTOTITLE](/actions/using-workflows/events-that-trigger-workflows#pull_request_target).
|
|
|
|
|
|
|
|
|
|
```yaml
|
|
|
|
|
name: Cleanup github runner caches on closed pull requests
|
|
|
|
|
on:
|
|
|
|
|
pull_request:
|
|
|
|
|
types:
|
|
|
|
|
- closed
|
|
|
|
|
|
|
|
|
|
jobs:
|
|
|
|
|
cleanup:
|
|
|
|
|
runs-on: ubuntu-latest
|
|
|
|
|
permissions:
|
|
|
|
|
actions: write
|
|
|
|
|
steps:
|
|
|
|
|
- name: Cleanup
|
|
|
|
|
run: |
|
|
|
|
|
echo "Fetching list of cache keys"
|
|
|
|
|
cacheKeysForPR=$(gh cache list --ref $BRANCH --limit 100 --json id --jq '.[].id')
|
|
|
|
|
|
|
|
|
|
## Setting this to not fail the workflow while deleting cache keys.
|
|
|
|
|
set +e
|
|
|
|
|
echo "Deleting caches..."
|
|
|
|
|
for cacheKey in $cacheKeysForPR
|
|
|
|
|
do
|
|
|
|
|
gh cache delete $cacheKey
|
|
|
|
|
done
|
|
|
|
|
echo "Done"
|
|
|
|
|
env:
|
|
|
|
|
GH_TOKEN: {% raw %}${{ secrets.GITHUB_TOKEN }}{% endraw %}
|
|
|
|
|
GH_REPO: {% raw %}${{ github.repository }}{% endraw %}
|
|
|
|
|
BRANCH: refs/pull/{% raw %}${{ github.event.pull_request.number }}{% endraw %}/merge
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Alternatively, you can use the API to automatically list or delete all caches on your own cadence. For more information, see [AUTOTITLE](/rest/actions/cache#about-the-cache-in-github-actions).
|
|
|
|
|
To manage your dependency caches, see [AUTOTITLE](/actions/how-tos/managing-workflow-runs-and-deployments/managing-workflow-runs/manage-caches).
|
Sam Browning