Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .github/actions/free_disk_space_android/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Reclaims disk on GitHub-hosted runners by deleting the preinstalled Android
# tooling. One action per ecosystem, deliberately: a workflow that never touches
# Android can call this and keep the toolchains it does use, instead of paying
# for a monolithic "free disk space" step that deletes everything and costs
# minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_android"
description: "Delete the preinstalled Android SDK/NDK from a GitHub-hosted runner (~9 GB)."

runs:
using: "composite"
steps:

- name: Remove Android SDK and NDK
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
sudo rm -rf /usr/local/lib/android || true
after=$(df --output=avail -k / | tail -n1)
echo "Remove Android SDK and NDK: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
35 changes: 35 additions & 0 deletions .github/actions/free_disk_space_apt/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Reclaims disk on GitHub-hosted runners by deleting large preinstalled apt
# packages. One action per ecosystem, deliberately: a workflow that has no large
# apt packages to spare can call this and keep the toolchains it does use,
# instead of paying for a monolithic "free disk space" step that deletes
# everything and costs minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_apt"
description: "Purge large preinstalled apt packages (browsers, LLVM, databases, cloud CLIs) from a GitHub-hosted runner."

runs:
using: "composite"
steps:

- name: Purge large apt packages
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
# Each pattern is a package family that is multiple GB installed and that
# almost no build needs. `|| true` throughout: the image contents drift
# between runner releases and a pattern matching nothing must not fail CI.
sudo apt-get remove -y --purge '^llvm-.*' '^libclang.*' '^mysql-.*' '^mongodb-.*' \
'^postgresql-.*' azure-cli google-cloud-cli firefox google-chrome-stable \
microsoft-edge-stable powershell mono-devel || true
sudo apt-get autoremove -y || true
sudo apt-get clean || true
after=$(df --output=avail -k / | tail -n1)
echo "Purge large apt packages: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
30 changes: 30 additions & 0 deletions .github/actions/free_disk_space_docker/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Reclaims disk on GitHub-hosted runners by deleting the preloaded Docker
# images. One action per ecosystem, deliberately: a workflow that does not use
# Docker can call this and keep the toolchains it does use, instead of paying
# for a monolithic "free disk space" step that deletes everything and costs
# minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_docker"
description: "Delete all preloaded Docker images from a GitHub-hosted runner (~4 GB)."

runs:
using: "composite"
steps:

- name: Remove preloaded Docker images
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
# Only safe before anything has been built or pulled: this removes every
# image on the runner, not just the preloaded ones.
sudo docker image prune --all --force || true
after=$(df --output=avail -k / | tail -n1)
echo "Remove preloaded Docker images: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
27 changes: 27 additions & 0 deletions .github/actions/free_disk_space_dotnet/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Reclaims disk on GitHub-hosted runners by deleting the preinstalled .NET
# tooling. One action per ecosystem, deliberately: a workflow that never touches
# .NET can call this and keep the toolchains it does use, instead of paying for
# a monolithic "free disk space" step that deletes everything and costs minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_dotnet"
description: "Delete the preinstalled .NET SDK from a GitHub-hosted runner (~2 GB)."

runs:
using: "composite"
steps:

- name: Remove .NET SDK
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
sudo rm -rf /usr/share/dotnet || true
after=$(df --output=avail -k / | tail -n1)
echo "Remove .NET SDK: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
29 changes: 29 additions & 0 deletions .github/actions/free_disk_space_haskell/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Reclaims disk on GitHub-hosted runners by deleting the preinstalled Haskell
# tooling. One action per ecosystem, deliberately: a workflow that never touches
# Haskell can call this and keep the toolchains it does use, instead of paying
# for a monolithic "free disk space" step that deletes everything and costs
# minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_haskell"
description: "Delete the preinstalled GHC/Stack toolchain from a GitHub-hosted runner (~5 GB)."

runs:
using: "composite"
steps:

- name: Remove Haskell toolchain
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
sudo rm -rf /opt/ghc "${AGENT_TOOLSDIRECTORY:-/opt/hostedtoolcache}/CodeQL" || true
sudo rm -rf /usr/local/.ghcup || true
after=$(df --output=avail -k / | tail -n1)
echo "Remove Haskell toolchain: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
30 changes: 30 additions & 0 deletions .github/actions/free_disk_space_swap/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Reclaims disk on GitHub-hosted runners by deleting the runner swap file. One
# action per ecosystem, deliberately: a workflow that is not memory-bound can
# call this and keep the toolchains it does use, instead of paying for a
# monolithic "free disk space" step that deletes everything and costs minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_swap"
description: "Disable and delete the runner swap file to reclaim its disk (~4 GB)."

runs:
using: "composite"
steps:

- name: Remove swap file
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
# Trades OOM headroom for disk. Only worth it for jobs that are disk-bound
# rather than memory-bound -- a large Rust link step is often both.
sudo swapoff -a || true
sudo rm -f /mnt/swapfile /swapfile || true
after=$(df --output=avail -k / | tail -n1)
echo "Remove swap file: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
27 changes: 27 additions & 0 deletions .github/actions/free_disk_space_swift/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Reclaims disk on GitHub-hosted runners by deleting the preinstalled Swift
# tooling. One action per ecosystem, deliberately: a workflow that never touches
# Swift can call this and keep the toolchains it does use, instead of paying for
# a monolithic "free disk space" step that deletes everything and costs minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_swift"
description: "Delete the preinstalled Swift toolchain from a GitHub-hosted runner (~2 GB)."

runs:
using: "composite"
steps:

- name: Remove Swift toolchain
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
sudo rm -rf /usr/share/swift || true
after=$(df --output=avail -k / | tail -n1)
echo "Remove Swift toolchain: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
30 changes: 30 additions & 0 deletions .github/actions/free_disk_space_tool_cache/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Reclaims disk on GitHub-hosted runners by deleting the hosted tool cache. One
# action per ecosystem, deliberately: a workflow that installs its own
# toolchains can call this and keep the toolchains it does use, instead of
# paying for a monolithic "free disk space" step that deletes everything and
# costs minutes.
#
# Hosted runners only. The self-hosted fleet is non-ephemeral, so a deletion
# here is permanent and affects every later job on that machine -- including
# jobs that need what was deleted. The guard is on every step, not on the
# caller, so misuse is impossible rather than merely discouraged.

name: "free_disk_space_tool_cache"
description: "Delete the hosted tool cache (preinstalled Python/Node/Go/Java versions) from a GitHub-hosted runner (~10 GB)."

runs:
using: "composite"
steps:

- name: Remove hosted tool cache
# Skipped on self-hosted: see the note at the top of this file.
if: ${{ runner.environment != 'self-hosted' }}
shell: bash
run: |
set -uo pipefail
before=$(df --output=avail -k / | tail -n1)
# Anything using actions/setup-* MUST run this before that setup step, or
# the version it wants disappears from under it and is re-downloaded.
sudo rm -rf "${AGENT_TOOLSDIRECTORY:-/opt/hostedtoolcache}" || true
after=$(df --output=avail -k / | tail -n1)
echo "Remove hosted tool cache: freed $(( (after - before) / 1024 )) MB, $(( after / 1024 / 1024 )) GB now free on /"
15 changes: 13 additions & 2 deletions .github/actions/npm/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ runs:
with:
node-version-file: 'package.json'

- uses: prosopo/github_actions/.github/actions/restore_npm_cache@main
- id: cache
uses: prosopo/github_actions/.github/actions/restore_npm_cache@main
if: ${{ inputs.restore_npm_cache }}

- name: install npm
Expand All @@ -34,11 +35,21 @@ runs:
# Commented until npm fixes itself https://github.com/npm/cli/issues/8757
# npm i -g "npm@$(jq -r '.engines.npm // "latest"' < package.json)"

# Skip `npm ci` when node_modules was restored from an exact-lockfile cache hit: the tree
# already matches the lockfile, and `npm ci` would only delete it and reinstall the identical
# tree (~2 min wasted). On a cache miss (or when caching is disabled) the output is not 'true'
# and `npm ci` runs as normal, so this is always safe.
- name: install project
shell: bash
if: ${{ inputs.npm_ci }}
if: ${{ inputs.npm_ci && steps.cache.outputs.node-modules-cache-hit != 'true' }}
run: |
# Unquoted on purpose: npm_ci_args is a list of flags and must word
# split into separate arguments.
# shellcheck disable=SC2086
npm ci ${{ inputs.npm_ci_args }}

- name: skipped install (node_modules cache hit)
shell: bash
if: ${{ inputs.npm_ci && steps.cache.outputs.node-modules-cache-hit == 'true' }}
run: |
echo "node_modules restored from exact-lockfile cache hit; skipping 'npm ci'."
21 changes: 21 additions & 0 deletions .github/actions/restore_npm_cache/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ inputs:
required: false
default: "false"

outputs:
node-modules-cache-hit:
description: "'true' when the node_modules cache was hit exactly (lockfile unchanged). When 'true', node_modules is already populated for the current lockfile and `npm ci` can be skipped (running it would only wipe and recreate the identical tree)."
value: ${{ steps.restore-node-modules.outputs.cache-hit }}

runs:
using: "composite"
steps:
Expand Down Expand Up @@ -37,3 +42,19 @@ runs:
key: some-unused-cache-key
restore-keys: |
npm-${{ runner.os }}-${{ runner.arch }}-

# node_modules is cached separately from the npm/turbo/nx caches above because it needs the
# OPPOSITE matching behaviour: an EXACT match on the lockfile hash only, never a fuzzy fallback.
# A node_modules tree from a different lockfile would be wrong, so there is deliberately no
# restore-keys here. On an exact hit the consumer skips `npm ci` entirely (it would only wipe
# and recreate the identical tree); on a miss the consumer falls back to `npm ci`, which is
# also why a miss is always safe. The cache is populated by save_npm_cache (run on main), so
# lockfile-unchanged PRs hit; lockfile-changing PRs miss until main re-primes after merge.
- name: Restore node_modules
id: restore-node-modules
if: ${{ runner.environment != 'self-hosted' || inputs.restore-on-self-hosted == 'true' }}
uses: actions/cache/restore@v4
with:
path: |
**/node_modules
key: node-modules-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/package-lock.json') }}
26 changes: 26 additions & 0 deletions .github/actions/save_npm_cache/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,17 @@ runs:
.nx/cache
key: npm-${{ runner.os }}-${{ runner.arch }}-${{ github.run_id }}-${{ github.run_attempt }}

# node_modules is saved under an EXACT lockfile-hash key (matching restore_npm_cache) so that
# lockfile-unchanged consumers get an exact hit and can skip `npm ci`. The key is stable across
# runs while the lockfile is unchanged, so actions/cache/save is a no-op (warns, doesn't fail)
# once the cache for a given lockfile already exists.
- name: Save node_modules cache
uses: actions/cache/save@v4
with:
path: |
**/node_modules
key: node-modules-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/package-lock.json') }}

- name: Cleanup npm caches
if: ${{ runner.environment != 'self-hosted' || inputs.save-on-self-hosted == 'true' }}
shell: bash
Expand All @@ -60,3 +71,18 @@ runs:
do
gh cache delete "$cacheKey" -R "${{ github.repository }}"
done

- name: Cleanup node_modules caches
shell: bash
run: |
# node_modules caches are keyed by lockfile hash, so stale hashes accumulate as the
# lockfile changes. Keep only the most recent (the one we just saved) and remove the rest.
echo "Fetching list of node_modules cache keys"
cacheKeys=$(gh cache list --sort created_at --order desc --limit 100 -R "${{ github.repository }}" --key "node-modules-${{ runner.os }}-${{ runner.arch }}-" | cut -f 1 | tail -n +2)
echo caches to be removed:
echo "${cacheKeys}"
set +e
for cacheKey in $cacheKeys
do
gh cache delete "$cacheKey" -R "${{ github.repository }}"
done
Loading