<Home>
← JournalEngineering · 7 min read

How we made our Next.js Docker build about 40% faster

Our Next.js production image built in 8m43s on a fresh spot VM runner. Three workflow changes brought it to 5m18s. Here is what we changed, with the YAML.

AN ORIGINAL HAKOPOD ILLUSTRATION

Short answer: our Next.js app’s Docker build in GitHub Actions took 8m43s. With three changes to the workflow it now takes 5m18s when the cache is warm. That is about 40% faster. The biggest single win was letting Docker use all four CPUs of the runner.

What were we building?

The app is a Next.js 16.3.4 site. It installs 2,424 npm packages and renders 219 static pages. We build its production image with next build --webpack inside a multi-stage Dockerfile.

The workflow runs on Hakopod Managed Actions. A job picks our runners with runs-on: [hakopod], and each job gets its own short-lived AWS spot VM: a c7i, c6i or m7i.xlarge with 4 vCPUs, in us-east-1. The VM comes with Docker installed and is deleted when the job finishes.

Here is where the 8m43s went:

StepTime
Install packages (npm ci)1m36s
Compile with webpack4m10s
Type-check1m10s
Render static pages and finishabout 50s
Export and push the imageabout 1m

Why was the build slow?

Every job started from zero. Three things caused that.

  1. Docker’s caches were always empty. Our Dockerfile caches npm downloads and the Next.js build cache with --mount=type=cache,target=/root/.npm and /app/.next/cache. Those caches live on the machine that ran the build, so a brand-new VM never had them.
  2. The shared layer cache was never refreshed from our runners. The workflow only wrote Docker’s GitHub Actions cache when it ran on GitHub’s own runners.
  3. Docker was limited to two CPUs. The workflow had been tuned for GitHub’s 2-core runner, with memory=5g, cpu-quota=200000 and max-parallelism = 1. On a 4-vCPU VM, half the machine sat idle.

What did we change?

1. Only limit Docker on GitHub’s own runners. The limits now apply only when runner.environment is github-hosted:

driver-opts: ${{ runner.environment == 'github-hosted' && 'network=host,memory=5g,memory-swap=5g,cpu-quota=200000,cpu-period=100000' || 'network=host' }}

The max-parallelism setting follows the same rule.

2. Carry the Next.js cache from one VM to the next. actions/cache@v5 saves a buildkit-cache folder. The cache key is the OS, the package-lock.json hash and the commit. If there is no exact match, it falls back to the same lockfile, then to the same OS. reproducible-containers/buildkit-cache-dance@v3.4.0 copies that folder into Docker’s cache mount and back out:

cache-map: { "buildkit-cache/next": "/app/.next/cache" }
skip-extraction: ${{ steps.next-cache.outputs.cache-hit }}

3. Save Docker’s layer cache from every runner. We now set cache-to: type=gha,mode=max on all runners. mode=max also keeps the in-between layers, including the one that runs npm ci.

How much faster did it get?

Docker buildTime
Before8m43s
After, first run with an empty cache8m05s
After, with a warm cache5m18s

The warm build is about 40% faster. Step by step:

StepBeforeWarm cache
Compile with webpack4m10s2m24s
Type-check1m10s8s
Install packages (npm ci)1m36s47s
  • Compile got faster because Docker could use all four CPUs. That gain shows up even with an empty cache.
  • Type-checking dropped to 8 seconds because TypeScript’s incremental build file now survives in .next/cache.
  • npm ci is now a cached layer. Docker reuses it instead of reinstalling. The 47 seconds left is downloading and unpacking that cached layer from GitHub’s cache.

What is still slow?

The compile step. A warm compile takes as long as a cold one, because webpack’s own cache is off on purpose.

Our Next.js config turns webpack’s cache off and builds one module at a time (config.cache = false, parallelism = 1) in container builds, so the build fits on small builders. We tried lifting that on a spot VM. On an 8 GB c6i.xlarge, full parallelism with a 5 GB heap ran out of memory, and the build stalled for more than 11 minutes. So we kept the memory-safe settings. The whole compile gain comes from the extra CPUs, and the type-check gain comes from the restored .next/cache.

We also tried Next.js’s experimental cpus: 1 setting. It only affects static page rendering, which takes about 6 seconds here, so it wasn’t the problem.

Would a bigger VM with webpack’s cache turned on help?

Not for this app. We tried it next.

The build checked how much memory its builder had, the smaller of RAM and any container memory limit, and sized itself from that. It gave Node 60% of it as heap. With 14 GiB or more, it turned webpack’s cache and full parallelism back on (“fast mode”). Smaller builders kept the memory-safe settings.

We ran it on a 16 GB m7i.xlarge spot VM. The build logged builder memory 15701 MiB: low-memory=0, heap=9420 MiB.

ModeCompileType-checkDocker build step
Fast mode, first build2.8 min15s6m17s
Fast mode, cache restored2.8 min47s6m42s
Memory-safe mode (what we ship)2.4 min8s5m18s

Even on 16 GB, webpack’s cache and full parallelism didn’t make the compile faster, and the whole build got slower. On an 8 GB c6i.xlarge, fast mode had already stalled for more than 11 minutes. So we kept the memory-safe settings on every VM and didn’t ship the memory-sizing change.

The lesson: measure before you assume a bigger heap and a warm compiler cache will help. Here the real wins came from letting Docker use every CPU and from carrying .next/cache between runs for TypeScript.

What else went wrong along the way?

Two runs failed for reasons that had nothing to do with the build:

  • A fresh VM timed out logging in to ghcr.io. Nothing in the build ran.
  • AWS reclaimed a spot instance 4 minutes into a build. Hakopod reruns a job lost to a spot interruption on its own, but only if the pool’s GitHub token has Actions: Read and write permission. Ours was read-only, so the rerun didn’t start. We’ve fixed that. If you run spot runners, check this permission first.

Does caching cost extra on spot VMs?

Yes, and it’s the part to watch. Each build uploads roughly 1 GB of cache from AWS to GitHub. Sending data out of AWS costs about 9 cents per build. The build itself uses about 1 cent of spot VM time. On GitHub’s 4-core hosted runner, at $0.012 a minute, a build this long costs about 10 cents.

So on spot VMs, compute is cheap and moving data out is not. Check how much time a cache actually saves before you upload it from every build.

Can I do this on my own runners?

Yes. If you build Docker images on bigger runners than GitHub’s 2-core default, look for CPU and memory limits like ours. They quietly waste the extra cores. On Hakopod you can run GitHub Actions jobs on spot VMs in your own AWS account and compare the cost with the calculator there. The setup guide shows how to connect an AWS account.

Questions people ask

Why is my Next.js Docker build slow in GitHub Actions?

Usually because every job starts on a clean machine. Docker's cache mounts for npm and .next/cache start empty on a fresh runner, so every build downloads packages and compiles from scratch. Builders tuned for GitHub's 2-core runner can also leave CPUs idle on a bigger machine.

Do Docker BuildKit cache mounts work on ephemeral GitHub Actions runners?

Not by themselves. A cache mount lives inside the BuildKit instance on that one machine, so a new runner starts with it empty. You have to save it to an external cache, for example with actions/cache and buildkit-cache-dance, and restore it in the next job.

How do I keep the Next.js build cache between GitHub Actions runs?

Save the directory behind your /app/.next/cache mount with actions/cache, keyed on your lockfile, and move it in and out of the BuildKit cache mount with reproducible-containers/buildkit-cache-dance. On a warm cache our TypeScript step went from 1m10s to 8s.

Does webpack's persistent cache make a Next.js Docker build faster?

Not for our app. On a 16 GB spot VM, turning webpack's cache and full parallelism back on left the compile at 2.8 minutes and made the whole Docker build slower: 6m17s on a first build and 6m42s with the cache restored, against 5m18s with webpack's cache off. On an 8 GB VM it stalled for more than 11 minutes. Measure it on your own app before you rely on it.

Why didn't my GitHub Actions job rerun after a spot interruption?

On Hakopod, the automatic rerun after a spot interruption needs the runner pool's GitHub token to have Actions: Read and write permission. With a read-only token, the interrupted job is not rerun.

Are spot VM runners cheaper than GitHub-hosted runners?

For this build, yes. The build used about 1 cent of AWS spot time, against about 10 cents on GitHub's 4-core hosted runner at $0.012 a minute. Uploading the build cache from AWS added about 9 cents of data transfer, so that is the cost to watch.

Have a use case we should understand?Share it on GitHub ↗

Keep exploring.

All stories↗
Engineering / 4 min read

What MyDuck’s two SQL protocols actually mean

How one DuckDB database serves MySQL and PostgreSQL clients, where compatibility stops, and what the managed release does and does not provide.

Read the story↗
Engineering / 8 min read

What Hakopod’s Managed Vitess release runs

A concrete look at Vitess shards, routing, private TLS, native backups, recovery and the limits of the first release.

Read the story↗