Earthly: Repeatable Builds That Run Everywhere

Imagine a build system that works the same way on a laptop, a CI server, or a Kubernetes pod—without you having to rewrite scripts for each environment. That’s the promise of Earthly, a modern, container‑native build tool that treats every step as a reproducible, cacheable Docker layer. By leveraging familiar Docker concepts and a simple, declarative syntax, Earthly lets developers write repeatable builds that run anywhere Docker does, while keeping the learning curve low.

Why Earthly Stands Out

Traditional build tools like Make, Gradle, or Maven often rely on the host OS’s shell, which makes builds fragile across platforms. Earthly abstracts the underlying OS by running each command inside a lightweight container, guaranteeing the same environment every time. This containerization also means you get built‑in caching, parallelism, and the ability to ship your entire build definition alongside your source code.

Another key advantage is Earthly’s Earthfile, a concise DSL that feels like a blend of Dockerfile syntax and Makefile targets. You define FROM statements to pull base images, then chain RUN, COPY, and CMD steps just like you would in Docker. The result is a build pipeline that’s both human‑readable and machine‑optimizable.

Getting Started: Installing Earthly

Installation is a single command on most platforms. On macOS or Linux you can run:

curl -sL https://github.com/earthly/earthly/releases/download/v0.8.0/earthly-linux-amd64 -o /usr/local/bin/earthly
chmod +x /usr/local/bin/earthly

Windows users can grab the .exe from the releases page or use Scoop:

scoop install earthly

Once installed, verify it with earthly --version. You’ll see the version number and a short help message, confirming that the binary can communicate with the Docker daemon on your machine.

Your First Earthfile

Let’s create a simple “Hello, World!” project to illustrate the core concepts. In a new directory, add Earthfile with the following content:

# Earthfile
FROM python:3.12-alpine

WORKDIR /app
COPY . .

RUN pip install --no-cache-dir -r requirements.txt

CMD ["python", "main.py"]

This Earthfile does exactly what a Dockerfile would do, but Earthly treats it as a reusable target. Save a requirements.txt containing requests and a main.py that prints a greeting. Then run:

earthly +build

Earthly automatically creates a +build target from the file name. The first run pulls the base image, installs dependencies, and caches each step. Subsequent runs are lightning‑fast because Earthly reuses the cached layers unless a file changes.

Understanding Targets

Targets in Earthly are prefixed with a plus sign (+) and can be invoked directly from the CLI. You can also define multiple targets in a single Earthfile, making it easy to separate compile, test, and publish stages.

# Earthfile
FROM python:3.12-alpine AS base
WORKDIR /app
COPY . .

FROM base AS test
RUN pip install --no-cache-dir -r requirements.txt
RUN pytest

FROM base AS package
RUN pip install --no-cache-dir -r requirements.txt
RUN python -m zipapp -p "/usr/bin/env python3" -o app.pyz .

Now you have three distinct targets: +test, +package, and the implicit +default (which would be the last defined stage). Running earthly +test executes only the test pipeline, while +package produces a self‑contained zipapp.

Pro tip: Use the --push flag with a target that builds an image to automatically push it to a registry, e.g., earthly --push +publish. This eliminates a separate Docker push step.

Real‑World Use Case: Multi‑Stage CI/CD Pipelines

In many organizations, the CI environment differs from the developer’s laptop, leading to “works on my machine” bugs. Earthly solves this by letting you define the entire CI pipeline in a single Earthfile that runs unchanged on GitHub Actions, GitLab CI, or any self‑hosted runner.

Below is a practical Earthfile for a Node.js application that compiles TypeScript, runs Jest tests, builds a Docker image, and finally pushes it to a registry. The example demonstrates how Earthly’s built‑in caching speeds up each stage.

# Earthfile
FROM node:20-alpine AS deps
WORKDIR /src
COPY package.json package-lock.json ./
RUN npm ci

FROM deps AS build
COPY . .
RUN npm run build   # Compiles TypeScript

FROM build AS test
RUN npm test       # Runs Jest

FROM node:20-alpine AS runtime
WORKDIR /app
COPY --from=build /src/dist ./dist
COPY package.json .
RUN npm ci --production

# Final Docker image
FROM alpine:3.18 AS final
COPY --from=runtime /app /app
ENTRYPOINT ["node", "/app/dist/index.js"]

To use this Earthfile in GitHub Actions, create a workflow step like:

- name: Build & Push Docker Image
  run: |
    earthly --ci --push +final
  env:
    EARTHLY_TOKEN: ${{ secrets.EARTHLY_TOKEN }}
    DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
    DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}

The --ci flag tells Earthly to treat the run as a CI job, disabling interactive prompts and ensuring deterministic caching. The +final target builds the image and pushes it in one atomic step.

Parallelism and Matrix Builds

Earthly can also orchestrate matrix builds without extra scripting. Suppose you need to test your library against Python 3.9, 3.10, and 3.11. Define a single target that accepts a version argument, then invoke it multiple times:

# Earthfile
ARG PYTHON_VERSION=3.12

FROM python:${PYTHON_VERSION}-slim AS test
WORKDIR /src
COPY . .
RUN pip install -r requirements.txt
RUN pytest

Run the matrix locally:

for v in 3.9 3.10 3.11; do
  earthly --var PYTHON_VERSION=$v +test
done

In CI you can map this loop to a matrix strategy, and Earthly will cache each version’s dependencies separately, drastically reducing total runtime.

Pro tip: Use the --cache-from flag to pull caches from a remote registry (e.g., GitHub Packages) when spinning up fresh runners. This gives you “cold start” speed comparable to a warm local machine.

Advanced Features: Secrets, Artifacts, and Remote Execution

Security is a common concern when builds need to access API keys or database passwords. Earthly provides a built‑in SECRET directive that injects values at runtime without baking them into the image layers.

# Earthfile
FROM alpine:3.18 AS secret-demo
ARG SECRET_API_KEY
RUN echo "Using API key: $SECRET_API_KEY"

Invoke the target with a secret sourced from your CI system:

earthly --secret API_KEY=$MY_API_KEY +secret-demo

Artifacts—files produced by one target and consumed by another—are handled automatically via COPY --from. This eliminates the need for intermediate archives or manual copy‑paste steps.

# Earthfile
FROM golang:1.22-alpine AS builder
WORKDIR /src
COPY . .
RUN go build -o /out/app .

FROM alpine:3.18 AS runtime
COPY --from=builder /out/app /usr/local/bin/app
ENTRYPOINT ["app"]

The binary built in the builder stage becomes an artifact that the runtime stage consumes directly. Earthly tracks these dependencies, so if the source code changes, only the builder stage is invalidated; the runtime stage is reused from cache.

Remote Execution with Earthly Cloud

For teams that want to offload heavy builds to the cloud, Earthly offers a managed service called Earthly Cloud. By logging in with earthly login, you can push caches and run builds on powerful remote workers, then pull the results back to your local environment.

earthly login
earthly cloud push +build
earthly cloud pull +build

This workflow is especially useful for resource‑intensive tasks like compiling large C++ codebases or creating multi‑arch Docker images. The remote cache is shared across the team, ensuring that each developer benefits from the work done by others.

Pro tip: Combine earthly cloud push with Git tags. Tag a commit, push the cache, and later any teammate can retrieve the exact same build artifacts by checking out the tag and running earthly cloud pull +build.

Testing and Linting Inside Earthly

Because Earthly treats every step as a container, you can run linters, static analysis tools, and security scanners in isolation without polluting your host environment. Here’s a concise Earthfile that runs flake8, mypy, and bandit on a Python project.

# Earthfile
FROM python:3.12-slim AS lint
WORKDIR /src
COPY . .
RUN pip install flake8 mypy bandit
RUN flake8 .
RUN mypy .
RUN bandit -r .

# Optional: produce a report artifact
FROM scratch AS report
COPY --from=lint /src /report

Running earthly +lint gives you a single, reproducible command that outputs all linting results. If any step fails, Earthly aborts early, making it perfect for CI gate checks.

Integrating with Pre‑Commit Hooks

Developers often want the same checks locally before committing code. By adding a small wrapper script, you can invoke Earthly from a pre-commit hook:

#!/usr/bin/env bash
set -e
earthly +lint

Place this script in .git/hooks/pre-commit and make it executable. Now every git commit runs the full lint suite inside a container, guaranteeing consistency between local and CI environments.

Best Practices for Scalable Earthly Projects

1. Keep Targets Small and Focused – Each target should do one logical thing (compile, test, package). Small targets improve cache granularity and make debugging easier.

2. Use Named Stages – The AS clause creates reusable stages that can be referenced with COPY --from. This reduces duplication and clarifies data flow.

3. Pin Base Images – Always specify exact image tags (e.g., python:3.12.4-alpine) to avoid accidental breakages when upstream images change.

4. Leverage Build Args Sparingly – Arguments are great for versioning, but overusing them can invalidate caches. Prefer separate targets for different configurations.

5. Store Secrets Outside the Earthfile – Use --secret or CI environment variables instead of hard‑coding credentials.

Pro tip: Enable Earthly’s --progress=plain flag in CI logs to get a clean, line‑by‑line view of each step. This makes troubleshooting failed builds much faster.

Migrating an Existing Dockerfile to Earthly

If you already have a Dockerfile, converting it to an Earthfile is straightforward. The syntax is almost identical; you only need to add target names and optionally split stages.

Original Dockerfile:

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS runtime
WORKDIR /app
COPY --from=builder /app/dist ./dist
CMD ["node", "dist/index.js"]

Equivalent Earthfile:

# Earthfile
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS runtime
WORKDIR /app
COPY --from=builder /app/dist ./dist
CMD ["node", "dist/index.js"]

The only change is the optional + prefix when invoking a target, e.g., earthly +runtime. By default, Earthly treats the last defined stage as the default target, so earthly alone will produce the final runtime image.

Performance Benchmarks

In a recent benchmark, a Go monorepo with 12 sub‑modules took 18 minutes to build from scratch using traditional Docker + Make. Switching to Earthly reduced the cold build time to 7 minutes and subsequent incremental builds to under 30 seconds, thanks to layer‑level caching and parallel execution.

Another case study involved a Python data‑science pipeline that required large model files. By storing those files as cached artifacts in Earthly Cloud, the team saved an average of 45 minutes per nightly run, freeing up compute resources for actual model training.

Common Pitfalls and How to Avoid Them

Cache Misses Due to Untracked Files – If you copy the entire repository with COPY . ., any unrelated change (e.g., a README update) invalidates the cache. Instead, copy only the directories needed for each stage.

Overusing ARG for Versions – Changing an argument forces a rebuild of all downstream layers. Use separate targets for major version changes.

Forgotten Secrets – Hard‑coding credentials inside an Earthfile will embed them into the image layers, exposing them in registries. Always use --secret or CI environment variables.

Pro tip: Run earthly ls --graph to visualize your dependency graph. Spotting unnecessary edges can help you refactor the Earthfile for better cache efficiency.

Extending Earthly with Plugins

Earthly supports custom commands via its plugin system. For example, the community‑maintained earthly-plugin-helm lets you package Helm charts as part of the build pipeline.

# Earthfile
FROM alpine:3.18 AS helm
RUN apk add --no-cache curl tar gzip
RUN curl -L https://get.helm.sh/helm-v3.12.0-linux-amd64.tar.gz | tar xz
COPY chart/ /chart/
RUN ./linux-amd64/helm package /chart -d /out

FROM scratch AS helm-artifact
COPY --from=helm /out/*.tgz /helm-charts/

Running earthly +helm-artifact produces a Helm package ready for deployment, all within the same reproducible pipeline that builds your application binary.

Conclusion

Earthly bridges the gap between container