restart: unless-stopped is the first thing everybody adds to a Compose file, and it is the least useful part of a self-healing stack. It restarts a container when the *process* dies. It does nothing when the process is alive but the application is wedged — the classic case for self-hosted services that leak connections, lose their database link, or hang on a dead NFS mount.
Healthchecks fix that, but only if something acts on them. That is what autoheal is for.
1. Write a healthcheck that means something
A healthcheck that curls / and accepts a 500 is not a healthcheck. Aim at an endpoint that touches the real dependencies:
services:
app:
image: ghcr.io/example/app:1.4.2
restart: unless-stopped
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8080/healthz"]
interval: 30s
timeout: 5s
retries: 3
start_period: 90s
depends_on:
db:
condition: service_healthy
Three details matter more than the command:
start_periodgives a slow-booting app time to warm up without burningretries. Without it, a 60-second JVM start counts as three failures and the container restarts forever.intervalshould be slower than your worst expected latency. A 2-second interval on a busy DB-backed endpoint creates its own outage.- The check must run inside the container.
wgetorcurlhas to exist in the image; if it does not, use a tiny shell probe instead:
test: ["CMD-SHELL", "timeout 3 bash -c '</dev/tcp/127.0.0.1/8080' || exit 1"]
2. Gate dependencies on health, not start order
condition: service_healthy is the single biggest reliability win in Compose. Without it, depends_on only waits for the container to be *created*, and every stack with a database races on first boot — which shows up later as an application that works after a manual restart and fails after a reboot.
If a service exposes no healthcheck, depends_on falls back to start order. Set condition: service_started explicitly so future readers know it was a choice.
3. Add autoheal
Docker does not restart unhealthy containers by itself. A one-line sidecar does:
autoheal:
image: willfarrell/autoheal:1.2.0
restart: unless-stopped
environment:
AUTOHEAL_CONTAINER_LABEL: autoheal
AUTOHEAL_INTERVAL: 10
AUTOHEAL_START_PERIOD: 60
AUTOHEAL_DEFAULT_STOP_TIMEOUT: 30
volumes:
- /var/run/docker.sock:/var/run/docker.sock
Then label the services it may restart:
labels:
autoheal: "true"
Using AUTOHEAL_CONTAINER_LABEL instead of all is deliberate: you do not want a sidecar restarting your database because a transient disk stall made it unhealthy for thirty seconds.
4. Prove it works
Break the app on purpose and watch the loop close:
docker compose up -d
docker inspect -f '{{.State.Health.Status}}' app
# healthy
# simulate a hang: freeze the main process
docker compose exec app sh -c 'kill -STOP 1'
watch -n5 'docker inspect -f "{{.State.Health.Status}} {{.RestartCount}}" app'
# unhealthy … then running with RestartCount 1
If RestartCount never moves, autoheal is not seeing the container. Nine times out of ten the label is missing or set on the wrong service; the tenth is a socket permission problem.
5. What healthchecks cannot do
- They do not fix crashes in the app config. A container that restarts every 30 seconds will also restart every 30 seconds with autoheal, only noisier. Add
restart: on-failure:5for anything that can flap. - They are not monitoring. A healthcheck is a local, cheap probe. It will not tell you the certificate expired, the disk is full, or the service answered slowly. Ship logs and metrics somewhere you actually look.
- They cost resources. Forty containers probing every 5 seconds is real CPU. Thirty seconds is a sane floor.
Healthchecks plus one gated dependency plus autoheal turns a stack that needs a human after every hiccup into one that mostly needs a human when something is genuinely wrong.