CI fails in a way that nobody sees until they need it. The server stays up, the web UI loads, and no pipeline has run since Tuesday because the one agent lost its gRPC connection after a reboot, or the forge stopped delivering webhooks when a token expired, or the queue filled with pending steps that no worker will ever pick up. The first symptom is a developer asking why their merge has been "pending" for forty minutes, usually right before a release. Both Woodpecker CI and Drone CI give you something to check from outside; neither gives you enough on its own.
This guide covers both engines because they are siblings. Woodpecker is the community fork of Drone, and Drone's source is now a snapshot branch inside the Harness repository, so plenty of teams run one while planning to move to the other. Status checks, exact expected status codes, and custom headers are on every CronAlert plan; keyword assertions and heartbeats are Pro ($5/month) and are marked. The examples use ci.example.com for the server behind a reverse proxy.
What each server exposes
| Woodpecker | Drone | |
|---|---|---|
| Health | GET /healthz: pings the datastore, 204 on success, 500 with the error on failure | GET /healthz: writes 200 and OK, checks nothing |
| Version | GET /version → {"source":"…","version":"…"} | GET /version → {"source","version","commit"} |
| Metrics | GET /metrics, only when WOODPECKER_PROMETHEUS_AUTH_TOKEN is set; bearer token (user API token). Or an unprotected copy on WOODPECKER_METRICS_SERVER_ADDR | GET /metrics, bearer token of an admin machine user, or anonymous with DRONE_PROMETHEUS_ANONYMOUS_ACCESS=true |
| Ports | HTTP :8000, gRPC for agents :9000 | HTTP on the server host; runners connect over HTTP(S) via DRONE_RPC_HOST |
| Agent / runner health | Agent: /healthz, /version, /varz on WOODPECKER_HEALTHCHECK_ADDR (default :3000), 200 or 500 | Docker runner: unauthenticated /healthz on its HTTP port (dashboard port, 3000 by default) |
| Scheduled pipelines | Repo settings → Cron; when: event: cron; schedules like @every 5m, @daily, 30 * * * * | drone cron add; 6-field expressions with seconds or @hourly/@daily; UTC; run in hourly batches |
The two health endpoints look identical and are not. Woodpecker's handler calls Ping() on the store before answering, so its /healthz is a database check that happens to be served over HTTP, and its success code is 204 No Content. Drone's handler returns 200 unconditionally, so it tells you the Go process is accepting connections and nothing else. Set your expectations, and your expected status codes, accordingly.
Step 1: the server health monitor
- Woodpecker: add an HTTP monitor on
https://ci.example.com/healthzwith the expected status set to204. CronAlert accepts any exact status from 100 to 599, and 204 is what a healthy Woodpecker returns; leave it at 200 and the monitor is red forever. If you run Woodpecker under a sub-path (WOODPECKER_ROOT_PATH), the endpoint is under that path too. - Drone: add a monitor on
https://ci.example.com/healthzwith expected status200. Because it checks nothing, pair it on Pro with a keyword monitor on/versionasserting contains"version", which at least proves the HTTP router and handlers are alive rather than a proxy's default page. - Timeout 10 seconds; both endpoints answer in milliseconds on a healthy server.
The check goes through your public hostname, so it also covers DNS, TLS, and the reverse proxy, and SSL monitoring is included on every plan. The interval is 3 minutes on Free and 1 minute on Pro; alerts go to your team's channels. If the server is reachable only inside your network, run these checks from a cron job on a host that can reach it and report with the heartbeat in step 3.
Step 2: the version endpoint as an upgrade tripwire (Pro)
Both /version endpoints return small JSON that changes only when you upgrade. A content changed monitor on it alerts once per upgrade, which is a cheap audit trail: if the version changed and nobody on the team deployed it, you want to know. It also catches the opposite, a server that was supposed to be upgraded and silently rolled back. Teams that upgrade weekly will find it noisy; teams that upgrade quarterly tend to like it. Content monitoring covers the match types.
Step 3: a scheduled pipeline that pings a heartbeat (Pro)
This is the check that matters. A green server with zero connected agents runs nothing, and neither health endpoint knows. The honest test of a CI system is whether it can run a pipeline, so make it run one on a schedule that does exactly one thing:
Woodpecker. In the repository's settings, add a cron job (the docs say you need at least push access) with a schedule such as @every 15m; Woodpecker's scheduler accepts @every, @daily, and standard five-field expressions. Then in the pipeline:
# .woodpecker/heartbeat.yml
when:
event: cron
cron: heartbeat # only this named cron job, not every cron
steps:
ping:
image: curlimages/curl
commands:
- curl -fsS --max-time 10 --retry 3 https://cronalert.com/api/heartbeat/YOUR-TOKENCreate a heartbeat monitor with an expected interval of 15 minutes and a grace of 15 minutes. If the agent is disconnected, the queue is wedged, the container runtime is broken, or the server cannot reach the forge to clone, the pipeline does not run and the ping does not arrive. If you run several agents with different labels, create one cron job per label with a labels: selector so each agent pool has its own heartbeat; one dead pool out of three is otherwise invisible.
Drone. Create the cron job with the CLI and match it in the pipeline trigger:
drone cron add octocat/infra heartbeat @hourly
# .drone.yml
kind: pipeline
type: docker
name: heartbeat
trigger:
event:
- cron
cron:
- heartbeat
steps:
- name: ping
image: curlimages/curl
commands:
- curl -fsS --max-time 10 --retry 3 https://cronalert.com/api/heartbeat/YOUR-TOKENDrone's documentation is explicit that the cron scheduler is approximate and executes jobs in hourly batches, so a job can run up to an hour late, and schedules are in UTC. That dictates the heartbeat: expected interval 1 hour, grace at least 1 hour, or you will page yourself on normal batching. An @every 15m-style schedule is not available; Drone accepts six-field expressions with seconds and the predefined @hourly, @daily, @weekly, @monthly, and @yearly. Cron is disabled on Drone Cloud, which does not matter for a self-hosted server. Grace periods explains why the grace should match the scheduler's worst case rather than its average, and diagnosing heartbeat alerts shows how to tell a late batch from a dead agent.
The Jenkins guide uses the same trick with a scheduled job per agent label, and the GitLab one with a scheduled pipeline; it is the only agent check that tests what you actually care about.
Step 4: agent and runner health on their own ports
The Woodpecker agent serves /healthz (200 when its internal state is healthy, 500 otherwise), /version, and /varz with counters, on WOODPECKER_HEALTHCHECK_ADDR, default :3000, enabled by WOODPECKER_HEALTHCHECK which defaults to true. The Drone Docker runner registers an unauthenticated /healthz on its HTTP port before the dashboard routes, so it exists whether or not DRONE_UI_PASSWORD is set. Neither should be exposed to the internet; use them from a cron job on the agent host and report with a heartbeat:
# every 5 minutes on each agent host
*/5 * * * * curl -fsS --max-time 5 http://127.0.0.1:3000/healthz >/dev/null \
&& curl -fsS --max-time 10 --retry 3 https://cronalert.com/api/heartbeat/AGENT-TOKENThis is a faster signal than the scheduled pipeline (five minutes instead of fifteen or sixty) but a weaker one: it says the agent process is alive, not that it can take work. Use both if you have the monitor budget; the pipeline heartbeat alone if you do not. Monitoring Docker self-hosted services covers the pattern.
Step 5: metrics as a heartbeat gate (Pro)
Both servers export Prometheus metrics, and the two numbers that describe a stuck CI system are pending work and available workers. Woodpecker exposes woodpecker_pending_steps, woodpecker_running_steps, woodpecker_waiting_steps, and woodpecker_worker_count; Drone exposes drone_pending_builds, drone_pending_jobs, drone_running_builds, and drone_running_jobs (and no worker count). Both endpoints require a bearer token by default (Woodpecker's is enabled only when WOODPECKER_PROMETHEUS_AUTH_TOKEN is set; Drone's wants an admin machine user's token unless DRONE_PROMETHEUS_ANONYMOUS_ACCESS=true). An HTTP monitor could send the token in a custom header, but do not put an admin credential in a monitor. Scrape from a script:
# every 5 minutes: Woodpecker has at least one worker and the queue is not stuck
*/5 * * * * M=$(curl -fsS --max-time 10 -H "Authorization: Bearer $WP_TOKEN" https://ci.example.com/metrics) \
&& W=$(echo "$M" | awk '/^woodpecker_worker_count /{print int($2)}') \
&& P=$(echo "$M" | awk '/^woodpecker_pending_steps /{print int($2)}') \
&& [ "${W:-0}" -ge 1 ] && [ "${P:-0}" -lt 20 ] \
&& curl -fsS --max-time 10 --retry 3 https://cronalert.com/api/heartbeat/QUEUE-TOKENHeartbeat interval 5 minutes, grace 10. For Drone, gate on drone_pending_jobs below a threshold, or on pending being nonzero while drone_running_jobs is zero for two consecutive runs. If you already run Prometheus, alert there and use a watchdog heartbeat to make sure Prometheus itself is alive; the script above is for teams that do not.
Step 6: the forge, and the backup (Pro)
CI only runs when the forge delivers a webhook, and a forge that is down or a webhook secret that was rotated produces the same silence as a dead agent. Monitor the forge with its own checks: Gitea and Forgejo, GitLab, or GitHub as a third-party dependency. The scheduled-pipeline heartbeat in step 3 catches the server-to-forge direction (clone fails, no run), but not forge-to-server (push event never arrives); a once-a-day commit to a canary repository that triggers a pipeline that pings a second heartbeat closes that gap if you need it.
For the backup, both servers keep their state in a database (SQLite by default on small installs, PostgreSQL or MySQL otherwise) plus secrets, and losing it loses pipeline history, repository activations, and secrets. Wrap the dump in a cron job that ends with a heartbeat ping, as in monitoring database backups; sqlite3 .backup for SQLite, pg_dump for Postgres, and the && before the curl so a failed dump does not ping.
Reading an alert
| Alert | Likely cause | First thing to check |
|---|---|---|
/healthz timeout or connection error | Host, container, or proxy down | Container status; proxy logs |
Woodpecker /healthz 500 | Database unreachable (the handler's Ping failed) | Database container; WOODPECKER_DATABASE_DATASOURCE; disk space for SQLite |
Woodpecker /healthz red with "expected 204, got 200" | Proxy serving something else at that path | The response body in the check result |
Drone /healthz green, /version keyword missing | Proxy default page or wrong upstream | Proxy config |
| Pipeline heartbeat overdue, server green | No agent connected, queue stuck, runtime broken, clone failing | Agent logs for gRPC or RPC errors; server queue in the UI |
| Drone pipeline heartbeat late by under an hour | Normal hourly batching | Nothing; widen grace if it alerts |
| Agent-host heartbeat overdue | Agent process down or host rebooted without it | docker ps / systemctl status on the agent host |
| Queue heartbeat overdue | Zero workers or pending steps piling up | Metrics by hand; agent registration secret |
| Version content changed | Upgrade or rollback | Who deployed it |
Frequently asked questions
Why does Woodpecker return 204 instead of 200?
Its handler sets http.StatusNoContent after the datastore ping succeeds and sends no body; the source comment says a 204 means everything is fine and a 500 signals an unhealthy state. Monitors that assume 200 will fail against Woodpecker, which is why the expected status is configurable.
Can I monitor the agent's gRPC port?
A TCP monitor on the server's :9000 proves the gRPC listener is open if the server has a public address, which catches a firewall change. It cannot tell whether any agent is connected; only the scheduled pipeline or woodpecker_worker_count can.
Will a pipeline every 15 minutes cost anything?
It runs a one-step container that exits in under a second. On a self-hosted agent that is free; the only cost is a row in the pipeline history, which both servers prune on their own schedules.
Which parts are free?
The server /healthz monitors with their exact expected status, SSL monitoring, and the TCP check are free. The /version keyword and content-changed monitors and every heartbeat (pipeline, agent, queue, backup) need Pro at $5/month.
Get started
One free monitor on /healthz with the right status code, and one scheduled pipeline whose only job is to prove CI can still run a job. Create a free CronAlert account, add the health monitor, and commit the heartbeat pipeline. Related reading: uptime monitoring in CI/CD pipelines, monitoring Jenkins, monitoring GitHub Actions scheduled workflows, monitoring a container registry, and cron job heartbeats.