Once a team puts a gateway in front of its models, the gateway becomes the thing that fails. LiteLLM routes every request, holds every provider key, enforces every budget, and keeps all of that in a Postgres database; when that database is unreachable, every virtual key stops working and every model is "down" at the same moment, with the models themselves perfectly healthy. Open WebUI is the face your team actually sees, and it fails in its own ways: a bad upgrade, a full disk under its SQLite file, a provider connection whose key was rotated. And behind both of them the model server can unload a model, lose its GPU, or hang, which no gateway health route reports.
This guide monitors the gateway in layers, from the process up to a real answer: is the proxy alive, can it serve, does a key work, is the UI up, does the UI see models, does a prompt come back, is the certificate valid, and did the backup run. Each is one CronAlert monitor. The unauthenticated HTTP checks run on the free plan; keyword assertions and heartbeats are Pro.
Layer 1: LiteLLM is alive and ready
LiteLLM ships three health routes with very different costs, and picking the right one matters more here than on most services. Two of them need no key and no dependencies:
URL: https://llm.example.com/health/liveliness
Expected status: 200
Keyword (Pro): contains "I'm alive!"
URL: https://llm.example.com/health/readiness
Expected status: 200 (503 = database unreachable)Liveness proves the process is up and nothing more; it returns 503 only during a graceful shutdown. Readiness is the one that catches the outage that matters: when LiteLLM is configured with a database and cannot reach it, readiness goes 503 and, in Kubernetes, the pod leaves rotation. Both routes are explicitly unauthenticated even with a master key set, and readiness's default response is deliberately low-detail so it is safe to expose; the detailed variant lives at /health/readiness/details behind a key if you want it.
The third route, /health, needs a key and makes a real request to every configured model on each call to report which are healthy. That is a diagnostic, not a probe. On a one-minute monitor against a dozen models it is twelve model calls a minute, slow enough to trip the timeout and expensive enough to notice. Leave it for the dashboard and the layer 3 heartbeat.
Layer 2: a key works and the config loaded
Readiness being green means the database answers. It does not mean a virtual key resolves, or that the config LiteLLM loaded on its last restart is the one you meant. The cheapest authenticated check is the model list:
URL: https://llm.example.com/v1/models
Expected status: 200
Custom headers: { "Authorization": "Bearer sk-monitoring-key" }
Keyword (Pro): contains "gpt-4.1-mini" (a model that must be present)Create a dedicated virtual key for monitoring with a small budget and access to one model. This request makes no model call; it exercises key lookup, team and budget checks, and the loaded model list. A 401 means keys are broken for everyone, a 200 without the keyword means the config lost a model, and both leave the readiness route green. The custom headers guide covers keeping that key in a header rather than a URL, and API rate-limit monitoring covers the related case where keys work but budgets are exhausted.
Layer 3: Open WebUI is up and sees models
Open WebUI serves a documented, unauthenticated health route that confirms the web server, application startup, and basic database connectivity:
URL: https://chat.example.com/health
Expected status: 200That covers "the UI is up". For "the UI can actually offer a model", use the authenticated model list with a user API key (generated under Settings → Account, if the admin has enabled API keys):
URL: https://chat.example.com/api/models
Expected status: 200
Custom headers: { "Authorization": "Bearer <open-webui-api-key>" }
Keyword (Pro): contains "llama3.1"Open WebUI builds that list by asking each configured connection (Ollama, LiteLLM, OpenAI-compatible providers) for its models, so a connection whose key was rotated or whose host is down drops its models from the response. If the model your team depends on is missing, this monitor goes red while /health stays green, and you know which connection to look at. Use a key belonging to a low-privilege user created for monitoring, and remember that admins can restrict which endpoints an API key may reach; this one needs only the models route.
Layer 4: a prompt comes back (the heartbeat)
Every layer so far asks the gateway about itself. None of them proves a model will answer, because an HTTP uptime monitor sends no request body and therefore cannot send a prompt. The check that fails for the same reasons your users' requests fail is a real, tiny completion, run on a schedule, reporting to a heartbeat only on success:
#!/bin/sh
# One-token completion through the gateway, 20s hard limit, ping only on success
curl -fsS --max-time 20 https://llm.example.com/v1/chat/completions \
-H "Authorization: Bearer $LITELLM_MONITOR_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4.1-mini","max_tokens":1,"messages":[{"role":"user","content":"ping"}]}' \
| grep -q '"choices"' \
&& curl -fsS --max-time 10 --retry 3 https://cronalert.com/api/heartbeat/<prompt-token>Run it every 5 minutes with the heartbeat's expected interval at 5 minutes and a grace of 5. The --max-time is your latency bound: a gateway that answers in 45 seconds is down for anyone waiting on it, and this is the only place in the stack where that gets measured. One token every five minutes is a few hundred tokens a day. If you route to several providers, run one line per model you care about, each to its own heartbeat, so the alert names the provider. Monitoring the inference server covers the same heartbeat against Ollama, vLLM, and llama.cpp directly, and the readiness traps on each; keep those checks too, so an alert here with the server's checks green points at the gateway rather than the GPU.
Layer 5: the certificate and the proxy in front
Gateways usually sit behind a reverse proxy with a certificate renewed by something other than the gateway, and OpenAI-compatible clients fail hard on an expired one with errors that mention SSL and not the gateway. Enable SSL monitoring on both hostnames; it is on every plan and alerts days ahead. SSL certificate monitoring covers thresholds, and monitoring the reverse proxy covers telling a dead proxy from a dead gateway. A gateway on a bare IP or a LAN address cannot be reached by public HTTP checks at all; give it a hostname, or rely on the outbound heartbeat in layer 4, or cover a bare public IP with a TCP monitor on its port.
Layer 6: the backups ran
Open WebUI keeps every user, chat, prompt, and setting in webui.db (SQLite by default) or a Postgres database you point it at. LiteLLM's Postgres holds every virtual key, team, and spend record. Both are state you will miss badly. Back them up with a heartbeat on the end of the job:
0 3 * * * sqlite3 /var/lib/open-webui/webui.db ".backup /backups/webui-$(date +\%F).db" \
&& pg_dump "$LITELLM_DATABASE_URL" | gzip > /backups/litellm-$(date +\%F).sql.gz \
&& curl -fsS --max-time 10 --retry 3 https://cronalert.com/api/heartbeat/<backup-token>Use SQLite's .backup rather than copying the file while Open WebUI is writing to it. Set a 24-hour interval and a 2-hour grace. Monitoring scheduled backups goes deeper on verifying what was written, and if the containers run on a single box, monitoring Docker and self-hosted apps adds the host layer.
Reading the alerts
| Alert | What broke | First thing to check |
|---|---|---|
| Liveness down | LiteLLM process, container, or the proxy in front | Container status; proxy logs |
| Liveness up, readiness 503 | LiteLLM cannot reach Postgres | The database; connection string; disk |
Readiness up, /v1/models 401 | Key auth broken, or the monitoring key revoked | LiteLLM logs; the key's status |
/v1/models 200 without the keyword | Config reload dropped a model | config.yaml and the last restart |
Open WebUI /health down | UI container or its database | Container; free space under webui.db |
Open WebUI /api/models missing a model | A connection's key or host | Admin → Connections |
| Prompt heartbeat overdue, everything else green | Provider key, model unloaded, GPU host, or latency over the bound | The script's last output; the inference server's own checks |
| Backup heartbeat overdue | Backup didn't run or failed | Cron log; backup target free space |
Alert channels in CronAlert are team-wide. If the platform team wants pages for the gateway and the people using Open WebUI want a Slack notice, put the gateway monitors in their own team; alert routing with teams explains the pattern.
Frequently asked questions
Can CronAlert send the prompt itself, so I skip the script?
No. HTTP monitors send a method, headers, and an expected status with one optional body assertion; they do not send a request body, and a chat completion needs one. The heartbeat script is the honest version, and it also gives you the --max-time latency bound that a GET check could not.
Our gateway is only on the internal network.
Run the prompt heartbeat and the backup heartbeat from inside that network; both are outbound and need no inbound access. For the HTTP checks, publish the gateway through a Cloudflare Tunnel, optionally with an Access service token in the monitors' headers.
What about other gateways: Portkey, Kong AI, or a custom proxy?
The shape is identical. Find the unauthenticated liveness route, find the authenticated no-cost route (usually the model list), add the prompt heartbeat, and monitor TLS. The names change; the layers do not.
Should the prompt heartbeat hit every model?
Hit every model whose outage you would act on, each with its own heartbeat, so the alert names the model. For a fleet of fallbacks you would not page for, one heartbeat on the primary is enough.
Which plan do I need?
The liveness, readiness, /v1/models, Open WebUI /health, and /api/models monitors are HTTP checks with custom headers and an expected status, on every plan at 3-minute intervals on Free and 1-minute on Pro, and SSL expiry monitoring is free. Keyword assertions and the prompt and backup heartbeats are Pro ($5/mo, or $4/mo billed annually).
Monitor the answer, not the port
A gateway that is up is the least informative fact about your LLM stack. Whether a key resolves, whether the UI can see a model, and whether a prompt comes back inside your latency bound are the facts that decide if anyone can work today, and each is one small monitor. Create a free account, add the liveness, readiness, and /health checks, and upgrade to Pro for the prompt heartbeat before the next silent key rotation. Related reading: monitoring AI APIs for the upstream providers, third-party dependency monitoring, monitoring MCP servers, monitoring internal tools, and choosing between HTTP, TCP, and heartbeat monitors.