Splunk On-Call, the product formerly known as VictorOps, is one of the standard incident-management platforms for teams that need real on-call rotations, escalation policies, and a shared timeline view of who did what during an incident. Like every incident-management tool, it doesn't monitor anything itself — it routes alerts that monitoring tools generate. Wiring up a real uptime monitor is the part that makes the whole thing useful.

This post walks through connecting CronAlert to Splunk On-Call end to end: creating the REST integration endpoint, adding the webhook channel in CronAlert, mapping the payload so the timeline view shows meaningful information, deduplicating repeated alerts so a flapping monitor doesn't page the on-call engineer ten times in a row, and routing different monitor types to different teams.

Why route uptime alerts through Splunk On-Call

CronAlert can deliver alerts directly to email, Slack, Microsoft Teams, Discord, webhook, and push channels. For a solo developer or a small team, that's enough. Splunk On-Call earns its slot when one or more of these become true:

  • You run a 24/7 on-call rotation. Splunk On-Call manages the schedule, time-zone math, handoff windows, holiday overrides, and PTO. Encoding any of that in CronAlert routing is technically possible and operationally miserable; an incident-management tool exists exactly to take it off the monitoring tool's plate.
  • You need escalation. Primary doesn't acknowledge in five minutes, page secondary. Neither acknowledges in fifteen, page the manager. CronAlert can fan out to multiple channels but can't enforce acknowledgment-based escalation — that's the job of the incident-management tool.
  • You want a shared timeline. Splunk On-Call's defining feature is the chronological timeline that merges alerts, chat messages, and incident actions across teams. After a real incident this is what makes postmortems writable instead of guessable.
  • You're already a Splunk customer. The integration ergonomics with the rest of the Splunk stack (Observability Cloud, ITSI, Enterprise Security) are tight; alerts from logs, metrics, and uptime can all land in the same on-call rotation.

If none of those apply, send alerts straight to the chat tool the team is already in and skip the indirection. Slack downtime alerts or Microsoft Teams alerts are a faster path for small teams without on-call rotations.

Step 1: Create a Splunk On-Call REST integration

Splunk On-Call inbound integrations are called "REST endpoints" in the UI. Each REST endpoint has a unique URL that contains a routing key, and every alert posted to that URL is matched to the escalation policy associated with that routing key.

  1. Open Splunk On-Call and go to Integrations3rd Party Integrations.
  2. Find REST Generic in the list and click Enable Integration. If it's already enabled, click it to view the existing URL.
  3. Copy the Service API Endpoint URL. It looks like https://alert.victorops.com/integrations/generic/20131114/alert/<api-key>/$routing_key. The $routing_key placeholder at the end is intentional — you'll replace it with the actual routing key when you create the webhook channel.
  4. Open SettingsRouting Keys and create a routing key for the team or escalation policy that should receive CronAlert alerts. Name it something descriptive like cronalert-prod or cronalert-frontend.
  5. Map the routing key to the appropriate escalation policy. Without this mapping the alert will land in the "default" team, which is rarely what you want past the testing phase.
  6. Save. Construct the final URL by replacing $routing_key in the Service API Endpoint with the routing key name you just created.

A final URL looks like https://alert.victorops.com/integrations/generic/20131114/alert/abc123-def456/cronalert-prod. Treat it as a secret — anyone with the URL can post alerts into your incident management.

Step 2: Add the webhook channel in CronAlert

CronAlert posts Splunk On-Call alerts through its generic webhook alert channel. The webhook channel sends a JSON payload to the URL you configure, and the Splunk On-Call REST endpoint accepts it directly.

  1. Open CronAlert and go to SettingsAlert Channels.
  2. Click Add channel and pick Webhook.
  3. Name the channel something the team will recognize, like Splunk On-Call (production) or Splunk On-Call (frontend team).
  4. Paste the full Service API Endpoint URL (with the routing key in place) into the URL field.
  5. Leave the Method as POST and the Content Type as application/json.
  6. Save the channel.

Splunk On-Call's REST endpoint expects a specific JSON shape: a message_type field that's one of CRITICAL, WARNING, ACKNOWLEDGEMENT, INFO, or RECOVERY, plus an entity_id for deduplication. CronAlert's webhook payload uses exactly those field names by default, so no custom payload template is required for a working integration. If you want to enrich the payload with the monitor URL, last response code, or the failing region, configure those as additional fields in the channel settings.

Step 3: Attach the channel to monitors

Adding a channel doesn't route any alerts on its own — each monitor decides which channels it uses.

  1. Open the monitor you want to page on, click Edit, and find the Alert channels section.
  2. Tick the Splunk On-Call channel you just created. You can also leave existing channels (email, Slack) ticked — Splunk On-Call handles the paging, the chat channels handle the situational awareness.
  3. Save the monitor.

For bulk attachment across many monitors, use the API endpoint PATCH /api/monitors/:id to add the channel ID to each monitor's alertChannelIds array, or do it in the UI in batches from the monitors list.

Step 4: Test the flow

Testing happens in two steps. First, post a payload directly to the REST endpoint to confirm Splunk On-Call is configured correctly:

curl -X POST \
  -H 'Content-Type: application/json' \
  -d '{
    "message_type": "CRITICAL",
    "entity_id": "cronalert-test-001",
    "state_message": "Test alert from CronAlert setup",
    "monitoring_tool": "CronAlert"
  }' \
  'https://alert.victorops.com/integrations/generic/20131114/alert/<api-key>/cronalert-prod'

Within a few seconds you should see the incident open on the Splunk On-Call timeline, paging whoever's on the rotation. Resolve it by re-posting with message_type: RECOVERY and the same entity_id.

Second, fire a real CronAlert alert through the channel. The cleanest way: temporarily change one of your monitors to point at https://httpstat.us/500, wait for it to fail, and confirm the alert reaches Splunk On-Call. Switch the URL back to normal and confirm the recovery alert closes the incident. Don't skip this step — the most common failure mode for a webhook integration is "the test curl works but the production payload from the monitoring tool doesn't match the expected schema."

Deduplication and recovery

Without deduplication, a flapping monitor would page the on-call engineer every time the status flipped. The fix is built into Splunk On-Call: incidents with the same entity_id are merged, with subsequent payloads appearing as timeline updates instead of new pages.

CronAlert sets entity_id to the monitor ID for every alert. This means:

  • Multiple consecutive "down" alerts for the same monitor produce one open Splunk On-Call incident with N timeline updates.
  • The recovery alert with message_type: RECOVERY and the same entity_id automatically resolves the incident.
  • A monitor that flaps once-per-minute generates one incident, not sixty.

Pair this with CronAlert's consecutive-check verification and multi-region quorum and you have three independent layers of false-positive defense before anyone gets paged.

Routing different monitors to different teams

A single CronAlert account can route monitors to multiple Splunk On-Call teams or escalation policies by using one routing key per destination:

  1. Create one routing key in Splunk On-Call per team (cronalert-frontend, cronalert-platform, cronalert-billing).
  2. Map each routing key to the appropriate escalation policy.
  3. In CronAlert, create one webhook alert channel per routing key, each with its own URL (only the trailing routing key differs).
  4. Attach the appropriate channel to each monitor — frontend monitors get the frontend channel, payments monitors get the billing channel, infrastructure health endpoints get the platform channel.

The result is that one monitoring account paginates correctly across the whole organization without the on-call engineers having to filter alerts that aren't theirs.

Priority and severity mapping

Splunk On-Call uses message_type to decide whether to page. CRITICAL pages; WARNING does not (by default); INFO posts to the timeline silently. CronAlert sends CRITICAL for all monitor down events by default, which is usually what you want for production monitors.

For staging and internal-tool monitors that shouldn't page anyone at 3am, change the channel's outgoing message_type to WARNING. The alert still lands on the timeline so it's visible during business hours, but it doesn't trigger the escalation policy. This is the cleanest way to distinguish "this matters now" from "someone should look at this Monday."

Common pitfalls

Routing key not mapped to an escalation policy

Alerts land in the default team and either nobody is on call for it, or the wrong people get paged. Map every routing key to a real escalation policy before going live.

Hard-coded routing key with $routing_key still in the URL

The URL Splunk On-Call shows in the integration UI contains $routing_key as a literal placeholder. If you paste it into CronAlert without replacing it, every alert posts to a routing key literally named $routing_key, which doesn't exist, and the alert silently goes nowhere. Always replace the placeholder before saving.

Forgetting to send recovery

If your webhook only fires on down events and not on recovery, Splunk On-Call incidents stay open until manually resolved. CronAlert sends both states automatically — make sure the channel is attached to both the "down" and "up" notifications on each monitor.

One routing key for the whole company

Works fine when there's one on-call rotation. Becomes painful when team A's payment monitor pages team B's mobile engineers. Split routing keys earlier than feels necessary; it costs nothing and the alternative is the wrong team waking up.

Frequently asked questions

Is Splunk On-Call the same as VictorOps?

Yes — Splunk acquired VictorOps in 2018 and renamed it. The URLs at alert.victorops.com still work, and the product is recognizably the same.

Does CronAlert have a native Splunk On-Call integration?

The generic webhook channel covers it cleanly with the default payload. No separate integration type is necessary, which means the same setup also works with Opsgenie, PagerDuty events API, FireHydrant, and incident.io.

How do I deduplicate repeated alerts?

Splunk On-Call deduplicates by entity_id. CronAlert sends the monitor ID as the entity_id, so repeated alerts from the same monitor produce one incident with timeline updates, not multiple pages.

Can different monitors page different teams?

Yes — use one routing key per team, one webhook channel per routing key, and attach the right channel to each monitor.

How does it compare to PagerDuty or Opsgenie?

Feature-comparable for paging. Splunk On-Call's distinctive feature is the timeline view; the integration shape is the same. See PagerDuty alerts and Opsgenie alerts for the equivalent setups.

Get started

Wiring CronAlert to Splunk On-Call takes about ten minutes once you have a routing key. Create a free CronAlert account, add a webhook channel pointed at your Splunk On-Call REST endpoint, attach it to your production monitors, and run the test curl. The next time something goes down, the right person gets paged, the timeline tells the story, and the recovery alert closes the incident automatically.

Related reading: PagerDuty uptime alerts, Opsgenie uptime alerts, Microsoft Teams alerts, reducing false positives, avoiding alert fatigue, and incident response for small teams.