Escalation & Approval

Who gets woken when nobody answers, and what a machine is not allowed to do without a human saying yes
At L1

18

first responder
At L2

5

escalated once
At L3

2

escalated twice
Ack overdue

4

past window

Triage queue

L1 → L2 → L3: each tier gets the routing rule's window to acknowledge before the next one is paged

AlertTierTeamOwnerWaiting / windowStatus
p99 latency spike on /v1/checkout

L3 · Infrastructure Team

L3Infrastructure Teamunassigned48m / 15mOverdue
Glue job customer_sync failed

L2 · Data Eng

L2Data Engunassigned22m / 15mOverdue
ChatGPT spend threshold

L1 · FinOps

L1FinOpsFinOps owner8m / 30mOpen
Apigee rate-limit storm

L1 · API Platform

L1API PlatformAPI on-call12m / 30mAcknowledged

No matching results

View allView less

Alert notifications

What was sent, to which channels, and why

AlertTierChannelsRecipientReasonSent
p99 latency spike on /v1/checkout

alert.escalated

L3PagerDuty, SMSInfrastructure TeamNo acknowledgement in 15m — escalated to L33m ago
Glue job customer_sync failed

alert.escalated

L2Slack, EmailData EngNo acknowledgement in 15m — escalated to L222m ago
EC2 cost anomaly

alert.resolved

L1SlackCloud OpsResolved: reserved-instance drift, rightsized1h ago

No matching results

View allView less

Pending approvals

Actions the automation engine has proposed and is holding until a human decides

Proposed actionRequested byConfidenceImpactWaitingPriority
Restart user-service in production

Production Restart Approval

Automation engine92%user-service, auth-service · ~3m downtime5mHigh
Scale orders connection pool 450 → 600

Database Performance Approval

Database monitor87%orders-db · no downtime12mMedium
Scale checkout-api to 6 replicas

Capacity Approval

Predictive scaler81%checkout-api · +$34/day21mMedium
Rotate the expiring Databricks PAT

Security Policy Changes

Credential monitor99%Data Observability, ETL collection2hLow

No matching results

View allView less

Escalation paths

Each level is given its timeout to acknowledge before the next one is paged

PathTriggerLevelsSuccessStatus
Critical Infrastructure

On-call → Team lead → Eng manager → VP

severity=critical AND category=infrastructure4 · 15m/30m/60m/120m94%Active
Security Incident

Analyst → Security lead → CISO → Exec

category=security4 · 10m/20m/30m/60m98%Active
Database Performance

DBA → Senior DBA → Architect

source=database AND response_time>5s3 · 20m/45m/90m91%Active
ETL Freshness

Data engineer → Platform lead

category=etl AND freshness_breached2 · 30m/90m88%Active

The routing rule a path belongs to carries its first timeout —Alert Managementis where that number is set.

Approval workflows

What automation may not do on its own

WorkflowApproversTimeoutPending
Production Restart

Emergency override allowed

Team lead (required), Eng manager30m3
Database Schema Changes

No override

DBA, Architect, Eng manager — all required60m1
Security Policy Changes

Emergency override allowed

Security lead, CISO — both required45m0
Cost-increasing scale-up

Auto-approves under $50/day

FinOps owner120m0

Escalation analytics

How far up the chain incidents actually travelled, last 5 days

DayLevel 1Level 2Level 3Level 4Escaped L1
Mon45123127%
Tue52154029%
Wed3882121%
Thu41113027%
Fri47132028%

An incident escaping level 1 usually means the first responder was already busy, not that the alert was hard. That ratio is the one worth watching.

LoveHeartBeat - A Product from RootVyana.

v1.24.0

Theme Customizer

Explore different styles according to your preferences

Color Scheme
RTL

Change text direction

Support Chat

Toggle support chat

Navigation Type
Vertical Navbar Appearance
Horizontal Navbar Shape
Horizontal Navbar Appearance
Purchase template
customize