Alert Fatigue: How to Cut Your Page Count
Alert fatigue is a volume problem with a number attached. Count pages per responder per week, then use four levers to cut the ones nobody acts on.
Alert fatigue is a volume problem, so start by measuring the volume. Count how many pages one person received last week, then count how many of them changed what that person did. The gap between those two numbers is the thing you are fixing, and almost nobody writes it down before they start tuning thresholds.
Count your pages before you tune anything
Every alert fatigue project should open with a spreadsheet. Pull the last four weeks of alerts into three columns: what fired, who it woke, and what they did about it. Total the pages per responder per week. That single number tells you whether you have a tuning problem or a design problem.
| Pages per responder per week | What it usually means | Where to start |
|---|---|---|
| 0 to 2 | Healthy, assuming coverage is real | Check for gaps, not noise |
| 3 to 8 | Tolerable, drifting | Trim the bottom third by actionability |
| 9 to 20 | People are muting things | Confirmation and routing, in that order |
| Over 20 | The pager is decorative | Stop paging on anything but confirmed outages |
The term came from hospitals rather than from operations. Wikipedia's entry on alarm fatigue describes clinicians becoming desensitised to safety alarms and missing the real ones, and the mechanism transfers exactly. A responder who has dismissed nine false pages this week will dismiss the tenth without reading it.
What causes alert fatigue on an on-call rotation?
Four causes account for most of the volume, and only one of them is a threshold. Single-probe false positives fire on network blips rather than outages. Duplicate alerts describe one failure from three tools. Non-actionable alerts report facts nobody can do anything about at 3am. Severity flattening sends everything to one channel at one urgency. Alert fatigue is what those four produce together.
IBM's write-up attributes the problem to infrastructure design, tool fragmentation and unrefined thresholds together, which matches what an audit usually turns up. Atlassian's on-call guide adds the consequence that matters commercially: slower response to the alerts that were real.
Alert noise is not evenly distributed either. In most rotations a handful of monitors generate the majority of pages, which is good news, because the fix is short.
The actionability test
Apply one question to every alert in your list. If this fires at 3am, is there something a human must do in the next fifteen minutes? If the answer is no, it is not a page. Demote it to a ticket, a daily digest or a dashboard tile.
That test kills more alerts than any threshold change. Disk at 70 percent is not a page, because nothing has broken and nobody needs to act tonight. Certificate expiring in 21 days is not a page. A checkout endpoint returning 503 from two regions is a page, because customers are failing right now and somebody has to choose a mitigation.
Keep the demoted alerts. Deleting them loses signal you will want during an incident. Route them somewhere that does not vibrate.
Four levers that actually cut the count
Tuning a threshold is the least effective of the four. Start with confirmation, because it removes the largest category of false pages without reducing coverage at all.
Confirmation across regions. One probe failing is a blip. Requiring a second region to agree before the state changes removes single-path network noise, which is the mechanism described in cross-region quorum confirmation. The state machine runs UP to SUSPECT to DOWN to RECOVERING. A probe error never counts as a down vote, so an infrastructure hiccup on our side cannot page you.
Consecutive failure thresholds. A single slow response is weather. Two or three in a row is a trend. In our testing on a small European API, requiring two consecutive failed checks from two regions cut weekly pages from eleven to three. The one real outage in that period still alerted within ninety seconds.
Severity mapped to channel. Not everything deserves the same delivery. There are 16 alert channel types, which is enough to give each severity a different level of intrusion.
| Severity | Example | Channel |
|---|---|---|
| Page | Customer-facing endpoint down, two regions | SMS or voice, then chat |
| Notify | Response time above budget for 15 minutes | Team chat only |
| Record | Certificate expires in 21 days | Email digest |
Scheduled silence. Deploys, migrations and nightly batch windows produce predictable failures. Schedule them as maintenance windows so the alert never fires, rather than muting the monitor and forgetting to unmute it. The mechanics are in maintenance windows.
Running an alert audit without a project plan
An hour a quarter is enough. Treat it as an alert fatigue review rather than a project. Sort the last ninety days of alerts by monitor, descending by count. Take the top five, because they will be most of your volume.
For each one, answer three questions in writing. Did every firing correspond to a real user-visible failure? If not, what did the probe see that a user would not? Would confirmation, a consecutive-failure threshold or a channel change have removed the false ones?
Then do the unpopular half. Look for outages that nobody was paged for, because alert fatigue also hides gaps. A team drowning in disk-space warnings usually has no check on the one endpoint that takes payments. Coverage and noise are the same audit, and fixing only the noise half leaves you quiet and blind.
Write the decisions down next to the monitor. Six months later nobody remembers why the threshold is 800 milliseconds, and an undocumented threshold gets widened by the next person who gets woken up.
What tuning cannot fix
Two limits are worth naming before you spend a quarter on this.
The first is human scheduling. An external monitoring service can decide which channel receives an alert and whether a state change is confirmed. The rota, the escalation ladder and a person's quiet hours live in a paging tool. Route into that tool rather than trying to reproduce it.
The second is depth. An outside-in check sees status, latency, content and certificates. Queue depth and heap pressure are only visible from inside the process. Alerting on those needs something running in-process, which is the saturation gap in the four golden signals.
Why a quiet rotation is worth money
Alert fatigue is usually argued on wellbeing grounds, which is true and also the weaker case. The stronger one is detection time. A responder who trusts the pager reads it immediately, and a responder who does not reads it after the second buzz, or after the customer email.
That delay lands directly in your mean time to recovery. Every minute spent deciding whether an alert is real is a minute added to the incident, and it compounds because the same hesitation appears at every handoff. Teams that cut page volume usually see recovery times fall without changing anything about how they fix problems.
There is a hiring argument too. A rotation with twenty pages a week is one people leave, and the person who leaves is the one who knew which alerts were fake. That knowledge is not written down anywhere, which is why the next rotation starts by widening thresholds again.
Common mistakes
Muting instead of demoting. A muted monitor is a deleted monitor with extra steps, because nobody ever unmutes it. Change the channel, keep the check.
Widening thresholds after every false page. Two rounds of that and the threshold no longer detects anything. Fix the confirmation logic instead of the number.
Paging on symptoms and causes at once. If the database alert and the checkout alert both fire, you have two pages for one incident. Page on the customer-facing one and let the rest inform.
Alert fatigue is fixable in a single afternoon of counting followed by four configuration changes, and the counting is the part people skip. Get the weekly page number per responder, apply the actionability test to every alert on the list, then add confirmation before you touch a threshold. Cross-region confirmation is included on the free plan, which covers 10 monitors at a 1-minute check interval as of September 2026.
FAQ
It describes responders becoming desensitised to alerts because too many of them arrive and too few require action. The term came from hospital alarm systems, where clinicians started missing real alarms among the false ones. On an engineering rotation the same pattern shows up as muted channels and slower response to the pages that mattered.
Count pages per responder per week first, then apply one test to every alert: if it fires at 3am, must a human act within fifteen minutes? Demote everything that fails the test to a ticket or digest. After that, require a second probe region to confirm a failure before the state changes, and route each severity to a different channel.
A useful split for on-call work is page, notify and record. A page wakes someone because customers are failing now. A notify goes to team chat for something that needs attention during the day. A record lands in an email digest or a dashboard, such as a certificate expiring in three weeks. Same data, three levels of intrusion.
Three to eight per responder per week is tolerable but drifting. Above nine, people begin muting channels, and above twenty the pager has stopped working as a signal. Track the number per person rather than per team, because one unlucky rotation can absorb most of the volume while the team average looks fine.
Only if you reduce it by widening thresholds. Requiring confirmation from a second probe region removes network blips without reducing coverage, since a genuine outage fails everywhere. The real risk runs the other way: a noisy rotation hides gaps, so audit for outages nobody was paged for at the same time you trim the false ones.
Related reading
- Cross-Region Quorum: How Confirmation Stops False Alerts - Cross-region quorum means a target is only called down once probes in more than one region agree. Here is the voting rule, its cost and its limits.
- Maintenance Window: Plan One Without False Alerts - A maintenance window is announced downtime you control. Here is how to size it, suppress the right alerts, and keep the data you will need later.
- Golden Signals: The Four SRE Metrics That Matter - Golden signals are latency, traffic, errors and saturation - the four metrics Google's SRE book says to watch when you can only watch four.