Silent faults
“Up” is not the same as “healthy”.
Two recent stories: an upstream link that looked fine, and a modem update that broke some services while speed tests stayed perfect.
An upstream circuit between Halifax and Toronto reported healthy while quietly dropping packets larger than about 1,478 bytes. Small packets got through, so basic checks passed. Larger ones vanished.
After a modem firmware update, Crave, TSN, parts of Facebook and some games stopped working for customers. Speed tests looked perfect, because a speed test is not the same traffic as a streaming app.
In both cases the first real signal was customers calling. The question is whether agents can notice the pattern sooner.
- Link status
- Up
- Speed tests
- Normal
- Customer reality
- Some apps broken
- Design
- Outage Detective