LOOK ACROSS THE INCIDENTS
Different failures.
Shared lessons.
Put cases side by side. Find the common assumptions—and the safeguards that could break the chain.
Loading comparison…
LOOK ACROSS THE INCIDENTS
Put cases side by side. Find the common assumptions—and the safeguards that could break the chain.
Loading comparison…
SHARED FAILURE PATTERNS
These editorial tags are common to every selected case. Compare the details below before drawing parallels.
On smaller screens, swipe the comparison horizontally.
| Case file | Knight Capital ↗August 1, 2012 | Cloudflare ↗July 2, 2019 |
|---|---|---|
| Failure pattern | Unsafe changes · Hidden assumptions | Unsafe changes · Shared dependencies |
| What happened | Knight Capital’s order router sent more than four million orders while attempting to fill just 212 customer orders. In the first 45 minutes of trading, the firm accumulated unwanted positions and lost more than $460 million. | A new web application firewall rule exhausted the CPUs serving HTTP and HTTPS traffic across Cloudflare’s network. Visitors to affected sites saw errors. |
| Why it spread | An incomplete deployment and the reuse of a flag activated obsolete trading logic. Controls failed to stop the resulting orders. | A regular expression performed excessive backtracking. Performance tests did not catch the behavior, and the rule went out globally without a staged rollout. The outage also complicated access to internal tools. |
| Impact in context | $460M+ — trading loss | 27 minutes — reported service outage |
| Recovery | Knight stopped the problematic trading and unwound positions. The incident exposed gaps in deployment verification and market-access safeguards. | Cloudflare disabled the WAF component to restore traffic, isolated and tested the problematic change, then re-enabled WAF functionality. Its incident report described changes to rollout, testing, and recovery processes. |
| Safeguards to discuss | Verify deployment consistency across every node. Remove dead code before reusing its controls. Enforce independent limits on automated actions. | Test computational cost as well as detection accuracy. Stage configuration changes even when they are not executable code. Rehearse recovery access when your own services are down. |
| Primary source | U.S. SEC · Enforcement release ↗ | Cloudflare · Technical postmortem ↗ |
Take one shared pattern into a rehearsal or review the evidence behind your own safeguards.
Review your safeguards →