Decision in brief
Start with separately monitored DNS, edge, origin and authentication paths. Add tested fallback only for the workflows that cannot tolerate vendor recovery time. A fallback needs independently reachable controls and it must preserve the same access and security requirements as the normal path.

Three outages with different symptoms
For November 18, 2025, Cloudflare reported that a database permission change changed the output of a Bot Management configuration generator. Its network propagated oversized configuration and affected proxied traffic and dependent services. The operational lesson is to test boundaries and rollbacks for configuration artifacts. Distinguish when user requests started working again from the later closure of an incident notice.
On December 5, 2025, Cloudflare reported a separate incident involving a configuration change during defensive work in its request-processing path. Cloudflare reports a 25-minute impact to a subset accounting for roughly 28% of its HTTP traffic. Use separate evidence for what failed, which workloads were affected, and what showed that recovery had propagated.
DENIC’s final report on the May 5, 2026 outage documents invalid DNSSEC signatures during a key rollover and access disruption to .de domains for around three hours. An application can be healthy while validating resolvers cannot reach it by name. Add provider dependencies such as authoritative DNS, certificate services and identity to the recovery inventory.
| Symptom | Check from a controlled path | Decision |
|---|---|---|
| Lookup fails while an authenticated origin health check passes | Compare resolution from independent validating resolvers and confirm authoritative status | Escalate the DNS dependency and use only a prepared recovery path |
| Edge requests fail while restricted origin health passes | Compare status, error rate and logs for the approved edge and origin probes | Preserve evidence and rehearse reverting your latest edge change before treating the origin as broken |
| Operator dashboard or authentication fails | Test company-approved emergency access and off-provider monitoring | Use the independent on-call channel and approved operator runbook |
| Both edge and origin fail | Check application logs, capacity and dependency health | Escalate application ownership while recording vendor symptoms separately |
How Hacker News observations change the drill
A_Venom_Roll spotted error messages across unrelated sites and reported that an external site-status checker appeared down as well. This was their observation during the November 18 incident. Use a monitor and alert delivery path that do not rely on the same provider as the service under test.
In the December 5 discussion, jacobgkau described seeing errors before the status page reflected the issue they observed. Use your own time-stamped probes for detection and the provider’s status updates and final report for attribution. Keep both records during the review.
In a third thread, emilburzo shared DENIC’s final report. The linked primary report supplies the details for a different dependency, while the HN submission is a discovery trail.
Inject a mock outage on a staging domain
- Choose a staging hostname and set explicit targets for detection, operator access and user-path recovery. Inventory the dependencies for each and mark those sharing a provider.
- Have one operator simulate a harmless 503 in a staging edge fixture while the restricted origin stays healthy. Ask a second operator to diagnose it from external monitoring, origin probes and a mock vendor status notice.
- As a distinct exercise, simulate a resolver failure against a test zone or local DNS fixture. Revert the fixture and time healthy resolution and application behavior. Test the alert channel with the main dashboard marked unavailable.
- Pass if the diagnoses agree with the injected faults, timing meets the targets and recovery preserves TLS and access policy. Stop if any fallback requires exposing the origin or weakening protections, or if a drill change could leave staging.