Your backup region is drifting right now. Here's the checklist that catches it.

Everything drifts. Test like the ground is moving, because it is.

At Silfra in Iceland you can float between two continental plates. They pull apart about 2 cm a year. You can't see it, but it never stops. Iceland expects earthquakes, so it builds for them.

Your backup region drifts the same way: config, versions, replication lag. So do AI agents, after a model update or a prompt tweak. This checklist catches the drift before an outage does.

A. Catch drift in your DR region

• Config diff: settings, secrets, feature flags and IAM roles match primary. Diff on a schedule and alert on any difference.
• Version diff: same images, schemas and dependencies deployed in both places.
• Capacity: quotas raised and instances warm in the DR region, so scaling doesn't fail the first time you need it.
• Replication lag: measured and alerted. You know your worst realistic lag and what you'd lose (RPO).
• DNS and TTL: TTLs low enough to move traffic in minutes; health checks test what users do, not just "port is open".
• Runbook: one page (who decides, exact steps, how to confirm, how to fail back), stored somewhere that survives the outage.

B. Catch drift in your AI agents

• Pin model versions (no silent "latest"). Record the model and prompt version with every answer.
• Eval suite on every change: any model, prompt, tool or retrieval change runs a fixed set of known-answer questions before it ships.
• Per-conversation tracing: you can see which model and path answered, and compare with last week.
• Fallback model or provider configured and eval-tested, not just listed.
• Timeouts and a step budget on every model and tool call, so it can't hang or loop.
• Degraded mode: when the model or a tool is down, answer "I can't verify that right now" or serve a cached answer. Never guess.
• Human approval for anything that writes, sends or deletes.

C. Chaos on purpose (the fix isn't hope)

• Pick a quiet window, a small blast radius and a stop button.
• State a hypothesis: "If primary is lost, users see errors for under N minutes" or "If the model changes, quality stays above our eval bar."
• Run the failover drill and the agent evals: break one dependency, follow the runbook, time it, note what surprised you.
• Fix the top three findings. Repeat every quarter, and after any big change.

Which of your backups have you never tested? Start there.

Generic good practice only. All opinions are my own.