India-built AI & blockchain engineering since 2017 · Serving clients worldwide

blog

How to Build a Disaster Recovery Plan That Actually Works (Not Just One That Looks Good on Paper)

A few months back, one of our fintech clients — an NBFC processing loan disbursals for nearly 40,000 customers a month — had a primary database server go down during a scheduled RBI audit window. Their DR plan, which had been signed off a year earlier, said failover would take “under 15 minutes.” It took four hours. The runbook referenced a server that had been decommissioned eight months prior, and the person who knew the manual override steps had left the company.

Nothing about that story is unusual. At Speqto Technologies, we’ve reviewed dozens of disaster recovery plans for banks, NBFCs, payment aggregators, and insurance platforms — and almost all of them share the same flaw: they were written once, filed away, and never pressure-tested against how the business actually operates today.

Here’s what actually separates a DR plan that works from one that just exists.

1. Start with RTO/RPO numbers tied to money, not guesses

Most plans we inspect have Recovery Time Objective (RTO) and Recovery Point Objective (RPO) figures that were picked because they “sounded reasonable” — not because anyone calculated what an hour of downtime actually costs.

For a UPI-based payments client we worked with, we ran the math: every hour of checkout downtime during peak hours (7–10 PM) cost them roughly ₹18 lakh in lost transaction volume, plus reputational fallout on social media. That number justified investing in a hot standby with a 5-minute RPO for their transaction database, while their internal HR and reporting systems were fine with a 24-hour RPO on cheaper cold backups.

The lesson: not every system deserves the same recovery tier. Tiering your systems by actual business impact — payments and core banking at Tier 1, reporting and internal tools at Tier 3 — saves money and focuses effort where it matters.

2. Write the plan for the 2 AM version of your team

A DR document full of architecture diagrams is useless to an on-call engineer who’s been woken up at 2 AM during a ransomware event. We rewrote a client’s 60-page DR document (a BFSI lending platform) into a one-page incident checklist with exact commands, exact contact numbers, and exact escalation order. Their mean time to initiate failover dropped from 45 minutes to under 10 — not because the infrastructure changed, but because the humans knew exactly what to do without thinking.

Practical rule: if a new team member couldn’t follow your DR runbook without calling someone for clarification, it’s not ready.

3. Account for the scenario where your DR site itself is compromised

Many plans assume the disaster is a server crash or a data center fire. But the disaster we see most often now is ransomware — and ransomware doesn’t respect your DR replication links. We had a case where a client’s secondary site was being replicated in near real-time, which meant the encrypted/corrupted files from the ransomware attack synced straight to the backup within minutes.

What actually works: maintain at least one backup copy that’s air-gapped or has a delay (12–24 hours), specifically to recover from scenarios where the primary is compromised by malware, not just hardware failure. For regulated entities, this is also increasingly an expectation under RBI’s cybersecurity and resilience frameworks, not just a nice-to-have.

4. Test it like it’s real — including the uncomfortable parts

Tabletop exercises are a good start, but they don’t catch the real gaps. One insurance-tech client of ours insisted their quarterly DR drills were “passing” — until we insisted on an unannounced failover test during business hours. It revealed that their DNS TTL settings were misconfigured, adding 20 unplanned minutes to every failover regardless of how fast the infrastructure recovered.

A few things worth testing specifically, not just assuming:

  • Can your team actually reach the DR environment without VPN access that routes through the primary site?
  • Does your payment gateway or third-party API provider know your DR IP ranges, or will they block you as “suspicious traffic”?
  • Do your compliance and audit logs survive the failover, or do they just stop recording?
  • Does someone outside IT — customer support, compliance, leadership — know what to communicate to customers and regulators during the outage?

5. Build in a communication plan before you need one

In BFSI, a technical recovery that works perfectly but comes with silence toward customers and regulators is still a failure. We’ve seen clients lose more trust from a vague, delayed customer email than from the outage itself. Draft your customer notification templates, regulatory disclosure steps, and internal leadership updates in advance — during the incident is not the time to be wordsmithing an apology.

6. Review it every time something changes, not once a year

DR plans go stale the moment you migrate a database, change a vendor, or onboard a new payment processor. We recommend tying DR plan reviews to your change management process — any major infrastructure or vendor change should trigger a DR plan check, not just an annual calendar reminder.

A disaster recovery plan isn’t a compliance artifact to show an auditor. It’s a rehearsed, living response that your team can execute under pressure, with numbers that reflect what downtime actually costs your business. At Speqto Technologies, we’ve found that the plans that hold up aren’t the most detailed ones — they’re the ones that have actually been broken, in a drill, before a real disaster got the chance to break them first.

Share this article

Leave a Reply

Your email address will not be published. Required fields are marked *

Keep reading

Related articles

Have an AI idea or a product to build? Let’s talk.

Tell us what you are building. A solutions architect will respond within one business day with next steps and a no-obligation estimate.

  • Free 30-minute consultation
  • NDA on request, before we talk details
  • Estimate within 3–5 business days
WhatsAppTelegramCall

Tell us what you want to build

A solutions architect replies within one business day. NDA on request.

We reply within one business day. Your details stay confidential.