This morning Plan B failed. I have high speed internet access into my office, but I pay a small monthly fee to keep a pay-per-minute dial-up account just in case my high speed internet provider ever goes offline. So, this morning, when I lost my internet access, I mentally started preparing myself for 56K upload and download speeds. What I had not mentally prepared myself for, was my phone lines also being down. Thankfully, I still had my cell phone and was able to reach the outside world and let some individuals know about my situation.
But, it occurred to me that this was fairly typical of how disasters go. Not that losing internet access or phone service is necessarily a disaster, but disasters are rarely neat and tidy, they never happen when it is convenient and you can generally count on them not to follow the plan you laid out.
In no way am I implying that companies should abandon either their data protection or disaster recovery planning efforts. What I am suggesting is that after you have backed up all your data, laid out your recovery plans and then tested them, introduce some reality back into the situation.
For instance, a concern that one records management provider recently expressed to me is that companies should evaluate their disaster preparedness after they have just finished a disaster recovery exercise. Tapes are out of order, the recovery environment is not properly configured and people are exhausted. How quickly and how well could your company recover in this situation if a disaster happened then?
Another important aspect to include in your plan is to identify someone who knows the plan but is not afraid to think outside the box. I was once in a disaster recovery situation where an entire production database had failed and there was not enough unallocated disk in the free disk pool at that site to recover the database. The plan called for us to recover to another site, but one individual asked “Do we have a SAN?” and “Can we move some allocated but unused disk on another server over to this one?” In both cases the answer was yes, and we were able to recover the application in 2 or 3 hours instead of 8 to 12 hours.
Disaster recovery plans are just that, plans — no more, no less. But like all plans, they were created at a past point in time and may not reflect the current reality. That is where having someone around who can assess the entire situation and not just follow the script becomes imperative if one is to turn the disaster into a recovery.