Backup fears: Success or failure?
Doing the right thing--like recovering backed up data--can get you into hot water if your company isn't on the up and up.
What are you most afraid of? Not being able to successfully recover your data or finding out that you can get it all back--but that you also recovered a smoking gun or two? Depending on your position within the company, you might be afraid of both scenarios.
And therein lies the dilemma. How do we succeed if success on one front means failure on another? You can't. Our motivations for backup and recovery are no longer synchronized throughout the organization. In fact, sometimes they're at direct odds with each other.
You, my storage friend, are trying to do the right thing. You're trying to create process from chaos, master tools that can't be mastered and respond to every request of "Get my data back." You're a hero, despite the absurd amount of capacity creation, the crippled tools and the lack of support. And while your odds of success are about the same as me beating Lance Armstrong in a bike race, you succeed most of the time. You keep us data-creating mortals from destroying the business with our fat fingers and lack of common sense. I'm awed by your abilities. You're the "Rudy" of IT.
A few years ago, I wrote an article in Storage and yelled at you for allowing "management" to run around with ridiculous expectations of what was and wasn't possible today. "It's up to you to correct their wildly wrong assumptions," I wrote, "or else you'll only be the goat, never the hero." I knew that if management realized they couldn't recover everything they needed a huge percentage of the time that smart, ethical business people would recognize this wasn't in their best interests and give you the resources you needed to fix the problem. That almost worked.
But a funny thing happened in the middle of this backup/recovery nightmare: Ignorance became bliss. Companies began getting data requests from outside (often regulatory) entities for legal discovery reasons. We non-morally bankrupt types thought "If they don't deliver, it will cost them big bucks," and it did. What we didn't figure on was that many of the muckety-mucks didn't care--even if the SEC fined them $87 million. That made me wonder. Surely $87 million in fines and the public relations nightmares would be enough to get management to pull their heads out of the sand?
Alas, the answer is "No." Companies that don't mind paying $87 million in fines have much more to hide. The only logical rationale behind not fixing a problem that can be fixed is that you don't want to find out what you have ... at any cost. I hate dirty CEOs and the fraud they commit. We aren't naive enough to think Ken Lay and Bernie Ebbers were the only dirty CEOs, but we don't know the magnitude of the problem because it's only spoken about at the highest levels. There are tons of businesses with "questionable" or illegal data sitting on production machines, in databases, on backup tapes and on the boss' laptop. The question is: Will we ever do anything about it?
So, while reality is a bummer, I'm also a perpetual optimist. I know that the house of cards built on the immoral behavior of some people will come crashing down. It won't happen fast enough, but it will happen. When it does, the guy who can find the data will be the big fish, and the muckety-mucks will leave in disgrace. Telling Congress "It can't be done" will eventually be unacceptable. It can be done--all of it. So you keep doing what you're doing, Rudy, and keep that white hat handy.