Marketing firm Blade lost 2G bytes of data when a backup went wrong, but a software update solved the problem.
On December 3, 2001, Blade Marketing Communications Inc.'s vice president Patrick McGovern found another reason to hate Mondays. On that Monday, he discovered that, over the weekend, a hard drive failure had erased 2G bytes of mission critical data. Even worse, the backup copy was gone. Things looked bleak. Could this day and this data be saved?
When McGovern first discovered the hard drive failure, he wasn't too worried. His Toronto, Canada-based advertising agency had been successfully using an online backup technology and service for years. However, at midday he discovered that, due to an oversight on his part, the backup copy had also been deleted. Outdated retention settings hadn't been changed. So, the online backup system did what it was told and erased the data after a certain number of backups.
The loss of 2G bytes of creative, accounting and account service data would be a big blow to the company. "Clients expect things to be available instantly," says McGovern, who also serves as Blade's co-creative director. "They'll say: 'Remember last year we ran the ad with the Christmas trees in it? Let's use that image again.' Then, we have to come up with that image."
Three years ago, when McGovern started with the agency, he faced blank stares when inquiring about the agency's backup. "Things were pretty primitive when I arrived,"
Adding "IT manager" to his job description, McGovern brought in a tape backup solution. For disaster recovery purposes, he stored two sets of backup tapes, one offsite and one onsite. Doing the backups on tape and shipping tapes out for storage "was a pain in the neck," he says. That pain increased when tape drive faults became commonplace.
Looking for alternatives to tape, McGovern did research on online backup. He was nervous about the idea. "We weren't even on the Internet at the time," he says. "It was all hocus pocus to us because it was totally invisible."
Representatives from Walnut Creek, Calif.-based EVault Inc. convinced McGovern to try online backup. "The EVault reps were very good at explaining how it all works," he says. "When they set up a test system, everything went so smoothly that we stopped worrying about it."
EVault backups happen at night. So, the process does not slow down the systems needed during the workday. The EVault VitalVault software, loaded onto Blade's Intel-based server, automatically dials EVault, making the connection to its server.
No one at Blade needs to facilitate backups, which is perfect for McGovern. "I wear a lot of hats here. So, I don't want backups to be a constant concern," he says. "I get a good feeling when I get the e-mail each morning that says backup happened automatically last night."
Initially, Blade installed an ISDN line that was dedicated to backup to EVault. That made things simple because the agency didn't have to worry about firewalls or getting an Internet service provider. Not long afterward, however, the agency began using the original ISDN line for network Internet access. McGovern set up the firewall with the help of an EVault consultant.
Everything went smoothly until the agency switched to an ADSL line for Internet access. That switch proved to be awful for McGovern. "The service providers weren't ready to provide a consistent ADSL service," he says. For eight months, he struggled with the problem of the service dropping out. When the service dropped out in the middle of an EVault backup, the entire backup had to be repeated when the service resumed.
To make sure that complete backups occurred, McGovern set up a schedule of eight backups each weekend. Once later backups were completed, the first backups made on the weekend would be deleted.
Jump forward to the weekend of December 1, 2001. One of Blade's hard drives failed, and 2G bytes of data were lost. That data had been backed up, but as subsequent backups occurred the earlier backup containing the 2G bytes of data from the failed hard drive was deleted.
Ironically, by December 3, 2001 Blade no longer needed to run eight backups each weekend. EVault's release of the VitalVault software in 2001 solved the problems associated with the interruption of a backup due to a lost connection. The new software allows a backup to resume midstream, after an interruption, rather than starting from the beginning.
On the afternoon of December 3, an EVault rep accessed Blade's data in EVault's own backup tapes, which had been archived. The rep hung in until 10 p.m., making sure that a complete restoration was achieved.
"When I came back in on Tuesday, all the data was there," says McGovern. Needless to say, he's changed his retention schedule and reduced the number of weekend backups. With those changes and EVault's support, Black Mondays should be a thing of the past.
For additional information about EVault, visit its Web site.
For more information about Blade Marketing Communications, visit its Web site.
For More Information
>>Ask searchStorage expert Jim Booth a backup/recovery question.
>>SearchStorage.com Best Web Links: Administrative Tools and Strategies >Backup
>>SearchStorage.com Tips: Backup
This was first published in January 2002