News Stay informed about the latest enterprise technology news and product updates.

Recovering data from a crashed drive using VMware

I was talking with a friend the other day about the prospect of multi-terabyte hard drives and how painful it would be to lose that much data. My friend — being my friend of course — countered that it’s not the amount of data, but where it resides and what the data is that’s important.

For instance, he went on, the EEPROM on your desktop motherboard isn’t more than 2MB worth of data. Yet without it the bazillion hours of work you have stored on your desktop hard drive, while safe and sound, is still useless to you because you can’t access it because your computer won’t boot.

After conceding the point, I rephrased the statement to emphasize the loss of multiple terabytes of data residing on a platter-based spinning medium, located in a computer or computer-like device providing data storage services to said computer, group of computers, or computer-like devices (whew!).

Without blinking an eye, he said he’d started a hard drive data recovery company. He built a clean room and had been perfecting his recovery skills on hard drives purchased on, all of places, eBay. As an aside, use a hammer and nail, or Sawzall, to properly delete all data from unwanted hard drives you dispose of.

A while back, I got a frantic call from a family member whose laptop hard drive had crashed. She was beside herself because on her hard drive were all the digital photos she’d ever taken. . .ALL of them. She’d meant to back up her stuff to a disk but never got around to it. She wanted to know was there anything I could do to help her.

That is when it hit me full force, I have brilliant and baleful friends.

My friend recovered almost all the data from her hard drive for me (at a very reasonable price) and now she has the first pictures of her child, some of her wedding photos and other very important moments in her life back, and on DVD this time. The whole saga got me thinking: Am I really protected from a hard drive crash? How about the executives I support? What would I do if my array at home failed where I have all of my photos!

Seeing the look on my relative’s face when I presented her with all of her photos was priceless. But it got me thinking about all the other people out there in the SMB world with the 0.5 person IT shop who don’t even know these services exist, much less who can afford the super-high cost of traditional data recovery. I don’t think today’s data protection schemes are going to be able to handle the eventuality of these super-sized drives making their way to the same SMB shops.

Do the math. A decent 100Mb pipe can push about 3TB an hour (this takes into account -25% for packet and transmission overhead). If you had three people with a terabyte drive, you’d saturate a 100MB uplink should they decide to back up to a device on the network. How are we going to back that up? The storage SaaS startups making their way to market aren’t going to be able to keep up either. Imagine backing up 400-700GB over your home Internet link where your upstream bandwidth is only 768Kbps.

I saw this coming a bit back when I got my grubby hands on the Hitachi Terabyte drive and have begun using a combination of VMware Player and VMware Workstation to mitigate my issues with capacious storage at home. I essentially virtualize the machine I want to use and deploy that on top of a generic OS install, replete with a pretty icon (in my case, Debian Linux), instructing the user to launch the player as their “desktop.” I’ll eventually get to a point where I will move upward from Player to Workstation for all my machines (right now cost is limiting me to using player for most of my machines), then run snapshots and back up the snaps to the same location as the original VMDK using RSync.

It sounds like a lot of work, but try explaining to your wife that she’s lost all her projects she’s been working on and you don’t have a recent backup because her drive is too big to back up quickly. You’ll appreciate the effort that much more when you can say, “I’ve got you covered, hon!!”

Here’s the visual I use when I explain this concept.

1) Fold a piece of paper four times (or use a folded napkin)

1a) Imagine the paper (napkin) as your physical hard drive

2) Tear off two or three 1-inch pieces of that napkin. Put them on the table next to the napkin.

2a) Imagine those pieces as virtual hard drives or volumes.

3) Reorder those 1-inch pieces of the napkin. Easy, isn’t it?

4) Peel apart the layers of those 1-inch pieces, 4x as much stuff to manipulate, making it take a little longer to move things around the table, no?

4a) Imagine those layers as individual files.

Take this one step further. Blow a soft puff of air at the three 1-inch pieces before you peel them apart (this works best with the napkin as they are slightly “stuck” together). Think of that puff of air as a failure or some sort of issue with storage. Do the same when you’ve peeled apart the pieces.

Now you have a great way to envision how your task of managing individual files (family photos) on a gargantuan hard drive (look how much napkin you have left!!). Multiply that out by a couple of napkins and you see why all of a sudden this problem of failed drives and how to protect against it becomes really hard in the TB-drive world. This can open eyes at the management level. It puts a real and appropriate understanding of why we as storage admins freak out at times when they refuse to allocate budget.

I started out talking about the advent of huge drives and what are you going to do to get the data back should they fail? I’ve developed my own solution to protect myself using some free and not-so-free tools from VMware, but I’m not sure it would scale well, or be easily manageable. Maybe a small challenge to the hardcore virtualizers out there may be in order. . . .

Join the conversation

34 comments

Send me notifications when other members comment.

Please create a username to comment.

Which command-line shell do you use most frequently?
Cancel
Never needed CMD in windows unless I needed to kill a program. In linux I use bash for almost everthing, it is just easier then using manially dealling with the FHS.
Cancel
Bash on linux. I have also used PowerShell in Windows, and I like it. It can really come in handy for some scripting.
Cancel
Of course the ultimate command-line processor is TAKE COMMAND, which has a 64-bit version now.
Cancel
I'm no Unix admin, but doesn't bash have much more going for it than CMD.EXE?
Cancel
Seriously? How can put Bash and CMD in the same camp? Without getting in to the boring details, Bash is incredibly powerful whereas CMD is, quite simply, not. I am not that familiar with Powershell so I am not in a position to critique it but I do know CMD and Bash. When I moved from using the former to the latter, my reaction was something along the lines of "Where have you been all my life!" The strength in the fact that it hooks in with so many vendor independent protocols such as SSH for all our network devices, such as Curl for troubleshooting SSL and TCP connections on HTTPS requests to our servers and ngrep for network connections and tcpdump.... etc, etc, Just means I can do it all from one place.
Cancel
Its cool
Cancel
Really, so we are comparing now, apples to cactus? They live in different ecosystems, have different underlying DNA and completely took a different evolution path. Both should do what they were designed for in their realm, although with convergence it becomes tricky, I ofter find myself frustrated on Windows so I install a bash interpreter and associated tools, I'm sure power-shell is powerful for windows that is, but I don't care to use it for business tasks.
Cancel
Bash is good and powerful.
Cancel
Let's see, what can we do with the Command Line Interface? What can we do in Windows? What can we do in Linux? Automated tasks? Remote tasks? Fully automated robots that execute "stuff" ? NIX has been in that culture since day 1, Windows is starting to learn the power of knowing how to speak other languages other than a GUI and extending that power to the final user. Will the user perform their critical backups this way? Will they ever monitor their systems this way? Will they roll-out software updates this way? If you begin answering yes to any of these, then you are a Windows person, don't worry too much, someone will become smarter than you and write a GUI program to do all of those hard-to-do sysadmin tasks, or as the note suggests, you may dare to become THAT smart guy that embarks in learning what *NIX users have known for quite a bit of time. It's all for your benefit.
Cancel
I want to use PowerShell but we don't have it on the still remaining few thousand windows 2003 servers and the few hundred remaining windows xp systems.
Cancel
Tried PowerShell in v1 days, but it wasn't on older systems, and cert requirements were not worth the effort. Also had concerns MS would abandon it. Now giving v3 a try.
Cancel
I wonder WHAT all those CMD users are actually doing with it, I mean, do they actually MANAGE a server like that? I mean ocasionally going into CMD to query something or copy is not that sofisticated is it? I think the angle of this note is misleading and again they succeed it in making it a "contest", by mentioning bash head to head to CMD and PowerShell. But here is a thought for Windows people, DISREGARD the bash chunk and you'll see clearly that the MOST majority is familiar with CMD not so much to PowerShell, but again, I'll bet that they do substantially different tasks as far as complexity goes. Oh well, another hit-honeypot.
Cancel
we did like to worn on CMD
Cancel
Bash rules, PowerShell sucks
Cancel
Ramping up...will certainly overtake CMD in the next foreseeable future
Cancel
I'm surprised that cmd is as high as it is, I use it more when I have to because of my time in bash. Obviously not as powerful, but better than stuck in GUI
Cancel
Bash is the shell, which is simple for automation.
Cancel
Power shell does have a steep learning curve, but it is definately worth it.
Cancel
bash is better suited for linux, as ksh,sh,csh for other unixes.
Cancel
No puedo creer que comparen powershell o cmd con una shell verdadera como bash,sh,ksh,csh...
Cancel
I would never put an enterprise critical app on any Windows box, and would take actions up to dismissal for anyone who does.
Cancel
I am somewhat surprised that you are not including JPSoft's 4nt, or the newer take command shell here.
Cancel
Mostly use the Unix Bash Sheel and Linux Bash Sheel
Cancel
I'm used to CMD; I often use it on Windows computers (clients and servers) for basic tasks. In my day-to-day administration tasks, I rarely need to use PowerShell (at least, until now). I also rarely work on *NIX systems, so I rarely use Bash.
Cancel
The facilities/possibilisties open to *sh users are almost endless
Cancel
where would VBscript fit into this?
Cancel
i use cmd a lot but am so much enthusiastic on the capabilities of powershell
Cancel
I use vbscript actually.
Cancel
bash is the vendor (protocol) universal option because Microsoft don't make my switches or firewalls.
Cancel
Can Powershell be used on Windows 7 systems - (A) locally and (B) remotely?
Cancel
Powershell has potential, but it has lots of limitations too, mostly dealing with syntax (at least to an ex-programmer-turned-admin). Bash and cmd are both well-trod, so there are tons of actually GOOD examples of how to do things with them. ANY of them are better than VBScript.
Cancel
I'm new to all this. So I decided to go with PowerShell. Too bad all these command line shells are limited to their respective operating systems.
Cancel
I use dos with unix tools. There are not enough useful Powershell examples to tempt me over
Cancel

-ADS BY GOOGLE

SearchDisasterRecovery

SearchDataBackup

SearchConvergedInfrastructure

Close