Data deluge could translate into an endless quest for capacity

The universe of data, according to a recent report, is expanding and if we're not careful we'll all end up in a data black hole.

This article can also be found in the Premium Editorial Download: Storage magazine: Hypervisor vendors up the ante with new storage features:

The universe of data, according to a recent EMC/IDC report, is expanding due to an endless data deluge. If we're not careful, we'll all end up in a data black hole.

We're doomed. Dead ducks floundering in a sea of data, and we're about to go under for the very last time. And it doesn't look like we're doing anything about it.

I just got a look at some stats from a recent EMC/IDC report entitled, The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things. I've written entire columns shorter than that title, and as interesting as it sounds from the press release, I just might skip the report and wait for the movie.

But this opus did offer up some juicy tidbits, such as how the size of the world's data (aka the digital universe) is doubling every couple of years. The brainiacs behind this report tell us that there are currently 4.4 zettabytes of data in the world. I wish I knew how they figured that out -- nobody contacted me -- but if that's the good news, the bad news is that the number will climb to 44 zettabytes. If we assume the 10x jump wasn't just conjured up because it's a lot more dramatic than say 7x or 8x, then there's one inescapable conclusion: That's a lot of data.

The press release helpfully puts all those zettabytes in perspective by pointing out that a single zettabyte is equal to a trillion gigabytes. But before you fire up your calculator app to figure out how many 16 GB USB thumb drives you're going to need to store just 4.4 zettabytes, check out this factoid courtesy of the EMC/IDC team: "The amount of information in the digital universe would fill a stack of iPad Air tablets reaching 2/3 of the way to the moon." Just two-thirds? It hardly seems worth having all that data.

They point to the Internet of Things (how about that trendy new acronym, IoT?) as one of the key reasons for the data deluge. I suppose that makes sense if we spend the next six or seven years plugging refrigerators and lawnmowers into the Internet. I just hope your company doesn't have too many "things" it wants to suck data out of.

What will we do with all that data? Sure, there's all that big data analytics stuff, but what happens when the data goes beyond big to be mammoth, colossal or gargantuan data? Aside from the likelihood that some people will inevitably be sacrificed to this data monster when their heads explode from trying to remember zettabytes of data (my prediction, not theirs), this report does have an oddly cheerier upside: "In 2013, only 22% of the information in the digital universe was considered useful data, but less than 5% of the useful data was actually analyzed," the press release explained. I can't tell you how relieved that made me feel.

These digital universe predictions are so much fun that it's easy to forget there's a serious side. That side is probably sitting in your data center right now, and it's bulging with terabytes or petabytes of data. It's been relatively easy to keep throwing iron at spiraling capacity as it's been a fairly cheap proposition to buy disks, tape, flash or whatever to store the data -- although something very un-Moore's Law is bound to happen long before most organizations pass from petabyte-land into the ether of exabytes. That familiar Moore's Law will finally veer off in the other direction and suddenly the economies of scale and super-capacity disks will turn into "We're out of capacity … and money!"

When we talk about having enough space, it's not something abstract like those vast black expanses dotted with stars -- the kind of space we're talking about can easily turn into a black hole for data. Our surveys tell us that the average company manages more than 300 TB of active data, with approximately three times that amount stashed away on tape, optical, disk or other media.

But that's only one dimension of the data growth picture. As EMC/IDC point out, it's only going to get worse over time, and time is the element we can't control. We can find more capacity and cram more and more stuff into it, but we can't slow down the clocks or stall time when it comes to backing up all that data. And with only so many hours in a day, how many of those hours do you want to spend just keeping track of what your company has stored on tapes, disks and so on?

Whether the EMC/IDC gang is right about zettabytes and yottabytes springing up, getting snared and saved all around the world doesn't really matter. IDC is probably pretty confident that by 2020 nobody will remember its prediction or everyone on the globe will be too busy buying EMC storage gear to store the 44 zettabytes to pay any attention. EMC and all the other storage vendors must chuckle with glee when they see predictions such as these, but you don't have to buy into the big data/gargantuan data/digital universe game.

No doubt the Internet of Things will grow, but there will be an Internet of Kinda Dumb Things and an Internet of Pretty Interesting Things. If we skip the dumb stuff -- as the EMC/IDC reports suggest we're doing right now -- maybe we can stem the seemingly relentless tide of data that threatens us. You have a choice: You can be selective about what you keep, or you can keep buying more capacity and hope that as the universe of data expands, there will be more hours in the day to manage it all.

About the author:
Rich Castagna is editorial director of TechTarget's Storage Media Group.

This was first published in June 2014

Dig deeper on Enterprise storage, planning and management

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchSolidStateStorage

SearchVirtualStorage

SearchCloudStorage

SearchDisasterRecovery

SearchDataBackup

Close