The new primary storage

A technology borrowed from backup may end up the biggest thing to happen to storage in a long time.

A technology borrowed from backup may end up the biggest thing to happen to storage in a long time.

When the Beatles sang "You say want a revolution" back in 1968, you can be sure they weren't singing about data storage. Changes to storage technologies happen so slowly that it's sometimes hard to recognize them even while they're happening. It's more like evolution, and at a Darwinian pace at that. Storage can be a real snoozer sometimes, so a couple of current developments are notable not only for the changes they're likely to bring, but for the pace of that change. Maybe "revolution" is too strong a word, but sleepy old storage is about to get quite a shakeup.

Solid-state storage clearly ranks as a game-changer that will undoubtedly alter the face of storage. But that story's going to take a little more time to develop. Data deduplication, on the other hand, is poised to rattle some cages right now.

Data deduplication is all the rage for backup. The level of interest in backup dedupe has been pinning the popularity meter for a few years now, even if our research shows that only about a quarter of all storage shops are actually using it. But even with an installed base that might be falling short of the hype around dedupe, the technology obviously has legs and there are a significant number of vendors offering a variety of products. So, the other 75% of data storage shops are bound to come around eventually.

But instead of coming late to the backup dedupe game, those potential dedupe users might skip backup and go directly to primary storage for their initial dedupe fix. I never would have thought that six months or so ago, but there's been so much happening on the data reduction in primary storage (or what I like to call "DRIPS") front that not only does it now loom as a bona fide game-changer for primary storage technology, but it could pick up momentum fast enough to slow down the backup dedupe express.

Of course, dedupe for backup and DRIPS are two entirely different things even if some of the technologies they share are essentially the same. Backup dedupe can help keep backups within their windows, provide faster restores, and cut down tape use and handling, and, in doing so, can save a few bucks by reducing the amount of disk capacity needed for backup data before it ultimately ends up on tape. All good stuff and in some environments the benefits might be considerable, but not all results are dramatic. And a lot of shops apparently aren't yet sold on the savings or don't think their backup operations are in need of such an expensive fix.

But everyone is grappling with primary storage. Our last purchasing survey showed that, on average, companies would add 40 TB of new disk storage this year, with bigger companies looking at adding more than twice that capacity. Even small companies are feeling the strain, looking at 25 TB of new disk in 2010. Cutting back on the amount of primary storage you need to add -- and even cutting back what you're using now -- will put the drama back in the dedupe story and give beleaguered budgets a break. And it will profoundly change primary and near-line storage systems.

The change is happening right now. Credit NetApp for putting DRIPS on the map, even though the firm didn't do all that much to promote it. But what was once a small field of players is now rapidly expanding. DRIPS products are popping up all over: EMC joined the fray by adding compression to its midrange arrays; HP says its new StoreOnce will run on its XP9000 storage systems within a year; and just about every other major storage vendor has added (or announced plans to add) data dedupe, compression, single instancing or some combination of these data-crunching methods to their mainline storage products.

Adding to the intrigue are recent announcements from two companies that know something about data reduction, with new products that have the potential to accelerate the adoption of primary data reduction. Ocarina Networks and Permabit Technology Corp. each essentially sucked the secret sauce out of their appliances and tricked them out with APIs that, according to both vendors, will make it easy for storage system vendors to add data reduction to their existing products . . . with emphasis on "existing." Permabit says it has a significant OEM partner or two in the fold, and Ocarina's relationship with prospective OEM Dell got so chummy that the Round Rock gang announced they're acquiring Ocarina. You can expect more of these experienced dedupe vendors to pursue packaging their code this way, and a lot more interest from the storage system vendors.

In the past, storage array vendors have been loath to integrate technologies that may chip away at their own new disk sales, so it took some time for things like thin provisioning to become widely available options. But DRIPS is different. Maybe it's just the result of a convergence of the constant capacity struggle meeting up with the success of backup dedupe, or a fundamental right of storage managers to get to use their storage arrays as efficiently as possible. Either way it's one of those rare moments when the tech industry is creating something that addresses a real problem, rather than creating solutions in search of problems that probably don't yet exist. Storage managers have gotten a taste of dedupe with backup and they want more.

BIO: Rich Castagna ([email protected]) is editorial director of the Storage Media Group.

* Click here for a sneak peek at what's coming up in the September 2010 issue.

Dig Deeper on Primary storage devices