This article can also be found in the Premium Editorial Download "Storage magazine: Hot storage technology for 2008."
Download it now to read this article plus other related content.
"Deduplication is the single biggest technology to come along in years," declares GlassHouse Technologies' Preston. Deduplication identifies and removes multiple copies of data that eat up storage capacity and lengthen backup windows.
By reducing the amount of data to store and move around, deduplication frees up capacity and bandwidth, and slows the rate at which a company needs to add storage. Less storage means lower costs. It also means reduced energy consumption.
Integral Capital Partners, a private investment firm in Menlo Park, CA, turned to deduplication to speed its overnight backups. The company operates facilities in Menlo Park and Baltimore, stores data locally and replicates data between locations nightly. "The process took eight hours," recalls Jason Paige, information system manager.
Paige began looking for faster options early in 2007. The few he found, however, "wanted disk to emulate tape," he recalls. "I didn't want that." Finally, Paige found Avamar, which has been acquired by EMC Corp. Avamar splits files into segments, applies deduplication and sends only the data that has changed, he explains. Integral Capital Partners replicates 300GB to 500GB nightly over a T1 line. The Avamar devices themselves can store approximately 2TB of data. "With Avamar deduplication, we were able to cut our backup time to
| about 45 minutes," says Paige.
"Deduplication already is game-changing technology," says Arun Taneja, founder and consulting analyst at Taneja Group, Hopkinton, MA. "It has gotten to the point where every VTL [virtual tape library] includes deduplication."
Despite enthusiasm for deduplication, there are some potential pitfalls. "It can slow recovery," says Greg Schulz, founder and senior analyst at StorageIO Group, Stillwater, MN. "Also, the data reductions are very dependent on your particular data."
This was first published in December 2007