One of the hottest storage topics these days is data deduplication, a broad term that includes several technologies such as single-instance storage and commonality factoring. While it can trim data capacities significantly and forestall buying more disk, deduplication has always been the exclusive province of nearline storage, most often used in conjunction with data protection applications such as archiving. But few dedupe vendors have tried to apply their techniques to primary storage largely because of the impact on performance.
A couple of pioneering vendors hope to change this. Storewiz recently rolled out the next generation of its primary storage compression appliance, which has been available for more than a year. FalconStor Software has a similar compression product on the drawing board. "We are expecting this special-purpose computing module to be available in [the] late 2007, early 2008 timeframe," notes ReiJane Huai, FalconStor's chairman and CEO.
The two companies are taking somewhat different routes, but the goal is the same--to put the squeeze on stored data using new compression algorithms and variations of existing algorithms. Storewiz's updated appliance line, the STN-6000 series, improves on the previous version's compression and failover capabilities. The appliances sit in the data path but compress the data on disk, so Storewiz claims there's little or no impact on performance.
"We are adding for each I/O request somewhere between 50 to 100 micro-seconds," says Tzahi Shahak, Storewiz's director of product management. But, he adds, there may also be a positive effect on performance because data size has been reduced. "Not only that, we are not hurting performance," he says. "We have many cases where storage performance improved."
"The performance boost that I would expect right off the bat would be the benefit of the storage array not having to move as much data," says Greg Schulz, founder and senior analyst of The StorageIO Group, Stillwater, MN. But Schulz is wary. "There is the risk of aggravating storage system performance bottlenecks as you increase the density of the stored data."
FalconStor is dabbling in hardware for its compression product. While Storewiz handles compression solely through software, FalconStor's Huai believes compression needs a hardware boost in the form of an ASIC or field-programmable gate array (FPGA) that's purpose-built to handle the job. "We are working with our partner to develop a module, which can be attached to the standard x86 server [storage appliance] to offload those CPU-intensive tasks from the main processor," says Huai.
Reliability and availability are key issues when compressing production data. "We have a very high MTBF [mean time between failures] appliance with no moving parts in it," says Storewiz's Shahak, noting that the Storewiz appliances boot from flash memory and don't have hard disks.
Though storage managers will have to wait to put the FalconStor product through its paces, Storewiz says that approximately 150 of its STN-5000 units are in use. (No current users were available to comment for this article.) The company claims the STN-5000 can provide up to 3:1 compression, yielding a capacity savings of 60% to 70%.
Given that Storewiz handles compression through software, and FalconStor's design calls for only a modest amount of specialized hardware, compression functionality needn't be limited to appliances. "We think the same idea can be applied to the switch as well," says Huai. It may even be feasible that some form of data compression could be built into arrays.
"For now, I would be surprised not to hear an array vendor explain that you don't need compression when you have low-cost, high-capacity drives," says Schulz.
- Distributed File Systems and Object Storage Vendor Evaluation –Western Digital
- NVM Express, NAND Flash Energize Storage Vendors –SearchStorage.com