Hifn offloads data deduplication to hardware

By Beth Pariseau, Senior News Writer 10 Mar 2008 | SearchStorage.com

Digg This!     StumbleUpon     Del.icio.us

"All of these are processor intensive features," Hifn vice president of business development John Matze said. "Up until now, all data deduplication products have done hashing calculations in software, and they haven't been able to offer encryption and compression with it in one pass."

Hifn will use a chip that provides network-based encryption and compression with two chip boards. The Express DR 250 board is an update to a product that has already been on the market. The DR-255 offers a faster four-lane PCI-e interface.

More on data deduplication HDS ships VTL bundles with Diligent deduplication FalconStor clusters VTL data deduplication U.S. Army drafts deduplication, sends tape on leave Copan adds data deduplication to MAID

"Right now, the fastest these products will go is 250 megabytes per second," Matze said. That limits their use to secondary storage, but a third card based on new silicon that could perform at 1.6 GBps is planned for later in the year. The faster card could be used for primary storage data reduction.

The two cards Hifn released today also are capable of deduplication, encryption and compression with one board, but not in one pass. It does MD5 hashing for data deduplication in one process and compression and encryption are done in a second pass. The card due out later this year will complete all three processes in one pass, Matze said. "Right now we're recommending as a best practice that two cards be used, one for hashing and one for compression and encryption, since they're already separate processes," he said. Today's cards will also leave data deduplication lookup -- the matching of identification keys produced by the hashing algorithm to existing data -- to software.

Hifn sells compression cards to OEM partners including Sepaton Inc., which uses it to add LZS-standard 2x compression to its VTLs with data deduplication. That increases its data-reduction ratio from 25:1 to 50:1 in some cases.

The new boards and software APIs for unlocking the hashing, compression and encryption features will ship to OEMs this week, and Matze estimated products could be available to end users by the middle of the summer. He was mum on whether Hifn will offer its own product based on the boards, or which OEMs have signed up to incorporate the hardware into products.

Data deduplication vendors have debated about whether inline or post-process deduplication is better. Inline deduplication vendors argue that their products finish the hashing process before data is written to disk, which more effectively reduces disk space requirements and allows data to be replicated immediately. Post-process deduplication vendors argue that inline deduplication creates a performance bottleneck. This new hardware could short-circuit that argument entirely, according to Arun Taneja, founder of the Taneja Group.

"Inline data deduplication vendors might be able to put this card in and equalize that performance situation," Taneja said. "Even if throughput isn't exactly the same as post-process products, it could at least come in close, and it could take one huge knock on inline deduplication off the table."

Taneja also pointed out that chip and component makers are porting more storage services into hardware, citing the storage controller chip with embedded encryption that PMC-Sierra launched today. The chip can also be placed into HBAs.

"We're going to have to start deciding as an industry where we want these services, especially encryption," Taneja said. "Right now you can do encryption at the disk-drive level, the controller level, the SAN switch level, the HBA level … the industry is in a bit of an overkill situation, and it's going to need to settle on the best approach."

Sound Off! -   Be the first to post a message to Sound Off!

Tags: Data reduction and deduplication,  VIEW ALL TAGS

Digg This!     StumbleUpon     Del.icio.us