itestro - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Why can't we perform primary storage dedupe on standard disk drives?

Most all-flash arrays perform primary storage dedupe and/or compression to extend capacity. Why are we are still waiting on that for traditional disk drives?

It's not easy. With memory technology, the systems can perform deduplication and/or compression a whole lot faster. There are more effective ways of implementing it in a flash environment when compared with a traditional spinning disk environment where there are a lot of synchronization issues to deal with.

If I was making an investment from a vendor side, I wouldn't invest [much] into primary storage dedupe or compression for spinning disks. I would probably only offer those technologies for flash and embed them into the flash controllers as much as possible, so it wouldn't take processing cycles away from the storage controller. I think the return on the investment [for dedupe and compression] is higher for flash than for spinning disk.

That being said, there are a number of implementations that perform data reduction for spinning disks. Some perform it post-process, which is problematic in some cases, but it does increase the capacity. If the data doesn't have a high-reference usage, that's probably very adequate. Also, there are some implementations that perform data reduction inline, before it's actually written. Those are very effective.

Next Steps

The importance of primary storage dedupe

How primary storage deduplication impacts backup

Data dedupe for primary storage

Dig Deeper on All-flash arrays