Does it matter whether multi-copy mirroring or erasure codes are used by my cloud provider?
In my experience, most users don't care whether their public cloud provider uses multi-copying mirroring (MCM) or erasure codes -- but they should. The short answer is that erasure codes offer a better guarantee that your data will be protected. But since the price per gigabyte is approximately the same for companies that use MCM -- Google and Amazon, for example -- and offerings with erasure coding -- such as Microsoft's Azure, many customers ignore this debate when choosing a provider. However, MCM generally uses more storage, and is less resilient, than erasure codes.
With MCM, you are protected against only two concurrent losses. MCM is most effective when the mirrored object has some form of autonomic healing that periodically checks data object health and corrects it if there are any issues. If a disk drive or node has failed, gone dark, isn't there or comes back corrupted, the storage system will make another copy from a healthy version of the data objects and write them somewhere else. This system works pretty well, but it has a high cost associated with it because protecting against a single data object failure requires 100% additional storage capacity and nodes. Protecting against two concurrent data object failures requires 200% additional storage capacity and nodes; three concurrent failures require 300% and so on. It's an extremely high overhead.
With autonomic healing, erasure codes change that equation by protecting against six or more concurrent drive, node, system, site or other failures. It does this by dividing a data object into a number of chunks, the total being referred to as the width. A common width is 16 chunks. The object storage has to read only a subset of those chunks (referred to as the breadth) to reconstitute the data object. A common breadth is 10 chunks. In this example, the object storage system can tolerate six concurrent failures of drives, nodes, sites and so on, and still read the data objects. Autonomic healing enables the lost chunks to be recreated and written elsewhere. This example demonstrates three times the resilience of MCM at one-fifth the overhead. You therefore should care which data protection method is implemented by a cloud provider because it is more likely that data objects will be there when they're needed if erasure codes are used.
However, nothing is perfect. A downside to erasure codes is that additional latency can reduce performance. Depending on how many other latencies you are introducing, this can be an important factor to consider.
Dig Deeper on Public cloud storage
Related Q&A from Marc Staimer
Network File System and Common Internet File System/Server Message Block were designed to work with any operating system, but NFS remains dominant in... Continue Reading
Object storage has unique features, including erasure coding and multi-copy mirroring, which may make it better suited to data protection than more ... Continue Reading
Why would you attach NAND flash storage directly to the memory channel? Isn't RAM much faster than NAND? Marc Staimer discusses this and more in this... Continue Reading