The massive data storage needs of cloud providers and Internet service companies beg the question: How can anyone keep available, back up and protect petabytes and even exabytes of data? Erasure code technology can help. We examine the ways in which erasure coding will also have an impact on enterprise IT shops.
Primer on erasure coding
Industry analysts predict erasure coding will become an increasingly important mechanism to protect exponentially growing volumes of unstructured data, such as documents, audio and video files, photographs and email. RAID 5 and RAID 6 are the most widely used erasure code forms, but can only protect against two failures, which isn’t enough for some organizations. The following link provides an overview of erasure coding.
Ethan Miller, a professor of computer science at the University of California at Santa Cruz, defines erasure coding, describes the different types of erasure coding, outlines the primary use cases and explores the distinguishing characteristics of storage products that support erasure coding. Continue Reading
Three leading storage experts dissect erasure coding
Erasure coding beyond RAID 5 and RAID 6 engenders debate on optimal use scenarios and the short-term and long-term potential for the data protection approach, which will have a profound impact on the IT and data storage landscape. We enlisted three leading storage experts to explore the topic of erasure coding, including the pros and cons, the potential for erasure coding to affect backup strategies and the decision on how much erasure coding to do.
Wikibon Chief Technology Officer David Floyer says that erasure coding can help reduce the amount of data needed for backups and significantly lower the cost of backup and recovery, compared to traditional models. Continue Reading
Professor Ethan Miller cautions IT organizations against using erasure coding to protect against device failures and excessive data loss. The tradeoff is the additional storage they'll require for the redundant information, and the need to update all of the disks they use for redundancy. Continue Reading
Marc Staimer, president of Dragon Slayer Consulting, asserts that the most critical factors in determining when to use erasure coding are the importance of the data and the length of time that the data needs to be readable -- not the amount of data. Continue Reading
Advice on erasure coding vs. replication
IT shops commonly choose between replication and RAID to protect data against catastrophic failures. Erasure codes beyond the often-used RAID 5 and RAID 6 provide an alternative to guard against more than two device or site failures.
Professor Ethan Miller says there's no benefit to doing erasure coding between multiple sites for most IT shops. RAID 5 or RAID 6 at each site, and replication between data centers, is often enough. Continue Reading