Facebook storage: Reviews vary for Blu-ray cold storage, M.2 SSD plans

Facebook's plans to use Blu-ray discs for cold storage, 'netbook-class' M.2 SSD for booting Web servers draw mixed reviews from analysts, IT pros.

The Blu-ray disc and "netbook-class" solid-state drive strategies that Facebook Inc. outlined at last week's Open Compute Summit in San Jose, Calif., drew mixed reviews in a survey of data storage industry analysts and IT professionals.

On the plus side, Facebook's storage strategy of using consumer-grade solid-state drives (SSDs) to boot Web servers received a collective thumbs-up. Some IT pros noted that they use consumer-grade SSDs to speed the boot process in their servers, although they haven't looked into the new, smaller M.2 form factor, formerly known as Next Generation Form Factor, which is designed for power-constrained devices.

In contrast, Facebook's decision to pursue optical Blu-ray discs for its cold storage left several industry analysts scratching their heads. Facebook showcased a prototype that stores 1 PB of cold data on 10,000 Blu-ray discs in a rack built to Open Compute platform standards. Jay Parikh, vice president of infrastructure engineering at Facebook, revealed plans to increase the capacity to more than 5 PB per rack.

I don't think Blu-ray is a good choice for any company to store data.
Russ Fellowssenior partner, Evaluator Group

"The vendor community is littered with the dead bodies of optical storage companies trying to sell optical [drives] to enterprise storage buyers," John Webster, a senior partner at Boulder, Colo.-based Evaluator Group, wrote in an email, citing InPhase and Plasmon as examples of failed optical storage vendors. "It just doesn't sell enough to justify an investment in the technology."

Parikh claimed the Blu-ray device would facilitate 50 years of durability for "really, really long-term cold data" and a savings of 50% in cost and 80% in energy consumption over the company's current cold storage infrastructure.

Facebook last year went live with its first cold-storage facility, and a second is due to go online soon as Facebook ramps to well over 150 PB of data, with an ultimate potential of 3 exabytes per facility, according to Parikh. The systems at its Prineville, Ore., and Forest City, N.C., facilities use hard disk drives (HDDs) for cold storage, according to a Facebook spokesperson.

Production testing of the Blu-ray prototype is expected to begin this year, the spokesperson said. A video demonstration of the Blu-ray system is available on the engineering team's Facebook page.

The Blu-ray rack consists of 24 magazines. Each magazine has 36 locked cartridges, and each cartridge contains 12 Blu-ray discs. A robotic picker goes to a specified magazine and locates a cartridge, unlocks it, removes the drawer and selects the specified disk, according to Giovanni Coglitore, director of hardware engineering at Facebook.

In the video, Coglitore said the robot consumes a fraction of the power that a normal magnetic spindle does. The power consumption is virtually zero in a cold state while awaiting an operation to occur; when retrieving data, the power consumption is less than 1,000 W, he said.

"It's a fraction of what you typically would spend in energy in a data center, and the reliability is really in the disaggregated nature of the design," Coglitore said. "Each one of these discs is actually certified for 50 years of operation. And because the media is separate from the drives, if you ever have a drive issue, you simply replace the drive. You won't have to replace the data within the disc."

Will Blu-ray strategy prove feasible over the long term?

Howard Marks, founder and chief scientist at DeepStorage.net, expressed skepticism about Facebook's Blu-ray strategy. He said the recordable 100 GB discs that Facebook likely uses are less than four years old, and their longevity can only be estimated through accelerated aging tests.

"Our experience with recordable CD-ROMs and DVDs has been that the projected life may have been achievable, but some significant number of disks fail earlier than that," Marks wrote via email. "The real problem with 50-year claims is that the advance of technology makes storing that data on that media [for] that long impractical.

"Even if the disks are readable 50 years from now, the drives can't last that long," he continued. "Capacitors leak, lubricants wear out or flow, and so on. So, to read those disks 40 years from now, you'll need a drive that's 10 years old or less to read them."

Russ Fellows, a senior partner at Evaluator Group, said that because there's little research and development into the technologies, "future improvements are doubtful at best." In addition, optical storage is typically more expensive than tape alternatives on a price-per-GB basis, he said.

"I don't think Blu-ray is a good choice for any company to store data," Fellows said. He noted that Google uses tape for long-term data retention, and he predicted that Facebook will regret its Blu-ray decision.

Tom Coughlin, president of San Jose, Calif.-based Coughlin Associates Inc., said Blu-ray technology is solid and capable of going to multiple layers to provide greater storage capacity. But he doesn't see how the economics of Blu-ray would be better than tape, both in terms of the overall purchase price and the operating cost.

"If I've got denser storage with tape, I don't need as big a library to store a certain amount of information," Coughlin said. "And therefore I don't have as big a footprint."

Stuart Miniman, a senior analyst and principal research contributor at Marlborough, Mass.-based Wikibon, called the Blu-ray prototype an interesting science experiment that "seems like a long shot for feasibility."

"Amazon Glacier is built on repurposed older technology," Miniman wrote in an email. "A specialized architecture for long-term storage built on optical media with robot arms doesn't seem to have the economies of scale to replace tape or other long-term options. I don't see this going beyond Facebook."

But Bob Plankers, a virtualization architect at University of Wisconsin-Madison, said he might consider Blu-ray for cold storage. The write once, read many nature of Blu-ray might be helpful for compliance needs and records archiving, he said. Plus, a move to Blu-ray might eliminate some of the headaches associated with tape, such as the need to convert data between different formats to avoid obsolescence, he added.

Plankers was sympathetic to the huge issue that power consumption can present to a company with the technology scale of Facebook. The Blu-ray option would consume little power, he said.

Wayne Pauley, a senior analyst at Milford, Mass.-based Enterprise Strategy Group, viewed Facebook's storage approach as an interesting alternative to a tape library system for cold storage. "Honestly, the mechanical aspects of optical have been around for so long, why not?" he said via email.

Pauley said the density, longevity and small form factor make sense, and the media format is cost-effective. But he added that Facebook would need to release its cataloging system to an open source community for the approach to have a chance at catching on beyond the company walls.

Will netbook-class M.2 SSDs extend beyond Facebook storage?

Whether the use of netbook-class M.2 SSDs extends beyond Facebook's storage is another open question. Michele Reitz, a senior research analyst of semiconductors at Stamford, Conn.-based Gartner Inc., said the M.2 SSDs are relatively new and need multiple sources on common specifications. She said they would replace 2.5-inch low-capacity HDDs, raising the question of whether servers will be able to readily accommodate them without modifications.

On the other hand, the small form factor can be appealing to save space and reduce power requirements, and like all SSDs, the M.2 variety would speed the boot process, Reitz noted.

"M.2 will have applications in SSDs in the enterprise as well as the client space," she said via email.

Matt Corddry, director of hardware engineering at Facebook, said the company examined the total cost of ownership (TCO) to boot a Web server, including the acquisition price, lifetime power draw and the operational cost associated with the drive failure rate. Its assessment showed that an SSD could work, but the purchase price for data center-class SSDs was far too high. The netbook-class SSDs offered lower TCO than spinning disk, according to Corddry.

Dig Deeper on Cloud storage