Hot projects such as OpenStack and Hadoop continue to fuel interest in open source storage software, but it has yet to catch on in a big way with rank-and-file enterprise IT organizations more accustomed to SAN and network-attached storage systems.
Statistics show an uptick in the use of open source storage software, with early stage adoption centering on cloud service providers, large-scale Web companies, universities and government agencies seeking to boost scalability for big data workloads at a lower cost than traditional storage, according to industry analysts and vendors.
The appeal of open source storage software is undeniable. But most users want an integrated and supported solution.
data storage analyst, StorageMojo
Matthew Kullberg, technical project and services manager at the San Diego Supercomputer Center (SDSC), said using OpenStack Object Storage, often known by its code name "Swift," gives the center flexibility. It enables SDSC to use existing and low-cost commodity hardware to provide cloud storage services to educational partners without sacrificing performance, he said.
But Kullberg is aware of the tradeoffs that keep many enterprise IT organizations from using open source storage software. He pointed out the additional labor required to instantiate and implement new functionality in Swift, and he noted that the software lacks some "refined" features that commercial enterprise storage products have.
"Open source often comes with fewer support options," Kemper Porter, infrastructure service manager for the Mississippi Department of Information Technology Services, wrote via email. He said the department does not use open source storage software and would need to see a compelling argument to consider it.
Dean Flanders, head of IT at the Basel, Switzerland-based Friedrich Miescher Institute for Biomedical Research, wrote in an email that his organization does not use open source software for data storage because "the performance numbers are not that great and you spend lots of money to get it running and maintain it." He said he may consider open source file systems such as ZFS or SAM-FS, but only their commercially supported variants.
"No time for science projects when you are protecting people's data," Flanders wrote.
TheInfoPro, a New York-based service of 451 Research, conducts in-depth interviews with 180 to 200 IT professionals at midsize to large enterprises for the market research reports it publishes every nine months. TheInfoPro has noted only rare mentions and limited exploration of open source storage software, according to Marco Coulter, a vice president and research director in storage at TheInfoPro.
"People are really just starting to look at it, for expansion of network-attached storage [NAS] in some cases," he said. "It's extremely early days."
Ashish Nadkarni, a research director on the storage systems team at Framingham, Mass.-based International Data Corp., breaks down open source software in storage into the following four categories:
- Traditional commercial storage products with no open source software. Examples include EMC's VNX and VMAX.
- Commercial products developed in the last 10 to 15 years that use open source software as an underlying technology, but add significant intellectual property (IP) from the vendor. Examples of these vendors include EMC (Isilon), Nexenta, Nexsan, Nutanix and Scale Computing.
- Commercially supported variants of open source software. A sampling of vendors, with the open source projects they support, includes Cloudera and Hortonworks (Hadoop), Inktank (Ceph), Red Hat (GlusterFS, OpenStack) and SwiftStack (OpenStack Swift).
- Open source software that an end user downloads from the project's website, deploys and self-supports. Examples include Ceph, Hadoop Distributed File System, OpenStack Object Storage/Swift and OpenStack Block Storage, also known by its code name Cinder.
He cited the second category as the fastest-growing area, where startups have been the most active, packaging up open source software with their own IP and selling it as a commercial product. "The vendors started off with an open source platform, but have layered so much IP on it that the underlying open source platform is not exposed," Nadkarni said.
Nadkarni said the fourth category is the fastest-growing for open source among public clouds, but the smallest for general IT use. "Nobody in their right mind, at least in the enterprise, is going to download a piece of software from the Internet and do whatever they have to do with it and use it for storing their enterprise data," he said.
Nexenta Systems Inc. claims to have approximately 5,000 customers for its NexentaStor, which became available in 2008. The ZFS-based software uses the open source Illumos operating system, a fork of OpenSolaris. NexentaStor's customer base includes roughly 1,500 service providers as well as organizations in education, healthcare, regulated industries, telecommunications and finance, according to CEO Tarkan Maner.
"The appeal of open source storage software is undeniable. But -- and this is big -- most users want an integrated and supported solution," Robin Harris, a data storage analyst at Sedona, Ariz.-based StorageMojo, wrote in an email. "The 'roll your own' crowd -- Amazon, Google, Azure, Facebook and the like -- isn't that large, although they have a big footprint. Partner programs, such as Nexenta with Dell or Super Micro, are good, but ultimately most enterprise customers still want a single vendor to take full responsibility."
International Data Corp.'s Nadkarni said commercially supported open source storage software, his third category, is relatively new and represents only a small percentage of the overall storage market. But there has been considerable activity with vendors, in particular with OpenStack and Hadoop.
Most end-user enterprises can't yet leverage open source storage software at large scale or for business-critical workloads.
research director, Gartner Inc.
For instance, the OpenStack Foundation has noted the project's considerable progress since the initial "Austin" release in July 2010 -- from 30,000 lines of code to more than 1.7 million, and from 20 contributors to more than 1,600.
OpenStack's website lists more than 70 user groups around the world, and the uptake is reflected in a survey of cloud operators and end users done last year by the OpenStack User Committee and Foundation. Based on 822 survey responses, the staff cataloged 387 OpenStack cloud deployments in 56 countries, with storage and backup ranking sixth among the top applications or workloads.
The survey indicated that 173 respondents use OpenStack object storage features. Eight deployments had more than 1 million stored objects, including one with more than 500 million, and 22 implementations had more than 100 TB of block storage.
The list of vendors with supported versions of OpenStack Swift includes Hewlett-Packard (HP), Rackspace, Red Hat, SUSE and SwiftStack. Plug-ins for OpenStack Block Storage are available from major vendors such as EMC, HP, IBM, Microsoft and NetApp.
Red Hat's support generated considerable attention given the company's history with open source software, but the jury is still out on the likelihood that Red Hat will be able to mimic the success enterprise Linux saw with open source software.
Widespread open source storage adoption is still an uphill battle
A mid-2013 survey of 50 Red Hat enterprise IT customers done by 451 Research suggested Red Hat's Gluster technology had yet to entice customers. TheInfoPro's Coulter said only four of the 50 surveyed Red Hat customers used the software. Among the IT pros who said no to current use of Red Hat's Gluster, 32% had evaluated it, and only one of them was likely to use it. The survey did not ask Red Hat customers who had not evaluated the technology if they were likely to use it.
In October 2011, Red Hat acquired Gluster Inc., which sold a supported version of the open source GlusterFS scale-out file system. Red Hat later renamed its Gluster-based product "Red Hat Storage Server." The non-vendor-sponsored survey done by 451 Research asked Red Hat customers if they used "Red Hat's Gluster technology."
Karin Bakis, a spokeswoman for Red Hat, said the company would not comment on the unpublished survey. She said the company is public and does not break out its storage numbers, so she is unable to provide adoption statistics. Red Hat customers who have spoken publicly about their use of the company's storage software include Amadeus Data Processing, Brightcove, Cornell University, Intuit and Saskatchewan Telecommunications.
Inktank claims its Ceph Enterprise customer base has grown from about 20 customers at the end of 2012 to roughly 80 customers. Customers include Bloomberg, Deutsche Telekom and the University of Alabama. Many use Ceph with OpenStack, according to Neil Levine, Inktank's vice president of product management.
Mario Blandini, SwiftStack's VP of marketing, said via an email that his company has gained a "few dozen" customers since product shipments began approximately six months ago. He said the split is roughly 50-50 between pilot and production use, with some companies storing more than 100 TB. But even when SwiftStack customers such as Disney Interactive use the software in production, the open source-based object storage represents only a small percentage of the total storage at the company, according to Blandini.
John Webster, a senior partner at Boulder, Colo.-based Evaluator Group Inc., foresees Hadoop providing a different model for storage. He said Hadoop is often viewed in the context of data analytics, but he views Hadoop as a storage platform providing analytics that users can program.
"The concept of programmable storage is one that I think will evolve and catch on," Webster said. "The evolution is toward data stores, which implies data services and intelligence defining what storage is as opposed to just a place you put things [in] and take things out of."
Jie Zhang, a research director at Stamford, Conn.-based Gartner Inc., expects adoption of open source storage software to increase, but he said the rate and pattern will be uneven depending on the size of the organization, its internal IT capability and its willingness to adopt new technology.
"It is still at a very early stage," Zhang wrote via an email. "Most end-user enterprises can't yet leverage open source storage software at large scale or for business-critical workloads. But there have been successful cases that serve as both inspiration and proof of concept."
TheInfoPro's Coulter said that even though open source storage software has found a home with leading-edge early adopters, he thinks it's still an open question as to whether it will gain significant uptake with midsize and large organizations.
"If you think of how it went in the operating system market, Linux crept into the colleges and then sort of crept into the enterprise. Then vendors arrived delivering support of it and making it able to be purchased from a vendor," he said. "We've never really seen that same pattern in storage. I don't think it's a certainty that it will fit for the enterprise."