BOSTON -- The open source community lags in developing applications optimized to take full advantage of flash performance, SanDisk's chief software architect said this week at the Linux Vault conference.
"The open source community has to get on board [with flash]," SanDisk's Allen Samuels told software developers at the Linux Vault storage and file systems conference Wednesday.
During his session, "At-Scale Data Centers and the Need for New Storage Architectures," Samuels prodded conference attendees to create more applications for flash.
"Open source is behind in creating products that are well-adapted for flash," Samuels said. "The performance gap translates into significant [inefficiencies] at the data center. Proprietary and in-house solutions are filling the gap but the problem at the end of the day is the wave of proprietary systems is creating fragmentation. This fragmentation is a drag on developers."
Proprietary solutions Samuels referred to include log structure file systems that work better on flash and are less efficient on hard drives. Log structure file systems are developed in Linux but are rarely used.
"The ability of people to build applications on multiple platforms is in danger during this transition to flash," Samuels said. "The third-party community has made this industry dynamic. The open source community has to get on board."
Storage admins at the conference are beginning to see the value of using flash alongside traditional hard disk drives. A systems administrator, who asked not to be identified because he does not have approval to speak for his company, said he has been testing solid-state drives. His genomics company is close to finishing a proof-of-concept implementation that uses SSDs for metadata and the peripheral services around the data. It still uses hard drives for data at rest.
"I don't think we will use it for primary data, because our data is very large," he said. "Our primary data is on platters because most of the time the data is at rest. I don't think hard drives are on the way out. They will be around for a long time. There still is a lot of large data and SSDs are still expensive."
Samuels said flash isn't the only architecture making gains in data center storage. Rackmount systems that scale-out storage and compute independently are also gaining traction for data centers that use Hadoop, Yarn, Shark or Spark for data analytics. The scale-out systems allow companies to balance price and performance based on the workload needs of applications.
"At the data center, that is how they are looking at the problem now," Samuels said. "It's a shift in the way people buy. It's a mind-set of how people look at the problem. If an architecture is scaled out, then the way you look at a problem is by determining: How many units do I need to solve the problem? Then I look at price. You hear lot from [customers] that they are optimizing around price performance. The best price performance solution is the winner."
Breaking down open source storage: OpenStack Swift
The next big thing in flash: Seamless data movement