News Stay informed about the latest enterprise technology news and product updates.

IBM stretches GPFS, Watson into Elastic Storage

IBM incorporates its General Parallel File System and Watson technology into Elastic Storage, a unified software-defined storage package.

IBM staked a claim in the software-defined storage software market this week when it unveiled Elastic Storage software, technology based on the company's General Parallel File System and the Watson supercomputer that defeated two humans on the Jeopardy! game show in 2011.

Elastic Storage uses Watson's cognitive technology to handle workloads generated from cloud, analytics, mobile and social media. The software can be deployed on-premises or via IBM's SoftLayer cloud.

"The world needs a new kind of storage that can automatically move data locally and globally," said Vincent Hsu, IBM's chief technology officer for storage in the IBM Systems and Technology Group. "Elastic Storage is extremely flexible. It will eliminate storage silos by allowing universal access to data files. It's very adaptable in managing different types of storage resources from flash to disk to tape."

IBM rolled out Elastic Storage one week after EMC launched its Elastic Cloud Storage (ECS) that incorporates its ViPR software-defined storage software. ECS and IBM's Elastic Storage both support file, block and object storage, and serve as an abstraction layer between the application and physical devices.

Elastic Storage works as a control plane that intelligently handles data. The software can automatically move dormant or infrequently accessed data to less expensive storage, while leaving faster and more expensive storage resources such as flash for hot data. The management is guided by analytics, using patterns, storage characteristics and the network to determine where to move the data.

It also includes data protection capabilities such as automated backup and snapshots. The software manages copies of data, using one copy of the data for snapshots and replication of snapshots. That reduces the amount of storage consumed and cuts costs. Elastic Storage supports OpenStack Swift so users can access and manage data across private and public clouds. It also works with Hadoop.

Elastic Storage has native encryption and secure erase features, as well as server-side cache to increase I/O performance.

"The notion of Elastic Storage is it allows the application to write to a software layer or the virtual control plane APIs [application programming interfaces], which will interface with the storage infrastructure whether it's on-premises or in the cloud," said Steve Wojtowecz, vice president of IBM Tivoli storage software development. "You can take Elastic Storage, insert any hardware box and allow it to take that data footprint and move it around and grow the workload."

Watson provides the analytics engine behind Elastic Storage's intelligent data movement. For Jeopardy!, Watson had access to 200 million pages of structured and unstructured data, and approximately 5 terabytes of its knowledge base was loaded into the computer's memory in minutes. IBM claims Elastic Storage can scan 10 billion files on a system in 43 minutes.

"Analytics is used to optimize the storage stack," said Tom Clark, IBM's distinguished engineer and chief architect for storage software, cloud and smarter infrastructure. "It's all about managing workloads that have become very dynamic. We are using analytics to manage that."

IBM plans to leverage Storlets, developed by IBM Research, inside of Elastic Storage. Storlets are computational modules developed to manage object storage. Storlets are built on top of OpenStack Swift and integrated with Hadoop and OpenStack Cinder, which provides block storage as a service. The Storlets software uses metadata to manage the behavior of objects for more detailed analysis to perform tasks such as extracting a single video frame for editing or identifying suspect pathology images.

"In the future, you can assume the capabilities available in Storlets will be features and functions in Elastic Storage software," Wojtowecz said.

Dig Deeper on Unstructured data storage