OpenStack Newton, the latest release of the open source software, includes at-rest data encryption and performance...
and scalability improvements across OpenStack Swift, Cinder and Manila.
OpenStack Newton is the 14th release of the open source software for building public and private clouds. The OpenStack community releases new versions on a biannual basis. The next OpenStack release, Ocata, is expected in February 2017.
Newton brings features found in commercial storage products into OpenStack, which is used mainly to build cloud services. The software's adoption has been limited, as many organizations find it complex to use and have a hard time finding IT pros with OpenStack experience.
The main new feature in OpenStack Object Storage, better known as Swift, is at-rest data encryption. The capability is often a requirement for large enterprises, government agencies and other organizations looking at new storage systems, according to John Dickinson, the project technical lead (PTL) of the OpenStack Swift project and director of technology at SwiftStack Inc., based in San Francisco.
Dickinson said Swift users can turn on at-rest data encryption when they create a cluster, or they can enable the feature on existing clusters to automatically secure object data and metadata values.
"If a hard drive that was used to store the object data in Swift leaves the cluster, you don't have to worry about information leaks," he said.
Dickinson cited an example of ingesting hundreds or perhaps even a thousand hard disk drives into a Swift cluster before learning an HDD batch is bad. He said the drive manufacturer or reseller often swaps the failing, warrantied drives for new ones. Encryption would guard against unauthorized parties gaining access to data when the user sends back the drives, he said.
Another example the Swift community sought to address with at-rest encryption is an inventory mistake. Dickinson said a staff member might remove a drive from a Swift cluster, misplace it and plug it into another server without first erasing the drive.
Swift's two-tier architecture has a proxy server and storage nodes. The proxy server handles the API and coordinates data requests, and the storage nodes persistently store data. The community implemented Advanced Encryption Standard 256-bit encryption in the proxy server, Dickinson said.
Swift encryption is not compliant with the Federal Information Processing Standards (FIPS), but Dickinson said it supports all of the appropriate encryption algorithms and methods to enable FIPS 140-2 certification in commercial products built on Swift.
Swift improvements in OpenStack Newton
Along with software-based encryption, the OpenStack Newton release of Swift features general performance and scalability improvements related to global clusters, concurrency and throughput.
Dickinson said the Swift community's work for the upcoming Ocata and future releases includes performance and scalability improvements, automatic data tiering and global erasure-coded clusters. He said Swift already supports erasure codes and global clusters, but using them at the same time can cause problems.
"With global clusters, the biggest problem you have to solve there is the fact that you have a WAN link between your two regions that is generally more expensive to use, more bandwidth constrained and more likely to have a failure. And you've got some high latency," Dickson said. "So, all of those things combined mean that if you decide to try to deploy erasure codes in a globally distributed system, it's very difficult to maintain high availability with your data."
Dickinson said the Swift project has grown to more than 600 contributors, including SwiftStack, Red Hat, Hewlett Packard Enterprise, Intel, Nippon Telegraph and Telephone Corp., Rackspace, IBM and Fujitsu.
Beyond the OpenStack Swift community, the broader open source ecosystem is working to improve the Amazon Simple Storage Service compatibility layer. Dickinson's employer, SwiftStack, is also finishing off an open source file system interface to OpenStack Swift. ProxyFS is a separate Swiftstack-developed technology, although it potentially could be folded into the OpenStack project, Dickinson said.
The Newton cycle of OpenStack Block Storage, known as Cinder, drew 215 unique code contributors and 366 code reviewers from more than 60 companies, according to Sean McGinnis, the Cinder PTL and a distinguished engineer at Dell EMC.
One key new feature in the OpenStack Newton release of Cinder is a message API and command-line interface to enable users to receive the results of an asynchronous task, beyond simply learning that a request kicked off. McGinnis said the feature could provide insight into instances where an asynchronous task fails.
The focus of Cinder's Ocata release will merely be bug fixes and further stabilization due to the compressed time frame for the development cycle, compared to the typical six months, McGinnis said.
McGinnis said the Cinder community is considering ways to make Cinder a more feature-rich storage management platform that can stand apart from OpenStack. No concrete plans have emerged to date, he said.
"We now have systems that are implementing our APIs in an attempt to be able to plug into OpenStack," McGinnis said. "We also have a couple drivers in Cinder to go the other direction and have Cinder be able to manage different storage devices through these storage platforms, so vendors can potentially just write a driver for one and leverage that for both environments."
New OpenStack Manila features
One of the main new capabilities in the OpenStack Shared File Systems project, known as Manila, is driver-assisted data migration. The OpenStack Newton-added feature facilitates efficient migrations between storage back ends from the same vendor, in addition to the typical type of migration that works from anywhere to anywhere, according to Ben Swartzlander, the PTL for Manila and a principal engineer and OpenStack architect at NetApp.
Additional new features include user interface support for share migration and share replication, Manila support in Red Hat's OpenStack distribution and an OpenStack Fuel management plug-in for Manila to enable support in the Mirantis distribution.
In the upcoming Ocata release, the Manila community hopes to remove the experimental tag from migration and/or replication. Other goals are to fix bugs related to highly available and scalable deployments, enable user messages to provide information on why a request failed, and improve snapshots and share groups.
"Contributions for Manila have reached the point where people are developing new features faster than we can review and merge them. This is great news from a community perspective, but a challenge for the core team," Swartzlander wrote in an email.
Most popular OpenStack storage is Cinder
In-depth look at Cinder, Swift
How storage fits into OpenStack