Storage standards: A progress report

The Storage Management Initiative-Specification (SMI-S), eXtensible Access Method (XAM), encryption key management and the Fabric Application Interface Standard (FAIS) are four standards that could radically change the way you manage storage systems and protect data. We look at the status of each of these standards and where they're headed.

Four storage standards (SMI-S, XAM, encryption key management and FAIS) promise to make it much easier to manage storage and data.

Several storage standards are emerging that will help storage administrators manage data more efficiently. According to Vincent Franceschini, chairman of the board of the Storage Networking Industry Association (SNIA), a "standards ecosystem" is slowly developing that will allow heterogeneous storage products to better communicate with each other, as well as to protect data and more easily find it. They're lofty goals and progress is measured in baby steps. What follows is an update on four important major standards: the Storage Management Initiative-Specification (SMI-S), eXtensible Access Method (XAM), encryption key management and the Fabric Application Interface Standard (FAIS).

Standard 1: SMI-S
The most comprehensive of the four standards discussed in this article is SNIA's SMI-S. The ultimate goal of SMI-S is to let applications manage heterogeneous storage environments. It provides standardized methods to discover, collect data from and manage storage devices. While ultimate realization of that goal is some years away, SMI-S is useful today and will be progressively more useful as it develops and additional vendors provide interoperable products.

According to SNIA, there are approximately 500 hardware and software products from more than two dozen vendors that conform to SMI-S. Many of those products that have gone through the SNIA compliance program bear the SMI-S logo, indicating SMI-S compliance.

SMI-S is an object-based standard built around the Common Information Model (CIM). There's a separate object (profile or subprofile) for each kind of device under SMI-S. Hardware vendors instantiate the object for each of their devices (called providers in SMI-S speak). Meanwhile, software firms can write their management software (called clients) to use the objects without having to worry about the characteristics of the particular device.

Since the release of Version 1.0, SMI-S has grown in the products and functions it covers (see "SMI-S roadmap," PDF). Versions 1.0 and 1.1 provided coverage of basic SAN equipment such as switches, host bus adapters and arrays. Only a few functions were covered in any of the products. SMI-S has since added more products, including NAS and much deeper coverage. Versions 1.2 (the newest adopted version) and 1.3 (in advanced development) cover or will add performance instrumentation, improved monitoring and virtual tape. Work is already starting on Version 1.4.

Click here for the SMI-S roadmap (PDF).

Among the features added to Version 1.2 are support for storage enclosures (as devices), host-based controllers, file-system quotas, consistency management for snapshots and replication. Disk partitioning, virtualization and a variety of Fibre Channel features are also included.

The standard is becoming broader by addressing different kinds of devices. It's also becoming deeper by providing additional capabilities for supported devices, says David Thiel, chairman of the SNIA Technical Council and a staff fellow in Hewlett-Packard (HP) Co.'s StorageWorks division.

"SMI-S 1.0 couldn't even spell file and didn't apologize for the fact," says David L. Black, a member of the SNIA Technical Council, as well as one of the chairs of the Internet Engineering Task Force (IETF) IP Storage Working Group. He's also a senior technologist at EMC Corp. "In Version 1.2, you find file management."

A work in progress
Despite SNIA's efforts, product interoperability is still an issue and will probably remain so for years. Just because two products have passed the SNIA certification process and bear the SMI-S logo doesn't mean they'll automatically work together.

"That would be putting the bar a little high," says Alan G. Yoder, vice chairman of the SNIA Technical Council and senior member of the technical staff at Network Appliance (NetApp) Inc. "It's not because we don't want it [certification] to mean that [absolute interoperability]; it's because getting it to mean that is a difficult and enormous proposition. EMC has over a billion-dollar lab for interoperability testing. I don't think any member of SNIA is going to demand an investment of that magnitude, but that's kind of what it takes to get bulletproof interoperability," he says.

"What we've got," adds Yoder, "is a lower bar, but one that's high enough to indicate that they're [the vendors] serious about it [SMI-S interoperability]." However, adds Thiel, if the products are certified, there's a much higher probability things will get along.

While SNIA doesn't have examples of specific product incompatibilities, it has identified some areas not covered by the standard or their testing that might cause problems. SNIA's testing doesn't cover the installation process for the storage management software because that's not part of SMI-S. Similarly, the way SMI-S-compatible software handles error scenarios isn't covered in the testing process. While SMI-S includes standards for reporting errors to the app, it doesn't specify how those errors should be handled or reported to the user. That's up to application vendors.

Currently, the highest version of SMI-S that SNIA is certifying is 1.1. Yoder says that more than 400 products have passed conformance testing at that level. Version 1.2 is just coming out and its certification process is just beginning, so there's no certification list yet. However, SNIA expects to have a short list by the first quarter of 2008, with more products to be added soon after. As a result, companies that want to use SMI-S features need to test carefully. "It always comes down to testing," says Yoder. "If you don't test, you're going to have a lot of disappointments."

"SMI-S [Version 1.2] helps by providing broad coverage but not much depth, which might not be adequate for a dynamic environment," says Noemi Greyzdorf, research manager, storage software at IDC, Framingham, MA. "You definitely get more granularity and more rich information from [proprietary] APIs."

Management framework
As big as SMI-S is, it's basically concerned with hardware. To use it, vendors need to write storage management software that works with the standard. To meet the need for fully SMI-S-aware storage management software, SNIA is working on what it's currently calling "the management framework."

"This [the management framework] is aimed at someone who is building a storage management application on top of SMI-S and needs component services and some agent infrastructure," says SNIA's Black. "The goal is to make it easier to do that kind of infrastructure." The management framework isn't part of the SMI-S spec, but a separate project that complements SMI-S. One piece of the framework, says Black, is a way to store the information the app discovers about the devices. By storing the information, the app doesn't need to constantly go back to the agents to retrieve the data.

Among the other pieces are a discovery module to find SMI-S agents (components) in the environment; in other words, what SMI-S services are available on the system, as well as finding Web service-based management interfaces. SNIA says the management framework has been under development for approximately a year and has at least 18 months to go before the first version of the spec is ready. It will probably receive a catchier name (see "Why SMI-S is taking so long," below).

Why SMI-S is taking so long
The Storage Networking Industry Association (SNIA) has been working on the Storage Management Initiative-Specification (SMI-S) for more than five years. Why isn't it finished yet?

Four years after the release of Version 1.0, a much expanded version of SMI-S 1.2 is finally gaining traction. SMI-S as it exists today is useful, but it's a long way from being complete. There are still a lot of things SMI-S doesn't do; over the next five or six years the SNIA committees will add functions in successive releases. Near-term plans include supporting performance instrumentation and NAS discovery. Enhanced security, beyond the simple security monitoring functions in Version 1.2, and IP storage are further out.

There are several reasons for the long development of SMI-S.
  1. SMI-S has turned out to be an extremely difficult standard to write because it includes so much. It supports a couple of dozen different classes of storage devices, and provides ways to collect hundreds of different kinds of data from tape library statistics to temperatures in array enclosures.

  2. It has been a moving target. SMI-S has constantly added support for new devices as well as new functionality, and will continue to do so.

  3. The process has been painfully slow as committee members went back and forth over the approach to take to the standard in general and to the various devices SMI-S included.

  4. Many vendors were lukewarm to the idea of a storage management standard in the beginning and it took time to get them fully on board.

  5. SMI-S was SNIA's first attempt at a storage standard. They were swinging for the bleachers in their first at-bat and the result was a major learning experience.

Standard 2: XAM
XAM is the other major storage management standard being developed by SNIA. XAM will make fixed content (data that doesn't change once it's created) easier to manage. It does this by freeing the data from dependencies on location, apps and storage devices. XAM attaches a unique identifier and user-defined meta data to each data object to make it faster to find. It virtualizes the interface between the storage devices and the apps to minimize the pain in changing hardware or software.

XAM meta data can range from standard information like date of file creation or file size, to more elaborate and project-specific information such as which project the data is associated with. The user (or system) can specify that some of the fields can't be changed while others can be modified.

By storing the meta data in separate fields and welding it to the data to create a single object, XAM makes it a lot easier to find, track and manipulate data. By making those fields fixed or modifiable, XAM protects the audit trail (which needs fixed fields) and updates other parts of the meta data as needed. New fields can be added as business requirements change.

In XAM, as in other content-based storage, every data item and its associated meta data has a unique identifier. "XAM supports unique IDs for the data," says David Martin, co-chair of SNIA's XAM committee and an information portfolio manager at HP. "No matter where the object gets transferred, that identifier is always going to be the identifier for that object. We can find that needle in the haystack, no matter what haystack the needle has been moved to."

XAM virtualizes the interface between app and hardware. That means apps can access hardware through a standard interface for any device that supports XAM, says Martin. That makes management easier, and removes much of the worry of migrating to new hardware.

The XAM architecture is based on the XAM Library, which implements the XAM API and sits between storage devices and the app. The XAM Library is part of the XAM Software Development Kit (SDK), which dynamically links apps that want to use the XAM system to the storage devices. The library contains Vendor Implementation Modules (VIMs), vendor-written code that translates XAM requests into device-specific action. One of the beauties of XAM is that vendors don't have to change their products to use it. Instead, vendors write VIMs for the library and XAM handles the rest.

SNIA expects to release Version 1.0 of XAM early in the first quarter of 2008. As of October 2007, the XAM standard was at 0.6 and Martin says he doesn't expect much change between the current version and V1.0.

At the fall Storage Networking World in Grapevine, TX, the XAM committee was handing out thumb drives containing a XAM emulator from Sun Microsystems Inc., as well as other tools and documents to let firms get a head start on their XAM efforts. "Once [Version] 1.0 is released, production-quality products will be coming to market at the same time," says Martin.

Standard 3: Encryption key management
Three separate standard groups are working on encryption key management standards. The IEEE Security in Storage Working Group (SISWG) 1619 committee, called P1619.3, is focused on storage management. A second key management standard, the Enterprise Key Management Infrastructure (EKMI), is being developed by the Organization for the Advancement of Structured Information Standards (OASIS) consortium and is more general. The third, IETF's Keyprov, is based around a list of best practices for key management and will eventually evolve into a standard according to IETF documents.

As storage security becomes widespread, managing encryption keys--especially those from different vendors' products--has become important. Standards for key management have lagged behind this need.

"As storage vendors, we got behind the curve and stayed behind the curve much longer than we should have," says Blair Semple, education and alliances officer for the SNIA Storage Security Industry Forum (SSIF) governing board and a security evangelist at Decru, a NetApp company.

Software conforming to any of the standards will have the ability to generate keys, store and replicate keys, authenticate keys, archive keys and, finally, destroy keys when they're no longer needed. While SMI-S will include key management starting with tape in Version 1.4, it isn't officially supporting any standards.

"SNIA is not backing any one of these activities, but rather it is watching all of them," says Eric Hibbard, chair of SNIA's Security Technical Work Group and vice chair of the SISWG. "It is true that many of the SNIA member companies are also actively involved with the P1619.3 standardization activity, so it is reasonable to assume that the P1619.3 standard will be better aligned to the needs of the storage industry."

He also notes that there isn't any indication of how these standards will work together. "At this point, the P1619.3, EKMI and Keyprov activities are all producing standards; it is still too early to say exactly how these standards will work together, if at all," says Hibbard.

One of the features of P1619.3 is that while it manages keys centrally, the key repository and management app doesn't have to be in the same place in every enterprise. For example, for some types of applications it may make more sense for the key repository to reside on a switch and for other types of applications it may be best for the key repository to be in the tape drive, says Semple, adding that it will probably take another 18 months before key management that conforms to an industry-wide standard is available in multiple vendors' products.

Standard 4: FAIS
FAIS is a standard for heterogeneous intelligent switches in a SAN. It provides a standard API to perform the functions of a SCSI initiator and SCSI target. Using the SCSI functions, you can perform a variety of storage management jobs, such as storage virtualization, snapshots, backup, data journaling and replication, and data migration, on the intelligent switches in the SAN. You can also issue some commands to the storage devices connected to the back end of the SAN.

Using the SCSI command set also means FAIS can communicate with SCSI devices without the need to write additional software. But the SCSI command set limits what you can do to manage the devices and which devices you can manage natively. Moreover, FAIS doesn't offer the sort of discovery, monitoring and management functions on hardware that SMI-S does. In fact, its ability to interact with hardware other than SCSI devices is very limited.

"FAIS is a data path interface for effective insertion of functions such as [storage] virtualization into an intelligent switch," says SNIA's Black. "SMI-S is about management of the [storage] infrastructure."

FAIS, developed by the T11.5 committee of the InterNational Committee for Information Technology Standards (INCITS), is much stronger at managing the SAN fabric and the data moving over the SAN than it is at storage management. For example, it does a good job of optimizing the flow of data over the fabric. FAIS is built on a split-path model that separates the flow of data from functions such as IO mapping in virtualization.

FAIS-2 (due out next summer) will add features such as enhanced separation of the control and data paths, more ways of routing exceptions to the control path, and better methods for discovering and configuring the data path capabilities. All of these will help to make FAIS-compliant applications much better at managing the SAN fabric and the data moving over it. Switches adhering to the FAIS standard are currently available from Brocade and Cisco Systems Inc.

Dig Deeper on Data storage compliance and regulations