It's not news that storage area networks (SAN) have quickly grown in size, complexity and heterogeneity, and that they've become increasingly difficult to manage. A SAN serving a company with more than 10,000 employees has an average of 317 storage devices, according to IDC. Managing a large SAN from a single pane of glass -- one consolidated console -- would be a godsend.
Although the prospect of consolidated management isn't likely to become a reality soon, the challenges of supporting heterogeneous storage include the following:
- A rapidly expanding, immature industry with many competing technologies
- The rise of iSCSI and the emergence of new storage protocols and disk technologies
- Various approaches for storage file systems and storage virtualization
But there's reason to hope. From emerging storage management standards to the consolidation of storage management tools, the storage industry is slowly -- albeit very slowly -- progressing toward unified storage management. "In the past couple of years, we've seen storage resource management [SRM] and SAN management converge," says Laura DuBois, research director, storage software at IDC.
Configuration management of storage components and the ability to provision storage -- from LUN creation and zoning to the presentation of storage to a host -- should also be among any storage management application's capabilities. Newer features, such as the ability to model changes and the visualization of the complete data path, are also turning up in storage management suites.
Vendor product support
Vendors that offer products in addition to a storage management suite have an advantage with customers using some of their other products. In addition to the existing customer vendor relationship benefit, most companies try to limit the number of vendors they use. For example, this gives Symantec Corp. a leg up with customers that already own its Veritas backup products because the products are tightly integrated. Similarly, Computer Associates (CA) International Inc.'s close integration of BrightStor with its Unicenter system management suite and ARCserve backup application, gives BrightStor an advantage in companies already using Unicenter or ARCserve. Storage management applications from EMC Corp., Hewlett-Packard (HP) Co. and IBM Corp. favor those vendors' own products. For example, IBM's TotalStorage Productivity Center (TPC) for Replication currently supports only IBM storage and does performance reporting on IBM's SAN Volume Controller virtualization product only; similarly, EMC's ControlCenter can only discover, map and monitor its own Invista storage virtualization product.
Companies like CreekPath Systems don't have this leverage and have been struggling to compete against major storage vendors. As a result, CreekPath is in the process of rebranding itself as a "storage business analytics vendor" with a focus on storage analysis and trending. AppIQ Inc., on the other hand, was among the best for heterogeneous storage support and was acquired by HP. The AppIQ product is now sold as HP Storage Essentials. AdventNet Inc. took a different path by focusing on usability and affordability, and limiting its suite to discovery, monitoring and reporting of heterogeneous devices targeting the small- to medium-sized market.
Today's storage management applications are far from being truly heterogeneous because they work with only a limited number of devices and protocols, requiring users to look closely at qualification matrices when shopping for a storage management application. Furthermore, customers should insist on a trial period to ensure that the storage management application fits into their environment and supports all storage management needs, recommends Babu Kudaravalli, director of enterprise systems at National Medical Health Card Systems Inc. in Port Washington, NY (see "From the field").
Slow standards adoption
As with many fast evolving industries, little attention has been paid to standards, resulting in a flood of proprietary implementations. Only in the past few years has the storage industry worked together under the Storage Networking Industry Association (SNIA) umbrella to create the Storage Management Initiative (SMI) to develop and standardize interoperable storage management technologies. But it takes time for a standard to evolve. "[The Storage Management Initiative Specification] SMI-S 1.0 was focused around discovery and basic management, but the latest version, SMI-S 1.1 ... goes beyond the basics, allowing a higher order of control," says Denis Kennelly, vice president of storage management product strategy at EMC.
In addition to slowly evolving standards, the adoption of existing standards has been hampered by incomplete implementations of SMI-S and vendors trying to protect their turfs. "One problem with SMI-S is that it can be implemented differently, and some vendors have opted for not implementing all features," says Eric Pitcher, CA's vice president of BrightStor product management.
To add insult to injury, "proprietary implementations protect vendors' investments and margins, and therefore prevent some vendors from truly supporting standards, and only supporting standards to the extent that it won't hurt them from a marketing perspective," says Brian Babineau, an analyst at Enterprise Strategy Group (ESG), Milford, MA.
That's why a single application can't manage the myriad storage devices and protocols in existence today. "Storage management deals with a huge number of variables," says David Russell, research director at Gartner Inc. "A large number of vendors, devices, applications and APIs make it a very complex problem that increases exponentially in complexity with the number of combinations."
To overcome the limitations of today's standards, vendors have added proprietary support for mainstream storage devices to their suites in addition to standards-based management. By supporting standards such as SMI-S, CIM and SNMP, their management applications can manage any device adhering to these standards; however, the hooks into these devices may not be very deep. Therefore, they also support leading storage devices from vendors like Brocade Communications Systems Inc., Cisco Systems Inc., EMC, Emulex Corp., HP, Hitachi Data Systems, IBM, McData Corp., Network Appliance (NetApp) Inc. and QLogic Corp. by turning to proprietary APIs and CLI commands when standards fall short. The latter method allows the management application to manage some leading storage devices at a much deeper level than by just using the present standards.
Storage management vendors are very selective when adding proprietary support for storage devices. "Proprietary device support requires testing whenever a vendor releases new firmware and software versions. Each time we release a new version of CommandCentral, we need to ensure that it works with all devices we support," says Symantec's Danny Milrad, senior product marketing manager for CommandCentral Storage. "It's very expensive and it gets more complex the more devices we support outside of standards."
"Instead of looking at heterogeneous storage management from a single pane-of-glass perspective, a better way to look at it today is from a user's pane-of-glass angle," notes Russell. It's all about the 80/20 rule: "Storage managers need to choose the tool that best matches their priorities and select a storage management suite that takes care of the top 80% of their needs," he adds. Obviously, standards are always lagging behind new technologies; as long as storage technologies evolve as rapidly as they have in the past 10 years, a heterogeneous storage management nirvana can only be approximated.
Heterogeneous storage management trends
Storage management vendors have been struggling to differentiate their suites from competitors and convince customers to spend money -- in some cases more than $1 million -- on an application that in many ways aggregates tasks otherwise performed by free tools that come with the various storage devices. To differentiate their products, storage management suite vendors are starting to move away from being just technical plumbing tools.
Mark Davis, CreekPath's president and chief executive officer, says his company is going to "stop building software that manages hardware and start building applications that provide true business value." In the next version of its suite, due in the first half of this year, CreekPath will add business analytics to its application that will help businesses better plan and forecast their storage needs. CA's BrightStor Storage Command Center, which is one of the BrightStor r11.5 components, provides similar capabilities.
In addition to closer business alignment, storage management suites are adding more advanced features, such as easier SAN management for tasks like storage provisioning and event correlation capabilities to identify problems. Wizard-based provisioning and event correlation in EMC SAN Advisor and HP Storage Essentials are examples of this trend. The Aperi reference platform (see "The Aperi initiative"), spearheaded by IBM and supported by other storage vendors, is focused on commoditizing and unifying basic element management. "Vendors shouldn't spend their time re-implementing the low-level plumbing, but should focus on higher level management like automation and ILM [information lifecycle management]," says Jamie Gruener, storage marketing manager, IBM Tivoli.
The ultimate storage management objective is the ability to react to storage events such as volumes running low on capacity or error conditions without human interaction. Although we find policy-based provisioning and process automation in Symantec's CommandCentral and EMC ControlCenter in the form of its automated resource management (ARM), ESG's Babineau believes most customers aren't ready for fully automated storage management. "In 2006, modeling will be big. Being able to model additions and changes to your storage environment but apply them manually is one step prior to full automation, for which the storage community is not ready yet," concludes Babineau.
As storage, server and network management grow closer, storage management suite vendors are extending their applications beyond storage management toward complete infrastructure management, including server and network management. For example, HP added storage management capabilities to the 5.0 release of Systems Insight Manager (SIM). Furthermore, HP's Storage Essentials and SIM are closely linked by a common security model, database of objects, as well as common monitoring and event handling. "As far as HP OpenView network management is concerned, it will stay separate for now," says Rich Escott, director, storage resource management software at HP. "Although there are interfaces between OpenView and SIM/Storage Essentials."
Along the same lines, IBM will couple storage and server management over time. EMC, on the other hand, will add network management features to ControlCenter by integrating the recently acquired SMARTS technology into ControlCenter by the end of 2006 to foster EMC's current focus on disaster recovery (DR) and business continuity. "For DR and business continuity, IP network management that's able to manage the network of connected sites is crucial," says EMC's Kennelly.
Although storage management vendors want their suites to be the center of the storage universe, managing heterogeneous storage components from a single console is only possible to a very limited extent today. For storage environments that have standardized on Fibre Channel using a minuscule number of storage elements from major storage vendors, single pane-of-glass management is well in reach. Beyond that, single pane-of-glass storage management of very heterogeneous environments is a fiction and will require storage management standards to mature before it can become a reality.
This piece originally appeared in Storage magazine.