Installing, configuring and maintaining Fibre Channel (FC) host bus adapters (HBAs) is the bane of many storage...
area network (SAN) administrators today. Often challenging to configure during installation and sometimes requiring server reboots after the initial deployment, the HBA becomes yet another maintenance nuisance for SAN administrators. But help is on the way: Some new cards offer better management tools that should simplify HBA administration tasks and extend the capabilities of this vital SAN component.
SANs already contain many points of complexity. Compatibility matrices require SAN administrators to double check HBA driver and firmware levels before doing even the most basic tasks. Storage arrays come and go from storage networks, necessitating that HBAs be reconfigured to discover the new logical unit number (LUN) assignments on the arrays. And as data continues to grow, LUNs must be allocated, so server reboots become mandatory for some operating systems in order for their HBA to discover these new LUNs.
With so many other administrative tasks, the last thing a SAN administrator needs is a long HBA to-do list. HBAs continue to improve, but SAN administrators can't assume that even the newest HBA will resolve all their existing issues or work in their environment. Each vendor's HBA comes with its own set of considerations that administrators must evaluate before looking to deploy and maintain.
The next generation of HBAs offers some features that shops may or may not want to introduce into their environments such as world wide name (WWN) spoofing and support for the recently proposed Storage Networking Industry Association (SNIA) common interface model (CIM) standard protocol. So let's take a look at what issues users face with HBAs today, examine how the latest HBAs try to answer these questions and look at what new features HBA vendors offer today in anticipation of tomorrow's challenges.
|Finding the right HBA|
|Sorting out the features|
With so many HBA features, knowing which ones matter and which ones don't can be a challenge for just about any administrator. Here's a short description of some of the features shipping today that may influence your buying decision.
DATA BUFFERS. Data buffers serve as a necessary and valuable component of Fibre Channel (FC) HBAs, but they shouldn't be thought of as a differentiator when selecting an HBA. This feature primarily becomes important in long-distance SANs, as higher buffer credits are needed to accommodate the longer latency periods when the data is sent and received.
FC SPEED. Two-gigabit FC HBAs are now available from every HBA vendor and make up the majority of HBA sales. They still autosense down to 1Gb to work with existing environments. The road map for next-generation HBAs is still uncertain, but 4Gb appears likely because, unlike 10Gb FC, it is backward-compatible with 1Gb and 2Gb FC, and will cost about the same as the 2Gb cards shipping today.
WWN SPOOFING. World wide name (WWN) spoofing is an interesting feature, but it has only limited application in today's SAN environment. Most HBA vendors offer it for customers who request it. Vendors see this feature primarily used in clustered server environments and feel the increased security risks outweigh the potential benefits of this technology.
CONTEXT SWITCHING. Context switching also serves as a necessary and valuable component of FC HBAs, but again, it shouldn't be thought of as a differentiator when selecting an HBA. This feature primarily takes on importance when the speed in routing a FC packet is a factor and SANs are already running at high capacity.
LOAD BALANCING/MULTIPATHING. These interrelated features serve as a requirement for any highly available server. Load balancing will balance the FC traffic over two or more paths through the SAN, while the multipathing feature enables the operating system to have more than one path to disk, should one HBA fail or a path through the SAN becomes unavailable.
CIM COMPLIANCE. Charles Kraus, HBA business unit director of Milpitas, CA-based LSI Logic, sees CIM functionality as one of the critical new features all HBA vendors must now offer. This feature will permit the management of heterogeneous SANs and enable certain functions (such as performance monitoring, remote firmware and driver upgrades and troubleshooting) across all HBAs, regardless of vendor.
Installing an HBA
Setting up any HBA requires answering various questions: What operating systems will it work with? What settings and configuration files need tweaking prior to, during and after installation? How does the HBA discover new LUNs on the SAN?
Probably the biggest initial task facing many system administrators centers on the installation and configuration of a new HBA in a server. Loading the correct driver, verifying the firmware, tweaking the configuration files, ensuring it is compatible with your SAN and discovering the LUNs for the first time can take hours--if not days--for new or experienced SAN administrators.
HBA vendors now ship HBAs that address many of the issues surrounding the initial install and configuration by eliminating or automating these tasks. Most vendors--including Atto Technology, Emulex, QLogic, and LSI Logic--now offer HBAs for Windows systems that are plug-and-play and require little or no intervention on the part of the system administrator to install and configure.
Amherst, NY-based Atto Technology's product manager, Pete Donnelly, lists interoperability of Atto's HBAs with Microsoft Windows as a top priority. Atto is currently working with Microsoft to ensure its HBAs are automatically recognized by Windows after being installed.
For Unix environments, installations are far from being hassle free, but there's encouraging work in this area. According to Charles Kraus, HBA business unit director of LSI Logic, Milpitas, CA, LSI Logic prepared a driver that will work under Linux and released that driver code to IBM and Hewlett-Packard, who then modified the driver for their specific versions of Unix (AIX and HP-UX). Kraus says none of the other HBA vendors offer native AIX or HP-UX drivers.
Sun Microsystems appears to be heading down the same path as HP and IBM. While all of the major HBA vendors currently provide drivers for Sun Solaris, Sun has included the leadville driver as part of its current SAN Foundation software. This driver offers full fabric support, along with failover and management capabilities, and replaces the one provided by the HBA vendors, though this driver currently only works with the newest HBAs provided by QLogic. The upside of this approach, according to Christopher Poelker, a storage architect with Hitachi Data Systems (HDS), is that the leadville driver removes the need to edit the sd.conf file because it runs at the kernel level and uses a different configuration file--sdd.conf--which needs no editing.
Managing an HBA
Once an HBA is installed, the focus changes to managing it. Here's where new struggles arise. In some large IT departments when upgrades occur on the SAN, administrators can find themselves at their wit's end trying to figure out which server has what HBA with what firmware and driver level.
Again, new features and tools from vendors such as Emulex, JNI and QLogic enable administrators to better control this part of their environment.
Costa Mesa, CA-based Emulex offers a central management console called HBAnyware that performs three critical management tasks that work in conjunction with their native driver kit.
HBAnyware grants administrators the ability to do local and remote discovery of Emulex HBAs. It also contains the ability to do local and remote management and real-time installation of firmware anywhere in a Fibre Channel (FC) SAN. Finally, it allows the administrator to gather FC HBA port status, attributes and statistics.
While having software like this becomes a powerful tool for administrators, knowing how to use it becomes just as critical. For instance, HBAnyware grants administrators the ability to upgrade Emulex HBA firmwares and drivers remotely. What the administrator needs to know is that upgrading Emulex's firmware can be done nondisruptively and won't require a reboot; however, changing its driver can be disruptive and may require a server reboot. So, administrators need to practice a think-first, act-second mentality when performing tasks like this.
The rest of the HBA vendors primarily provide tools that are locally installed and managed on each server. Atto has its ExpressPCI Configuration Tool for driver and firmware upgrades for its HBAs. Atto Technology's Donnelly says that while its firmware and driver updates may be done on the fly, he doesn't recommend that approach--especially for novice users. He sees the management of these functions being done locally for the time being, though when CIM becomes universally available, there will be more opportunities to perform these tasks remotely.
JNI and LSI Logic also provide similar utilities that simplify HBA installation and configuration for their specific HBA cards, though the other major vendors lack the central management control consoles offered by Emulex. LSI Logic's Charles Kraus says that interoperability with third-party SAN management software tools such as EMC's ControlCenter is one of LSI Logic's priorities for 2003. LSI Logic also plans to have a tool similar to Emulex's HBAnyware released by the end of 2003 to centrally manage LSI Logic HBAs.
QLogic provides a management tool similar to HBAnyware called SANblade Manager, which simplifies the configuration and setup of QLogic HBAs. This tool works in conjunction with its SANblade Control FX utility, which is controlled by the SANblade Manager software. The SANblade Control FX utility enables a faster HBA installation and configuration through a wizard-like interface, includes diagnostic tools that report on the firmware and driver levels and also includes a display that shows attached devices and LUN properties.
|How HBAs and operating
systems work together
The degree of difficulty or ease in configuring and using LUNs that an HBA discovers largely depends on the OS supporting the HBA.
Under AIX and HP-UX, a system administrator only needs to run the AIX cfgmgr or the HP-UX SAM utility to discover new LUNs. Using this utility, it doesn't matter whether or not the LUNs reside on the same storage array ports as other LUNs it already has discovered. However, you need to check with IBM or HP before purchasing just any HBA, as none of the HBA vendors provide drivers for either AIX or HP-UX. Both HP and IBM develop all of their own HBA drivers, so if they don't provide a driver for the HBA you are interested in, you're out of luck.
Sun Microsystems is in a period of transition right now. Under older versions of Sun Solaris or when using older HBAs, the system administrator must set up the corresponding configuration files. This requires the administrator to obtain the storage array port world wide name (WWN) and the list of LUNs that the storage array port will present to the server. However, Sun is in the process of changing this approach: In Solaris 8, the leadville driver eliminates many of the tedious manual configuration tasks though leadville currently only works with the QLogic 2342 HBA. The good news is that most HBA vendors provide drivers for Sun Solaris and these drivers actually offer most--if not more--functionality than what Sun's leadville driver offers.
The degree of support for LUN discovery under Windows varies quite a bit, depending on the version of Windows you're using. Under the Windows NT 4.0 operating system, the system administrator is always forced to do a reboot to discover a new LUN. Complicating the matter, the LUN is presented as a drive letter, so as a result NT has a built-in limitation of 26 LUNs. Windows 2000 addresses these specific limitations of NT 4.0. It offers the ability to add new volumes or span volumes across several disks without taking the server offline using its new dynamic volume management (DVM) capabilities. It also now offers the ability to manage up to 2,500 LUNs, breaking the previous 26 LUN limitation of Windows NT substantially.
As part of ensuring HBA interoperability with its operating system, Microsoft now also offers driver signing for HBA vendors who go through their certification process. Customers will be able to ascertain an HBA vendor's compliance with a specific release of Windows by looking for the appropriate Windows logo on each HBA vendor's technical documentation.
Discovering new LUNs
The other major issue is the ability to discover new LUNs on the SAN, whether on storage array ports where the HBA already sees LUNs or on different storage array ports on the SAN that will present new LUNs to the server. The degree of difficulty in discovering and configuring LUNs behind these ports will vary widely, depending on both the OS and HBA. (See "How HBAs and operating systems work together," above.)
The newer HBA cards' ability to autodiscover LUNs dramatically reduces the time administrators spend managing which LUNs the server has access to. Before this important new feature, some storage administrators chose to set up configuration files to recognize every possible LUN behind every storage array port on the SAN, even if the HBA didn't have access to them. This workaround allowed system administrators to discover new LUNs without server reboots.
Yet using configuration files to recognize LUNs has a big downside. During a reboot, for example, some operating systems will probe each of the LUNs described in its configuration file. This is a problem when a reboot is required and the server does not have access to all of the LUNs in its configuration file (and in large SANs, this number could be significant). In these circumstances, server reboots would take forever while the server timed out on LUNs it didn't have access to. New HBAs from Emulex, JNI and LSI Logic, along with more intelligent server operating systems, eliminate this gotcha.
Consider the QLogic QLA2342 HBA. On Sun Solaris systems, it autodiscovers any LUNs on any storage array on the SAN for which it has the appropriate permissions, without any additional configuration tweaks or reboots. On Windows 2000 systems, QLogic provides system administrators a GUI utility to discover and report on the new LUNs though Windows itself detects these new LUNs under the Disk Administrator component of Computer Manager.
For HP-UX and AIX operating systems, as long as they support the vendor's HBA, the autodiscovery of LUNs feature should be available. Each ships with utilities native to its operating systems that allow new LUNs to be discovered without requiring reboots of its respective systems, regardless of the HBA manufacturer.
With these advances in simplifying the maintenance and configuration of HBAs, vendors are now turning their attention to address the next generation of FC HBA challenges. One such feature that's been available but whose future acceptance is still debated is the WWN spoofing functionality. To date, few--if any-- HBA vendors natively ship this feature, even though they have it available in their labs. Atto's Donnelly cites lack of customer demand and that fact that WWN spoofing introduces security risks into the SAN as reasons for not promoting this feature more aggressively.
Yet with the number of servers coming onto the SAN growing consistently, the growth of clustered servers and the desire by some to either simplify LUN security or establish a meaningful corporate HBA naming schema, this feature continues to generate some attention.
Currently, SAN administrators have to reapply the LUN security of the new HBA on every LUN the server sees if the old HBA fails. With the deployment of WWN spoofing, system administrators may now re-assign the old HBA WWN to the new HBA, reducing any risks involved with reapplying LUN security. Atto's Donnelly also sees this feature being used by customers who have clustered server environments. The HBAs of the failover server inherits the HBA WWNs of the server that failed, ensuring that failover server receives access to the data of the server that failed.
However, a more important benefit comes in the potential to come up with a meaningful WWN naming standard for all HBAs similar to what has occurred in the TCP/IP networking space. Right now, each server--regardless of its make, model or underlying operating systems--assumes the WWN that comes with the HBA. With the introduction of WWN spoofing, system administrators may now assign Windows servers one set of WWNs, Sun servers another and Novell servers yet another. This could enable a higher degree of QoS and better recognition of the nature of the FC traffic traversing the SAN.
All HBA vendors will provide this WWN spoofing function to customers who ask. Also, there are some third-party drivers available that replace the HBA's vendor driver with their own driver to deliver this sort of functionality. For instance, the leading storage array and tape drive vendors have driver code they use that replaces the HBA driver software on their respective device arrays that permit WWN spoofing for easier SAN management.
However, use WWN spoofing on servers with extreme caution. If, for example, an HBA on a Windows server is assigned the same WWN as has been assigned to a Unix server, the Windows OS could see the Unix storage on the SAN, which will result in a loss or corruption of the data associated with the Unix server. Worse yet, depending on the management and security tools deployed within the SAN itself, this situation could be difficult to diagnose and could occur again.
Fibre Channel speeds
Another feature that seemingly every SAN environment wants is more speed. The move to 2Gb-compatible HBAs now seems a done deal. Atto reports that more than 80% of its sales now come from 2Gb-compatible cards, while all of the HBAs LSI Logic ships are now 2Gb compatible. Atto's Donnelly says with 1Gb and 2 Gb cards costing the same, there's no longer any compelling reason to buy 1Gb cards (see "Finding the right HBA").
Yet lurking around the corner are the 4Gb and 10Gb FC standards. Here's where the next generation of HBA speed remains a little murky, though LSI Logic's Charles Kraus sees momentum starting to build for 4Gb. Kraus points to simple economics as the reason 4Gb FC HBAs have a future. The 4Gb HBAs will be backward compatible with 1Gb and 2Gb protocols, and they will autosense the older protocols and cost about the same.
Meanwhile, 10Gb HBAs aren't backward compatible with 1Gb, 2Gb or 4Gb because the optics they use are different, and it will cost about $10,000 per port to deploy.
Yet Kraus says one shouldn't assume 4Gb is a done deal, as the HBA vendors are still waiting for a major switch vendor to join the 4Gb fray. While Kraus notes the industry hasn't made a decision on 4Gb or 10Gb yet, he suspects that 4Gb will appear in the marketplace in about 12 months.
The final feature that's generating attention among the HBA vendors is the CIM standard that was just recently proposed by the SNIA. Atto Technology's Donnelly sees CIM putting all HBA vendors on equal footing in terms of being able to manage HBAs. While he doesn't see this standard as necessarily making HBAs a commodity, it will give SAN administrators the ability to perform 70% or more of the needed management functions on any HBA, regardless of which vendor they are dealing with.
The introduction of CIM support, LUN autodiscovery capabilities, remote management and faster FC speeds will all help to increase the value of the next generation of HBAs. These features will help SAN administrators make their SANs faster, less complex and easier to manage.