Host bus adapter upgrades and replacements

SAN expansion best practices include maintaining control over HBA selection, eliminating bottlenecks and boosting availability.

Host bus adapters (HBAs) provide the network interface that connects host computers and storage systems to various switch ports, enabling communication on the storage area network (SAN).

HBAs are defined by their connectivity, speed, port count and system interface. One popular HBA provides two 4...

Gbps optical ports as a PCI Express (PCIe) expansion card. Another provides four 4 Gbps Fibre Channel optical ports as a PCI-X card. Expansion card HBAs are found in servers and other PC-based SAN hosts. HBAs can also be integrated directly into the main board or backplane of a storage system, but such "hardwired" HBAs are more time-consuming and difficult to replace.

Even though HBAs are not frequently updated, there are circumstances when HBAs must be changed. Reasons for upgrading HBAs include:

  • to get faster connectivity
  • to add ports for aggregation or failover
  • to achieve better reliability
  • to get more stable or mature drivers and management software
  • to get superior RAID
  • to get virtualization support.

Best Practice No. 1: Don't overbuy HBAs

Storage architects and administrators have the least control over HBA selection and implementation. HBAs are normally purchased as part of a new server with an understanding of the switches and other SAN infrastructure that will be in place. Consequently, it's poor practice to overbuy an HBA during an initial purchase unless other SAN expansion plans exist. There's no reason to purchase HBAs with 4 Gbps connectivity if the SAN speed is not expected to evolve beyond 2 Gbps. No is there any reason to buy more HBA ports than necessary unless you plan to implement port aggregation or failover schemes.

Best Practice No. 2: Standardize on a primary HBA vendor

Storage administrators can reduce potential interoperability problems with HBAs by relying on vendor compatibility matrices and using the same HBAs whenever possible. This will also reduce the amount of maintenance needed for firmware updates. However, it's also smart to qualify HBAs from a second vendor in the lab, so an alternate HBA can be used in cases where the original vendor's HBA is unavailable or has changed in an unexpected way that introduces interoperability or performance issues in your environment.

Best Practice No. 3: Eliminate bottlenecks in the HBA itself

Select an HBA that provides local processing electronics that are dedicated to each port. This practice eliminates HBA ports "sharing" the same chip and forming potential bottlenecks for network data. It also improves performance in the HBA and reduces common failures. For example, if four ports are serviced by the same chip and that chip fails, all four ports can be disabled. However, if a chip only drives one port and that chip fails, only that port is affected, improving overall availability and reliability on the SAN. "The mere presence of ports on an adapter doesn't necessarily mean more performance," says Greg Schulz, founder of the StorageIO Group.

You should select a host bus (e.g. PCI, PCIe or PCI-X) with enough bandwidth to support all the port traffic you expect to handle. Today, PCIe is the fastest readily available PC-based bus, offering a maximum theoretical data transfer rate of 8 GBps between expansion cards and the computer backplane. By comparison, the PCI-X supports data transfers to 1,064 MBps, while the antiquated PCI bus tops out at 532 MBps.

Best Practice No. 4: Use multiple HBAs for higher availability

A single HBA may provide adequate performance when connecting a host server and SAN, but it can present a single point of failure that is an unacceptable practice for reliable SAN design. Using a single adapter and allowing failover to another SAN cluster is also a poor design choice. It's equally bad practice to connect multiple HBA ports to the same switch or even to interconnect the switches. "Keep your separate paths," says Schulz. "Don't mistakenly create one single fabric by interconnecting the different devices."

Best Practice No. 5: Test new drivers and firmware before deployment

HBA drivers and firmware are updated periodically. Unfortunately, system administrators and IT technicians often make updates without first establishing their stability and performance. New does not always mean better, and there are circumstances when a new HBA driver or firmware version can actually impair SAN performance. It's important to identify new drivers and firmware updates, but be sure to test those updates in a lab environment. Quantify any differences in interoperability, performance and stability, then decide whether to roll out the updates to the corresponding HBAs in your environment.

Best Practice No. 6: Embrace NPIV for virtualization

N_Port ID virtualization (NPIV) allows a physical HBA port to be partitioned to support multiple virtual machines, enabling multiple virtual fabrics through the same physical HBAs and switches. It's not an essential feature for nonvirtualized environments. However, with many organizations moving toward server and storage virtualization, future HBA upgrades may require features like NPIV. Note: NPIV does not directly support VMware's file system at this time, though this should change as NPIV development continues.

Best Practice No. 7: Look ahead to FCoE

Storage industry leaders recently started an initiative intended to combine network protocols, dubbed Fibre Channel over Ethernet (FCoE). This new layer 2 networking standard allows Fibre Channel information to reside within the data payload of Ethernet packets. The FCoE standard should merge Fibre Channel, enhanced Ethernet and high-performance computing (HPC) clustered traffic within the data center. FCoE support will require new HBAs and switches and will likely rely upon a 10 Gbps Ethernet infrastructure. The formal proposal for FCoE is already in the hands of an ANSI T11 committee. FCoE standards and products should appear sometime in 2009. Users watching the development of FCoE may want to limit their SAN expansion plans until new FCoE HBAs and other components become available.

Here are seven best practices for HBA expansion or replacement.

Dig Deeper on SAN technology and arrays