8 Gbps Fibre Channel storage: Use arrays, switches and server for gradual move to 8 Gbps

Learn why upgrading your fabric switch and core switch, server, host bus adapter and cables can help you gradually move toward 8 Gbps Fibre Channel storage.

IT shops interested in advancing to 8 Gbps, the latest iteration of Fibre Channel (FC) storage networking technology, won't need to do an instant overhaul of their entire infrastructure. Instead, they'll likely shift to 8 Gbps Fibre Channel storage as they update their servers, switches and/or disk arrays because the components can automatically negotiate down to the slowest speed of the equipment they're using.

One of the biggest considerations with 8 Gbps when products started shipping more than two years ago was the price differential with 4 Gbps. Now that the components are nearing cost parity, there's little reason for Fibre Channel customers not to go with the higher bandwidth as they refresh the pieces of their infrastructure.

Once an IT shop has new equipment in place from end to end, 8 Gbps Fibre Channel technology can help to improve response time and increase throughput, especially with I/O- or bandwidth-intensive applications such as backup, data analysis and server virtualization.

Storage arrays are usually the final component to add support for the latest storage networking technology, but all of the major vendors now offer support for 8 Gbps.

EMC Corp., for instance, supports 8 Gbps in its Symmetrix V-Max, Clariion and most of its Celerra models, although it continues to give customers the choice of 8 Gbps or 4 Gbps.

Fibre Channel storage networking vendors are planning to start rolling out 16 Gbps products for next year, and they're already working on a standard for 32 Gbps Fibre Channel.

For the latest 8 Gbps Fibre Channel storage networking technology, here's what you'll need:

Switches: Fabric switch and core switch

New switches might be the first purchase an IT shop makes in its move to 8 Gbps Fibre Channel, especially now that the price differential isn't a major obstacle. Brocade Communications Systems Inc., Cisco Systems Inc. and QLogic Corp. all supply 8 Gbps fabric and core switches, although users typically buy them through resellers or storage vendors.

The per-port cost gap between an 8 Gbps and a 4 Gbps FC fabric, or fixed configuration, switch was $62 in the first quarter, according to Dell'Oro Group Inc., a Redwood City, Calif.-based market research company that tracks the networking and telecommunications industries. The per-port delta for 8 Gbps vs. 4 Gbps core, or director-class, switches was only $36 during the same period.

Although Dell'Oro Group doesn't track end-user pricing, the data illustrates the general trend in switch prices. The average per-port price paid to manufacturers for an 8 Gbps fabric switch declined from $265 a year ago to $214 in the first quarter, while the average per-port cost for a core switch dropped from $595 to $493.

Fibre Channel shops that want the full 8 Gbps speed on all ports need to think about their level of comfort with oversubscription as they weigh the largest of the chassis-based director-class switches from Brocade and Cisco.

Similar to the way an airline overbooks a plane on the likelihood that some passengers will not make the flight, oversubscription is the practice of allotting more potential demand than a switch can handle on the assumption that every port won't need to use all of its bandwidth at the same time.

Cisco's MDS 9513 Multilayer Director appears to rely more heavily on oversubscription than Brocade's 384-port DCX Backbone core switch does. Although all 528 ports on the MDS 9513 are capable of 8 Gbps, only 88 of the ports can simultaneously deliver the full 8 Gbps.

"There is no huge demand for 8 Gbps full rate on all the ports. We haven't seen a single customer of ours require more bandwidth than we can deliver," said Paolo Perazzo, a senior product line manager for Cisco data center solutions. Cisco does note that its new MDS 9148 Multilayer Fabric Switch can run at 8 Gbps with no oversubscription.

The Brocade DCX Backbone is available in eight-slot and four-slot versions, with 256 Gbps of bandwidth available per slot. The company recently announced a 64-port blade to go with its 16-, 32- and 48-port models, raising the maximum port count per Brocade DCX chassis to 512. Although Brocade claims a customer using 384 ports, for instance, on its eight-slot DCX Backbone that can run all ports at full 8 Gbps, it acknowledges the scenario assumes a certain degree of local switching.

Erik Pounds, a product marketing manager at Brocade, explained how it works via e-mail: "If the traffic enters one blade and exits another, it needs to travel across the core switching blade and will consume some of the 256 Gbps slot bandwidth. If the traffic enters and exits the same blade, local switching is used and does not consume any of the slot bandwidth. If the network is architected to use just a small amount of local switching, non-oversubscribed 8 Gbps performance is obtained on all ports."

The bottom line is that Brocade's core switches are lightly oversubscribed, or not oversubscribed at all, whereas Cisco's core switches are more heavily oversubscribed, according to Robert Passmore, a research vice president at Stamford, Conn.-based Gartner Inc. "That means the Brocade [switches] have significantly higher internal bandwidth."

Brocade was first to market with an 8 Gbps blade for its 48000 Director in late 2007 and followed up in early 2008 with the 8 Gbps DCX Backbone, which customers had the option to purchase with 4 Gbps small form-factor pluggable ( SFP) transceivers. Those that bought the Brocade DCX with 4 Gbps optics merely need to replace the SFPs to move to 8 Gbps, according to Brocade. Otherwise, Brocade customers need to buy new switches to upgrade to 8 Gbps Fibre Channel.

Scott Shimomura, a product manager at Brocade, acknowledged it's generally a forklift upgrade for customers on 1 Gbps, 2 Gbps or 4 Gbps switch or director platforms. Upgrading to 8 Gbps with Cisco's MDS Multilayer Director requires changing the line cards and shifting to the new Cisco MDS 9513 CrossBar Switching Fabric2 Module if the customer had a Fabric1 Module. The Fabric1 Module started shipping as the default in 2006, and the Fabric2 Module became the default in 2008. Fabric modules are located at the back of the chassis, so no re-cabling is required to upgrade it, according to Cisco.

Server and host bus adapter upgrades

IT shops typically move to 8 Gbps Fibre Channel storage networking in tandem with the purchase of a new server. For many, those new servers are equipped with Intel Corp.'s Xeon 5500 processors (formerly code-named Nehalem), which enable higher I/O throughput and more virtual machines (VMs) per server.

Users need a new host bus adapter (HBA) with 8 Gbps Fibre Channel, and they generally get them from their server vendors or resellers. The major HBA manufacturers are Brocade, Emulex Inc. and QLogic.

The per-port prices paid to manufacturers for an 8 Gbps HBA have plummeted from an average of $419 in the first quarter of 2009 to $235 in the same time period this year, according to Dell'Oro Group. The statistics reflect the per-port price that OEMs and resellers paid for a mix of single-port, dual-port and quad-port HBAs, noted Tam Dell'Oro, founder and president of Dell'Oro Group.

The $235 average per-port price for an 8 Gbps HBA is nearing parity with the $215 HBA manufacturers received on average for 4 Gbps HBAs during the first quarter, Dell'Oro noted.

Emulex's current manufacturer's suggested retail price (MSRP) guide lists an 8 Gbps enterprise dual-port Emulex HBA at $1,785 and a 4 Gbps dual-port HBA at $1,700. QLogic noted the MSRP for its 8 Gbps dual-port QLogic HBA at $2,598 and a comparable 4 Gbps dual-port HBA at $2,474.

Cables and connectors

Fibre Channel users can keep their existing fiber-optic cables as they upgrade to 8 Gbps, but at that speed those cables work over shorter distances than they do at 4 Gbps. For instance, orange OM2 fiber-optic cables support distances up to 150 meters at 4 Gbps but only 50 meters at 8 Gbps; aqua OM3 fiber-optic cables support 380 meters at 4 Gbps and 150 meters at 8 Gbps.

"If you're doing OM3 cabling already, you're in good shape," said Stuart Miniman, principal research contributor at The Wikibon Project. "If you were OM2 before, you might want to upgrade because your distances are going to be dropping significantly."

Moving to 8 Gbps Fibre Channel also requires replacing the SFP transceivers used with 4 Gbps Fibre Channel with SFP+ transceivers. Brocade's product literature notes that customers can use either 4 Gbps SFPs or 8 Gbps SFP+/optics with its latest 8 Gbps SAN switches and upgrade to the latest SFP+ as they're ready.

"Most of the switch manufacturers went to that newer [SFP+] form factor because if we can have all the same optics, it's cheaper," Miniman explained, noting that SFP+, unlike SFP, can be used with both copper and fiber-optic cables.

Users need to be careful about which SFP/SFP+ transceivers they use because some might work only with products from their switch or HBA vendor, according to Dennis Martin, president of Demartek LLC, an analyst firm that operates its own test lab. Brocade's 8 Gbps SAN switch installation manual, for instance, warns that the switches require Brocade-branded SFPs. QLogic is the only switch vendor "that does not lock you into its own brand of optics," according to Marty Holmes, senior product marketing manager in the company's network solutions group.

Editor's Picks: Read the following articles for more information on Fibre Channel storage.

8 Gbps Fibre Channel promises bandwidth and data storage port boost

iSCSI storage vs. Fibre Channel storage: A SAN tutorial

8 Gbps FC: Price driving adoption of the storage technology

Dig Deeper on SAN technology and arrays

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.