Here comes 8Gig Fibre Channel

UNLIKE THE PREVIOUS Fibre Channel (FC) specification transition from 2Gb/sec to 4Gb/sec in 2004, there may be a legitimate enterprise need for 8Gb/sec FC.

"There are some select applications that require 8Gb/sec FC right now, like high-definition video," says Tim Lustig, solutions architect at QLogic Corp.

However, "the key driver for 8Gb/sec is virtualization and server consolidation," says Kyle Fitze, director of SAN marketing at Hewlett-Packard (HP) Co. Large companies are adopting virtualization to consolidate thousands of servers. Bob Gill, managing director, servers at TheInfoPro, a New York City-based research organization, says "95% of respondents [to our latest survey] state that virtualization is critical to achieving their business objectives."

Not like 4Gb/sec
When Storage first wrote about the transition from 2Gb/sec to 4Gb/sec FC, the big question was what organizations would do with the extra performance. At that time, almost no organization, regardless of size, was saturating its 2Gb/sec pipes, said Tony Asaro, then a senior analyst at Enterprise Strategy Group (ESG), Milford, MA.

The transition promised to be transparent and painless, and there was no price premium attached to 4Gb/sec components. At the time, Framingham, MA-based IDC Corp. predicted that after the industry began 4Gb/sec FC general shipments in 2005, it would achieve 90% market penetration within two years.

Today, 8Gb/sec FC products are being announced or released. By virtualizing three, four or more servers within a single physical server, organizations will generate considerably more I/O. "In that case, you'll need more ports and more bandwidth," says ESG analyst Bob Laliberte.

Virtualization can benefit from bigger network pipes, just as it benefits from increased memory. However, "even with virtualization, you're not automatically going to saturate your pipes," says Greg Schulz, founder and senior analyst at StorageIO Group in Stillwater, MN. Most of the apps virtualized aren't high I/O database transaction apps that would benefit from the performance kick of 8Gb/sec FC.

"We tested four applications on one processor and we hit an HBA [host bus adapter] bottleneck at 50,000 IOPS," says QLogic's Lustig. With 8Gb/sec, the HBA bottleneck could be pushed off until 200,000 IOPS.

End to end
At this point, 8Gb/sec FC is available for the server and switch components of the SAN. Here's the rub for the storage architect: "Behind the switch, you still have 4Gb/sec storage," notes Lustig. Even with HP, which has been actively cheerleading 8Gb/sec FC, the storage arrays remain at 4Gb/sec. "For complete 8Gb/sec throughput end to end, you'll need 8Gb/sec to the disk array," says the firm's Fitze.

There's usually a lag between when new technology components are introduced and when they're built into general-purpose storage arrays. HP expects to release an 8Gb/sec EVA array in 2009. Other vendors will probably follow in the same timeframe.

The industry will follow what it did with the previous FC speed transition; the 8Gb/sec components will automatically sense slower components and ratchet down the speed. As a result, the best an organization can do, end to end, today is 4Gb/sec--unless it wants to aggregate pairs of 4Gb/sec array ports.

Specialty storage vendor Facilis Technology Inc. demonstrated an end-to-end 8Gb/sec FC SAN for use in the high-resolution video industry in April using 8Gb/sec FC components from Atto Technology Inc. But like general-purpose storage vendors, Facilis is taking time to do the final engineering. "We're waiting for boards so we can do more testing and benchmarking," says Kathy Kane, director of business development.

New SAN offerings
Brocade Communications Systems Inc. recently announced the availability of 8Gb/sec FC switches and HBAs. Beyond speed, Brocade is promising to build in advanced capabilities for QoS and support for virtual server mobility through dynamic state migration of link and port personality profiles.

IBM Corp. has announced three new 8Gb/sec switches: IBM System Storage SAN24B-4, SAN40B-4 and SAN80B-4. Pricing starts at $5,360, and the switches have higher port counts (up to 24, 40 and 80 ports, respectively) and, according to IBM, consume less power than previous models. IBM is also releasing three new 8Gb/sec switch blades for the IBM TotalStorage SAN256B director. And Cisco Systems Inc. will offer a "transparent" upgrade to 8Gb/sec for its MDS 9500 Series directors in Q4.

				InfiniBand update
				InfiniBand is an industry standard, channel-based, switched-fabric interconnect protocol for servers and storage. Today, it offers three levels of performance: 2.5Gb/sec, 10Gb/sec and 30Gb/sec for low-latency, high-aggregate throughput. InfiniBand is deployed primarily between servers in clusters for the purposes of high performance or failover. It's positioned to be complementary to Fibre Channel and 10GbE. The InfiniBand Trade Association envisions storage networks connecting into the edge of the InfiniBand fabric to communicate with the high-performance compute resources found there. "InfiniBand is mainly for high-performance computing. It has limited opportunities in enterprise storage," says Mike Karp, senior analyst at Boulder-based Enterprise Management Associates. "How many protocols do you really want to support in your data center?"

Price premium
The move to 8Gb/sec FC will entail a price hike, at least initially. "The 8Gb/sec components require new optics. The price won't drop to 4Gb/sec levels until the volumes ramp up," says Scott McIntyre, VP of software and customer marketing at Emulex Corp. Adds Mario Blandini, Brocade's director of product marketing: "There will be a 20% to 30% price premium over 4Gb/sec components initially."

For the extra money, 8Gb/sec components will bring some new capabilities. "These involve how data corruption is handled and how you authenticate the host to the FC fabric," says McIntyre. For example, there'll be more Cyclic Redundancy Checks (usually a mathematical checksum), which detect data alteration during transmission or when stored by comparing the data stream going in and coming out.

One upshot of the transition to 8Gb/sec FC may be improved energy conservation. "You may be able to turn off lanes on the PCI bus and power off lanes in the HBA and ports," says QLogic's Lustig. These green capabilities, however, will come from new firmware and software, not from the 8Gb/sec HBA alone.

FCoE
Although 8Gb/sec FC doubles FC performance, it still comes up shy of 10Gigabit Ethernet (10GbE). However, FC enterprises can achieve 10GbE speed through Fibre Channel over Ethernet (FCoE). With FCoE, the FC protocol and FICON run native on a premium lossless, low-latency and QoS-enabled Ethernet. FC becomes just another network protocol running at 10Gb/sec alongside IP on the 10GbE network.

FCoE also allows convergence at the cabling level through a single Ethernet cable handling both FC and IP. FCoE will appeal to enterprises with FC SANs but with no plans to migrate to iSCSI and Ethernet. "It lets organizations with FC and FICON preserve their FC skills and tools," explains StorageIO Group's Schulz. They can converge their network to a fast 10GbE backbone and reap savings by running and maintaining only one set of cabling for all traffic. (For more, see "FCoE: Coming to a data center near you".)

Because FCoE doesn't use TCP/IP, it isn't routable. In addition, at this point FCoE is considered local technology only. "FC shops that want to follow strategies requiring long distance, such as remote mirroring, should look to FCIP [Fibre Channel over IP] or other protocols," says Schulz.

To use FCoE, organizations will have to deploy a Converged Network Adapter (CNA) that will look to the server as both an FC HBA and an Ethernet NIC. A number of vendors have declared support for FCoE, including switch makers like Brocade and Cisco and component makers like Emulex and QLogic. Despite industry support, "this technology is still in its infancy," adds Schulz. "It will be 2010 before FCoE is ready for use by anyone except early adopters." (See "Fibre Channel over Ethernet roadmap," PDF below.)

Click here for the Fibre Channel over Ethernet roadmap (PDF).

Beyond 8Gb/sec FC
The Fibre Channel Industry Association, which manages the FC roadmap (see "Fibre Channel roadmap," PDF below), is signaling the next performance jump to 16Gb/sec. Work on the 16Gb/sec FC specification is underway with completion slated for 2009. Actual products should be delivered by 2011. After that, the future gets murky with development of 32Gb/sec, 64Gb/sec and even 128Gb/sec FC dependent on market interest.

Click here for the Fibre Channel roadmap (PDF).

With storage arrays incorporating 8Gb/sec technology still months away, there isn't much for a storage manager to do right now. If an organization needs the performance of 8Gb/sec FC, it can begin upgrading to available switches and HBAs, or it can ponder a future jump to FCoE. When the time comes to finally make the move to end-to-end 8Gb/sec FC, prices should have dropped. Otherwise, sit tight.