One of the key decision points for any organization building a SAN is the switch. A business considering a SAN for the first time might consider an iSCSI option, which can make use of the existing Ethernet switching and IP infrastructure. But most companies that already have deployed a SAN have a substantial investment in Fibre Channel switches to connect their servers and storage devices.
Fibre Channel SAN switches serve the same general purpose as any other network switches: automatically connecting senders and receivers. The switch inspects a data packet, determines where it came from and where it's going, then forwards the packet to the intended storage destination in the data center.
But Fibre Channel switches were designed specifically to handle heavy transactions loads over high-performance Fibre Channel networks. These switches vary by port count, performance, architecture and price. Brocade Communications Systems Inc. is the market leader for Fibre Channel switches, followed by Cisco Systems Inc. QLogic Corp. runs a distant third.
There are two main types of Fibre Channel SAN switches: the high port-count, modular director and the smaller fixed-port or semi-modular switch, alternately known as a basic switch, fabric switch or just a switch. Most basic switches are fixed-port, and the port count can range from eight to 80. A small business might start by buying eight of the 24 ports on a low-end, fixed-port Fibre Channel switch
Below its director class, Cisco offers what it calls a "modular fabric switch" with a fixed first slot and an open slot that allows users to expand their SAN environments with additional Fibre Channel ports or intelligent services. Mark Allen, manager of storage technical marketing at Cisco, calls this switch a "middle-sector" product designed for SMBs or small data centers.
Like the more expensive directors, the basic switches can handle enterprise-class applications. Some midrange switches offer dual power supplies. But the basic switch has only one CPU, and because it's not modular, the whole switch needs to be replaced if an active component fails.
Basic switch ports start at around $500 or less; per-port costs can double or go even higher for director-class switches. Users typically gravitate to directors as their SANs grow.
Director-class switches are modular chassis-based systems with slots into which line cards or Fibre Channel blades with ports can be plugged. The slots allow users to choose different connectivity options. The blades look much the same as basic switches, but when plugged into the director, multiple blades can create much greater port counts.
Directors technically start at 16 ports, but they're generally used in much higher port-count scenarios. For example, QLogic's high-end model has 16 to 128 ports in a single 4U chassis. Brocade sells 16-, 32- and 48-port blades, and its high-end director scales to 384 ports. Cisco supports up to 528 ports in a single chassis.
Director-class switches have no single points of failure in the chassis; redundant components include power supplies, fans, control processors and core switching. If there's a problem with the CPU in one side, it can be replaced without any system downtime. If a blade fails, only that blade needs to be replaced.
"As you get into the director class, you can't afford to have them go down, so there are all sorts of redundancies built in, which makes the cost go up," says Bob Laliberte, an analyst at Enterprise Strategy Group.
The director's bladed architecture allows for adding intelligent services, such as storage virtualization, storage data mobility, data encryption and continuous data protection. Brocade refers to the add-ons as specialty blades,while Cisco calls them service or line cards or modules.
"That's where the future is, adding more of those value-added services in the network layer," says Dan Iacono, a senior SAN systems engineer in multivendor systems engineering at Hewlett-Packard Co. He says that the switch is a good place to build in more intelligence since it lets users reduce the hardware in their environments. He adds, "When it's embedded in the switch, you get more throughput."
The advent of director-class switches led to larger SANs, and as SANs expanded, they required more planning. One of the more common architectural approaches is a core/edge design, in which smaller basic switches are positioned at the edge of the network and the bigger director-class switches are at the core. Large enterprise environments sometimes use modular switches at both the edge and the core.
A high-capacity backbone switch at the core can provide even more potential for growth. Brocade claims its DCX Backbone delivers four times the slot bandwidth of its SAN directors and all 384 ports at full 8 Gbps speed. The product's interchassis link lets users double the port count from 384 to 768. The modular chassis provides flexibility to add more robust specialty blades than are available in the director class.
Another point of differentiation is the product's support for the emerging Converged Enhanced Ethernet (CEE) standard and Fibre Channel over Ethernet (FCoE) protocol, based on 10 Gigabit Ethernet. "You just add a blade that will support FCOE/CEE capabilities," says Bill Dunmire, a senior product marketing manager at Brocade. The option will not be available in the director-class switches, he says, noting, "This is an optional capability for certain types of large environments. All customers environments are not going to move to FCoE/CEE."
Fibre Channel was designed to handle heavy transaction loads reliably with low latency and no data loss. Ethernet, in contrast, can drop data packets. High-overhead TCP/IP is used to compensate Ethernet's shortcomings, providing for retries and acknowledgements and flow control. The industry-led CEE initiative – or what Cisco prefers to call Data Center Ethernet (DCE) – aims to bring Ethernet to the level of reliability of Fibre Channel.
But the industry's efforts to enhance Ethernet for data center use aren't expected to be finalized until next year, and products supporting it may need a year or two more to mature. "Assuming it really takes off, I'd be surprised if, by 2015, we had had half the data centers" using switches that support enhanced Ethernet, says Robert Passmore, an analyst at Gartner Inc.
Howard Goldstein, president of Howard Goldstein Associates Inc., doesn't think FCoE is a good investment. According to Goldstein, "iSCSI over TCP over IP over Ethernet is a better model, because the services are not married to hardware."
Cisco claims that its Nexus 7000 Series modular switching system already can support DCE, since the switch's fabric is lossless. Cisco's Web site notes that the product will need new line card modules and an upgrade to its NX-OS Software, and that most current Ethernet switches will require new hardware and software upgrades, since several of the DCE extensions are enabled in hardware. Cisco's Nexus 5000 Series switches also support DCE.
The other trend with Fibre Channel SAN switches is the slow shift from 4 Gbps to 8 Gbps. Since most applications don't even use 2 Gbps of bandwidth, many users see no pressing reason to rush to update their switches.
Companies with high-end data mining and video editing applications might benefit from the shift to 8 Gbps, as might firms with virtual server deployments. Users of server virtualization technology find that formerly underutilized servers are now running multiple applications in virtual machines on a single physical box,and their bandwidth needs in turn increase.
"We're in the early phases of migration from 4 gig Fibre Chanel to 8 gig Fibre Channel," says Passmore. "What customers generally do is, when their switches come to end of life and need replacement, they buy the highest speed available."
This was first published in December 2008