This article can also be found in the Premium Editorial Download "Storage magazine: Lessons learned from creating and managing a scalable SAN."
Download it now to read this article plus other related content.
Switches and directors
FC switches are a key component of SANs, connecting initiators like servers with SAN targets (arrays and tape libraries). With FC switches available from a range of vendors, including Brocade, Cisco, Emulex Corp., McData Corp. and QLogic Corp., the selection of an appropriately sized switch should be driven by your current needs, growth expectations, and redundancy and performance requirements.
|Tiered SAN design|
In a tiered SAN design, servers are connected to one tier of the SAN fabric while storage is connected to another tier.
First, determine the number of currently needed FC ports and then project the number of ports that will be needed within the next couple of years. "Typically, we see companies size their infrastructure up to a factor of two of today's needs. Sizing infrastructure beyond a factor of two is expensive and, in most cases, uneconomical," says Gartner's Opfer.
Combine the number of required ports with the best-practice guideline of limiting the number of switches to as few as possible and you'll get a rough idea of what type of switch to consider. Generally, if the number of switch ports you need is more than 64, you should seriously consider a director-class switch. While directors are expensive, their passive backplane, redundant hardware components, variable port-count FC blades (that can operate in 1Gb/sec, 2Gb/sec or 4Gb/sec FC modes), and aggregate backplane data throughput capacity of more than 1TB makes FC directors the ideal switching workhorse, especially for the SAN core. Besides HA, the chassis form factor of directors provides scalability by simply adding FC blades. For instance, Cisco's MDS 9513 will scale to 528 ports if fully populated with 48-port FC blades.
For port-count requirements of fewer than 64 ports, including some room for growth, director-level switches are overkill and in most cases not affordable. Although most stackable FC switches are available only with up to 32 ports, Brocade's SilkWorm 4900, and QLogic's SANbox 5200 and 5600 scale up to 64 ports. In fact, Opfer sees an increasing demand for high port-count switches in the future. "With both switches and directors deployed in pairs, the high-availability design of directors seems like overkill," comments Opfer. "High port-count switches are especially attractive for small to midsized companies that can't afford expensive directors."
One of the drawbacks of 1U and 2U form factor switches is their fixed port count. The key feature to look for in stackable switches is port scalability within the switch. In other words, does a switch support activating additional ports by procuring additional licenses in the future, or do all ports have to be purchased outright? While both Brocade and McData support ports-on-demand in all their switches, Cisco and QLogic typically don't. Ports-on-demand lets you pay for ports when they're needed rather than paying for idle ports that might be used in the future.
A well-designed SAN will take advantage of port oversubscription as a tool to balance performance requirements with cost. Oversubscription relies on the fact that not all ports within the same port group are fully utilized at the same time. For instance, Brocade's SilkWorm 48000 director will only operate at full line-rate speed on all ports when using 16-port blades, but it's oversubscribed 2:1 if 32-port FC blades are used.
Similarly, the Cisco MDS 9513 director with 48 4Gb FC ports is oversubscribed 4:1. "Compromises like oversubscription make a SAN somewhat more complex, as it becomes more difficult to predict the available bandwidth per port," says Mario Blandini, Brocade's product marketing manager. From a storage design perspective, ISL links and performance-sensitive servers shouldn't be connected to ports that are part of an oversubscribed port group.
Another tool for balancing cost with performance requirements is harnessing quality of service (QoS) to achieve defined performance goals. For instance, a port within an oversubscribed port group can be configured using port bandwidth reservation to ensure that it will always get the reserved bandwidth. It's also conceivable to operate a blade at a lower FC port rate. For example, a Cisco 48-port blade in the MDS 9513 director will operate at full line speed at 1Gb/sec FC mode. Furthermore, switch features like traffic classification and zone-based QoS, as well as the ability to assign more credits to certain switch ports, provide storage architects with the ability to assign guaranteed capacity where required.
To grow SANs horizontally, most switches and all directors provide the ability to trunk multiple ISL ports into a single data-aggregation pipe. For instance, Brocade supports trunking of as many as eight ISLs into a single 32Gb/sec trunk group, balancing data loads across trunk groups using a combination of standard route balancing via the fabric shortest path first (FSPF) protocol and optimized vendor-specific techniques such as Brocade's Dynamic Path Selection (DPS) protocol.
This was first published in July 2006