What you will learn from this tip: Everything you need to know to buy the best switch for your storage environment. Need a downloadable version? Go here.
- In most cases, the switch manufacturer makes small switches (32 ports and below) and large switches (64 ports and above).
- Small switches cost less per port than large switches; it is harder to design a big switch than a small switch. But keep in mind that you lose ports when connecting switches together.
- Most SAN switches today are 2 Gbps.
Before you buy, ask these questions:
- How large a switch makes sense?
- How large a SAN can or should I build from small switches?
- For a small SAN, should I use a single large switch rather than a collection of small switches?
- For a large SAN, why would I use a core-edge design rather than core switches only?
- Know the major switch vendors. Who are the major switch vendors? Our product roundup on switches has information on all of the major switch vendors.
Bigger isn't always better
- When connecting 1 Gb switches together, the 8-, 16-, and 32- port switches can be readily networked without hitting performance problems and without having to worry about which devices are talking together. When we get to 1 Gb 64-port switches, it becomes very hard to design a SAN with significant amounts of data moving switch-to-switch.
- What about 2 Gb switches? Start from a simple fact -- a server with a 2 Gb HBA does not give two times the bandwidth of a 1 Gb HBA. It will be a bit faster, it will give more I/Os per second, but this can be attributed to the HBA being newer and more intelligent. Having made this assumption, you can interconnect switches of 64 and maybe 128 ports in a reasonable fashion using 2 Gb ISLs, particularly if there is good load balancing or trunking. However, even at 128 ports, start thinking about localizing traffic. Any larger and life gets very difficult.
- Why not make my switches even bigger? Things start to get really tough! The bigger the switch the more it will cost per port, assuming that it is a true single switch with non-congesting performance for any-to-any-port connectivity with all the ports running full speed. After all, servers do not/can not actually use all the bandwidth of 1 Gb, let alone 2 Gb ports. By definition you over-subscribed connections on the disk arrays, and so on.
Not all switches created equal
- If we look at the IP network world, we know not all switches are equal. We choose whether or not to pay more for a switch that runs more I/Os per second. In addition, there is a limit to how large most people are comfortable taking a single fabric. Without some way of splitting a SAN into separate subnets for manageability, big SANs can be challenging. They can be difficult to manage with the current state of management tools, they can lead to scalability concerns, etc.
- Unless you are lucky and have lots of structured optical cabling throughout your data center, then having one big switch in the middle of the room can be a cabling nightmare. Whereas, having a number of switches in different locations in the data center, we can consolidate the cabling, reduce cable complexity and have a more usable physical environment. While we talk about heterogeneous SANs, there can be advantages to some level of homogeneous design, such as having all the Microsoft servers connected to one switch, all the UNIX servers to another.
- Small switches cost less per port, so a core-edge design may well reduce the average-cost-per-user port. Depending on your environment, this may be more or less critical. In a Wintel environment, the cost of a Fibre Channel port as a proportion of the cost of the server may be quite high, whereas in a Unix environment this may be less of an issue. You could reuse small switches that have been purchased over the last few years. Even if you are starting to deploy SANs now, you may feel you want to start with smaller switches to dip your toe in the water. That being said, there are still some cases where SANs can be constructed using only large switches.
There is no one topology or approach that applies to everyone. Always keep in mind that this is a network. It will grow over time. When choosing a small or large switch, and a topology, think about the long-term implications. In any environment, you already have a lot of servers and storage, you probably know what types of servers you will typically purchase in the future and you can probably make a good guess as to what systems would make sense to incorporate in the SAN. Knowing these parts, you can fairly easily build out your SAN.
For more information:
Product roundup: Switches
Advice: What switches to use when implementing an iSCSI SAN
Advice: What is an intelligent switch?
About the author: Simon Gordon works for Nishan Systems. He has worked with storage area networking technology, particularly with Brocade-based SANs for over four years. Simon has been working in the IT industry 15 years in software development, systems integration, Unix and open systems, Microsoft infrastructure design and most recently storage area networking. Find out more about Simon at his Web site: http://homepage.ntlworld.com/simon_gordon.
This was first published in November 2004