Tip

Maximize locality when setting up a SAN

If the three guiding principles in real estate are 'location, location, location,', the guiding principles in designing a SAN are 'locality, locality, locality.' Because high locality produces significant cost savings and performance improvements, it is worth setting up a SAN to maximize locality.

Locality refers to localizing SAN traffic on a single switch or in one area of a SAN. It is not the same as topology, which refers to how the servers, switches and storage devices are connected together. Instead locality measures how the traffic flows over the SAN's topology.

The guiding principle of locality is that as much as possible all the storage traffic between a server or a group of servers should flow through only one switch. Obviously this requirement is automatically met in simple SANs having only one switch. Things get more complicated, and potentially a lot less efficient, when the SAN has a fabric of several switches.

Non-locality costs in multi-switch fabrics because each Inter-Switch Link (ISL) between switches takes at least two ports (one on each switch) and ports are usually the critical resource in a SAN fabric. Since each device on one switch that communicates with a device on the other takes an ISL, a traffic pattern that has four devices communicating across switches takes four ISLs. If locality isn't considered in laying out the SAN's traffic patterns, much of each switch's capacity can be eaten up by ISLs.

Locality considerations

Requires Free Membership to View

become more important as the complexity of the SAN grows. Finding the best arrangement of servers and storage is fairly easy in a two-switch fabric, but it becomes more difficult as the number of switches grows to ten or 20 or more. A core-edge topology can ease the problem by providing central switches or directors at the core of the fabric to switch between devices connected to switches as the edge of the fabric.

The key to maximizing locality is analyzing traffic patterns across the SAN. The SAN design team should consider carefully how to group the storage devices and servers to keep them in the as logically close as possible, even though they may be separated by physical distance. If the SAN is replacing systems with locally allocated storage the relation between existing servers and storage devices can provide a pattern for connecting them via SAN. In general you want to keep the storage now connected directly to a server on the same switch as the server in the SAN. In the event of a completely new system it is usually easy to allocate storage to servers on the same switches or in the same part of the SAN. This gets trickier when the design becomes more complex, as when the SAN incorporates existing devices as well as a substantial number of new ones. Often setting up a spreadsheet will help the designers visualize the relationships between the servers and storage and aid in keeping as much traffic local as possible.

Locality isn't just a matter of SAN design. SAN administration plays a vital role in maintaining locality by allocating storage and servers in such a way as to keep them on the same switch or in the same area as much as possible. This is a balancing act that tends to change over time as storage requirements grow and more servers are added. The need to maintain locality becomes another factor to consider in allocating storage or adding servers. It can act as a constraint on other factors, such as maximizing storage utilization. If the under-utilized storage device is on a different switch that the servers needing more storage, it may actually make more sense to accept the under-utilization and add a storage array to the switch with the servers than to try to use the existing, under-utilized storage and have to add more ISLs, and possibly additional switches, to carry the load.

Brocade Communication Systems Inc. discusses locality and its importance in a white paper titled "Locality In SAN Design", which is available at the company's Web site at: www.brocade.com.


Rick Cook has been writing about mass storage since the days when the term meant an 80K floppy disk. The computers he learned on used ferrite cores and magnetic drums. For the last twenty years he has been a freelance writer specializing in storage and other computer issues


This was first published in January 2003

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.