Sane strategies for SAN growth


This article can also be found in the Premium Editorial Download "Storage magazine: Expanding SANs: How to scale today's storage networks."

Download it now to read this article plus other related content.

Different approaches
Currently, there's two common approaches to SAN expansion: one that can be thought of as "storage-centric" (see "Credit Valley Hospital runs out of room")

Requires Free Membership to View

and another that is "network-centric" (see "Building on the old SAN" and "Guardian Life swaps the new for the old"). Depending on the environment, choosing the right approach can have a significant impact on the scalability and cost of the SAN.

The storage-centric approach is seen most often in small (fewer than 50 ports) to midsize (50 to 100 port) SAN environments. Administrators typically purchase their new FC switches as part of a storage upgrade that also includes larger capacity storage arrays, additional connected hosts and the latest in tape drive technologies. These users expend minimal effort evaluating the different technologies available from the leading FC switch providers. Essentially, the storage systems are the primary purchase and the switch is treated as an ancillary component. In these cases, the switch recommendation of the selected storage vendor is followed with little further consideration. Because only one or two of the current director-class switches are needed to accommodate this number of nodes, the "Day 1" SAN design considerations are minimal.

In contrast, we have found that administrators of large SANs (greater than 100 ports or more than four switches per fabric) often take a network-centric approach. They evaluate, select and design their next generation SANs independently of additional server, storage and tape purchases. This is likely because their long-term requirements are less clear. They need to have the flexibility to address a wider range of possibilities. Some of the considerations that they must weigh include:

  • Cost vs. flexibility: The high Day 1 costs associated with scalable multiple-switch designs. This cost is frequently magnified by low Day 1 utilization of available ports (see "Merrill Lynch's three- year refresh").
  • Interoperability and heterogeneity: Interoperability between switches--even that of like manufacturers--is limited. For example, Brocade trunking isn't supported between the vendor's new model switches and its popular 2xxx series switches. Therefore, a significant consideration when planning a major upgrade frequently involves deciding whether the entire existing switch infrastructure needs to be replaced.
  • ISL bottlenecks: As the port count increases, so does the aggregate fabric I/O. More I/O increases the probability of ISL congestion.
  • Excessive hop counts: In poorly planned SAN configurations, expanding port counts often lead to an excessive number of hops between host and storage, significantly increasing latency. As a rule, the lower number of hops between the initiator HBA and target storage devices the better overall throughput.
  • Complex connectivity requirements: Larger SANs are more likely to have multiple targets for a single initiator. These targets include multiple storage arrays and tape devices. Greater care must be taken to ensure that configuration best practices such as vendor fan-out recommendations continue to be followed.
  • Extending beyond the data center: Geographical considerations introduce a host of issues, including distance limitations and extension costs, which can have a far-reaching impact on the usefulness of the SAN. Connection issues such as increased latency, the risk and impact of isolating part of a SAN and link state changes over a single, wide-area fabric can be limiting factors in SAN design.
Given the greater levels of complexity associated with larger SANs, administrators find that separating network from storage expansion allows them to better focus on these complexities and make wiser design decisions.

Technology for large SANs
If you're building a large SAN infrastructure, what technologies should be considered? Vendors have been listening to their largest customers and are adding features to switch offerings that can help to overcome a number of expansion hurdles. Here are some of the most important.

Cisco Systems Inc. was the first to introduce autonomous regions (ARs) into FC switches with its Virtual SAN (VSAN) feature. ARs create independent fabric services for each set of ports that are configured within an AR. Each AR is a logically independent SAN within a physical storage network. Changes made within one AR don't impact other ARs. For example, re-zoning is one of the riskiest configuration changes that can be made in a production fabric environment (due to its ability to impact every node on the fabric). With ARs, a zone change can now be limited only to ports within a specific AR.

Today's SAN administrator typically has full access to the entire fabric. Switches are beginning to offer multiple login groups with more granular access. When combined with autonomous regions, this allows a departmental administrator to be given access to make changes only in his own AR. This added level of security makes it easier to restrict the ability of one administrator to affect the entire SAN.

Early FC switches had inefficient algorithms for routing I/O across multiple ISLs. Previously it was done on a per connection basis, with all I/O between a single initiator and target combination traveling across a single ISL between two switches. If two connections had substantially different I/O loads, traffic would be unequally distributed across the available ISLs. The latest technologies permit striping at the frame level across multiple ISLs. This increases the overall utilization of available ISLs, reducing the number of required ISLs, which in turn results in more useable node ports.

One hundred-plus ports per switch are now common on the director-class switches. Vendors such as McData Corp. are already bringing to market 256-port devices and have roadmaps for 512-port devices. These higher port-count devices go a long way toward simplifying large SAN designs within the data center. The greater the number of ports per switch, the lower the number of ISLs required, resulting in a greater number of useable ports. In addition, there are fewer design considerations when a smaller number of high port-count switches are utilized.

The ability to have an IP blade option in a switch makes it easier to expand SANs across greater distances. Previously, specialized channel extension equipment was required to create a single fabric between two switches more than 10 km apart. Today, using a company's existing IP WAN infrastructure, fabrics can be more easily expanded across multiple, geographically separate locations. This is particularly useful for data replication.

The cost of scalability
Unfortunately, designing a SAN for scalability with minimal disruption in a production environment comes at a price. The high Day 1 cost can't be justified in many environments, despite advantages in manageability.

For example, consider the problem of designing a SAN to support a future goal of 2,500 connections. The classic three-tiered SAN design (see "A three-tiered switch SAN design model") would be one way to approach the problem. Consisting of a host tier, a connectivity tier and a storage tier, this configuration enables fabric scalability to a large number of nodes with minimal reconfiguration. A three-tiered SAN could be designed around two 140-port core switches supporting 20, 140-port edge switches (with 14 ISLs per edge switch).

However, if only a few hundred ports are required initially, the economics don't make sense. The Day 1 purchase requires the purchase of two core switches plus a minimum two edge switches. The resulting cost per port would be too expensive for most organizations. In fact, it's even worse when you double the number to support an independent redundant SAN, a recommended best practice and often a requirement in production environments.

The result is that many companies compromise with a less scalable two-tiered architecture (see "A two-tiered switch SAN design model"). This design supports nearly half as many nodes as the three-tiered architecture, but requires two switches per fabric to meet Day 1 requirements. While the two-tiered design can later be converted to three-tier, significant reconfiguration would be required, causing a major disruption. Redundant fabrics can mitigate some of this disruption, but the process of planning and executing this expansion would still be a considerable effort.

Looking ahead
As technologies mature and solutions evolve, many of today's SAN shortcomings will subside, and the trend toward larger SANs will continue (see "How big is too big?"). Just as the early adopters of SAN technology realized benefits over the previous technologies, further benefits will be realized by building larger networks.

So what should be done today to enable successful expansion in the future? If you aren't already doing so, start taking a network-centric approach to SAN design and purchasing decisions. Begin by thinking of SAN infrastructure independently of storage, and develop a set of requirements for SAN infrastructure that supports potential expansion plans. Logically decouple SAN purchases from storage purchases, and the new equipment should have the features you need. As much as is practical from a cost perspective (see "Are smaller switches an option?"), design for minimal disruption. Act now to position yourself to take advantage of it.

This was first published in November 2003

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: