Managing and protecting all enterprise data


Virtual SANs bring order to chaos

What will Cisco's embedded virtual SAN technology in its new MDS switch line mean to storage managers? For starters, a new way to manage SANs as they spread across the company.

Cisco has embedded virtual storage area network (VSAN) technology in its new Multilayer Director and Fabric Switch (MDS) line. This new technology will let you construct fabrics as collections of ports--not collections of switches. You can also segregate not only just traffic, but fabric services, between the virtual SANs.

Integrating VSANs into your fabric
An MDS switch can act exactly like a non-VSAN switch by using a single VSAN and using only E ports and ISLs to communicate with other switches. This single VSAN and its fabric would communicate with all the non-VSAN switches in exactly the same way they do and participate fully in whatever topology exists.

A VSAN switch could also have several VSANs and connect to multiple non-VSAN switches over multiple E ports and ISLs. It is possible that these non-VSAN switches could all have their own independent fabrics. In this case, the topology would look like a hub with spokes, where each spoke represented a unique fabric connection. It is conceivable that an MDS switch could be used to connect to several different fabrics and act as a centralized point for implementing changes to all attached fabrics. For example, an MDS switch connecting to four different non-MDS fabrics could be used to manage changes to zones in all four of these fabrics, relieving the administrator of managing zoning changes on four different administrative sessions.

The result is nothing less than a new way to plan, implement, operate and manage SANs. VSANs are destined to become one of the most important advances in SAN technology for many years.

VSANs shouldn't be confused with storage virtualization. Instead, VSANs are all about Fibre Channel (FC) network communications. While it's true that MDS switches have excellent IP integration options, the benefits of VSANs can be achieved in a homogeneous FC SAN.

The most powerful aspect of a FC fabric is the way it spans multiple network switches to create a consistent network context for all nodes in the SAN. In multiswitch--also called multielement--fabrics, fabric service information is relayed over interswitch links (ISLs) and aggregated in each participating switch to form a complete, replicated instance of the fabric.

With their port-level granularity, VSANs allow a single switch to have multiple fabrics. They also allow a fabric to combine some ports from several switches. Without VSANs, you can only add to a fabric by adding an entire switch. With this flexibility, VSANs could be assigned based on the management goals of the IT organization. For instance, different business units or server/application groups could be assigned to different VSANs and managed according to cost and service level targets.

With VSANs, IT organizations can architect storage networks with distinct SAN service groups and service levels. There's no reason to have a one-size-fits-all fabric for every node that connects to a switch or director when it's possible to have application- or server-specific fabrics for each group of communicating nodes. A fabric that's established for a specific purpose can operate without interruption from other changes made in the SAN.

Other VSANs operating in the same switch could be created, removed or have properties changed or be connected to remote SANs through FCIP gateways--all with minimal impact on the original transaction processing VSAN. To limit the scope of changes in the fabric, VSANs create a more stable environment for all SAN users.

Segregating traffic and services
VSANs go beyond FC's current traffic segregation mechanism--zoning--to include fabric services.

In the implementation in "VSANs enable more flexible fabric design" (this page), no traffic or fabric service information is exchanged for ports belonging to VSANs 1 and 2. Only one-third of the normal fabric service information is transferred over an enhanced interswitch link (more on that later). More bandwidth becomes available for storage I/O data transfers. In a local SAN this isn't so important, but in SANs connected over WANs, it could make a difference.

VSANs enable a more flexible fabric design

Here's how a SAN could be made from two 16-port switches that have three VSANs: VSAN 1, which is unique to switch A, VSAN 2, which is unique to switch B and VSAN 3, which has ports in both switches. The two switches are joined by an enhanced interswitch link (EISL), which carries traffic routed between the switches as well as all fabric service information needed to operate VSAN 3.

Current zoning technology is inadequate in an environment that doesn't tolerate errors well. Zoning is adequate as an optional connectivity service, but not robust enough to be an architectural element in SANs. It has no mandatory or default behaviors, which leads to confusing scenarios. For instance, a node that's not restricted by belonging to a zone should be able to communicate with all other nodes in the SAN, yet another node that does belong to a zone should supposedly only be able to communicate with other members of the same zone. This creates a rogue zone of non-zoned ports (the zombie zone) that's difficult to understand, troubleshoot and manage.

All ports in a SAN should be assigned to a zone to avoid confusion and configuration errors. In data management applications that depend on changing zone sets in a switch, the potential for confusion and error significantly increases. The burden of managing zones falls on administrators with relatively few tools to verify that zoning is applied correctly. Mistakes can happen, even with highly skilled staff.

In Cisco's implementation, every port must belong to a VSAN. All ports are assigned to a default VSAN unless they are explicitly assigned to another VSAN. There's no way to have undefined membership status.

Fabric service isolation
Each service in each VSAN fabric is a separate software process tracked individually by the switch's operating system. You can identify services that have stopped running and attempt to restart them, which makes it less likely that faults will ripple over to other fabrics and switches. Let's say a transaction processing system is connected to its storage through an MDS switch and isolated on its own VSAN. Now suppose that a service failure occurs with a zoning service in another VSAN fabric. The fabric that experiences the zoning service failure might encounter some difficulties, but the VSAN used for the transaction processing system continues to run without interruption or loss of functionality.

Each switch has at least one VSAN running and can actively participate in as many as 1024. The total number of possible VSAN IDs is 4096, starting at VSAN 0 and continuing to VSAN 4095. This fabric address space can simplify management in an enterprise SAN environment with many different entities using SANs. Each business entity could be assigned different VSAN IDs or ranges of IDs.

Consider two manufacturing plants located in different cities. Let's say they both create SANs and use the VSAN IDs 20 and 30 for their material requirements planning systems. A year later, they decide to mirror SAN data between the two locations. At this point, they could either establish a new VSAN ID for mirroring or change one of the VSAN IDs at one of the locations to create a common VSAN ID. With so many VSAN IDs available to the SAN administration team, there are many ways to get the job done with minimal impact on the operations of other VSANs running in the organization.

Each VSAN has its own zoning service that supports multiple zones and interchangeable zone sets. Nesting zones within VSANs give administrators a hierarchical method of creating safe methods for dynamic data access requirements. For instance, two servers could share a disk subsystem through mirroring and snapshot where one server writes the data to a subsystem and the other accesses it for testing a new application. Both servers could belong to the same VSAN and make use of changing zone sets to ensure no access mistakes occur when creating the snapshot or testing the data. Any problems would be contained to this VSAN and not affect the operations of other storage I/O processes in other VSANs.

Work that still needs to be done
While VSAN technology offers significant management capabilities that haven't been available before, there's still room for improvement in the future. One area that needs to be addressed is VSAN membership by port, as opposed to by some other identifier, such as Fibre Channel (FC) worldwide names (WWNs) that identify the SAN node uniquely. Another area to be developed is administrative restrictions based on VSANs.
The most obvious holes in the first edition of VSANs is that membership is based on the physical port used and allows membership based on WWN or other possible identifiers. The situation is reminiscent of the development of Ethernet VLANs, where membership by port was introduced prior to other methods that could recognize the end node such as Ethernet MAC (media access control) addresses.
The problem with port-based membership is cables can be moved from one port to another, potentially changing the VSAN that the node belongs to and circumventing the isolation scheme. Clearly, VSANs will need to have the ability to recognize WWNs in order to reach their potential. I expect Cisco will address this shortcoming soon and it is undoubtedly aware of the requirement, having done virtually the same thing in the Ethernet/IP world previously.
MDS switches have multilevel role-based security enabling a master administrator to restrict the actions and views of other administrators. However, in the initial release of the product this doesn't include restricting administrators according to the VSANs they control. In other words, an administrator may be able to start and stop specific services in the switch, but there's nothing preventing them from doing so for any of the VSANs. This isn't quite the level of isolation that some SAN users demand--such as DBAs--who might feel exposed if an administrator from another department could impact their operations. This doesn't seem like the sort of improvement that should be very far out of reach for Cisco, but it's something that needs to be addressed for the technology to thrive.
The concepts embodied in FC's class of service haven't panned out because it's so expensive to provide the delivery guarantees specified in Class 1 service. FC SANs have generally defaulted to using Class 3 service--a best effort datagram service without any delivery guarantees.
However, just because class of service hasn't materialized, it doesn't mean that QoS in SANs isn't needed. Cisco plans to release VSAN QoS prioritization soon after the initial availability of the product. Like the rest of VSAN technology, the VSAN prioritization scheme was borrowed from the 802.1p standard used in Ethernet VLANs where a three-bit field determines priority.

VSAN routing
In most SAN switches, there's a single routing table shared by all ports in the switch. With VSANs, there are individual fabric shortest path first (FSPF) routing tables for each VSAN. The routing tables in SANs are small, but with enterprise SANs they grow and could affect switch latency.

One of the advantages of VSANs is they can each have their own weighting schemes applied to them for route selection. FC's FSPF routing mechanism provides a way to influence the route used by assigning different weights to different network routes. This provides a mechanism for failing over to a secondary route if the first route fails. By virtue of having their own independent routing tables, different VSANs in a switch can be directed to use different routes that may have different characteristics. The usefulness of VSAN weighted routing becomes more obvious when combined with MDS port channeling.

Port channeling in MDS switches
Port channeling is a technology for aggregating ISLs in MDS switches. It isn't a part of VSAN technology, per se, but VSANs take advantage of it. With port channeling, several ISLs can be combined to form a virtual link with much higher bandwidth and better reliability than a single link. For instance, a port channel made up of three ISLs has approximately three times the bandwidth and reliability of a single ISL.

Port channels have their FSPF weighting automatically adjusted to reflect the higher bandwidth available. Without establishing special weight values, routes with port channels in them will be selected over routes without them by the FSPF.

By changing weighting for VSANs you can assign weights to servers and applications that give them preferential route selection over less-important or time-sensitive applications and servers. A company could have separate VSANs for their transaction processing applications, all of them weighted to use a port channel comprised of four physical ISLs.

Less time-critical application servers could be placed in VSANs that are weighted in the opposite fashion to avoid using the port channel and offer more economical single link ISLs. If their ISLs failed, they could be switched over to use the port channel ISL.

Port channels can be formed from any combination of ports in an MDS switch and don't have to be formed from ports belonging to the same communication module. This adds an element of reliability as the port channel doesn't have to depend on the availability of any single module for functionality.

VSAN trunking
VSAN communications between switches using frames with VSAN tags are transmitted over links called enhanced ISLs, or EISLs. In Cisco's nomenclature, EISL links are considered trunking links. A single trunking EISL can be used as part of a SAN route or multiple trunking EISLs can be combined to form a port channel.

As FC has well-defined port types, Cisco had to invent a new type of E port, called a trunking E port (TE) to carry VSAN tagged frames. A TE port only communicates with other TE ports. For use with other vendors' switches, an MDS port can be configured to operate as a standard E port. MDS switches have a number of ways to configure port functions in a switch, including auto-detection--TE-only, E-only and F-only--to ensure mistakes don't occur when connecting switches to other switches, servers and storage subsystems.

An MDS switch can have both TE and E ports running simultaneously. An EISL can support multiple VSANs simultaneously because the TE ports have the ability to interpret VSAN tags. However, an ISL link doesn't carry VSAN tags, so it's not possible to have VSAN trunking. That means an MDS switch is restricted to mapping a single VSAN over a non-trunking ISL. The fabric for this VSAN becomes part of the fabric running in the other switch.

It will be interesting to see if Cisco attempts to create a standard around their VSAN technology, as they have with so many other networking technologies. I assume they will, but that assumption is based purely on wishful thinking for the SAN industry. The sooner VSANs are made into an industry standard, the better off the whole industry will be, including all the other FC switch vendors. The adoption of Cisco's VSAN is the shortest, most efficient path to moving to a new level of functionality.

Article 4 of 21

Dig Deeper on Storage for virtual environments

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Storage

Access to all of our back issues View All