This article can also be found in the Premium Editorial Download "Storage magazine: Inside the new Symmetrix DMX model offerings."
Download it now to read this article plus other related content.
|Downsides of switch virtualization|
The virtualization operator
A concept that can help understand how virtualization can work in switches is the virtualization operator. The virtualization operator is the logic that masks the details of storage I/O from the host--or some other upstream operator. Virtualization operators are storing-level functions located in the I/O path that manipulate the address spaces of the devices, subsystems and virtual devices that are downstream. From another perspective, the virtualization operator exports virtual devices to upstream I/O requestors (see "Virtualization operator").
Of course, there can be multiple virtualization operators in sequence along the I/O path. For example, a virtualization operator in a disk subsystem controller can provide one type of virtualization and a virtualization operator in host volume management software can provide the same or a different favor of virtualization. See "Multilayered virtualization scheme," which shows a host volume manager and several downstream disk subsystems. One way to think about multilayered virtualization is that the I/O path fans out into multiple downstream paths after it is processed by the virtualization operator.
Storing functions in SAN switches
Now it's time to examine how virtual storage techniques can be implemented in SAN switches. In general, this involves incorporating a virtualization operator for certain I/O paths that pass through the switch.
The first order of business for virtualization in a SAN switch is finding a way to identify the target virtual devices created by the switch's virtualization operator. Virtual storage devices in SAN switches need to be configured by an administrator. The storage administrator must first select the form of virtualization for the virtual device, and then discover the downstream target storage IDs that will comprise the virtual device. The next step is to create the exported target ID of the new virtual device, and assign this new virtual device to a zone or virtual network. Once the virtual device and its exported IDs are created, they can be accessed by any initiators in the SAN that have access to it.
The virtualization operator must have a processor to run on. There are many ways this could be implemented in SAN switches with a wide variety of choices for processors. For instance, the virtualization operator could run on a processor that the switch has for providing other services. It could also run on its own dedicated processor, which may be on a separate adapter or blade. Another alternative is to have a separate system work in tandem with the switch to provide the virtualization function. While this wouldn't necessarily be switch-based virtualization, there are possible implementations where the distinction would be splitting hairs.
Downstream storage IDs
The whole idea of virtual storage is based on making some number of storage entities or address spaces appear to be something different, or virtual. In the case of switch-based virtualization, the storage entities being virtualized are some type of SAN-resident storage. In the remainder of this article, I refer to this as SAN-resident and virtualized storage as downstream storage IDs. "Downstream" in this case doesn't necessarily imply a physical or even a relative location in the network topology. Instead, downstream here is intended to refer to the relative location on the I/O path. Entities closer to the host are upstream and entities closer to storage are downstream.
A close analysis of the virtualization process begins by understanding what happens when an incoming I/O command from an initiator is addressed to the switch's exported target virtual device ID. The switch port identifies the egress port for this virtual storage ID as the virtualization operator, then queues the command for processing by it. This doesn't have to be an actual egress port, and depends entirely on the design of the switch and its method for integrating virtual storage functions.
|Multilayered virtualization scheme|
The I/O path fans out into multiple downstream paths.
The virtualization operator then will read the entire command and process it according to the form of virtualization being used for this virtual device. There's a subtle, but important point: The initial I/O command is terminated by the virtualization operator and reinitiated it to downstream storage IDs. In other words, the initial I/O no longer exists, but has been reconstituted and retransmitted to a number of downstream IDs, through some combination of egress ports in the switch. See "Reinitiating an I/O command," which illustrates the termination and reinitialization process.
This terminating and reinitiating of I/Os isn't necessarily a bad thing, but it's important to understand the implications. First, this process adds latency to all I/Os addressed to virtual devices. Queuing, terminating, processing and reinitiating the downstream I/Os will be slower than simply forwarding an I/O through to an egress port based on the destination ID in the header.
Second--and perhaps more importantly--error processing for I/O operations becomes much more involved. If there's an error on any of the downstream I/Os, the initiator function in the virtualization operator has to become aware of it and attempts to retry the operation again or passes the error back to the upstream initiator for a retry. The virtualization operator then needs to determine if it will continue working with the problematic downstream ID. If so, it can try again--if not, it can operate in reduced mode according to the form of virtualization being used for the virtual device.
When failures occur, it's essential that the virtualization operator in the switch notifies the administrator of the failure, so it can be rectified in a timely fashion. This is a high-priority problem because a subsequent network or device failure on any of the I/O paths for the virtual device results in a loss of data. The situation is no different than for any other virtualization product, such as volume managers and disk subsystem controllers, except that the notification, diagnosis, troubleshooting and repair of the problem is done through switch management interfaces, instead of host or subsystem interfaces. Customers will need to verify that the timeliness of reporting problems, the severity level assigned and the reparation methods available through the switch are adequate for their operations. If best practices have been defined for storage I/O failure operations, then the switch should be expected to comply with them.
Initially, storage application-enabled switches aren't expected to have local storage connections for directly-attached storage devices, which implies that any storage functionality will involve virtual storage techniques. The most common forms of storage virtualization are mirroring, striping, concatenation and RAID.
This was first published in February 2003