Of course, there can be multiple virtualization operators in sequence along the I/O path. For example, a virtualization operator in a disk subsystem controller can provide one type of virtualization and a virtualization operator in host volume management software can provide the same or a different favor of virtualization. See "Multilayered virtualization scheme," which shows a host volume manager and several downstream disk subsystems. One way to think about multilayered virtualization is that the I/O path fans out into multiple downstream paths after it is processed by the virtualization operator.
The first order of business for virtualization in a SAN switch is finding a way to identify the target virtual devices created by the switch's virtualization operator. Virtual storage devices in SAN switches need to be configured by an administrator. The storage administrator must first select the form of virtualization for the virtual device, and then discover the downstream target storage IDs that will comprise the virtual device. The next step is to create the exported target ID of the new virtual device, and assign this new virtual device to a zone or virtual network. Once the virtual device and its exported IDs are created, they can be accessed by any initiators in the SAN that have access to it.
The virtualization operator must have a processor to run on. There are many ways this could be implemented in SAN switches with a wide variety of choices for processors. For instance, the virtualization operator could run on a processor that the switch has for providing other services. It could also run on its own dedicated processor, which may be on a separate adapter or blade. Another alternative is to have a separate system work in tandem with the switch to provide the virtualization function. While this wouldn't necessarily be switch-based virtualization, there are possible implementations where the distinction would be splitting hairs.
A close analysis of the virtualization process begins by understanding what happens when an incoming I/O command from an initiator is addressed to the switch's exported target virtual device ID. The switch port identifies the egress port for this virtual storage ID as the virtualization operator, then queues the command for processing by it. This doesn't have to be an actual egress port, and depends entirely on the design of the switch and its method for integrating virtual storage functions.
The virtualization operator then will read the entire command and process it according to the form of virtualization being used for this virtual device. There's a subtle, but important point: The initial I/O command is terminated by the virtualization operator and reinitiated it to downstream storage IDs. In other words, the initial I/O no longer exists, but has been reconstituted and retransmitted to a number of downstream IDs, through some combination of egress ports in the switch. See "Reinitiating an I/O command," which illustrates the termination and reinitialization process.
This terminating and reinitiating of I/Os isn't necessarily a bad thing, but it's important to understand the implications. First, this process adds latency to all I/Os addressed to virtual devices. Queuing, terminating, processing and reinitiating the downstream I/Os will be slower than simply forwarding an I/O through to an egress port based on the destination ID in the header.
Second--and perhaps more importantly--error processing for I/O operations becomes much more involved. If there's an error on any of the downstream I/Os, the initiator function in the virtualization operator has to become aware of it and attempts to retry the operation again or passes the error back to the upstream initiator for a retry. The virtualization operator then needs to determine if it will continue working with the problematic downstream ID. If so, it can try again--if not, it can operate in reduced mode according to the form of virtualization being used for the virtual device.
When failures occur, it's essential that the virtualization operator in the switch notifies the administrator of the failure, so it can be rectified in a timely fashion. This is a high-priority problem because a subsequent network or device failure on any of the I/O paths for the virtual device results in a loss of data. The situation is no different than for any other virtualization product, such as volume managers and disk subsystem controllers, except that the notification, diagnosis, troubleshooting and repair of the problem is done through switch management interfaces, instead of host or subsystem interfaces. Customers will need to verify that the timeliness of reporting problems, the severity level assigned and the reparation methods available through the switch are adequate for their operations. If best practices have been defined for storage I/O failure operations, then the switch should be expected to comply with them.
Initially, storage application-enabled switches aren't expected to have local storage connections for directly-attached storage devices, which implies that any storage functionality will involve virtual storage techniques. The most common forms of storage virtualization are mirroring, striping, concatenation and RAID.
Mirroring and striping
Mirroring is one of the most simple, yet most powerful virtualization techniques. The latency injected by the virtualization operator in mirroring is minimal as there's no parsing of the command and no block address manipulation to do. Additionally, error recovery is simplest for mirroring compared with other forms of storage virtualization.
Striping takes advantage of parallel operation on storage to provide faster performance. Incoming I/O operations from initiators would be terminated by the virtualization operator and a number of reinitiated I/O operations would be created, corresponding to the number of downstream storage IDs in the virtual device.
Striping requires that the incoming I/O operation be split up among the member storage destinations of the virtual device. While striping without parity redundancy is used solely for performance purposes, it's not clear that using striping in a switch would result in a net performance gain, after the latency of the virtualization operator is taken into account. Also, considering the lack of protection from failures, it's likely to be used only in cases where flexibility in creating striped arrays of different sizes is the highest priority.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Reinitiating an I/O command |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 
The terminal and reinstallation process.
|
 |
 |
 |
 |
 |
 |
 |
RAID
RAID in a switch uses parity or mirrored redundancy to provide data protection from downstream storage and link failures. As with striping, the incoming I/O would be terminated by the virtualization operator, and the number of subsequent downstream I/Os would be determined by the number of member storage IDs in the array.
Parity RAID adds a significant level of overhead to the virtualization operator. For example, the read, modify and write penalty of RAID-5 would be a much higher burden on a switch processor than mirrored redundancy such as used in RAID-1 or 0+1. As RAID-1 was previously discussed as mirroring earlier, RAID-0+1 should be the RAID level of choice for switch virtual storage implementations to minimize the negative impacts of unanticipated write and switch workloads.
Concatenation
Concatenation merges two separate storage address spaces together and utilizes them as a single, virtual contiguous storage address space. Like striping, it has no built-in redundancy to protect against downstream link and storage failures. Unlike striping, however, a failure in a concatenated virtual device doesn't necessarily result in the loss of all data because data that's stored completely on any downstream storage IDs may still be able to be accessed. However, there's no guarantee that data objects were written completely on any one device of a concatenated storage volume. The overhead of concatenation is relatively minor, involving simple algebraic processes for translating storage addresses.
Fierce competition
In the heyday of the technology bubble when everything seemed possible, storage switch vendors expected the market to gobble up virtualization switches like hotcakes. Times have certainly changed and the prospects of switch-based virtualization don't seem like such a sure thing anymore.
The issue for IT professionals is whether or not switch vendors will provide virtualization functions that can add enough value to make it worthwhile to pay a premium for it. This can only happen by making storage easier to manage and less expensive to own. Already, volume management software and disk subsystem controllers provide complete functionality for the most useful types of storage virtualization.
Beyond competition with volume management software and disk subsystems, there are new virtualization products and architectures to contend with. Switch-based virtualization is another instance of in-band virtualization, a concept that hasn't been too warmly received by the market yet. If the market decides that out-of-band virtualization is a better way to go as Gartner Inc. has suggested, that would leave the switch virtualization players holding a mostly empty bag.