Getting ready for IP SANs

IP SANs promise benefits to groups within your organization that up until now haven't had access to these kinds of capabilities. But before you even think of deploying an IP SAN, read this article.

This article can also be found in the Premium Editorial Download: Storage magazine: Storage products of the year 2003:

Understanding iSCSI host connections
Because iSCSI sits on top of TCP/IP, any standard network interface card (NIC) could be used to connect a host to an IP storage area network (SAN). However, special-purpose iSCSI cards are becoming available that can reduce CPU utilization and potentially provide greater levels of performance--albeit at a higher price--than a general purpose NIC (See "Network interface card options," below).

With a conventional NIC, all functionality above the GigE level is provided in software, including all SCSI, iSCSI and TCP/IP functions. The impact on the host depends on the CPU load and the I/O characteristics of the application. A lightly loaded CPU may have sufficient headroom to provide good iSCSI performance. A moderately busy CPU may even be able to support a light to moderate iSCSI load.

There are two alternatives for offloading the iSCSI processing overload: TOE NICs and iSCSI adapters. A TCP/IP Offload Engine (TOE) NIC offloads all TCP/IP operations. This type of interface isn't specific to iSCSI at all, and could be used as a replacement for a general-purpose NIC card in any application deemed appropriate. When using a TOE NIC for storage, SCSI and iSCSI processing would still be performed by the CPU.

An iSCSI adapter is directly analogous to a Fibre Channel host bus adapter. In this case, all iSCSI and TCP/IP operations are offloaded to the interface card. All of these options are currently available. Make sure you don't draw conclusions about performance without testing specific applications on specific platforms.

Network interface card options:

Here's the commercial: iSCSI and other related IP storage technologies promise to open doors that will extend the benefits of networked storage to segments of the organization that up until now haven't had access to these capabilities. A number of features make IP storage networking attractive, including its lower cost relative to Fibre Channel (FC) products.

So, is 2004 the year to build an IP storage area network (SAN)? As with most storage adoption questions, the answer lies in understanding the current capabilities and limitations of the technology, and having a solid understanding of your storage networking needs. If your organization hasn't done so yet, it should begin asking--and answering--questions on how best to deploy an IP SAN. Among the areas that need to be evaluated are your application storage characteristics, network infrastructure and security requirements.

iSCSI design considerations
First and foremost, application considerations will impact design. Some questions to answer are: Is the application transaction-oriented or throughput-oriented? What are the I/O performance requirements of the application? What platforms do the application support? Ideal candidates for iSCSI today are low- to moderate-level I/O applications that run on Windows or Linux platforms. This typically includes applications such as Microsoft Exchange and many SQL Server-based applications.

The leading network consideration relates to the bandwidth required. An important decision to make is the method of host connection. Can a software initiator with a standard gigabit network interface cards (NICs) be used, or is a special-purpose adapter required? (See "Understanding iSCSI host connections" on this page.) There's no one right answer to this question. The major benefit of iSCSI host bus adapters (HBAs) and TCP/IP offload engine (TOE) NICs is the decrease in CPU utilization relative to a standard NIC, with performance improvement as a secondary benefit. If there's enough CPU headroom available and the application I/O requirements aren't extraordinary, it's worth considering foregoing a special-purpose card and using a standard NIC, given the potential cost savings.

Another network consideration is the segregation of traffic. It's generally recommended that iSCSI activity be isolated from general network traffic. For example, this means dedicated host connections and separate virtual LANs. Additional bandwidth considerations may be based on performance optimizations for routing storage traffic.

Making the IP to FC connection
Administrators responsible for configuring Fibre Channel (FC) storage area networks (SANs) have become familiar with the concepts of zoning and LUN masking to ensure that SAN storage is only presented to the appropriate hosts. Zoning involves segregating sets of ports or world-wide names (WWNs) to isolate access. (For an in-depth discussion of zoning, see W. Curtis Preston's series of three articles, beginning with "Protect Your SAN From Attack" in the August 2003 issue of Storage). LUN masking involves building the appropriate mapping tables of host WWNs with corresponding storage WWNs.

How do you do this for an iSCSI host? "iSCSI host to FC storage" shows an iSCSI host connected to an FC storage array through a gateway switch that supports both FC and iSCSI protocols. The gateway provides a virtual iSCSI target for each physical FC SAN target device. To successfully provide the host with access to the storage requires implementing iSCSI and FC access control mechanisms.

On the iSCSI side, there are no WWNs or zones. Instead, several other constructs are utilized, including some that are standard within TCP/IP networks and others that are unique to iSCSI. For addressing, iSCSI uses TCP/IP addresses and ports. Each initiator has an IP address. In addition, for discovery purposes, each initiator also has an iSCSI name, which is a dot-separated structure that is roughly analogous to a URL. iSCSI zoning is accomplished by two mechanisms: virtual LANs and discovery domains, which are similar to FC WWN zones.

To enable an iSCSI host to access an FC target, a mapping between an initiator TCP/IP address/port and a WWN must occur within the switch. While these mappings can be dynamic or static, in most cases static mapping is used to present a persistent view of a device to a host. On the FC side of the gateway, traditional zoning and LUN masking can be configured using the mapped address. On the IP side, VLANs and discovery domains are created. Discovery from the host perspective is transparent, but ensuring that appropriate mappings and access controls are in place requires careful management.

Security and access control in an IP SAN potentially represent a greater concern than in traditional FC SAN environments. This is largely because LANs tend to be more open and widely accessible than SANs. The good news is that iSCSI was designed to take advantage of existing network security capabilities. Standard protocols such as CHAP for authentication and IPsec for encryption can be employed to make a SAN secure. This functionality can actually be implemented transparently to iSCSI. It's anticipated that in many environments, iSCSI hosts will require access to storage in existing FC SANs. This will require a combination of IP and FC access control (see "Making the IP to FC connection").

Among the fundamental lessons learned from the design of FC SANs is the importance of redundancy. In a direct-attached storage (DAS) environment, loss of connectivity only impacts a single server; in a storage network, the impact can be widespread. Therefore, avoiding single points of failure and understanding the impact from a storage perspective of the loss of key components are critical design considerations.

Another lesson learned from FC SAN deployment is to carefully adhere to vendor support matrices.

iSCSI SANs are new, and interoperability issues are certain to arise. Components that logically should work together may not, or may only partially work. Any design should be validated to ensure that all hardware and software components are certified to interoperate.

The case for IP SANs
Unfortunately, traditional FC SANs have one substantial downside. The cost to acquire SAN technology, including HBAs and switches, the cost to implement this technology and not insignificantly, the cost to manage this infrastructure are limiting factors to widespread deployment. When smaller organizations carefully evaluate the TCO for a storage network proposal, they can't justify the expenditure. Even within larger organizations, cost has often been the limiting factor in the extensive deployment of storage networks.

IP SANs, which are based on iSCSI technology, promise to address all three of these cost issues, making widespread storage networking a reality. iSCSI SANs can be implemented using standard Gigabit Ethernet (GigE) cards and IP switches. In most environments, this technology is already in place and well understood. Even if this equipment must be purchased, the costs--compared to FC--are substantially lower (see "Sample cost comparison").

Because most organizations already have network management capability, implementation and management costs are also considerably lower.

Furthermore, IP SANs promise additional benefits beyond the capabilities of FC. They can span far greater distances and can provide prioritized service capabilities through functions such as IP VLAN tagging and TCP/IP QoS. They also provide the potential for greater security through the use of standard network security capabilities such as IPsec.

So, do IP SANs give you more functionality at a lower cost? While that sounds attractive, don't put your FC gear up on eBay just yet. FC is a more mature technology and--at least in the short term--will have a performance advantage over iSCSI. Rather than work as a replacement, IP SANs offer the opportunity to extend the storage network beyond its current boundaries.

Sample cost comparison:
connecting 100 hosts via an IP SAN and an FC SAN

More info
Special-purpose adapters for iSCSI, a sample of native iSCSI storage offerings and more.

Where is iSCSI technology today?
2003 witnessed two major milestones in the technical evolution of iSCSI: The Internet Engineering Task Force approved the final draft of the iSCSI standard, meaning that vendors now have a defined target to focus their development efforts. The other major event was the release of officially supported iSCSI software drivers from Microsoft Corp. This support for a major commercial platform provided a validation to many that the technology is truly viable and will likely have a promising future. In addition, some major storage vendors such as EMC Corp. and Network Appliance Inc. (NetApp) have started offering some level of direct iSCSI attachment to their storage systems. Microsoft recently announced the first group of devices that have been qualified with its driver.

Is iSCSI technology ready for prime time? The short answer is yes and no. Yes, because more products and standards are entering and stabilizing the market. No, because IP SANs bring many implementation issues to the table that are new to storage. Consider, for example:

Host connectivity: In iSCSI (and traditional SCSI) parlance, the host requesting service is an initiator and the device providing that service is a target. An iSCSI initiator can be either a hardware device or a software driver. Today, drivers for host iSCSI connectivity are readily available for Windows and Linux hosts. Although at least one vendor, Cisco Systems Inc., offers drivers for some Unix platforms, most midrange and high-end systems haven't started providing support in the iSCSI arena. These software initiators require only a NIC. For performance reasons, it's generally expected that GigE will be the common connection for iSCSI, but aside from performance, traditional 100Mb NICs should also work.

Hardware interfaces are available from a number of vendors. These devices include iSCSI HBAs, as well as gigabit NIC cards equipped with TCP/IP offload engines, known as TOE NICs. (See "Special-purpose adapters for iSCSI")

Networking: Because iSCSI is a TCP/IP-based protocol, there's no need for special switches to support it. Traditional IP switches and routers process iSCSI packets in much the same way that they handle other network traffic. However, one significant opportunity for iSCSI deployment is to provide broader host access to existing FC storage. To accomplish this, an IP/FC gateway or switch is needed. Some vendors, such as Cisco, McData/Nishan and Sanrad, provide products to perform this function. These devices include some GigE ports with at least one FC E-port (and possibly N-ports). See "IP-FC gateway devices" for details on some of the options available for IP-FC interoperability.

Storage: Until recently, the iSCSI storage market consisted primarily of emerging vendors. While some major storage vendors have introduced iSCSI support, others have been slow to enter the market (see "Sample of native iSCSI storage offerings").

Where to use iSCSI
It's clear that the technology needed for successful deployment of an IP SAN is available today. The next consideration is what the appropriate uses for the technology are, and where it will provide the most benefit.

Prior to iSCSI/IP SAN technology, the conventional way to retrieve and write data via the IP network was by file access (conventional file servers and network-attached storage appliances). Applications--such as databases and some e-mail products--that access data by blocks preferred DAS or an FC SAN. With iSCSI, these applications can now be supported over the LAN.

Providing file- and block-level data over the same transport can simplify storage management. Utilization can also be improved because storage can be more readily shared and allocated by function (see "Shared storage in an IP SAN"). Additional benefits can also be realized in improvements to availability. With LAN access to storage, a variety of options relating to clustering of nodes and replication of data become more viable.

IP SANs also present new options for backing up data. Administrators now have the option to do traditional file-based backups across the network to a backup server or to perform image-based backups directly to shared tape devices. While similar to traditional LAN-free backup, the increased flexibility of IP provides a greater opportunity to deploy this option.

Management questions
As IT organizations begin to deploy iSCSI, they will need to develop additional capabilities to manage these new technologies. However, organizations actually start from an advantage that early adopters of FC SANs didn't have. Today's IT networking staff has already developed strong capabilities to configure, maintain and monitor IP network infrastructures. Likewise, expertise previously gained by managing storage in FC SAN environments can be readily applied to iSCSI storage.

Many comprehensive management tools are available to monitor and configure an organization's IP network infrastructure. However, expect tools that assist in managing the iSCSI layer to be relatively limited in the early stages. As IT organizations implement IP SAN technologies, new tools will emerge and existing SAN management solutions will integrate iSCSI configuration and monitoring capabilities into their offerings.

Getting started with IP SANs
The promise of iSCSI to offer greatly expanded functionality at a reduced cost is highly compelling, and most organizations should be investigating ways that they can pilot this technology. As "Bringing IP and Fibre Channel together" illustrates, the opportunity to expand functionality at a dramatically lower cost and to move toward an any-to-any storage architecture becomes a real possibility. While FC SANs will continue to be the domain of tier-one production applications, many tier-two and tier-three applications can leverage some of the same functionality with IP SANs.

While the adoption of any new technology requires a cautious approach, much of the functionality within IP SANs has already been proven: IP networking is mature and the steady maturation of FC storage has been leveraged in much of the development of IP storage.

By selecting appropriate applications for storage networking, following the design and implementation guidelines discussed in this article and applying solid test and evaluation processes, IP SANs can be introduced within organizations and begin to play a major role in storage environments.

Bringing IP and Fibre Channel together
This was first published in December 2003

Dig Deeper on SAN management



Find more PRO+ content and other member only offers, here.



Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:



  • Flash technologies remain hot in 2016, experts predict

    Experts predict solid-state technology will remain hot in 2016, leading to the demise of high-speed hard disk drives, as ...

  • Tintri VMstore T5000

    Like all of its VM-aware storage systems, Tintri’s first all-flash array -- the Tintri VMstore T5000 -- allows admins to bypass ...

  • SolidFire SF9605

    The high-capacity SolidFire SF9605 uses SolidFire’s Element OS 8 (Oxygen) to deliver new enterprise features such as synchronous ...