Although iSCSI technology enables cost-effective block-based storage networking, its deployment has generally been restricted to SMB/SME and departmental workgroup users -- mainly due to the limitations inherent with Ethernet networks such as IP overhead, latency, port oversubscription and iSCSI target initiator software. So, while an iSCSI storage area network (SAN) "could" be assembled with common and inexpensive Ethernet components, it probably wouldn't work very well under real operating conditions. In order for iSCSI to compete and flourish in the enterprise, storage and network administrators must address these traditional problems.
One part of the solution is improved Ethernet hardware, including Ethernet network interface cards (NICs) that incorporate a TCP/IP Offload Engine (TOE). TOE chips offload some or all of the work required to process TCP/IP data, reducing workload on the local CPU and improving Ethernet efficiency. TOE cards have not been very popular because of their added expense, but their deployment is expanding as iSCSI SAN traffic rates increase. TOE NICs are used most often in the storage system (target) where the I/O traffic from multiple servers (initiators) will come together and cause congestion. TOE cards are available from just a handful of vendors, including Alacritech, LeWiz Communications Inc., and QLogic Corp.
Common Ethernet switch ports tend to introduce latency into iSCSI traffic, and this reduces performance. Experts suggest deploying high-performance Ethernet switches that sport fast, low-latency ports. In addition, you may choose to tweak iSCSI performance further by overriding "auto-negotiation" and manually adjusting speed settings on the NIC and switch. This lets you enable traffic flow control on the NIC and switch, setting Ethernet jumbo frames on the NIC and switch to 9000 bytes or higher -- transferring far more data in each packet while requiring less overhead. Jumbo frames are reported to improve throughput as much as 50%. High-performance IP storage switches/routers are available from Brocade Communications Systems Inc., Cisco Systems Inc., Emulex Corp., Hewlett-Packard Co., SANRAD Inc., and Woven Systems Inc.
Switch port performance can also be enhanced by eliminating "oversubscription." There's no harm in oversubscription when bandwidth is underutilized -- multiple devices can help to fill underutilized bandwidth and achieve more value per port. But as utilization goes up, devices can begin competing for the port, introducing latency and performance problems for an iSCSI SAN. Rather than allowing multiple devices to compete for one switch port, establish a limit of one device per port.
It's important to consider the performance of your iSCSI initiator (server-side) software. As with any device driver, the quality and integrity of your iSCSI initiator software can vary dramatically depending on the vendor, their experience in the iSCSI market, and the maturity of their iSCSI product -- some initiators simply work better than others. It may be worthwhile to test the performance and robustness of several iSCSI initiators before deciding on the best initiator. TOE cards and other hardware devices include their own initiator firmware, eliminating the need for separate initiator software.
Finally, iSCSI SAN performance can be improved through careful logical and physical separation. For example, iSCSI SAN traffic should never be mixed with ordinary Ethernet user traffic. This not only impairs SAN performance, but also creates a potential security risk since storage data is accessible on the user LAN. Instead, iSCSI SAN traffic should be isolated from the everyday user traffic. The most common means of separation is a virtual LAN (VLAN), limiting iSCSI traffic to the virtual LAN and keeping regular traffic out. The same effect can be achieved physically by creating a new LAN segment for the iSCSI SAN (often using high performance NIC and switch hardware), and keeping that segment isolated from other regular Ethernet segments.
Check out the entire iSCSI vs. FC handbook.
