One of the great advantages of iSCSI SANs is that they are built on standard gigabit Ethernet. Another is that many iSCSI components are cheaper than their Fibre Channel counterparts. However, it is possible to carry these advantages too far.
Theoretically, you could build an iSCSI SAN using standard Ethernet Network Interface Cards (NICs) to connect the components to the network. It works in practice, too -- but at a performance penalty that is unacceptable in most applications.
An iSCSI SAN has to wrap data in Ethernet packets to send it over the network and unwrap it at the other end. It also has to make sure that the packets are presented to the servers and storage in the correct order. ISCSI Host Bus Adapters (HBAs) do those jobs on the HBA with specialized hardware. An iSCSI SAN using NICs has to rely on the server to handle those jobs. This kind of housekeeping can eat 20% or more of the processor's power.
Although iSCSI HBAs are more expensive than NICs, it's worth the extra money to maintain system performance.
Similarly, it is not a good idea to try to combine your iSCSI SAN with your regular Ethernet network. Again, it works, but the SAN traffic will quickly load up a regular network and produce a major performance hit.
For more information:
About the author: Rick Cook has been writing about mass storage since the days when the term meant an 80 K floppy disk. The computers he learned on used ferrite cores and magnetic drums. For the last 20 years, he has been a freelance writer specializing in storage and other computer issues.