This article can also be found in the Premium Editorial Download "Storage magazine: Good match: iSCSI and vSphere."
Download it now to read this article plus other related content.
Best practices for using iSCSI storage with vSphere
Once iSCSI disks have been configured, they're ready to be used by virtual machines. The best practices listed here should help you get the maximum performance and reliability out of your iSCSI data stores.
- The performance of iSCSI storage is highly dependent on network health and utilization. For best results, always isolate your iSCSI traffic onto its own dedicated network.
- You can configure only one software initiator on an ESX Server host. When configuring a vSwitch that will provide iSCSI connectivity, use multiple physical NICs to provide redundancy. Make sure you bind the VMkernel interfaces to the NICs in the vSwitch so multi-pathing is configured properly.
- Ensure the NICs used in your iSCSI vSwitch connect to separate network switches to eliminate single points of failure.
- vSphere supports the use of jumbo frames with storage protocols, but they're only beneficial for very specific workloads with very large I/O sizes. Also, your back-end storage must be able to handle the increased throughput by having a large number (15+) of spindles in your RAID group or you'll see no benefit. If your I/O sizes are smaller and your storage is spindle-bound, you'll see little or no increase in performance using jumbo frames. Jumbo frames can actually decrease performance in some cases, so you should perform benchmark tests before and after enabling
- jumbo frames to see their effect. Every end-to-end component must support and be configured for jumbo frames, including physical NICs and network switches, vSwitches, VMkernel ports and iSCSI targets. If any one component isn't configured for jumbo frames, they won't work.
- Use the new Paravirtual SCSI (PVSCSI) adapter for your virtual machine disk controllers as it offers maximum throughput and performance over the standard LSI Logic and BusLogic adapters in most cases. For very low I/O workloads, the LSI Logic adapter works best.
- To set up advanced multi-pathing for best performance, select Properties for the iSCSI storage volume and click on Manage Paths. You can configure the Path Selection Policies using the native VMware multi-pathing or third-party multi-pathing plug-ins if available. When using software initiators, create two VMkernel interfaces on a vSwitch; assign one physical NIC to each as Active and the other as Unused; use the esxcli command to bind one VMkernel port to the first NIC and the second VMkernel port to the second NIC. Using Round Robin instead of Fixed or Most Recently Used (MRU) will usually provide better performance. Avoid using Round Robin if you're running Microsoft Cluster Server on your virtual machines.
|VMFS volume block sizes|
By default, VMFS volumes are created with a 1 MB block size that allows a single virtual disk (vmdk) to be created up to a maximum of 256 GB. Once you set a block size on a VMFS volume, it can't be changed. Instead, you need to move all the virtual machines (VMs) from the volume, and then delete it and recreate it with a new block size. Therefore, make sure you choose a block size that works for your configuration based on current and future needs.
The chart shown here lists the block size choices and related maximum virtual disk size.
Choosing a larger block size won't impact disk performance and will only affect the minimum amount of disk space that files will take up on your VMFS volumes. Block size is the amount of space a single block of data takes up on the disk; the amount of disk space a file takes up will be based on a multiple of the block size. However, VMFS does employ sub-block allocation so small files don't take up an entire block. Sub-blocks are always 64 KB regardless of the block size chosen. There is some wasted disk space, but it's negligible as VMFS volumes don't have a large number of files on them, and most of the files are very large and not affected that much by having a bigger block size. In most cases, it's probably best to use an 8 MB block size when creating a VMFS volume, even if you're using smaller volume sizes, as you may decide to grow the volume later on.
iSCSI guides available
VMware provides detailed guides for implementing iSCSI storage for vSphere. Two useful guides available from VMware include the iSCSI SAN Configuration Guide and the iSCSI Design Considerations and Deployment Guide.
BIO: Eric Siebert is an IT industry veteran with more than 25 years of experience who now focuses on server administration and virtualization. He's the author of VMware VI3 Implementation and Administration (Prentice Hall, 2009).
This was first published in August 2010