While hyper-converged architectures promise to decrease the cost and complexity of managing a virtualized server...
or desktop environment, they also need to deliver performance that is acceptable for the applications that they host. To meet the performance expectations of their environments, most HCAs leverage a flash storage SSD for some or all of their storage infrastructure. The techniques used to support flash affect both the performance and cost of HCAs, so it is important to understand how the various architectures implement flash data storage.
HCA storage basics
HCA storage infrastructures typically come in one of two designs:
- VM data stored on the same node that hosts the VM. To protect data and enable virtual machine mobility, each VM's data is replicated as it changes to other nodes in the cluster. The advantage of this design is that initial I/O is local when the VM reads and writes data; however, this replication design consumes at least two to three times the capacity of a typical RAID 5 design, and there are some limitations as to where VMs can move.
- VM data stored in a virtual volume created from an aggregation of capacity across the nodes in the cluster. This design essentially creates a scale-out storage system out of the disks inside each node in the HCA. It allows every node in the cluster to access any VM's data store and provides parity-based data protection. The result is that this scale-out design is more efficient from a capacity perspective, but requires more CPU overhead to create parity.
How HCAs use flash storage
These storage architecture designs determine how HCAs implement flash data storage. Both designs can use a flash storage SSD, either in an all-flash or hybrid fashion.
All-flash HCA. All-flash implementations are relatively straightforward from an architecture perspective. In the replication design, there may be cost concerns because of the design's additional capacity requirements. However, the replication design enables very high performance because a local flash device serves the majority of read traffic, eliminating or at least reducing network overhead. This design does not require a finely tuned network, reducing the hard cost and time involved supporting the network.
The scale-out design requires less capacity, but needs more overhead from both the network and the CPU to calculate parity. The result is latency, which affects flash performance. Unless additional CPU power and efficient networking compensates for this latency, it may be noticeable in some environments.
The downside to an all-flash HCA is cost, as all data needs to be on a flash device at all times. Most all-flash HCAs cannot add hard disk tiers to complement the all-flash tier.
Hybrid flash HCA. Hybrid flash HCA implementations are more intricate because hybrid systems try to efficiently use HDDs alongside flash. In the replication design, flash can be used locally within each node as read cache, so the VMs on those nodes experience low latency and high performance. The writing and replication of changed data occurs to disk drives on other nodes. A small flash storage SSD in each server can store the majority of the active data and cost-effective hard disks can store net new and inactive data. This implementation provides an excellent balance of high performance and lower costs.
There are two options in the scale-out design. First, two tiers -- a flash tier and an HDD tier -- can be created across the nodes, and then data can be moved back and forth between these tiers. The storage software component of the HCA should provide the ability to perform this movement automatically based on access. Additionally, the parity-based redundancy that the scale-out design creates should allow flash data storage to be used to store both data that is being actively read and data that is newly written. In other words, this design improves read and write performance. The downside to this approach is that data has to be written across the network, introducing some latency, which could keep flash from achieving its maximum performance.
The other option in scale-out storage HCAs is to maintain an aggregated hard disk tier for the scale-out component, but manage dedicated flash drives inside each of the nodes for local read performance. HCA storage software needs to understand data locality and cache the active data for a given node local to that node. The advantages of this design are similar to the replicated design -- lower cost and high performance for read access. However, this option has lower write performance.
When flash data storage makes sense
HCAs use a flash storage SSD in a variety of ways. It is important to understand the latencies that using an HCA may introduce compared to using a dedicated storage architecture. The need for HCAs to spend resources on maintaining VM performance and creating a shared and protected storage resource pool affects flash performance. The networking, which is often not dedicated to the storage function, also introduces latency, which affects performance.
For many data centers, these combined latencies have little noticeable impact on performance. However, some data centers with demanding VMs or a high number of nodes may see performance degradation and may decide that a more traditional, dedicated storage architecture is better for their needs.
How much flash storage to use in hyper-converged systems
Determine if an all-flash array is right for your needs
How hyper-converged storage helps manage virtual desktop infrastructure