Managing and protecting all enterprise data


Caching software adds IOPS to solid-state storage

A new category of storage software is emerging with apps that optimize solid-state storage to help increase I/O performance and fully realize the benefits of flash-based storage.

A new category of storage software is emerging with apps that optimize solid-state storage to help increase I/O performance and fully realize the benefits of flash-based storage.

Balancing all the elements of a computing infrastructure is a bit like squeezing a balloon: push one side in and another side pops out. Throughout the evolution of computing, server, network and storage technologies have leapfrogged one another in terms of performance. As the technology of one component improves, the others become bottlenecks in the overall system performance.

When it comes to mechanical storage devices, however, it seems the Rubicon has been crossed. Although mechanical hard disk drives (HDDs) continue to be enhanced, most of the advances are in areal density. Capacities within a given form factor continue to improve significantly, yet increasingly, mechanical HDDs simply can’t keep up in terms of IOPS throughput. Fortunately, solid-state drive (SSD) technology has allowed storage to again jump ahead of server throughput requirements. Whereas a 600 GB 15K rpm SAS drive can deliver approximately 145 IOPS, a 512 GB SSD card can deliver 50,000 IOPS or so. While the SSD cost per GB is higher, its cost per IOPS is much lower.


All major storage vendors, as well as many startups, have added an SSD layer to their arrays and permit provisioning of hybrid storage pools. Adding even a small slice of solid-state storage to a pool can dramatically increase its I/O potential. Array-based SSD, however, is still limited by the network latency.

In response, a number of traditional storage vendors, such as EMC Corp. and NetApp Inc. plus startups like Proximal Data and VeloBit Inc., are offering server-based flash storage software. Software stitches all server flash and array SSD together (though technically speaking either technology can be deployed in both places). One important distinction is between true extended CPU cache and solid-state cache. The first is purely accessible and usable by the CPU, whereas solid-state cache functions much like direct-attached storage (DAS): solid-state management software coordinates the movement of data between the server flash and array storage pools.

The rapid adoption of server virtualization has concentrated and exponentially increased the required IOPS per physical server. Server-based flash is an ideal candidate for this use case. For example, NetApp’s recently announced Flash Accel is targeted specifically at VMware and Windows environments, and NetApp claims it can yield a 90% reduction in latency and an 80% improvement in IOPS.

Boosting performance

Although both server- and array-based solid-state storage can deliver high IOPS rates, proper deployment is the key to optimizing them. Generally, array-based SSD can be thought of as “general purpose” in that it can improve I/O to all applications by increasing the array’s IOPS rate. Of course, it can be provisioned to specific applications (LUNs) as well. In any case, SSD in an array is well suited to random I/O workloads where large fetch rates benefit the cache hit rate.

Server-side flash is best suited to read-intensive applications. When it becomes necessary to read data from disk, network latency negates some of the SSD benefits and all is for naught if mechanical disk reads are required frequently.

Benefits of solid-state software

  • Eliminates the I/O performance bottleneck common in virtual machine configurations.
  • Maintains data consistency between server-based flash and array storage pools.
  • Locates “hot” data in server flash and keeps it there.
  • Manages and extends the life of solid-state storage.
  • IOPS and latency improvements.

Thus, the central emphasis for solid-state software is getting the right data into flash and keeping it there. The key is predicting what will be the high-access data and retaining it in flash. In a nutshell, that’s the purpose of solid-state software. VeloBit’s HyperCache uses “content locality caching” that manages the contents of data blocks. HyperCache works with both server- and array-based solid-state storage. Proximal Data’s AutoCache software product uses what it terms “merit-based” allocation, in which data is cached based on I/O access patterns at the physical server level. In contrast, NetApp’s Flash Accel can be allocated to specific virtual machines (VMs) for targeted performance gains.

For most products then, the secret sauce is all about finding “hot” data using sophisticated algorithms. The effectiveness of these algorithms can be measured in the cache hit rate. Normal and good cache hit rates are in the 40% range, while 80% hit rates are phenomenal.


Optimizing cost

Fewer than 5% of servers have a flash tier deployed today and cost is likely a major reason for that small figure. In some cases, the cost of the flash is a significant percentage of the blade it serves. Even so, adding solid-state storage can help improve the density of a server environment. If the physical server is I/O bound with a certain number of VMs installed yet has more CPU capacity available, flash will enable more VMs to be deployed on that server.

EMC’s recently released VFCache 1.5 has added data deduplication technology, leveraged from other EMC products. Deduplication can be a game changer for some IT organizations because its benefits to flash are twofold. It decreases the cost per GB stored, an important metric to most IT organizations; and it requires fewer write operations to the flash card, thereby extending the life of the physical solid-state storage. This reduces the tradeoff IT organizations must make to balance the higher cost per GB of SSD versus the lower cost per IOPS of SSD. Similarly, VeloBit’s HyperCache enables compression on flash. Compression stores more data in the same physical media, also lowering the cost per GB. “Your mileage may vary” always applies, but IT organizations shouldn’t expect the 20:1 deduplication/ compression ratios achieved with backup appliances. EMC suggests a more conservative 20% expectation for deduplication of VFCache.

Proximal Data takes, perhaps, a more conventional and straightforward approach to offering a compelling value: low price. AutoCache starts at $999 for up to 500 GB of solid-state capacity.

Maintaining data coherence

Because virtual server and virtual desktop environments are the key targets for server flash, IT managers need to understand how a solid-state software product will function in that environment. One key value of a VM is its ability to migrate to recover from hardware failures or for performance improvement. How solid-state software behaves during a migration is an important factor in determining the best product for a given situation.

Key features to look for in solid-state software

  • Seamless support of vMotion for business continuity.
  • Compression and/or deduplication to reduce the cost per GB stored.
  • Deployment flexibility in device type and workload targeting.
  • “Hot” data algorithm and cache hit rate.

Of the factors needed to support virtual environments, perhaps the most important is data coherence, or consistency, between the server flash and back-end storage. The solid-state software algorithms copy the “hot” data from the primary storage to the flash and ensure the data remains consistent between the two repositories. Although some vendors plan to support active-active clustering in the future, support is currently limited to active-passive configurations. As such, data consistency issues arise when data is modified out-of-band (i.e., a snapshot restore) or when a volume is unmounted/remounted as may occur during a VMDK move.

EMC VFCache supports live migration with vMotion in a VMware environment. As such, it doesn’t need to “re-warm” the cache when a VM is migrated and it fully supports business continuity. NetApp Flash Accel caches the VMDK at a block level and uses what the company calls “persistent endurability” that avoids cache re-warming in the event of a reboot or VM failure. VeloBit HyperCache currently requires a re-warm, though it has persistency on the roadmap. Proximal Data AutoCache algorithms adapt in real-time to move metadata with the VM to “pre-warm” the cache.



Just because storage vendors now support server-side flash with software doesn’t mean they’re getting into a whole new hardware line -- at least not yet. EMC packages VFCache with qualified PCI Express (PCIe) flash products, such as a 700 GB single-level cell (SLC) card. EMC also supports the LSI Nytro WarpDrive in the Cisco UCS B-series blades. These come in 400 GB and 800 GB multi-level cell (MLC) cards. VFCache supports multiple cards for larger workloads and split card implementations.

NetApp has developed a strategic reseller agreement with Fusion-io to bundle Flash Accel with Fusion-io’s ioDrive line of NAND flash cards. NetApp also qualifies other flash hardware vendor offerings as partner alliances. Both Proximal Data AutoCache and VeloBit HyperCache support a variety of PCIe and SSD hardware products, sold separately. It’s a matter of installing the hardware and downloading the software.

An end to wasted capacity

Prior to solid-state storage, array vendors and storage managers devised creative ways to spread data sets across multiple devices to harness the aggregate performance of the grouped devices. To avoid contention, active data sets can’t be located on the same devices. The result can be adequate performance at the cost of gross inefficiencies in capacity utilization. When storage devices are only 20% utilized, then the cost per GB stored is five times the purchase price. Moreover, the cost per IOPS is also five times.

IT managers need to examine where the performance bottleneck is in their environment. If the problem is I/O, then solid-state storage is likely the answer. In VM deployments, I/O is very likely to be a culprit and server-based flash with the appropriate solid-state software can be a viable solution. Both EMC and NetApp have professional services and software tools to help identify performance issues and usage patterns, as do numerous third-party organizations. Proximal Data offers a dashboard to illustrate I/O by host and guest OS on a storage device or LUN. Properly implemented, server I/O problems can be solved, ahem, in a flash.

BIO: Phil Goodwin is a storage consultant and freelance writer.

Article 3 of 8

Dig Deeper on Solid-state storage

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Storage

Access to all of our back issues View All