Flash storage brings many benefits to the data center, not the least of which, of course, is performance. That...
performance boost also comes with greater density and reduced power consumption, but flash does have a few dark sides.
First, there is an ongoing concern about durability. There are also concerns about generating consistent performance while under load. The purpose of a flash controller is to manage flash so that the media lasts as long as possible and performance is as consistent as possible. While many NAND flash products look very similar, the way they handle flash management may differ. Understanding those differences is critical to selecting the right flash system for your enterprise.
Flash has always had a durability problem. No matter the NAND type, it can only sustain a certain number of writes before it eventually fails. The durability problem actually gets worse as the technology advances. Though 3D NAND TLC flash may be more cost-effective than SLC flash, it is far less durable.
Storage systems and flash management
To combat this, both flash management vendors and storage system vendors have added capabilities to make flash last longer.
On the storage system side, features like high amounts of flash capacity, deduplication and compression extend flash life. All-flash arrays and hybrid arrays with a large flash tier extend flash life simply because there is more flash capacity. Extra flash capacity allows flash wear to be distributed more broadly.
Storage systems also use data efficiency technologies like deduplication and compression to help extend flash life. When these data efficiency technologies are implemented "in-line," they eliminate some of the potential writes that flash can incur. For example, if a system offers a 5x data efficiency rate when deduplication and compression are used together, it is reasonable to expect that flash drives should last five times longer than they would have without the technology.
Flash controller technology
Most of the storage systems mentioned previously use drive form factor flash, also known as a solid-state drive (SSD). Each SSD is made up of many flash NANDs, and each SSD has a flash management module on it, known as a flash controller, which is designed to extend the useful life of the SSD. Flash controllers can extend flash in ways that flash systems alone cannot. The first responsibility of a flash controller is wear-leveling, which makes sure that writes are written across flash cells evenly. Without wear-leveling technology, certain NAND cells within an SSD may be written to more frequently and wear out sooner.
Flash controllers use wear-leveling in conjunction with another life-extending capability: over-provisioning. SSD manufacturers place extra flash capacity in each SSD that is not recognized by the rest of the flash system. The flash controller then uses this excess capacity as part of its wear-leveling routine. Because there are more flash cells on which to write, the SSD offers higher write endurance at the expense of total SSD capacity.
Most SSD manufacturers set the percentage of flash that is reserved for over-provisioning. Recently, some flash vendors have allowed customers to tune the amount of flash that is allocated for over-provisioning. With this capability, IT professionals can adjust the characteristics of the drive for the particular circumstance. For example, flash SSDs assigned to a write-intensive workload can have the flash over-provisioning allocation set higher to increase SSD life. Or, alternatively, a set of flash SSDs assigned to a read-intensive application can have the allocation set very low to increase flash capacity and decrease costs.
Another capability that new flash controllers have is the ability to more granularly mark out sections of flash that have failed. A basic function of the flash controller is to monitor the flash it is managing, and as cells fail, mark them as read-only. However, sometimes this means marking some good areas of the NAND "bad" as well, wasting capacity. Modern flash controllers are more granular and only flag the exact cells that have gone bad, saving flash capacity.
An SSD is under the most duress when it is trying to handle a high amount of inbound writes. During this time, the flash controller charges the cell with a high amount of voltage so that the write can complete faster. New flash controllers can monitor the inbound I/O, and, when there is less I/O, they can write slower and soften the charge to the cell. Doing so should enable the drive to last longer.
Flash controllers do more than just increase drive durability, though. They can also improve performance.
One way that flash controllers can deliver better performance is by buffering inbound I/O in RAM on the SSD. Writes to flash are much slower than reads. Buffering to RAM allows writes to be processed faster, and the flash controller can use the RAM to better organize the write to flash, also increasing performance.
The problem with RAM is that if power is lost, the data in RAM is also lost. To get around this problem, enterprise SSDs use a capacitor to protect the RAM in the event of power loss. The flash controller also can initiate an emergency flush of the DRAM buffer to the SSD if power is lost.
Software-defined flash management
The next wave of flash management will not happen in the flash controller. Instead, the storage software or operating system will actually manage the flash. Previously, storage software and operating systems had no visibility into how the flash controller was managing the SSD. But now, SSD manufacturers are opening up the flash management process via the use of an API that will allow external control over the flash. This concept is known as "host-managed flash."
Host-managed flash allows the storage software or operating systems to have a better understanding of the data. In theory, if given control, a storage system should be able to better manage when functions like garbage collection are activated, or be able to automatically adjust over-provisioning rates. The result is more effective use of flash capacity and better control over performance.
While hidden from the user, flash management is an important function for IT professionals to be aware of. How a flash controller manages NAND flash can directly impact both longevity and performance.
PCIe SSDs can help with flash storage management
Predicting flash management's future
Server-based SSDs grow in popularity