What is memory channel flash storage and what does it offer that PCIe connected flash doesn't?
Memory channel flash storage, also known as "in-memory storage," is flash storage that inserts or connects directly into the DIMM slots of a server motherboard. Currently, in-memory flash storage is only available from SanDisk as ULLtraDIMM. SanDisk has an exclusive agreement with Diablo Technologies that marries their DDR3 DIMM-based flash storage to SanDisk's Guardian software. PCIe flash connects to the PCIe channel. In-memory flash storage has several advantages over PCIe flash storage.
The number of server DIMM slots greatly exceeds the number of PCIe slots. Take the example of Intel's latest E7-8800 v2 family. A quad socket motherboard delivers 96 DIMM slots. At least 2 DIMM slots per socket are required for DRAM, leaving 88 slots for in-memory storage. That same motherboard has 4, 8 or 16 PCIe slots. Some of those slots must be utilized for network interface cards, storage adapters, clustering or other peripherals. That leaves only a small number for PCIe flash storage cards. There are almost always surplus DIMM slots in a server, whereas PCIe slots are precious and rare. Advantage: In-memory flash storage
Because in-memory flash storage looks like DDR3, writes and reads are performed in parallel. Parallel writes and reads means consistent high performance. Multiple PCIe flash storage cards with parallel writes and reads is still a work-in-progress. Advantage: In-memory flash storage
The general advantage of server flash over external flash storage systems is lower latency. Lower latency comes from the shorter distance from the application to the flash storage. PCIe flash storage has lower latency than embedded SATA or SAS flash SSDs because it has less distance to travel. The SAS or SATA SSDs must traverse an additional controller sitting on the PCIe channel and then the PCIe channel. In-memory flash storage has lower latency than PCIe flash storage because it does not have to traverse the PCIe controller or deal with contention from other cards for bandwidth on the PCIe channel. Advantage: In-memory flash storage
Flash latency consistency is an issue for many high performance applications such as high-frequency trading, life sciences, HPC applications in general, seismic data interpretation, reservoir modeling, fluid/flow simulation, aircraft simulation, 3-D imaging/modeling, and more. In-memory flash storage has tested out to have a much more consistent and smaller latency range (an order of magnitude smaller range) than the best PCIe flash cards. Advantage: In-memory flash storage
Cost per gigabyte tends to be a little higher for in-memory flash storage than PCIe flash storage. Advantage: PCIe flash storage
PCIe flash storage is a more mature technology today than in-memory flash storage, with far more vendor products and choices. There is also more software available written specifically for PCIe flash storage. Advantage: PCIe flash storage.
PCI Express access for flash arrays
Flash PCIe solid state shareable among physical servers
Server PCIe flash cache trend catching on with vendors
Is PCIe SSD right for you?
Dig Deeper on All-flash arrays
Related Q&A from Marc Staimer
Network File System and Common Internet File System/Server Message Block were designed to work with any operating system, but NFS remains dominant in... Continue Reading
Object storage has unique features, including erasure coding and multi-copy mirroring, which may make it better suited to data protection than more ... Continue Reading
Why would you attach NAND flash storage directly to the memory channel? Isn't RAM much faster than NAND? Marc Staimer discusses this and more in this... Continue Reading