CenturionStudio.it - Fotolia

DDN enlarges capacity of ExaScaler Lustre file system array

Data Direct Networks' ExaScaler array embeds the Lustre file system software within its controller. The flagship ES14K array scales to petabytes of capacity in two racks.

Data Direct Networks Inc. has doubled the raw capacity of its Lustre file system-based ExaScaler storage array, adding the ES14K flagship to replace its previous high-end ES12K.

The ES14K targets enterprises with petabyte-scale deployments, particularly research and academic teams whose applications require extremely high throughput and low latency to analyze large data sets. The ES14K is a scale-out and scale-up NAS, with two controllers in a 4U enclosure, which contains slots for 72 hard disk drives or solid-state drives (SSDs). 

One ES14K appliance scales to 274 TB of raw storage and is rated to deliver 50 Gbps throughput through the Lustre Parallel File System. One rack of ExaScaler arrays provides 7 PB of raw capacity, more than twice the 3.45 PB of the ExaScaler 12K storage, and a throughput rating of half a terabyte per second. Up to 14 PB of capacity -- two racks -- is available as a single ExaScaler system.

DDN: Our Lustre file system storage sets density record

Customers scale ES14K storage by adding up to 20 expansion trays, each with 84 slots, to create an 84U configuration with up to 1,752 8 TB drives. The ExaScaler array embeds the Intel Enterprise Edition for Lustre software within the controller for an end-to-end view of data and metadata across Data Direct Networks' (DDN) storage systems.

This is the largest capacity Lustre file system appliance on the market.
Laura Shepardsenior director of vertical markets, DDN

"This is the largest capacity Lustre file system appliance on the market," said Laura Shepard, DDN's senior director of vertical markets. "If you want to build a really superlarge system, you could use it as a single building block underneath, scale up the capacity and manage it behind a single file-system view. Or, you could scale it out to terabytes [of capacity] in two racks just by adding more controllers behind the file system."

ES14K arrays are available in all-flash, hybrid and disk-only configurations. The base ES14K array supports 2.5-inch form factor drives. Expansion trays can hold 3.5-inch drives. The all-flash ExaScaler version supports 72 SAS-connected SSDs for 274 TB of raw storage in 4U. Adding 20 fully populated flash drive trays pushes maximum flash capacity to 5,746 TB in 84U. DDN rates the all-flash version for performance of 1,050 GBps.

The hybrid model adds eight 480 GB SATA SSD drives to the SAS SSDs, divided evenly as system drives and application drives, and scales to 13,714 TB in 84U of rack space with rated throughput of 50 GBps. A single ExaScaler outfitted only with hard disk provides 130 TB of raw storage in 4U and scales up to 13,570 TB, with rated throughput of 9 GBps.

Standard features in the ES14K include two compute controllers, four 14-core Intel Haswell processors, 256 GB of DDR4 memory and redundant power supplies. As a scheme to reduce network latency, the ES14K allows some Haswell cores to be dedicated to boot virtual machines that contain Lustre-based applications running alongside transactional data.

PCIe flash, multiple caching levels address storage latency, performance

ExaScaler connectivity is supported with eight EDR/FDR InfiniBand ports, an interface for 10 Gigabit Ethernet (GbE), 40 GbE and 100 GbE, and DDN Omni-Connect, which is designed on Intel's Omni-Path storage architecture. The Lustre file system embeds applications with internal PCIe networking. The PCIe flash is designed as an over-provisioned backplane to deliver maximum bandwidth to the available drives.

ES14K supports data tiering between flash, disk, cloud and tape. System-level caching is included as a base feature with the vendor's Storage Fusion Architecture (SFA) React technology, which directs fully formed I/O straight to drives. SFA only uses the cache to remedy malformed I/O before sending writes to storage.

ES14K arrays can be used as a persistent tier of file storage and run in tandem with DDN's GridScaler array, which is based on the IBM-developed General Parallel File System. The SFA layer provides a persistent tier of block storage underneath the file storage.

DDN's Infinite Memory Engine (IME) doubles as a compute burst buffer and a flash acceleration layer. IME is available as software-only or as an NVMe hardware client that sits between compute and Luster file system storage to manage data thrashing and realign out-of-core computing. IME data protection uses erasure coding calculated at the client level.

Deployment of Lustre file system-based ExaScaler arrays has largely been in academia, government, and the oil and gas industry, although Shepard said use cases are increasing in the financial services sector. 

"We're seeing rising demand in hedge funds and trading shops to gain competitive in high-frequency trading. They are using Lustre as a consolidator with in-memory databases to compute their algorithms," Shepard said.

DDN said list pricing for the ES14K appliance starts at $60,000.

Next Steps

Lab boosts supercomputer with DDN, Lustre file system

DDN delivers Lustre to midrange customers

Oil company uses Lustre file system for HPC

Dig Deeper on Solid-state storage