Add Apeiron Data Systems to the list of vendors peddling ultra high-performance, low-latency storage systems based...
on emerging nonvolatile memory express (NVMe) technology.
The Folsom, Calif., vendor launched in 2013 and quietly released the Apeiron Data Systems (ADS) 1000 in late 2016. The company plans a formal product launch early next year after securing Series A funding, according to Jeff Barber, Apeiron's executive vice president of sales and marketing.
Apeiron Data Systems architecture
The Apeiron Data Systems architecture consists of three components: the array/storage enclosure, compute node and host bus adapter (HBA). Of the three, the HBA is where the bulk of Apeiron's intellectual property resides, Barber said.
"We're more of a storage network company than we are a storage company," he said.
Each 40 Gbps ADS40G-HBA, manufactured by NEC, contains two 40 Gbps Ethernet ports and plugs into the application server's PCI Express slot. The HBA adds a four-byte wrapper to each NVMe command and moves the block-based data packet via Layer 2 Ethernet to a set of three field-programmable gate arrays (FPGAs) from Intel -- via its Altera acquisition -- in the storage enclosure, Barber said.
Customers don't need a separate network infrastructure. Each of the two I/O modules in the 2U ADS1000 storage enclosure embeds a 40 Gigabit Ethernet (GbE) switch from Mellanox and 16 40 GbE ports. The modules are connected with eight dedicated lanes between the switches.
Each I/O module also includes an Apeiron Storage Controller (ASC), consisting of the three Intel FPGAs, and a storage management chip. But Barber said the ASC is not a traditional storage controller that would handle protocol processing and functions such as compression and RAID parity checks. The ASC's main function is stripping the four-byte wrapper from the data packet, he said.
"There is some storage intelligence obviously and some monitoring going on," said Barber. "But the real point there is that we rely on the application and the operating system to manage our storage."
He noted, for instance, that one of Apeiron's key target applications, Splunk, does compression and manages replication.
Apeiron claims the ADS1000 can hit 18.4 million IOPS and a maximum bandwidth of 72 GBps based on 4K random read tests in a fully loaded appliance. NAND latency is about 100 microseconds, and Apeiron's NVMe over Ethernet (NoE) is less than three microseconds per round trip, depending on the workload and NAND flash, according to the vendor.
Barber said Apeiron has found a way to move the NVMe command from the server to an externally attached SSD that writes data faster than using an internal disk. "You can only do that when you have an ultra-fast network," he said.
Jeff Barberexecutive vice president of sales and marketing, Apeiron
He said upcoming Intel 3D Xpoint Optane drives would reduce the latency to less than 10 microseconds.
The 2U ADS1000 appliance holds 24 NVMe SSDs (12 per I/O module) and offers options for write- and read-intensive drives. Barber said Apeiron has qualified NVMe SSDs from Intel, Micron, Samsung and Toshiba, with others expected in the future. Current SSD raw capacity options are 800 GB, 1.6 TB, 3.2 TB, 6.4 TB and 7.68 TB. The ADS1000 maxes out at 192 TB when filled with 7.68 TB SSDs, and the limit will increase when 11 TB SSDs become available.
Apeiron's single rack-unit compute node is optional. Barber said the ADS-N8101 is a standard x86 dual-processor server with Apeiron's HBA integrated. He noted that early customers have preferred to buy the Apeiron compute node rather than risk breaking their server vendor's warranty by opening up the box to install the Apeiron HBAs.
Target workloads for the ADS1000 include Splunk, Apache Spark, Hadoop and other applications designed to ingest massive amounts of machine data for analysis, SAP HANA, NoSQL databases, electronic discovery and elastic cloud.
Barber said an ADS1000 with six Linux application servers, an Apeiron storage fabric switch, fabric management software and two dozen 3.8 TB NVMe SSDs would have a list price of $598,000. An ADS system with eight Linux application servers and 7.68 TB NVMe SSDs would list at $945,000. The Apeiron HBA has a list price of $3,500, and the Apeiron enterprise-class 1U dual-processor server with 32 cores and 128 GB of RAM sells for $15,200, according to Barber.
Apeiron's engineers hail from network adapter vendors Emulex -- now owned by Avago/Broadcom -- and QLogic -- now part of Cavium -- as well as Intel and NetApp. NEC Japan provided two years of engineering, manufacturing and logistical support in exchange for access to Apeiron's Layer 2 Ethernet transport technology, Barber said.
Apeiron took its name from the Greek word meaning limitless. The array provides networked access to an external pool of potentially thousands of high-performance NVMe solid-state drives (SSDs). But the design goal is for the ADS1000 to look and act like internal block storage.
"The entire point of this system is throughput and speed – getting out of the way of the data," Barber said. "Because we remove the [traditional storage] controller and the external switch, we remove a tremendous amount of latency."
Apeiron Data Systems competition
Eric Burgener, a storage research director at IDC, noted via an email that most applications don't yet need the level of "blazing fast performance" of "rack-scale flash" products. He said there were fewer than 30 customers using those types of rack-scale flash systems as of Dec. 1, 2016. But he predicts the market will grow rapidly over the next three to four years as real-time big data analytics become more prevalent.
Howard Marks, founder and chief scientist at DeepStorage LLC, envisions the technology trickling down to more mainstream storage vendors that would offer more features at lower cost and support the emerging NVMe over Fabrics (NVMe-oF) specification. NVMe-oF is designed to extend the benefits of NVMe across a network. Marks said Apeiron's proprietary NVMe over Ethernet technology might offer better performance, but the NVMe-oF-based products will be more accessible.
"Standard approaches almost always win," Marks said.
Marks said he has witnessed other attempts to "take storage and view it as a networking problem and integrate networking into the storage solution," as Apeiron has done. "I have yet to see that be successful," he said.
Apeiron submitted its NoE architecture to NVM Express for consideration as an alternate transport protocol with NVMe-oF. The firm's Barber said NVMe-oF's current remote direct memory access-based options for Ethernet are designed to support legacy storage systems with rich feature sets, but they carry greater overhead than Apeiron's NoE, require more storage processing, and could limit linear scaling and performance.
"We believe we address two different needs," he said.
Amber Huffman, chair of NVM Express, wrote in an email that the organization received Apeiron's proposal too late for consideration with the NVMe-oF 1.0 specification. She said the group would begin planning NVMe-oF 1.1 in the first quarter of 2017. "If there is ecosystem pull for this [Apeiron] feature, it will be added to the scope for the next version of the specification," she wrote.
NVMe will get hot in 2017
NVMe over Fabrics emerges
Pure Storage readies its arrays for NVMe