Manage Learn to apply best practices and optimize your operations.

ATA over Ethernet protocol overview for data storage managers

The ATA over Ethernet (AoE) protocol, backed by Coraid, is used by some data storage managers for low-cost, high-volume storage. Learn 10 AoE protocol facts

What you'll learn: Get an overview of ATA over Ethernet (AoE), discover why the fate of the AoE protocol is joined to the business prospects of Coraid Inc. and learn 10 must-know facts about ATA over Ethernet.

ATA over Ethernet (AoE) was designed by a group of techies from Bell Labs and is an open-source community project. The primary corporate backer of the AoE protocol is Coraid Inc., the main commercial vendor of AoE storage arrays. A few smaller companies also support the protocol, and it has a dedicated following among hardcore open-source storage hackers.

Proponents claim AoE delivers reliability and performance at very low cost thanks to its basic design. The guiding principle behind ATA over Ethernet is simplicity and performance over a local network, and many of the archaic assumptions of data storage have been abandoned in favor of a new approach.

Of course, converging storage over Ethernet is nothing new: IT architects have relied on network-attached storage (NAS) protocols like NFS and SMB [now known as CIFS] for two decades, and iSCSI is now firmly established as a Fibre Channel (FC) alternative for storage-area networks (SANs). Fibre Channel over Ethernet (FCoE) is a darling of the convergence proponents, but these aren't the only choices: A scrappy alternative called ATA over Ethernet has been around even longer than FCoE.


Although it differs a great deal from FCoE in philosophy, ATA over Ethernet shares some aspects with that new data center protocol: It translates the existing Advanced Technology Attachment (ATA) storage protocol directly to Ethernet networking rather than building on a high-level protocol, as iSCSI does with TCP/IP.

ATA over Ethernet as alternative to SAS

SCSI, the core data protocol of both iSCSI and FCoE, was designed to run over a single reliable channel, not a multiconnection unpredictable network like Ethernet. Much of the technology behind FCoE exists merely to make Ethernet reliable and predictable enough to stand in for Fibre Channel, and iSCSI relies on TCP to retry in the event of an error.

ATA over Ethernet, in contrast, was designed for Ethernet. AoE isn't connection based, so I/O can flow across the network in as many packets and links as possible, increasing performance. AoE has its own TCP-like congestion avoidance system and adapts to changing network conditions.

Stacking Coraid arrays for massive capacity

The dominant use case for this technology is low-cost, high-volume storage. ATA over Ethernet clients can seamlessly map storage to multiple arrays and the protocol requires less processing power than FCoE, iSCSI or NFS. This simplifies the storage arrays and host drivers, driving down cost.

The few AoE users I have met use it as an alternative to SAS in volume storage segments, stacking Coraid arrays high to provide massive capacity. But they also praise its scalable performance, claiming it makes maximum use of every Ethernet link they throw into the mix.

An administrator at a financial company (which requires him to remain anonymous) compared their ATA over Ethernet infrastructure to the notorious invasive plant, kudzu. "It grows like crazy, pulling in applications and data like nothing else in the data center."

And Coraid EtherDrive products provided the City of North Canton, Ohio, with a lesson in ATA over Ethernet storage when it was trying to deal with the increased demands of a growing school district and capacity-sucking multimedia files. This case study is a good example of how AoE can win customers who can't consider FC or even iSCSI based on cost.

The fate of the ATA over Ethernet protocol is inextricably joined to the business prospects of Coraid. Regardless of technical capabilities, is it wise for an enterprise to become reliant on an unusual protocol mainly supported by a single small vendor?

Coraid has lasted longer than many pundits thought, and a new management team recently took in additional VC investment. Coraid's arrays are inexpensive and simple to implement. Although migration to and from new storage systems always poses challenges, ATA over Ethernet is no harder to integrate than a SAS, iSCSI or FC solution.

ATA over Ethernet support, integration remain challenges

Server support is one key area of concern. Software drivers are widely available, but ATA over Ethernet isn't natively supported by operating systems other than Linux and OpenBSD. VMware ESX would seem to be an ideal match for the high performance and scalability of AoE, but there's no driver in ESX. Similarly, storage virtualization and data protection systems don't directly support ATA over Ethernet, though products like Symantec Corp.'s Veritas Storage Foundation should work fine with it.

Far more critical is the lack of any sort of integration with other products. VMware administrators want vCenter integration and vStorage APIs for Array Integration (VAAI) support, but these aren't to be found. Coraid's arrays also lack thin provisioning, deduplication and tiered storage capabilities. The company would counter that the intrinsic low cost and scalability of the solution removes the need for these capacity optimizations, but the lack of snapshot and replication capabilities is more concerning.

Last year, Coraid launched a series of 10 Gigabit Ethernet (10 GbE) SANs that support SAS, SATA and solid-state drives (SSDs) on the same system, which the company is targeting for highly virtualized and cloud storage environments. This Coraid offering is a higher performing platform. Coraid also has an entry-level SR Series, which supports only SATA and Gigabit Ethernet.

Still, if the ATA over Ethernet protocol is to gain broad acceptance, it must be more than just a niche offering for low-cost capacity. But it isn't clear if the open-source community and a single revitalized company can make this happen.

BIO: Stephen Foskett is an independent consultant and author specializing in enterprise storage and cloud computing. He is responsible for Gestalt IT, a community of independent IT thought leaders, and organizes their Tech Field Day events. He can be found online at,, and on Twitter at @SFoskett.

Dig Deeper on All-flash arrays