Coraid Inc.'s EtherDrive products provided the City of North Canton, Ohio, with a lesson in ATA over Ethernet (AoE) storage when it was trying to deal with the increased demands of a growing school district and capacity-sucking multimedia files.
With more than 5,000 administrative, teacher, student and remote users, network systems administrator John Fano needed a system that was available 24/7, simple to use and accessible to multiple types of servers.
But as public and private funding dried up, Fano's small budget ruled out Fibre Channel (FC) and even iSCSI solutions. "We just don't have the budget," Fano said. "So cost was a big issue."
Fano researched the little used AoE data transfer protocol, and was set to build his own system when he accidentally ran across Coraid's ATA over Ethernet-based storage solutions during a Google search.
Fano acquired two Coraid Inc. EtherDrive SR1521 storage appliances, one EtherDrive VS21 virtual storage appliance, and five EtherDrive SAN HBA 2 Gbps PCI-E cards. Each EtherDrive SR1521 shelf has 15 7,200 rpm 250 GB SATA disks chunked into RAID 5 arrays. The list price for the EtherDrive SR1521 is $7,495 ($3,995 without disks).
"The [VS21] storage virtualizer allows us to mirror the shelves together," Fano said. "So the shelves are set up with individual RAID 5 arrays, and then one is mirrored to the other. We did it that way for disaster recovery."
Fano installed the appliances himself with phone and email support from Coraid. The only additional work required to allow the Coraid AoE to work with Fano's system was enabling jumbo frames.
Brantley Coile, Coraid's founder and chief technology officer, was involved with the AoE protocol development. Coile describes it as "an Ethernet-based protocol that uses remote procedure calls [RPCs] that ride directly in Ethernet frames, not on top of IP, not on top of TCP." Because the data protocol doesn't depend on TCP/IP, it can use all available links and paths from the client initiator to the target storage device.
"The objective in developing the protocol was to have a SAN system that wasn't burdened with Fibre Channel culture requirements," Coile said.
There are two Ethernet connections from each EtherDrive SR1521 running to the VS21 virtual storage appliance, Fano said, and two Ethernet connections from the virtual storage appliance to a Cisco Systems Inc. Catalyst 4503 switch.
Fano said he uses the Coraid system for the district's file services, and it houses all user data and long-term archives. He sends backups offsite to a county facility. The only problem he's had during the last three years was with a Microsoft Windows' driver unrelated to the Coraid system. "[The driver] was reporting single-drive failures in the RAID 5 array as a failure of the array," he said. "We got that straightened out pretty quickly."
Coraid tech support reviewed the logs to make sure it wasn't an issue with its equipment.
Fano just started using the EtherDrive host bus adapters for his virtual environment, which he said is almost the entire school district system. He was a part of the HBA beta program and reports no problems with the Coraid system or the VMware virtual servers.
The increasing use of multimedia files, such as student-made videos, high-resolution pictures for district newspapers and yearbooks, and streaming video from Discovery Education has pushed the Coraid system to its capacity limits. The multimedia files now make up 75% of Fano's data. The district's daily backup increases between 2 GBs and 5 GBs per day during the school year. Fano plans a full upgrade this summer, swapping the 250 GB drives for 1 TB drives. "We'll just break the mirror, upgrade one shelf, transfer the data, and then re-mirror the data when we bring the second shelf up," he said.