Raisor said a system comprised of a 20 TB Hitachi Data Systems AMS1000 connected to the lab's 630-node Linux farm using Ibrix's SAN file system kept crashing. "There were many problems we encountered -- many were resolved, some were not," he said. "Our real problem was that we were spending a lot of our time chasing issues and losing productive time."
Raisor and his staff tired of constant troubleshooting and finger pointing among
Raisor discovered BlueArc at the Supercomputing Conference last November, before he learned about its HDS relationship. "I didn't realize Hitachi and BlueArc had a partnership until then," he said.
Before installing two Titan 2200 NAS heads attached to commodity and AMS1000 disk this February, the lab evaluated other NFS-based clustered file systems, including the open source Lustre product acquired by Sun Microsystems late last year. The lab avoided proprietary file systems this time around because some required host agents to mount servers and NFS will connect to practically any operating system.
A question of support
It came down to BlueArc and Lustre. "Supportability was a concern [with an open source file system]," Raisor said. "Our testing and discussion with other users of BlueArc also showed we'd get the best performance with small files."
With other file systems, Raisor explained, the software-based metadata layer can lead to latency when files are accessed, but BlueArc runs NFS on hardware within a field-programmable gate array. So far, Raisor said the 2200s have shown a 1.2 GBps throughput rate.
That, and a peak of 42,000 IOPS in production are fine for now, but Raisor said the lab will need more performance by this summer when the Linux cluster will expand by 50%. To accommodate the surge in servers, Fulton is planning to upgrade to BlueArc's newer Titan 3000 NAS head. "That upgrade was part of our purchase agreement," Raisor said.
Also on Raisor's wish list going forward is support of Posix open source access control lists, which he expects BlueArc to support by the third-quarter of this year.