The University of Florida in Gainesville is advancing everything from biomedical engineering to physics, but innovation doesn't stop with the researchers. When a new data center was built to house a high-performance supercomputer, the university
As the home base for research across 85 different disciplines, the University of Florida (UF) has 175 different faculty/student research groups that often collaborate with others in their fields around the world.
"Everybody is collecting data and [working on] complicated problems that single investigators cannot solve, so you have collaborations that are going to be across the world and nation," said Erik Deumens, UF's director of research computing. "So, all of a sudden, there's a much bigger need to store data."
In January, UF upgraded from a 10 Gbps switch to a 100 Gbps switch with a grant it received from Internet2, an organization comprising more than 200 research universities dedicated to advancing research through innovations in technology. Meanwhile, the university was building a 16,000-core supercomputer called the "HiPerGator" to support its research; it would be housed in a new $15 million data center that had been in the works for five years.
"We wanted to make something that really stood out, and that's HiPerGator," Deumens said. "So in May 2012 we started to work with several vendors, and eventually we picked the partnership between Dell and Terascala."
Building the infrastructure for the new data center was a staggered process, and the Terascala storage appliance was the first part to be implemented. It provides a parallel file system with a usable capacity of 2 PB, and it's connected to a Dell cluster using InfiniBand from Mellanox. The data center opened in May 2013, and Terascala supports all the research computing done on campus. It currently stores 200 TB of the campus-wide total of 8 PB.
"Our main concern was that the disk system had to be very responsive and very robust in the sense that many jobs were going to ask for data randomly because they're completely different jobs from different users. So we wanted to be sure the system wasn't going to fall over," Deumens said.
He said UF looked into other high-performance storage appliances, but because university engineers had previously custom built a smaller appliance similar to that of Terascala, their familiarity with it, as well as its management capabilities, influenced the purchase decision.
In addition to Terascala and the custom storage system built by UF engineers, Deumens said UF has several systems from Nexenta. Those are housed among three older, smaller data centers on UF's main campus and are largely used for data protection and backup. The data stored on Terascala, however, is not automatically backed up. Instead, researchers are responsible for backing up their own data, and have the option to purchase space on the Nexenta systems for that purpose.
"The cost of buying tapes to keep up with that capacity would keep going up, and 90% of that data is never read again. We don't have an automatic process because we'd run out of space," Deumens said.
Currently, because researchers often strike up collaborations with other universities, UF is working with the State of Florida and the Sunshine State Education & Research Computing Alliance to implement state-wide storage for long-term use, which they hope to carry out in the next three to six months.