"It's beyond daunting," says Gary Grider, computer scientist at the Los Alamos National Laboratory, of the challenge facing the new Petascale Data Storage Institute (PDSI).
The U.S. Department of Energy (DOE) has awarded a five-year, $11 million grant to the PDSI to find ways to manage the torrent of data produced by next-generation supercomputers, modeling phenomena such as global weather and nuclear stockpile safety.
"They'll require up to hundreds of thousands of magnetic hard disks to handle the data required to run these simulations," says Garth Gibson, associate professor at Carnegie Mellon University in Pittsburgh.
But to get meaningful results, the supercomputers must run for months without downtime, says Grider.
The PDSI is working with vendors to develop "self-star" systems--software-based techniques for automatically managing relationships between supercomputers and storage. "Think of computers fixing computers," says Gibson.