The X4500, which can pack 24 terabytes
As a server, the X4500 features two dual-core AMD Opteron x64 chips. The 48 SATA disk drives attach via a passive disk plane, for 100% in-rack servicing. The server runs Sun's ZFS file system, which presents a single file system and stripes the data across all the available drives.
Matsuoka also acknowledged that his environment is particularly suited to these huge, self-contained boxes.
"Large supercomputers need massive I/O bandwidth and already have massive internal network bandwidth for parallel communication," he wrote. "I can't really think of any meaningful application [in our environment] that does not use common storage on large servers and supercomputers, and the [X4500s] provide just that."
For the institute, Matsuoka said, cost was also a factor, and when it came to the network infrastructure, multiple cheap X4500 boxes connected via InfiniBand was a far more attractive option for him than the switches and networks required for traditional networked storage.
"The addition of a mere additional 42 InfiniBand ports for storage to our existing compute network is next to nothing," since there are already 1,310 InfiniBand ports in his environment, he said.
Matsuoka also said there were a few drawbacks to putting together HPC clusters this way. The chief issue, he said, was that the X4500 has packed more compute power and I/O into its control boards to support such a large amount of hard drives in each box, placing more strain on the controller. At the same time, the control boards in each server remain a single point of failure.
"If there's a failure, data may not be lost but still may become inaccessible until the controller board and/or the network is replaced," Matsuoka said. "For time-critical applications, or where failure cannot be tolerated, the solution would be to [implement] RAID, not only internally but across the boxes" -- something he said he is still working with Lustre to implement.
Analysts weigh in
"In the storage industry, people have been talking about putting general-purpose computation close to storage for 10 years now," said Jonathan Eunice, analyst with Illuminata Inc. "It's kind of a weird product, because there's so much storage in there that it's like a new type of storage box -- not quite storage and not quite a server."
Eunice acknowledged that scaling the system out would be tricky for users, at least in the beginning. "Right now you need some sort of scale-out middleware or application to make it work," he said.
"But a lot of industries, including the financial and medical industries, need archival boxes, storing data about the workflow. This targets those types of applications."
According to David Lawler, director of product definition and strategy for the systems group at Sun, the X4500 began as a video-streaming box and was based on its acquisition of LSC Inc., a high-performance file system startup based in Eagan, Minn., back in 2001.
"If you want to analyze data, you need to take it off storage and onto the server," Lawler said. "The problem in video applications is that the amount of data you need to analyze is huge."
Generally, Lawler agreed with Eunice's statement that the X4500 is somewhere between a server and storage.
"It's almost a cache type device," he said. "It's not really enterprise class storage. Sun is looking into making this a 'true' storage device with a more complete management toolset."
Other steps that would have to be taken to make it a "true" storage appliance, Lawler said, would be to put in a specialized processor for RAID. "Right now, it has a general processor doing everything."
For now, the X4500 has two key advantages over its enterprise NAS and clustering competitors -- price and simplicity. A typical enterprise NAS system, such as BlueArc Corp.'s Titan, begins at $100,000. Other enterprise storage systems require not only a much pricier box but proprietary networking equipment as well.
"I think Sun wants to present a very attractive price for the combination of their server and large amount of SATA storage and QFS to get some business they have lost to NetApp [Network Appliance Inc.] and companies like PolyServe [Inc.] and Isilon [Systems Inc.]," said Arun Taneja, founder and analyst with the Taneja Group. "I think they will do well with it. But right now, it is still a DAS system with single point of failure."