According to Pixar's vice president of technology Greg Brandeau, the making of Cars included computer animation effects that were unprecedented -- and required unprecedented amounts of throughput from a new parallel file system, which the company wound up deploying in the midst of the movie-making process.
So rich was the imagery involved in Cars that in the beginning, "it broke all our systems, basically," Brandeau said. By the time the film was finished, it had used 23 CPU millennia, or the
Previously, the company had fed data to its animators and technical directors working on images through a Network Appliance Inc. (NetApp) FAS 960 filer. This powerful system had handled the load of even its most recent film, Monsters, Inc., but proved overmatched by Cars.
On Monsters, Inc., each NFS cache delivered 20,000 NFS operations per second with a 95% hit rate on the RAM cache. When work began on Cars, the NFS caches strained under the load, delivering only 2,000 NFS operations per second. It was clear the storage system would need to be switched out. Pixar disconnected the filers and went with a parallel file system from Ibrix Inc..
Brandeau said he mounted the Ibrix file system onto a dozen 32 GB NAS heads using commodity Linux servers from Dell Inc. Four of the 12 feed into the approximately 675 technical directors' and animators' workstations, and the other eight process exporting images from the model farm to the render farm, which is where the images are finished and put into animated movie form. The new system is backed by a 4 terabyte EMC Corp. Clariion CX700 SAN.
The parallel file system allows access to files simultaneously, giving the Pixar animators a bigger pipe to draw from. "We actually found that the access is faster than local disk," Brandeau said. "And balancing the file system across multiple nodes helped us avoid disk hot spots."
Brandeau emphasized that his move to an Ibrix file system was not a knock on NetApp. "They're a great company and we love them," he said. "But we had gone from an enterprise computing operation, which is their market, to a high-performance grid computing operation, which is outside their main expertise."
Still, he admitted, the Ibrix file system had been a lifesaver. At the rate Pixar had been going, Cars would not have made its production deadlines.
"It wouldn't just impact our revenues," Brandeau said. "We'd told everyone we were making a movie, and there were other people out there busy making Happy Meals and clothing and toys to support it -- we estimate that data was worth about a billion dollars."