Creating the animated film "Planet 51" was a seven-year project for Ilion Animation Studios, and putting together...
a networked storage infrastructure was a big part of that adventure.
The Madrid, Spain-based studio was founded in 2002, and got its first full-length movie into production when "Planet 51" opened last month. Ilion turned to BlueArc Corp.'s Titan clustered network-attached storage (NAS) three years ago as its animators began intense work for the feature.
Ilion Animation Studios had no data storage expertise when its artists began creating "Planet 51," but had specific needs. Chief technology officer Gonzalo Rueda said the studio needed storage that could grow as the project intensified, handle large files, migrate data between tiers of storage and support CIFS because Ilion is a Windows shop.
He said the studio evaluated NAS systems from EMC Corp., Hewlett-Packard (HP) Co. – Ilion uses HP workstations – and NetApp Inc. before picking BlueArc. He said BlueArc was priced better than competitors and built specifically for files. "For vendors that do not live in the NAS storage space, it's more like configuring a SAN [storage-area network] and putting NAS heads in front of it," Rueda said. "BlueArc fulfilled our needs for bandwidth and IOPs. We had one Titan head to start, while other vendors were talking about clustering two and three nodes. I feel the less devices you have in your infrastructure, the less that can break."
Rueda said it also helped that BlueArc's sales and support people had a better understanding of the needs of a media company.
Ilion Animation Studios installed its first Titan head three years ago. "It was a two-phase project," Rueda said. "We're a startup, so our first deployment was a single Titan head with SATA drives; then we upgraded to add a second Titan head with Fibre Channel drives."
The studio didn't have to upgrade as early as anticipated. Rueda said Ilion was up to 160 artists with 200 render nodes before performance started to degrade with a single Titan head and SATA drives. Ilion added the second head and Fibre Channel approximately a year ago, two years after the original purchase. During the final phase of production, Ilion had 90 TB clustered on the Titans and used BlueArc's Data Migrator software to move files between data storage tiers. The Titans supported 300 artists and 300 rendor nodes working around the clock compiling frames.
He estimated Exchange and business data take up less than 1 TB of storage on Ilion's Titans. The rest of the capacity was used for animation.
"We've done some things in brute force ways," Rueda said. "On the storage where we store frames, we have a 50 TB file system with 30 million files shared as one folder. That workflow poses issues, such as backup of a 50 TB file system. And changing permissions in Windows can take hours. We were looking for a way to add not so much more disk, but more clusters to distribute storage and ease infrastructure strain. We wanted to split up our storage and network bandwidth. We do move quite a bit of data around, but when we upgraded to Fibre Channel disk we stopped having bottlenecks."
Ilion takes a two-pronged approach to data backup. It uses a Sun Microsystems Inc. StorageTek LTO-4 tape library for data generated by human artists and backs up computer-generated data to disk.
"If part of the computer-generated data is lost, you can regenerate it," Rueda said, explaining why those files are kept on disk. "The stuff that the artist generated gets backed up to tape, so we can restore it if we have a disk failure. At the peak, we had 10 terabytes and six million files on tape. As for our final frames, that was trickier and backup Windows grew larger. We backed up only sequences as they were finished and closed, then once a month we would do a full backup."
As Ilion Animation Studios moves on to its next project, Rueda said he has one item on his wish list for BlueArc: "I would love to see the ability to shrink file systems, although I don't know if anybody can do that. And anything that tunes performance is always good."