San Rafael, Calif.-based Whiskytree does visual effects and post-production for major motion pictures, including Thor and Terminator Salvation. It uses one network for both visual effects and rendering, but Whiskytree CEO and creative director Jonathan Harb said performance on his Xsan began to degrade when he first tried to connect more than 100 render nodes to it in addition to the company’s Fibre Channel (FC) workstations.
“We use a minimum of a 100 render nodes at any given time,” Harb said. “The bandwidth that's required ranges from hundreds of megabytes to several gigabytes of data. You kick off the rendering and you're going to have 100 nodes asking [the file server and the Xsan] for the same data at the same time. The number of IOPS hitting the Xsan was high. The file server started freaking out. It actually stopped working.”
The company’s artists work on FC-attached workstations that depend on the rendering farm to build and compose final shots. Whiskytree’s rendering farm comprises dense servers operating on dual, six-core processors and 24 GB of RAM connected to Ethernet switches. That server farm is connected to a NAS-based file server that, in turn, is connected to a FC-based Xsan. A NAS gateway provides the translation between the storage-area network (SAN) and NAS protocols.
“The file server is a Fibre Channel client of the SAN,” Harb said. “We map the SAN as an X drive on Windows and it appears as a volume. The file server gets access to the volume and re-shares the volume with CIFS protocol. That allows all the Ethernet-connected rendering nodes to have access to the SAN via the file server.”
Whiskytree installed a clustered pair of Avere FXT 2300 appliances that sits between the file server and the rendering farm. The Avere appliances offload data requests from the rendering farm, reducing the I/O workload hitting the file server. An Arista 7124SX 10 Gigabit Ethernet switch provides a non-blocking backbone connecting the SAN to the render servers. Harb said the new setup has more than quadrupled performance and scales the amount of render nodes needed to process special effects in short time frames.
Rebecca Thompson, Avere Systems’ vice president of marketing, said Whiskytree’s rendering work had grown to the point where they would have had to implement two architectures, one for the special effects design and the other for rendering. “In many production houses, these are separate architectures, but Whiskytree wanted to keep the architecture unified because it’s easier to manage and not as costly,” she said.
“We need a lot of data to be accessed over and over quickly in this organization,” Whiskytree's Harb said, “and the system has to be rock solid. We have such reliable performance now; it's outstanding [and] so few IOPS hit the file server now. It’s such a wise way to approach this function. At some point, the Avere will get overloaded but we haven't reached that point yet.”