EMC, IBM and Hewlett-Packard are planning to ride a wave of growth among Web 2.0 and multimedia companies in the coming year with clustered 'massive scale-out' storage systems.
But while these data storage heavyweights are still developing new clustered storage systems, multimedia companies like Turner Broadcasting System Inc., ESPN and DISH Networks have deployed a grid storage system from Omneon Inc., which offers features like transcoding and automated replication on top of clustered storage.
Until last year, Omneon's product was the Spectrum Media Server, used for playback at TV stations owned by Turner, Discovery Networks, Viacom, PBS and DirecTV. The Spectrum server comes with direct-attached RAID storage, but increasing demands for storage capacity from its customers led Omneon to create a new clustered storage system called MediaGrid.
MediaGrid consists of 1U servers with four drive bays and dual Gigabit Ethernet connections. The grid is managed with a single metadata server, but, according to director of storage marketing Dave Frederick, MediaGrid does most of its processing on the nodes in parallel; the metadata controller doesn't handle all the throughput of the system.
MediaGrid's software automatically replicates files across nodes, and, like most clustered systems, adds processing power as it adds capacity. MediaGrid also creates more copies of files that are requested the most frequently, then shrinks the number of copies back down when they're no longer in demand. This technique is widely used in Web content delivery systems and is also a feature of a grid storage product from startup Parascale. Online storage hosting provider Nirvanix also offers this feature.
The first customer for MediaGrid was Turner Broadcasting, which has had MediaGrid in production at its London facility for about a year and is planning to expand the deployment to multiple locations. According to Clyde Smith, senior vice president of global broadcast technology and standards at Turner Broadcasting, MediaGrid's flexible replication was a big selling point.
"There are certain patterns to a media lifecycle," Smith said. At the beginning of its life, a multimedia file is heavily accessed by technical producers looking to edit and code it for delivery and salespeople looking to design pitches to potential advertisers. Once the file has had its first run on television, it goes all but dormant. . .but if it's later promoted again or reused, performance needs will spike again. "This doesn't fit today's storage models very well—we're not seeing bandwidth being added to storage in proportion with drive capacities, or the ability to scale the number of replications up or down."
Elsewhere among its divisions, Turner has EMC equipment, and Smith said Turner isn't opposed to considering EMC's clustered storage system, code-named Hulk and Maui and due out this fall. He noted, however, that "Omneon has been adding features while others are still getting products off the ground," citing a partnership with Pro-Bel Ltd. that allows Omneon's transcoding software to run directly on the grid, so files can be transcoded while they're also being accessed for playback or editing.
Omneon also recently acquired Castify Networks, a maker of file transfer software maker, which will allow management and movement of content across multiple grids, something Turner is considering for multi-site disaster recovery.
"The ability to have a half-hour show ready in three minutes is great in the broadcast industry," said another Omneon user, Chuck Stanley, chief engineer for Retro Television Network (RTN), a Texas-based "library" network that runs syndicated shows such as Laverne and Shirley as well as movies from the 1960s and 1970s on a regularly rotating schedule. MediaGrid allows RTN to keep multiple seasons of shows handy, but access them quickly when they're needed.
Stanley said that Omneon understands the multimedia business better than HP, whose servers and desktop PCs are the mainstays of his company's IT environent. "HP has a huge support organization, but you might not get the quickest response from them," he said. "In an industry where time is so important, you have to have someone understand that."
This isn't to say all the bugs have been worked out of a relatively new product—or that it has all the features users are looking for. According to Smith, Turner is doing ongoing development with some playback and transcoding applications so they get the most performance out of the grid. "A lot of the applications we use need better optimization for multithreaded environments so they'll get the maximum benefit."
Stanley said he's hoping for more applications that can run on the grid like Pro-Bel's, such as a spider that could correct or flag video and audio levels, which are held to strict standards by the FCC, in the background as the system runs.
Still, according to Brian Garrett, technical director of Enterprise Strategy Group (ESG) Lab, "There's a really good chance big companies will have a run for their money" when they release systems into this market. "The place they're running towards is where Omneon grew up."
Omneon has also announced eTech Ohio, Mississippi Educational Television, Norwegian state broadcaster NRK, Pappas Telecasting Companies, Spanish-language broadcaster Televisa, and The Weather Channel as customers. The vendor is expanding its marketing to reach more mainstream enterprise companies as well, and is in its "quiet period" in anticipation of an IPO this spring.
"Over time, companies like EMC and HP will find ways to deliver the same services," Garrett added. "It's just a matter of time." Ultimately, "it's hard to tell where this market will end up. There's still room for standalone companies to create their own brands and billion-dollar markets, and right now both big and little companies are fighting for that spot."