Managing and protecting all enterprise data

itestro - Fotolia

Video storage takes its turn in the spotlight

While organizations create high-definition video for a variety of uses, they turn to today's storage systems to house and protect it.

Back in 2010, Frost and Sullivan projected massive growth in the sale of non-linear video editing software worldwide, claiming that market size would jump three-fold to top $1 billion by 2016. They undershot the estimate. By 2014, the video editing software market had produced $1.61 billion in revenues and Frost & Sullivan's new estimate shows that the market for video editing wares will climb to $3.17 billion by 2020.

The business community at large has embraced video, and demand for these products goes far beyond broadcast media and entertainment and large post-production editing houses. In fact, the real growth in the video storage market has been happening horizontally -- across virtually every kind of business and in most agencies of government. If you spend any time on the Internet or social media, the proliferation of video storage is obvious.

Real world video challenges

Aaron Brenner, director of production for the Los Angeles Kings hockey team from 2008 until 2015, watched corporate video use explode firsthand.

Today, you would be hard-pressed to find any team that isn't producing their own content and telling their own stories.
Aaron Brennerformer director of production, Los Angeles Kings

"When I started with the Kings, there were only a handful of teams in pro sports who were investing in producing their own stories. Sports teams relied on media and entertainment outlets," he said. "Today, you would be hard-pressed to find any team that isn't producing their own content and telling their own stories. In sports, there's a new story every day: stories about players, about big picture stuff, about what happened in last night's game and about what will happen tonight. It just makes sense to bring the workflow in-house."

For Jose Eli Villanueva, multimedia designer and producer with the County of San Diego's Communications Office, his work with video began with a local cable channel program, covering board of supervisor meetings and occasionally developing training programs. "That effort morphed into video for a website," he said. "Now, we create video material for several departments of the government and video presentations for county officials [in addition to the] website and TV access channel."

At the University of Notre Dame in South Bend, Ind., the investment in media capabilities was intended to help the University "tell the story of Notre Dame through the lens of athletics," said Scott Rinehart, program director for Fighting Irish Media. "There is more to what these student-athletes accomplish than just on the field."

Bottom line: Video is everywhere. And with video comes a need for storage.

SAN for speed and sharing, tier for cost

Fighting Irish Media uses a SAN for primary storage, according to Rinehart. When it was originally deployed, the system totaled about 64 TB in capacity, "but we blew through that rather quickly," he said. "Producers were gathering between 75 GB and 150 GB of video data per game, and it quickly became apparent that we couldn't afford to keep all of the data on disk storage. It was unsustainable."

So, Notre Dame offloads video to a secondary storage tier in the form of an enterprise tape library from Spectra Logic, the T950. "We needed a way to keep a local archive for fast access, in case a news organization wants some video in a hurry to help illustrate a breaking news story," said Rinehart. "That happened recently when a Notre Dame athlete was named NCAA Woman (Athlete) of the Year. LTO tape with the Linear Tape File System provides the access speed at a low cost."

We needed a way to keep a local archive for fast access, in case a news organization wants some video in a hurry to help illustrate a breaking news story.
Scott Rinehartprogram director, Fighting Irish Media

At the County of San Diego, Villanueva said that his 32 TB FC SAN from Studio Network Solutions delivers the right amount of speed for video sharing, provided that he backs older project data off of the primary storage on a routine basis. "It's the only way [to do this economically.]"

To accomplish this task, Villanueva selected StrongBox from Crossroads Systems, the V48 model that combines a server head and a 75 TB LTO tape library as a secondary storage repository.

Breaking tape tradition

"Smaller companies are less likely to need tape in their infrastructures, even as tape becomes more relevant in large environments requiring low costs and long-term archiving capability. One exception is M&E companies that use LTFS-based tape in an almost film-like fashion. In this environment, video data is pulled from P2 flash cards and copied to LTFS tape. Once the files are ingested, the original LTFS tape is removed from the library and stored in a vault. The use of tape in the non-traditional M&E shop usually depends on the ecosystem partners present in the environment and the organization's chosen application of tape."  -- Matt Starr, CTO, Spectra Logic

Villanueva said, "The StrongBox solution expands our FC SAN disk storage seamlessly by extending the file system directly out onto high-capacity, low-cost tape." The tiering strategy, he says, has helped contain costs for his video production efforts. Plus, when he needs data back for a project, he can retrieve it conveniently from his tape subsystem.

He noted that storage clouds were considered, but dismissed. "We looked at clouds but determined that the strategy is unsustainable. For one thing, we could not retrieve data fast enough over links that we could afford. For another, the response time of an Amazon Glacier service negated the prospective cost savings from its two cents per gigabyte per year. We asked how long it would take to get our video back if there was a sudden need and we heard that the request would be handled within five hours of the request. That wouldn't work for us."

According to Brenner, an FC SAN is the optimal choice for shared access to the video storage assets by editors and producers, but "collecting 500 GB of video highlights per game, or 25 TB to 30 TB of video from 35-plus cameras for the Stanley Cup Finals games, [is] a lot of data to store on a SAN."

Brenner said that the L.A. Kings embraced both tape and cloud for secondary storage. "In addition to our 500 TB SAN, [the team] maintains a 500 TB archive on tape, which is copied to the Dternity cloud service from FUJIFILM," he said. "That way, if there were ever a need for a big restore, involving an amount of data that we can't handle with the network connections to the cloud, they can ship us a tape-based copy of the data on request."

Object storage making inroads

The City of Austin's Public Safety Division maintains 650 police car dashboard video storage systems producing 2 TB to 4 TB of data each day. After only two years, nearly 1.5 PB of data had accrued, which was being stored on network-attached storage appliances, and the limitations of file system name spaces became uncomfortably within reach, according to network systems architect Tony Williams.

Williams researched cloud-scale storage options by examining cloud service providers Amazon, Apple and Google, and reached the conclusion that object storage provided the best fit for his needs. The city went with Caringo's object storage product, Swarm, to provide scaling of both capacity and file system namespace. Williams installed Caringo Swarm on six commodity Dell servers, each with 84 4 TB SATA disks on direct-attached storage shelves, and created a back-end archival storage repository for his more expensive NAS storage appliances.

Today, adding capacity to the Public Safety Division video storage system is comparatively simple: Roll out more servers with storage and Caringo Swarm. Caringo also recently announced FileFly, a set of agents that integrate Windows or NetApp file systems directly to Swarm platforms, maintaining a familiar method of access for users who are more comfortable with traditional file system storage.

We looked at clouds but determined that the strategy is unsustainable. For one thing, we could not retrieve data fast enough over links that we could afford.
Jose Eli Villanuevamultimedia designer and producer, County of San Diego Communications Office

Object storage and video go hand-in-hand going forward, according to Jason Pan, senior director of product marketing for Promise Technology. Promise has built up a significant presence in the media and entertainment vertical market in recent years, partly by building high-performance local storage for video editing workstations from Apple that feature the high-performance Thunderbolt and Thunderbolt 2 interconnect.

The company's Pegasus 2 arrays are used widely at the workstation level and, while they can share their data between different workstations, they are admittedly "not ideal for networking." However, the company offers SANLink bridges to interface the devices to a FC network and to their larger-capacity VTRAK A-Class storage arrays that deliver more than adequate speeds and feeds for shared video editing devices.

That's a wrap … Until the sequel

In the final analysis, whether video data sits on disk or tape, on premises or in a cloud, seems linked to the familiar criteria:

  • Does the infrastructure fit the workload requirements in terms of speeds and feeds?
  • Is it within budgetary reality?
  • Can the infrastructure be deployed and operated using the skills and knowledge available on staff?

Whichever option provides the best fit with these requirements will usually get the nod.

David Cerf, Crossroads Systems EVP of strategy and business development, agrees with this conclusion and adds that he is seeing increased demand across all market segments for "intelligent storage with intuitive knowledge of how to meet workflow requirements while delivering storage capacity optimization and performance," he said. "Everybody needs this kind of solution: those with big video data workflows are simply getting there first."

Article 1 of 8

Next Steps

How video storage systems benefit from SSDs

USC stores petabytes of video on private cloud

video archiving benefits from LTFS products

Dig Deeper on Data center storage

Get More Storage

Access to all of our back issues View All