Scanrail - Fotolia
Published: 12 Jan 2008
With digital video on the rise in enterprises, storage teams will play a key role in accommodating the petabytes of data that can be generated by video surveillance systems.
Industry researcher Frost & Sullivan, Palo Alto, CA, sees IP as the next phase for the surveillance industry and one that will have a far-reaching impact on storage. According to company research, "the most important advantage in an IP surveillance network is the capability of storing large quantities of video footage in a relatively limited storage space with much higher image quality than analog networks."
"Companies doing surveillance are generating a lot more data, 24/7. The data is increasingly high rez. Now they want to see a recognizable face," says Rick Bentley, CEO at video surveillance system integrator Connexed Technologies Inc., Palo Alto, CA (see "Multiterabyte video production," below).
For enterprise storage managers, this has the makings of a good news-bad news joke: The good news is the new source of deep pockets--the security department--ready to buy terabytes or even petabytes of networked storage. The bad news is that the storage group will have to quickly get up to speed on how to store this new stream of data. Video surveillance requires a different approach to storage requirements, capacity planning, data protection and more.
"Storage stands to receive a large chunk of that money, currently about 25%," reports Dilip Sarangan, research analyst in the North America AutoID & Security Practice at Frost & Sullivan. "Storage is the largest and most expensive component in a video surveillance system."
As soon as the video surveillance stream has been digitized and directed across a network, companies can implement shared, scalable network storage instead of being limited to the storage typically attached to each digital video recorder (DVR). "DVRs create islands of storage," says Lee Caswell, chief strategy officer and founder of Pivot3 Inc. in Spring, TX. Islands of video storage suffer from the same inefficiencies and unnecessary management overhead as the islands of data storage that drove enterprises to implement SAN and NAS to consolidate their storage.
Of course, companies don't need the latest IP video cameras to take advantage of digitized video. "Companies still use a lot of analog cameras," says Caswell. "They aren't going to replace those cameras that fast because today there is still about a $200 price delta between an IP camera and a conventional video camera." By using an encoder, companies can capture and digitize the incoming analog video data stream before shipping it over a LAN to networked storage.
The importance of IP video can be seen in the arrival of a new class of video product, the network video recorder (NVR), which captures data feeds from multiple cameras for storage on directly attached disks. "The NVR is like the old DVR except that it attaches to the network. You can log in remotely and access data or manage it over the network," says Bentley.
IT in the video spotlight
From a storage standpoint, video surveillance was totally uninteresting. Initially, the data was stored and viewed on VCRs, and tapes were continually rerecorded in a rotation. Rarely did organizations keep surveillance data for more than a month; often, it was kept for a week or less. Things became more interesting when the DVR, which contained a few hard drives, came on the scene. As disk capacity increased, the amount of storage in a DVR grew. With the advent of IP video and NVRs, the video surveillance effort required the services of the storage group.
"The problem is that most video surveillance purchasers are not IT-sophisticated," says Norall at Taneja Group. "They went with packaged systems based on direct-attached storage. But that is hard to scale and expensive to manage." The scale of video surveillance storage can be staggering. A retailer can have dozens of cameras per store, while a single large casino can have thousands of megapixel cameras generating high-resolution video streams 24/7.
The typical enterprise IT storage infrastructure isn't optimally set up for surveillance video storage. "This is absolutely not an FC [Fibre Channel] market, and clusters of NAS filers won't do the job either," says Norall. "Block-based iSCSI storage is what makes the most sense," he adds, while the security industry has shown a preference for packaged proprietary systems.
Storage managers who think giving each email user 2GB of storage space is a big deal will be shocked. Consider the following formula: Take the number of cameras generating video streams 24/7 and the megapixel resolution of the new cameras and an acceptable frame per second (fps) rate (3fps to 5fps won't cut it--think 10fps, 15fps, 20fps) with medium compression and multiply that by the length of time you want to store the data.
The total is a very big number. The need to store 30GB to 40GB per camera per day isn't unusual. That comes to 1TB or more per camera per month. Now, if a retail organization has 30 to 40 cameras per store and 1,000 stores, you're looking at multipetabytes of storage, even if you're rewriting over some old data (see "Surveillance storage capacity factors," above).
Another challenge is the storage workload. "This is not your typical enterprise storage workload," says Jeff Whitney, VP of marketing at Intransa Inc. For example, surveillance storage is 95% to 98% writing. "It is hard to keep up with a near constant write load," he notes. "The storage system doesn't have any time for the usual housekeeping."
Most SAN storage arrays are optimized for business workloads that use small blocks and present a balance of read/write activity. "With video surveillance, you are constantly writing large sequential blocks," adds Whitney. And you don't keep it long; often just a week, a month, a year at most. And skip the backup; RAID 5 is all the protection that's required.
Finally, video storage has to be simple. The systems are usually deployed and operated by security and facilities people who don't understand storage provisioning and networking. LUNs, zones, masking, RAID--forget it. "You need fast performance to the drive, linear scalability and utterly simple management," says Caswell at Pivot3. That's why DVR/NVR appliances have been such a hit; however, they can't scale in the face of the storage capacities required.
Limited storage options
"Cisco grabbed lots of mindshare, but nobody knows what products they are offering," says Bentley at Connexed Technologies. Adds Frost & Sullivan's Sarangan: "It looks like Cisco is planning to converge of lot of technologies, but it hasn't announced products yet."
The surveillance market would seem ideal for low-cost storage vendors, but that hasn't been the case. "This is a market for cheap iSCSI, but companies like LeftHand Networks and EqualLogic [recently acquired by Dell Inc.] are focused on the SMB [small- to medium-sized] market. NetApp [Network Appliance Inc.] is aiming more at the data center," says Taneja Group's Norall. Products optimized for those markets won't meet surveillance storage requirements.
"EMC, IBM and HP [Hewlett-Packard Co.] are exploring surveillance storage, but haven't done much yet," adds Sarangan.
At this point, only a few iSCSI array vendors are actively targeting surveillance storage: Infortrend Technology Inc., Intransa, Pivot3 and Nexsan Technologies Inc. "We were using DVRs, each with about 500GB of storage. Then we went with Intransa," says Derrick Wright, manufacturing security manager at Baxter Healthcare International in Cherry Hill, NJ, which uses approximately 120 cameras for security, workflow supervision, safety and quality control. The company took 8.2TB (11.25TB raw) of Intransa storage to replace 7TB of storage attached to its seven DVRs. It planned to add another 9TB before the end of 2007 and 11.25TB in the first quarter of 2008.
"The [video] information comes into the DVR and is written directly to Intransa," explains Wright. "The DVR is used to search and serve the video. Bypassing DVR storage improved performance." The company had to equip each DVR with a network interface card to connect with the Intransa system via iSCSI.
For Connexed Technologies, the storage options were so limited it actually started building an iSCSI array. "We looked at SAN and NAS appliances. You can make any array work, but cost and scalability are problems," says the firm's Bentley. Then he found Pivot3. "Pivot3 really scales. It actually gets faster as it gets bigger because each [expansion] box has its own processor."
Pivot3 uses a different approach to RAID it calls RAID Across Independent Gigabit Ethernet (RAIGE). Pivot3 delivers storage in units containing disks and a processor connected by 1GB Ethernet. RAIGE provides what amounts to block-based virtualization and RAID-like data protection across all the RAIGE storage units in what becomes an iSCSI cluster. If a data unit fails, it doesn't affect the operation of the other units--the DVRs keep pumping out data to store on the units still available.
Maybe the biggest challenge storage professionals face will be working with the group that's mainly responsible for surveillance video. "The security people may not be so welcoming," says Caswell at Pivot3. "They see storage as just a small part of the overall solution." They're quite comfortable buying complete solutions from one vendor or integrator.
No matter who controls the video surveillance budget, the storage team will be asked to build out the necessary capacity to store video. And they'll have to learn the idiosyncrasies of surveillance storage because it's not going away.