This column discusses the technologies available today that can help you conform to government regulations regarding record retention. There are always a number of different ways to solve a particular problem and in this case, your best solution would be one that lets you conform to the regulations as simply and cheaply as possible. After all, this is an expense item, not a revenue generating function. The cheaper the better, but it also must LAST for the entire retention period.
So, let's look at what's out there. Currently, there are solutions from a number of vendors in the compliance space. Some take the hardware approach, some take a software approach and others combine the two.
Let's look at the different methods used for e-mail retention that cover SEC rule 17a-4 which was modified under 17 CFR Part 241 [Release No. 34-47806] covering the electronic storage of broker-dealer records.
The rule was modified in May to take advantage of new technology and allows for storage of electronic records on media other than write once read many (WORM)-based optical storage. In one section, the rule states:
"One method using such a system stores a specified expiry or retention period with each record or file system. The system blocks record deletion or alteration by any manner of intervention until the expiry is reached or the retention period has lapsed. At expiry, or after the retention period, the records may be deleted from the system, thereby freeing space for reuse."
This allows for storing data on cheap magnetic disks for the entire retention period, as long as the system enforces that the data cannot be deleted until the retention period expires. The amendment goes on to say:
"A broker-dealer would not violate the requirement in paragraph (f)(2)(ii)(A) of the rule if it used an electronic storage system that prevents the overwriting, erasing or otherwise altering of a record during its required retention period through the use of integrated hardware and software control codes. Rule 17a-4 requires broker-dealers to retain records for specified lengths of time. Therefore, it follows that the non-erasable and non-rewriteable aspect of their storage need not continue beyond that period."
The new ruling can make data retention cheaper by not forcing you to buy expensive optical storage devices and allows you to re-use the storage space for other data when the retention period is over. The new rule also makes available the ability to retrieve data much faster. If everything were stored on CD's and you were looking for a specific e-mail, you would first need to locate the CD that contains the data, then copy it back into your email system for retrieval. With the newer systems that use content addressing and indexing software on top of magnetic disks, the data can be instantly made available for review.
There are a few things you need to consider before making a buying decision on new compliance systems.
1. The solution has to be compatible with your current applications. The solution should be transparent as possible. (You may not want to be forced to bring in consultants to integrate the solution with your email system!)
2. The solution should not have to store multiple copies of your data. In some cases, offsite copies are required to conform to the regulations, but the solution should store only the original (and an offsite copy if needed) and that original should never need to be touched again.
3. There should be a way to migrate the data to other media if needed. Let's face it, things change. Six years from now, newer cheaper higher density disks will be available. You don't want to be stuck. The solution should also let you migrate a copy to WORM tape. Spinning disks fail. Tapes can be stored off-site.
4. If you're using a software only solution, make sure the company will be around six years from now.
5. The ability to delete data and reuse the storage can reduce costs down the road. Make sure the vendor's solution conforms to the regulations if they give you the ability to delete data once the retention period is up.
Here are a few of the solutions available today. This is not a complete list, just a sampling of what's available:
StorageTek: WORM tape solution (lets you store data in read only format on tape).
EMC: Centera (object-based storage solution uses digital signatures to make sure content is not overwritten, stores data on magnetic disks).
Hewlett-Packard: Offers a complete integrated hardware/software solution.
Mirapoint: Appliance approach (Sits in the data path in front of your email server, intercepts SMTP messages, and archives them on magnetic disks).
Hitachi: Integrated hardware solution (allows magnetic disks to be turned into WORM disks in firmware).
Persist: Uses off the shelf blade servers with "storage cell" technology for storing messages with its software.
NetApp: Uses a NAS head attached to cheap ATA disk in conjunction with "lockable snapshots" for conformance to the regulations.
Legato: Software solution that works with any magnetic media. (Data would need to be archived to WORM devices to conform to non-erasable rule).
CommVault: Integrated data backup, archive, and migration solution that can provide single object storage and indexing.
IXOS: Global vendor who can also integrate with applications like SAP, instant massaging and other data types.
Persist: Very scalable index and data retention into storage "cells" based on server blades.
Zantaz: Uses the Persist software approach with a total consultative solution.
Spector: Allied with SIAC, provides outsourced retention solutions for its customers.
Read Chris' previous column "Regulation compliance driving storage demand, new business strategies and technology."
This was first published in June 2003