2003 will be remembered as the year that financial services companies were forced to archive their e-mail records. Media coverage, several high-profile SEC fines confirm the word on the street is clear -- all electronic messages and documents must be safely stored in an electronic archive for the same period of time as traditional business records.
The SEC is not the only one pushing for better archive compliance. The FDA, for example, mandates that all electronic records be safely archived and remain accessible for years. The Sarbanes-Oxley Act of 2002 and the Health Insurance Portability and Accountability Act (HIPAA) define strict rules for the retention of all electronic records and for the first time authorize severe penalties for non-compliance.
Optical and Tape Media
Organizations have traditionally relied upon optical media and write-once, read-many tape (WORM) media to comply with regulatory requirements for "non-erasable" and "non-rewriteable" storage media. Optical and tape media continue to be used, but they are not the ideal solution for electronic archival. The SEC, for example, requires that electronic records be easily accessible for a period of two years -- making it difficult to use tape media. The explosion in regulated data storage is pushing the limits of storage capacity and terabytes (TB) of archive data are now the norm -- pushing the performance and capacity limits of optical media.
New "WORM Disk"
To solve this dilemma, new cheap disk arrays combined with WORM safeguards deliver a "WORM disk" storage solution that is ideal for regulated data storage. WORM Disk uses cheap disk arrays (typically ATA) to create large economical online storage arrays. ATA-based storage arrays deliver TBs of online capacity at a price that is less than or equivalent to tape. Prices for ATA-based storage arrays are commonly in the one-to-two cents per megabyte (MB) range -- a price level that was unheard of as recent as two years ago.
Two WORM disk storage solutions have been recently been introduced and offer some unique features for regulated data storage. EMC's Centera Compliance Edition was introduced this April and delivers an online WORM disk solution that satisfies new archival regulations. The Centera uses a specialized content addressed storage (CAS) system specifically designed to meet and exceed today's strictest records retention regulations.
Also in April, Network Appliance introduced its new disk-based SnapLock software solution. SnapLock provides easy-to-deploy, robust, long-term data archival protection through open industry standard protocols. SnapLock combines with the NearStore Filer to provide online archival and long-term protection for very large amounts of data.
WORM disk's primary function is to provide long-term, "read-only" protection for data. Applications can write data to the disk but are not allowed to make any changes after the data is frozen. Locking mechanisms are controlled by the application at either the file-level or the volume-level and freeze the data as "read-only". The data remains secure and easily accessible and cannot be altered for the life of the record, a mandatory requirement of SEC Rule 17a-4.
The Centera uses a method referred to as content addressed storage (CAS) to store its records. Centera calculates a hash value for each object and returns a unique identifier to the application. Using this unique identifier, or "fingerprint", the record object can be quickly accessed later on. The record remains secure and is unalterable for the period of time assigned by the application. The retention period can be increased at anytime, but it can never be decreased.
SnapLock is an add-on feature of the DataONTAP Operating System and makes records "read-only" on a file-by-file basis Files that are either created or copied to the SnapLock volume are changed to "read-only", they become frozen and remain unalterable permanently. SnapLock is controlled by a simple command line interface.
Protecting data integrity is a unique feature of the Centera. Many types of reference information, medical x-rays for example, require that the data not degrade over time. Using the fingerprint that Centera creates for each record, a periodic check can verify that the data is still in its original condition. Many archive applications also calculate "fingerprints" which are stored in the archive to manage data integrity. The SEC does not require that any "fingerprint" technology in its regulations. Rather this technology is used when archiving reference data that must maintain its quality over time.
Following expiration of a record's retention period, certain regulations such as the Freedom of Information Act require that the record be completely expunged from the disk. The EMC Centera allows the application to delete the record after the retention period has expired. It is the responsibility of the application to completely erase the expired record. SnapLock stores records permanently and does not manage a retention period.
WORM disk storage is ideal for storing large amounts of electronic data and fits nicely between online primary storage and offline secondary tape storage. Primary online storage is the storage of choice for performance-critical OLTP applications. Offline tape storage remains useful for archival of records that are infrequently accessed and to protect against disasters.
With capacity in the terabytes, WORM disk storage arrays are the perfect solution for regulated data storage. They offer fast online search and retrieval of data and huge capacity -- at a lower total cost of ownership. New regulated data storage solutions from EMC and NetApp deliver capacity and performance for storing large amounts of data and provide WORM safeguards that satisfy strict regulations such as SEC Rule 17a-4.
About the author:
Bob Spurzem is a Principal Analyst with Contoural Inc. an independent provider of storage consulting and storage education services. He has been closely involved in the storage industry for the six years as a Product Marketing Manager with leading storage software vendors.
Do you want to see more articles and insights from noted industry observers? Visit the complete Bits & Bytes column library.
This was first published in July 2003