This article can also be found in the Premium Editorial Download "Storage magazine: What you need to know about data storage provisioning."

Download it now to read this article plus other related content.

Dos and Don'ts for creating a long-term archive
  • Test the entire process

Requires Free Membership to View

  • of recovering archived data. Refresh the data format and storage medium.
  • Think about updating data formats to Adobe Systems Inc.'s PDF/A, OpenDoc or some other data format that's likely to be readable in 70 or more years.
  • If your archives need to be encrypted, be sure to plan for recovering old keys even if the encryption software/appliance is no longer in active service.
  • Compare the actual cost of archiving to offline media, managing the data and renewing/refreshing it every five to 10 years against the cost of keeping the data online.
  • Assume you'll be able to read archived data just because the media shelf life is supposed to be longer than the retention period.
  • Assume your storage vendors will still be around in 50 years.
  • Throw tapes in a box and hope you won't need them.

Planning for the future
To ensure you'll have a reasonable chance to recover archived data 30 or more years from now, you must first identify the different requirements for data retention and then use those requirements to define policies (see "Dos and don'ts for creating a long-term archive," this page). Then decide what kind of data management application you'll need and start a test bed to validate rule sets and policies. You'll need to work out a way to pull old unstructured archives into the new management system, bringing all the old archives under the same system. Consider rewriting data formats and backup formats on a regular basis to avoid orphaned data that can no longer be read.

For the long term, data formats such as Adobe Systems Inc.'s PDF/A, Microsoft Corp.'s Microsoft Office Open XML file format and XML-based standards like OpenDoc should ensure that data continues to be readable. Developers are beginning to embrace these long shelf-life data formats, so there's a good chance the applications your company uses can be updated or enhanced to add these capabilities.

On the storage format side, it's critical to create meta data describing what files are being stored and how they were created. According to Forrester Research's Balaouras, the Storage Networking Industry Association (SNIA) will be instrumental in creating standards for both information lifecycle management (ILM) and the eXtensible Access Method (XAM) standard, which gives ILM applications a standard interface and meta data structure to communicate with object-based storage systems. Meta data stored with each object identifies the owner, the application that created the file, data format and so forth. The standard specifically addresses both long-term retention standards and data security.

Storage management products from companies like CA Inc. and IBM Corp./Tivoli can use the meta data associated with files to determine how long a piece of data is archived and what policies apply. This is increasingly important when administrators are faced with archiving millions of e-mails, as well as all of the other content created within an organization. There's no way a human can individually set policies for that much data.

There should also be policies in place for long-term storage of encryption keys. As more regulations designed to protect customer and proprietary data require some form of encryption, the practice will undoubtedly become more prevalent. While it's possible that data-recovery companies will be able to bypass current encryption standards 50 years from now, it might still be more expensive to re-create the data. Archiving the necessary keys as part of the overall archive process should prevent this problem.

This was first published in October 2006

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: