Regulatory compliance requirements confronting financial service firms triggered a rush of email archiving products more than a decade ago, and today the need to retain emails for legal discovery is driving a new wave of updates and improvements in the e-discovery market.
Whether it's high-profile, high-penalty court cases involving corporate defendants, or simply the amount of data storage that email and its attachments eat up, storage managers are increasingly putting email archiving at the top of their priority lists.Table of contents
Enterprise Vault's Discovery Accelerator helps construction firm with search
Save, delete: The email archiving policy problem
Storage vendor response to changing demands in email archiving
Hosted email archiving options gaining traction in SMB space
On-site email archiving options still most popular
IT organizations that purchase email archiving products chiefly to address storage management headaches often also wind up seeking additional tools or add-on modules to respond to litigation-related demands. As a result, the email archiving vendor landscape is evolving so vendors can compete on reporting tools, data classification capabilities, and search functions.
The San Mateo, Calif.-based construction company Webcor Builders LP purchased KVS Inc.'s Enterprise Vault in early 2004 because its Microsoft Corp. Exchange Server was slowing, Outlook was crashing and users were having trouble finding information. Even though the company has only 1,200 mailboxes, the messages tend to be large, with many attachments containing an architectural drawing, a photograph or a blueprint.
The addition of Enterprise Vault, which Veritas Software Corp. acquired later that year and Symantec bought in 2005, "really improved the performance of the Exchange [data] stores," said Webcor's CIO, Gregg Davis. "By moving all of this email out of the Exchange store, it allowed other data to fill in. It's like pruning a tree," he added.
But, by year's end, Davis found himself confronting a new email-related challenge in the form of a lawsuit. Attorneys started asking for information out of Enterprise Vault, and Davis realized he needed to contact his archiving vendor.
Fortunately for Webcor, a Discovery Accelerator module was already available for Enterprise Vault. The add-on module would let the construction company put litigation holds on any emails that turned up in keyword searches and provide an internal link to a Web page where legal counsel and paralegals could find and review all of the flagged emails.
"It makes it phenomenally easy to view the search results," said Davis. "IT doesn't have to make determinations of what's valid and what's not valid. We just set up the initial search terms and set up a link."
One challenge for IT shops is identifying an email archiving policy that makes sense. Saving everything does not, experts say. Having an email retention and deletion policy for email that is based on preservation dates set by state and federal regulations, for example, is a good starting point. Once a litigation hold is placed, or an e-discovery request is made, a company must often change that policy, and start retaining many more documents. But a company is not likely to be held liable for emails or attachments that were eliminated as part of a reasonable retention and deletion policy prior to the existence of a litigation hold -- as long as it was applied regularly across the board without exception.
Best practices for email archiving often start with the prickly issue of establishing retention policies for users. The shorter the retention period, the harder it will be to enforce. A 90-day retention policy, for instance, sounds good in theory, until users balk and seek out other ways to save what they consider important emails.
At the other end of the spectrum, the "save everything" mentality has its share of potential pitfalls, from clogging the system to making it more difficult and more expensive to search for information in the event of litigation, potentially exposing the organization to added legal risk.
A March report from Gartner Inc. "Q&A: Planning for e-mail archiving" recommended a "selective save" email retention policy that the report acknowledged is also the most challenging strategy. It requires "integration to records retention policies" and "good classification and metadata management."
Organizations can benefit from archiving emails after a 30- to 90-day period and then setting a timeframe when the messages will be deleted, from one to three years or longer, depending on the regulatory requirements or needs of the specific organization, according to the Gartner report.
The report, authored by Gartner vice presidents Kenneth Chin and Debra Logan, noted that many organizations lack the enterprise records management policies and infrastructure to support the approach. But, the Gartner executives also noted that vendors have been working to improve their software and classification methods.
In response to this need for standards and policies, vendors have added functionality to help users do things like automatically move archived messages and attachments to different storage systems based on accessibility or cost requirements.
"You've shifted the storage management burden to the archive environment," said Brian Babineau, a senior analyst at Enterprise Strategy Group, "so that puts the onus on the archive vendors to make it easy to manage."
Symantec, for instance, late last year added a new analytics tool to Enterprise Vault to prioritize search results and global deduplication to enable the archive to keep single copies of emails that span multiple servers.
In April, EMC Corp. replaced its EmailXtender product with a newly architected SourceOne line of products that includes a more scalable, highly available Email Management offering and an add-on Discovery Manager to enable users to identify, collect and place a legal hold on emails associated with ongoing litigation.
"No one imagined the amount of emails that would be managed, or the fact that people would begin to use their email as a repository," said Sheila Childs, director of product marketing for compliance and archiving products at EMC, explaining the need for the product overhaul. "No one really understood that e-discovery would become such a predominant activity, something that all corporations face penalties for if they don't comply."
Other email archive offerings include Autonomy Zantaz Enterprise Archive Solution, CA Inc.'s Records Manager, CommVault Systems Inc.'s Data Archiver, Hewlett-Packard (HP) Co.'s Integrated Archive Platform, IBM Corp.'s CommonStore and Mimosa Systems Inc.'s NearPoint.
Still, many IT organizations have yet to adopt email archiving products. Babineau estimated that 50% of companies don't archive email, despite clearly demonstrated benefits over the last 10 years.
"With companies that haven't done anything yet, the decision they have to make is Software-as-a-Service [SaaS] versus product," said Babineau. "They should not be deciding whether or not to archive email."
Even IT organizations in less litigious industries increasingly are finding reason to consider e-discovery technology. Effective Dec. 1, 2006, amendments to the Federal Rules of Civil Procedure governing civil actions in U.S. courts stipulated for the first time an obligation to disclose and preserve electronically stored information. Although the rule applies to all electronic information, email tends to be the most requested.
The impact of the 2006 amendments has been profound, from the largest to some of the smallest of IT shops. Miami Country Day School, for instance, has no pending lawsuits and no pressing need for e-discovery capabilities. But, Donna Lenaghan, the private school's director of technology, said she wanted to be proactive as soon as she learned of the legal requirements through an education technology journal.
Lenaghan didn't consider an in-house, or on-premise, email archiving product, because the school's four-person IT staff already has a full plate, with 1,000 students and staff members. Instead, Miami Country Day School chose to outsource email archiving to LiveOffice, a Software-as-a-Service provider that charges customers on a per-mailbox basis.
"They are the experts, and we cannot do it as well as they can," said Lenaghan. "There is no advantage for us doing that."
LiveOffice emailed the steps to configure the journaling functionality in Exchange Server, and within an hour, the service provider started receiving the school's forwarded email. Miami Country Day School still hasn't had occasion to use LiveOffice's e-discovery capabilities for legal discovery, but it has found the tools helpful for locating inadvertently deleted emails. In the past, ferreting them from backup disks might have taken a day or more.
"It's a very handy day-to-day operation tool," said Lenaghan. "But it's also a piece-of-mind insurance policy if you have a legal issue."
Although on-premise email archiving tools are still far more commonly used than outsourced options, email archiving service providers have been gaining traction among small- and medium-sized businesses (SMBs) in recent years.
Laura DuBois, a program director at International Data Corp., noted that IDC market research showed that 37% of the email archiving market was delivered as SaaS or online services during 2007, and she expects the number to be higher when IDC wraps up its numbers for 2008.
"It's certainly part of the bigger cloud trend," DuBois said, linking online email archiving to the increasingly popular term for Internet-based services. "We're bullish on cloud, so I think it will continue to be something that we see increased spending on."
Some large enterprises find it's helpful to outsource divisions. "In highly litigious areas, you might find that certain parts, like the broker-dealer community, is handled as a service, and the rest of the organization does policy-based archiving that captures selected messages and manages the mailbox size," said Carolyn DiCenzo, a research vice president at Gartner Inc.
More than 20 service providers offer hosted email archiving services. They include Autonomy Corp. (Zantaz), Dell Inc., Global Relay Communications Inc., Google Inc., Iron Mountain Inc., LiveOffice, Microsoft, Mimecast North America Inc., Mirapoint Inc., Proofpoint Inc. and Smarsh Inc.
"We're definitely seeing Software-as-a-Service solutions catch up to product solutions as far as feature functions go," in technology areas such as mailbox management and enhanced search, said Enterprise Strategy Group's Babineau.
Still, most storage shops prefer to control their own email archives.
"You lose a certain amount of control" with a service-based email archiving system, said Curtis Rawlings, assistant CIO for DeKalb County (Ga.) government, which uses Symantec's Enterprise Vault. "And do you really save money? If you start growing exponentially, your [outsourcing] costs can spiral out of control."
Today's on-premise archiving products typically allow IT organizations to delete email from the active data stores, shift it to lower cost disk, leave a stub for the archived messages and provide users access to historical data through Web browsers.
Many of the archiving products increasingly allow users to capture and archive not only email but other content types as well, such as instant messages, files and Microsoft SharePoint data.
Rawlings said he first proposed retention policies in 2006, but significant pushback forced him to take a "save everything" approach. Only recently was he able to revisit the proposal after a two-day email system failure (unrelated to its Symantec Corp. Enterprise Vault archiving product).
This time, Rawlings said he was able to get the necessary buy-in from senior management, which includes some new members who took office after the last election. Plans now call for a 25 MB active store and two-year retention policy for the email archive.
"If I had to do it over again," Rawlings said, "I would have made sure I had my policies approved and ready to implement at the time we bought Enterprise Vault."