This article can also be found in the Premium Editorial Download "Storage magazine: Tame the e-mail beast: Valuable e-mail management strategies."
Download it now to read this article plus other related content.
Breaking old habits
Changing the way e-mail is managed has taken time, particularly since e-mail vendors have built their systems on the assumption that the best way to keep pace with e-mail volumes is to add more disks. That approach was becoming a real problem for Peel Children's Aid (PCA), in Ontario, Canada, a statutory body charged with protecting the welfare of children in Canada's million-strong Peel region.
|Average number of e-mails stored|
The organization found its e-mail requirements were exploding after installing a Lotus Notes-based workflow and e-mail system in 1998. While the capabilities of Notes made it invaluable for the organization, Chris Harbour, manager of information services with PCA, says the system's demands on storage space quickly became an issue.
"Because of the service that we're in, we're accountable for keeping track of information," he says. "That was the reason for installing Notes: There were documents all over the place and managing them was difficult. We were looking at making better use of the servers and offloading a lot of that information."
PCA installed OTG's EmailXtender running against the Notes server, with 320GB of online Maxtor disk array available to the system. The setup archives the organization's e-mail, with the indexed database growing at around a gigabyte per month.
Having the e-mails so comprehensively archived, and so immediately accessible, has made life much easier for PCA. "In the past, a funding realignment meant we were going through workers very quickly and generating, deleting and regenerating e-mail databases," says Harbour. "We were losing messages that way. But cases are audited frequently, and the archiving has been a lifesaver a couple of times when we were trying to do an audit and the Notes pointers had lost track of where a thread had gone. The only way we could have found them was by going through the e-mails one by one, but by centralizing we have been able to manage them."
While proprietary products staked out the early market for intelligent e-mail management, mainstream e-mail vendors are coming to the party too. Just as many corporate manufacturing interests are being charged with cleaning up toxic byproducts of their business, e-mail vendors are building better e-mail storage management into coming system upgrades.
IBM subsidiary Lotus Software, of Cambridge, MA, has restructured its message structure, replication protocol and compression techniques to allow records to be managed with a much higher degree of granularity. Called the Streaming Replication and Attachments model, the new technique will be embodied in the upcoming Notes R6, according to Will Raabe, IBM's director of product introduction for Lotus products.
Notes R6 will include Single Copy Template (SCT), an SIS-like feature in which a single copy of each document template will be stored in the Notes database. Documents built using that template will be stored with a pointer to the template, rather than storing the entire template in the document itself.When applied to the existing e-mail databases of IBM test users, says Raabe, SCT immediately trimmed the size of those databases from an average 9MB to around 1.5MB.
Microsoft, which is readying a major upgrade of Exchange for release next year, has refrained from delving too far into e-mail storage management. Although it includes single instance storage features for reducing e-mail overhead and deleted items retention features for automatically deleting messages after a certain time, Microsoft has left more sophisticated HSM-like features to third parties, says Microsoft program manager Ken Ewert.
"I'm seeing that these solutions that [third party] vendors have are being used in very creative ways," he says. "Storage in general is undergoing a great radical change, since it's the one thing that people are finally trying to get a hold of. Single instance storage is very complex ... we haven't seen a lot of pressure until this year in this space."
Far more eager to make changes is SendMail Inc., Emeryville, CA, which offers an inline mail filtering API that lets IT staffers screen messages as they pass through the mail server. SendMail has previously partnered with UK firm ArchiveIt, which offers automated e-mail archiving using SendMail's hooks. But as the need for integrated e-mail management grows, Jeff Morris, product line manager with SendMail, concedes it is likely to become part of the e-mail server itself.
"For now we're doing it as a third party thing, but we are investigating becoming that third party," he says. "We're getting more and more granular about the types of messages we want to pass to storage, and doing things now to enhance control over mail as it's in transit. I'm quite interested in packaging our message server as a device with a single instance message store with digital signatures to ensure the identity of the e-mail's owner."
This was first published in October 2002