Published: 08 Feb 2004
|How Attachstor works for internal e-mail|
When asked why, the storage admin replied: "very long backup and very long recovery times."
That client--like a lot of other Exchange users--found that if an Exchange database gets to more than 25GB to 30GB, it takes too long to back it up and even longer to restore. This is especially true if you aren't using a mailbox-level backup and recovery system. In that case, the only solution for recovering a single mailbox is to recover the entire information store to another server, then move the desired mailbox using standard Exchange tools.
Therefore, every time my client's Exchange information stores gets larger than 25GB to 30GB, they make another Exchange server. The result is an incredible waste of money, as each one of these servers requires its own hardware and software, including licenses for Windows, Exchange, backup software, antivirus and firewall software and all of the maintenance fees that go along with software packages.
There's a clichÉ that says someone's problem is someone else's opportunity. The AttachStor Suite from AttachStor Inc. in Scottsdale, AZ, solves the problem of overgrown Exchange and Domino information stores by limiting the number of e-mail attachments.
Everyone knows that attachments consume a lot of storage. According to AttachStor, 21% of e-mail messages have attachments which consume 94% of a typical organization's e-mail resources, and 60% to 65% of those attachments are either duplicates or revisions of existing attachments. The company says this is the case even if your mail product is using single instance store where each attachment is supposed to be stored only once.
Single instance store
Exchange and Domino have single instance store, but apparently it doesn't work the way we think it should. It works from the message level. This means that if a single attachment is sent to 100 users on the same server, that attachment will only be stored once. However, if someone replies "to all," and attaches the same message again, it will again get stored once. Think of the number of times an attachment floats around your organization; compound that by the number of times that attachment is modified and you'll begin to see why single instance store in Exchange and Domino doesn't live up to the hype.
Attachments cause other problems, too--especially for remote users. Who hasn't received an e-mail with a multimegabyte attachment and waited several minutes for it to download? Then, what happens if you need to modify the attached file and send it back? Upload speeds are even worse than download speeds with many Internet connections.
|Attachstor at a glance|
The AttachStor solution
These are the problems that AttachStor sets out to solve, and its solution is relatively simple. It pulls the attachments out of the Exchange or Notes information store, and stores them in a separate database that supports true single instance store and delta-level incremental storage for all attachments. There's an Outlook and Notes client that integrates with AttachStor to provide more functionality to remote users.
Then there's the AttachStor server, which is typically installed on a dedicated server. It uses a Messaging Application Program Interface (MAPI), a Microsoft program interface that enables you to send e-mail from a Windows application, attach a document to the e-mail to communicate with Exchange or Domino and extract the attachments from the information store. This means that if it's installed on a separate server--which is how it is normally done--no software needs to be installed on the Exchange server.
AttachStor extracts, compresses and encrypts each attachment, stores them in its own file system, then replaces it with a 1K HTML file, or what AttachStor calls a Storfile. A Storfile is a special file that is used as a pointer to the real attachments in the AttachStor file system. Attachments with the same name are compared to verify that they are indeed the same file. If they are, then only one instance of that file will be stored in the AttachStor file system, regardless of how many messages or information stores it was used in. So, if you have multiple information stores on a server or multiple Exchange or Domino servers, it will provide true single instance store for a given file across all of those servers.
If AttachStor finds that a given attachment is a modified version of another attachment, it will store the bit-level differences between the two files. For example, suppose you had a 10MB attachment that is actually a revision of another 10MB file stored in the information store, but only 1MB of that file has changed. In that case, the 1MB differences would be stored in the AttachStor file system, significantly reducing the space required for versions of other attachments.
However, there's a slight downside to extracting all attachments. Once an attachment is extracted, an Outlook or Domino client user that doesn't have the AttachStor plug-in will have to click twice to receive that attachment. That is, they will receive an attachment they will have to open containing an HTML link to the real attachment in the AttachStor file system. While this minor issue doesn't apply to Outlook/Domino users that have the plug-in, some organizations won't deploy the plug-in on every desktop/laptop because of the additional cost.
Some companies say this inconvenience is worth it, compared to the amount of space they save by extracting all attachments. Other companies extract only attachments older than 30 days, leaving their most recent attachments in the information store. The AttachStor analysis tool breaks down attachments by age, so storage admins can decide what's the best policy for their organization.
If a user is running the AttachStor client, the movement of these files is invisible. When a user requests an attachment that would normally be in the information store, AttachStor automatically retrieves that attachment from the AttachStor database, without having to click twice. If the file that the user requested is a revision of another file, AttachStor merges the original file stored in the AttachStor database with the changes found in the Storfile, thereby presenting to the client the complete, revised file.
The user's desktop/laptop running the client has its own local cache of received attachments. If the user receives an attachment that is a duplicate of another attachment, the AttachStor client tells the AttachStor server that it has it, and the second attachment would not need to be downloaded to the user again. Even better, if the user receives a later revision of an attachment that's already in a local cache, the AttachStor server knows that and will send the user a Storfile that contains only the bit-level differences between the two versions. Then, when the user opens the attachment, the AttachStor client automatically merges the differences with the original that is in the client cache.
Imagine being able to send a 20MB PowerPoint file back and forth between remote users who are connected to the Internet via dial-up. If users are doing a POP3 or IMAP session, all they're downloading in their initial session is the message with the Storfile attached, and the Storfile is small.
|Corporate e-mail traffic, 2002 and 2006|
Will Microsoft add this functionality?
At first, I wondered if this wonderful functionality might be something that Microsoft might add to its product at a future date, making an investment in AttachStor a waste. However, I doubt that will ever happen. Using this functionality allows a storage admin to significantly reduce the number of information stores, reducing licensing costs for both the operating system and Exchange.
There are areas for improvement, of course. One of the other issues with mail is remote PST files. Think about all of the intellectual property that resides in attachments embedded in files. Wouldn't it be wonderful if AttachStor could use its software to solve this problem?
I also wonder about those companies who don't use Exchange or Domino, and use all kinds of products that make mail accessible via IMAP. Consider my company's anti-spam system. Because installing an inline spam filter was a problem, we used a free tool called IMAP Spam Begone to open our IMAP in boxes, checked them for spam using SpamAssassin and then moved those messages to a spam folder--all from about 600 lines of Python code. Surely AttachStor could apply its technology in the same way.
Areas for improvement aside, it looks like the AttachStor Suite is a great tool that could save Exchange and Domino environments lots of money by reducing the number of servers they have, and reducing the storage that attachments use, making things much easier for remote users. AttachStor does this for about $20 per user. If you're using Exchange or Domino, I'd be giving these guys a call.