This article can also be found in the Premium Editorial Download "Storage magazine: Email storage lessons learned from Citigroup."

Download it now to read this article plus other related content.

Simply put, it's imperative to match the kind of storage to the value of the application to lower storage costs. Put another way, cost-effective storage--especially for e-mail and applications on plain vanilla file servers--should provide service levels just above what's needed. Newer technologies, such as Serial ATA (SATA) drives and iSCSI connectivity, offer more ways to fine-tune your storage environment and save money by not overprovisioning storage.

Requires Free Membership to View

Best practices for e-mail storage
Optimize capacity over performance
Use pooled SAN storage (avoid DAS)
Use storage virtualization for easy and fast expandability
Use bigger disk drives at any performance level
Use RAID 5 for best cost/capacity ratio
Name directories and files with historical context, change locations periodically
Multiplex (oversubscribe) e-mail servers 15:1 on SAN storage
Use e-mail archiving software
Document archiving operations
Use WORM tape or CAS disk technology for e-mail archives

Tuning e-mail storage
E-mail is one of the fastest growing applications in terms of storage capacity. With Sarbanes-Oxley and other regulations, the problems of managing e-mail have increased. (See "Regulations squeeze storage.") Because of the requirements for data retention, e-mail storage will have two components: primary (online) storage and secondary (archival) storage.

E-mail storage should be tuned for simple capacity scaling and long-term data retention. I/O performance and availability are secondary requirements. Most e-mail messages are accessed within the first few days they are received and then they are accessed infrequently--if ever again. Users may search their old messages for specific information, such as another person's e-mail address, but most old messages are never opened and read after several days.

This is partly due to the fact that e-mail replies and discussion threads often have the message text from a previous message copied in subsequent messages. E-mail storage can be described as a write-infrequently, read-rarely facility. It's lightly used, although that might not be obvious, based on the capacity problems that it causes. The characteristics of archived e-mails are different--the archive must be designed to last for many years, and in some cases, prove that e-mails haven't changed.

First and foremost, e-mail primary storage should be cheap, but scalable. Disk drives running in server cabinets are not the least bit scalable, although this is probably the most common type of e-mail storage used today. Virtualization in storage area networks (SANs) is a much better option. Virtualization's ability to change storage capacity on demand would dramatically lower the cost of primary e-mail storage.

E-mail iSCSI SAN
Another way to lower the cost of e-mail storage is to use cheaper SAN connection technology for e-mail servers. This sounds like a job for iSCSI, and it is. Of course, this may require creating a new iSCSI SAN or using iSCSI routers to provide iSCSI servers access to Fibre Channel (FC) storage subsystems.

Implement your iSCSI SAN independent of other Ethernet networks--leveraging an existing Ethernet network only invites problems and there's not much synergy to be had uniting legacy LANs and iSCSI SANs. Do not bother with TCP offload engines (TOE) because e-mail servers don't generate enough I/O traffic to justify the additional TOE cost.

If you plan to leverage an existing FC SAN, you will need an iSCSI router. Cisco Systems Inc., Crossroads Systems Inc., FalconStor Software Inc., McData Corp. and Sanrad Ltd. all offer iSCSI storage routers. Some of these products also provide the virtualization function that you need for optimal e-mail storage. You can put the virtualization product on the FC side of the router using any number of SAN virtualization products. I recommend putting the virtualization system on the FC side so it can be leveraged by other servers in the SAN. If implemented successfully, virtualization will increase your storage capacity levels more than any other storage technology on the market today.

Additionally, you should oversubscribe or multiplex server connections for e-mail servers. A single 2Gb storage subsystem port should be able to accommodate between 15 and 20 e-mail servers. Surprisingly, you might find that this number could be even higher. A fast LAN connection is necessary to migrate data transparently between storage subsystems.

This was first published in July 2004

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: