While users are finding clever ways to make the Exchange store work for them, (See yesterday's Part one -- "Users turn to SANs to manage Exchange")
"With a 4K block size, that's quite a few writes to be searching for," McCall said.
Ray Mohrman, technical product manager of Exchange for Microsoft, said customers have been bringing the idea of a move to SQL as the basis for the Exchange database to his attention frequently of late.
"We've heard a lot of requests from customers for improvements to the core of Exchange, and a lot of them want a switch to SQL," he said. But "when it came to Exchange 2007, what people were really looking for in Exchange was increased stability and higher availability," Mohrman said. "We found we could deliver those capabilities and do so sooner by adding features to this newest release rather than overhauling the database."
Mohrman added, however, that the next move for Exchange could be a move to a SQL platform. "With some of the features we've already added, we're in a better position to move to the SQL server," he said. "We've integrated more of SQL Server's text search engines. We're continuing to look toward that option for the Exchange store."
Microsoft already claims that Exchange 2007 will address some of the common storage, replication and backup headaches without a change in database. On the BladeMail product, which Microsoft has officially endorsed, the company expects to be able to support up to 17,500 Exchange 2003 email users on a single BladeCenter chassis, and up to 70,000 Exchange 2007 email users per BladeCenter when Exchange 2007 becomes available. (A regular version of Exchange 2007 will jump up to 2000 to 5000 users per server, rather than the 2003 limit of 200.) The company also plans to move to a 64-bit operating system on Exchange servers, increasing the memory from 4 gigabytes (GB) to 8 GB.
"Today the memory restrictions on 32-bit hardware limit customers from fully utilizing the space on their storage hard drives," Microsoft spokesperson Ray Mohrman said in an email. "The move to 64-bit systems will help to alleviate this by providing the memory required to drive down IOPS by 70%, enabling customers to not only get more usage out of their storage systems today, but reduce the amount of hard drives they may need to purchase as inboxes grow."
But until the database changes, McCall said, the sequential reads/random writes problem will remain to some extent. The best way, then, to optimize storage for an Exchange database is to have lots of spindles, as having to do sequential reads on large disk platters only makes performance issues worse, according to McCall. And, he added, if more spindles are better, where better to put the Exchange store than on multiple 16-drive shelves in a SAN?
Is it the product? Or how people use it?
But while the improvements anticipated in Exchange 2007 and a future move to a standard relational database would lighten the load, others think the burden of managing Exchange comes from the way end users have used it, rather than how it's architected -- and that no amount of re-engineering will render Exchange manageable on its own.
As email has supplanted the telephone and remains ahead of instant messaging as the most popular form of work-related communication, Lockhart said, workers have begun to use email in a way the designers of Exchange could never have anticipated.
"People use email for huge amounts of communication, and they use it as a personal filing cabinet," Lockhart said. "Think about it -- when was the last time you created any kind of document at work and didn't email it to someone? It's become a knowledge management tool as well as a communications tool, and that's what starts to overwhelm a mail server running Exchange."
Then, Lockhart said, email becomes a battle between mail administrators and end users who want to keep everything and access it all instantaneously. Huge inboxes clog up the Exchange server, "but users complain bitterly if quotas are enforced to save performance."
In response, Lockhart said, more and more users are settling the tug-of-war by offloading Exchange store onto the SAN, by outsourcing it to companies like Postini, archiving old messages onto third-party systems such as that offered by Mimosa, or some combination of all three. "That way, the IT department is just having to manage the Exchange server, while employees get what they want, which are fewer quotas and immediate access to all their data," said Lockhart.
Microsoft may have solved some of those issues with Exchange 2007, Lockhart conceded, but he predicted they won't be able to keep up with the way Exchange is growing. "The amount of bytes a typical user sends and receives doubles every three years," he said. "It's almost like a force of nature -- a better database or four-year product refresh only gets you so far."
"People have created a file archiving system within email," said Bill Augustadt, CTO of application managed services for ACS Inc., a service provider that manages Exchange for corporations like Disney as well as its own internal group of 38,000 mailboxes. "That kind of thing makes attachments multiply like rabbits. And there's collaboration -- shared calendars, reminders, attachments to those reminders.
"Is email really going to be this central repository, or is it just a communications system?" he said, saying it was up to users, not Microsoft, to decide the answer to that question. "And remember," he added, "Just because something's easy doesn't mean it's right."
A cottage industry around email
A relational file system, such as WinFS, which up until recently was in its own separate development at Microsoft, could do something to address the problem of email as file archiving system by giving users a separate file system archive with the search and indexing features that currently make it easier to store so many files as email. But according to an announcement in the past month, the standalone system has been discontinued (see also "Microsoft kills standalone WinFS file system", SearchWinComputing.com, June 26). However, the functionality is now reportedly going to be folded into the next version of SQL server, which then could theoretically make its way into Exchange.
Meanwhile, there remain a number of companies kicking around the Exchange problem. XOSoft, for example, has a tool called InMotion, announced in October. InMotion uses a wizard-based interface to walk users through basic email migration steps. The product has chiefly been used to migrate between Exchange 2000 and Exchange 2003, but it can also be used to move the Exchange store from direct-attached disk to NAS or to the SAN.
Hewlett-Packard Co. also has a product, called Application Storage Manager (ASM), which is included in its ProLiant server iSCSI pack. ASM has a number of other functions for managing Microsoft SQL Server 2000 databases and Exchange 2003 storage groups on HP Storage Servers, but it can also be used in a manner similar to XOSoft's product, to move Exchange stores from DAS to NAS.
EMC Corp. also has a plug-in for Clariion that works with Exchange System Manager, called Storage Administrator for Exchange, which automates underlying storage tasks including converting quotas and the number of mailboxes into SAN storage units, applying RAID levels, LUN creation, binding and masking.
All this is aimed at reducing the amount that companies pay to manage Exchange, which industry estimates peg at $15 per month, per user. "When I told our CFO how much we pay for email systems in a year, he was astonished," Augustadt said. "People are becoming aware of the crazy things people do to get to their data easily and looking for ways to mitigate them."