Published: 07 Aug 2002
Microsoft has made no bones about its designs on the storage industry. In the take-no-prisoners style that typifies its entrance into new markets, the Redmond, WA, giant has rapidly been recruiting consulting partners, riding shotgun with ISVs and talking to customers as part of a crash course on the ins and outs of storage management.
The fruits of all this work will become evident when Windows .NET Server finally debuts next year. The launch will be crunch time for Microsoft's recently formed Enterprise Storage Group (ESG) - headed by Microsoft senior vice president Bob Muglia - which will play its hand by introducing several storage-related .NET features that should improve Windows' still-developing storage credentials.
At the core of ESG's storage management plans is the Virtual Disk Service (VDS), a Windows .NET Server component that provides block-level virtualization of stored data. VDS includes a host of APIs that let applications manage those blocks with high granularity and control. Using these APIs will give storage management applications better control over file system operation. Although VDS interfaces will remain somewhat vaguely documented at first, Microsoft promises the first update to Windows .NET Server will fill in VDS documentation gaps.
VDS's major claim to fame is its promise to simplify the management of multiple elements within the storage infrastructure. According to Keith Hageman, U.S. program manager for core file services, VDS will give management applications a way to look through the multiple layers of, for example, a RAID implementation to affect changes at the OS level, the controller level, software RAID level, and the disk level, from a single console.
In addition, a utility called Diskraid will ship with the .NET Server resource kit that will allow granular control of which spindles will be used by a given LUN. Microsoft claims that Diskraid and VDS will simplify the process of writing scripts to control various processes on different vendors' arrays, since VDS will abstract some of the hardware-specific details.
Most anticipated among the new features is Volume Shadow Copy Services (VSS), a service that builds on VDS to provide instantaneous, full or incremental backups of chosen NTFS volumes or specific applications. Data is dumped into a file, and the mirrored application is free to resume operation. VSS files can then be backed up onto locally-attached tape or disk, or sent to another dedicated server for archiving - without increasing the CPU load on the primary server. Microsoft's design goal is to support multivendor snapshots with a total application downtime (freeze to thaw) of less than 10 seconds.
The VSS architecture consists of writers and providers. VSS providers, which receive snapshots and store them, are embedded in management applications such as NTbackup or third-party systems. Providers get their data by communicating with writers embedded in each target application. Once a signal is received, the VSS writer initiates actions such as freezing a database while the snapshot is taken, resuming its operation afterwards and confirming the snapshot was taken correctly.
One thing VSS promises to do, at least for the Windows world, is bridge the gap between backup software packages, application-specific backup modules, and device-specific snapshot capabilities. VSS-enabled backup software would automatically be able to backup any application with VSS writer on any device with a VSS provider (see "VSS simplifies application backup"). With backup software support from some of the majors (Veritas and Legato, for example) and hardware support from EMC, HDS, HP, and XIOtech as of this writing, many users would have the potential of simplifying backup of key applications.
Windows .NET Server will ship with writers for Active Directory, NTFS file systems and SQL Server. Microsoft and third parties will work to develop other VSS writers for specific applications. This approach gives storage administrators a high level of control over specific backup operations and let ISVs develop specific tweaks to optimize interaction with particular applications. For example, a custom VSS writer might provide advanced archiving of Exchange e-mail systems, allowing better long-term management of e-mail communications as mandated by recent SEC policies. In fact, Microsoft says, the Exchange writer will be part of the next version of the e-mail server (now code-named Titanium).
Adding snapshot capabilities to .NET Server will help Microsoft slowly inch into the high-end storage management area it wants so desperately to inhabit. The company has already won over major ISVs by offering its shadow copy methodology as a consistent standard for data imaging. This is a major sweetener for ISVs that have traditionally had to develop their own proprietary snapshot technology.
"In the past, if a vendor wanted to make functionality like this available on Windows, they had to write a complete solution," says Eric Burgener, senior director of platform strategy with Veritas. "Now we'll be able to make system calls into .NET, which means it's going to be that much easier to provide these features in Windows."
VSS is about more than backup, however. Because it's tightly integrated with NTFS, VSS allows administrators to give users live access to old snapshot images. Shares on Shadow Copy folders operate just like any other volume. For example, with Shadow Copies for Shared Folders, users can directly access old versions of their files through a Windows Explorer-like interface. This capability is available today for XP users.
"Users with [Shadow Copies for Shared Folders] enabled can see old versions of files, and they're managed just as they would manage ordinary files," says Jason Goodman, product manager for storage technologies in Microsoft's Windows .NET Server Solutions Group. "This allows for user-initiated undeletes of previous versions of files. Imagine this as an enabling technology for HSM-like near-line storage capabilities. Because it's enabled through the file system, it's transparent to the user."
Administrators of Windows .NET Server systems configure VSS through the Disk Management or Shared Folders Snap-Ins within the Computer Management console. The amount of disk space dedicated to shadow copies is set by the administrator, with older images automatically deleted as necessary to free up space for new snapshots.
Sharing restrictions have meant Windows NT and Windows 2000 Server lack the ability to back up open files currently locked for use by specific end users; such files are unceremoniously skipped during backup. Within .NET Server, however, VSS will allow backups to archive works in progress by backing up even open files; unsaved but backed-up versions will be available to users alongside old versions of properly saved files. Customer response to VSS has been positive. "A quick review tells me there are some very strong business cases to be made to take advantage of .NET Server's new features," says Larry Scott, manager of server support with Dollar Rent-A-Car of Tulsa, OK, which is using .NET and Web services standards to rebuild many of its core business applications. "Now we do traditional backup from servers onto tape; it's always laborious to have to retrieve tapes to do a restore. [.NET] features could be very time-saving if we implement them properly." .NET Server will also ship with a utility called volsnap.sys that creates copy-on-write shadow copies.
Snapshots aren't .NET's only enhancement to Windows storage technology. For example, improvements to the Diskpart.exe command-line tool - copies for W2K and WNS (beta) are now available at http://www.microsoft.com/windows2000/techinfo/reskit/tools/new/diskpart-o.asp - which replicates the functionality of the Microsoft Management Console Disk Manager Snap-in. This allows administrators to share clustered volumes and grow basic disks online. This is a boon for companies using Windows Clustering Services, since clustered servers can't currently be expanded without taking the system down. Using volume mount points, administrators can graft a new namespace onto an existing folder, seamlessly add the volume to the cluster, and use Diskpart to automate scripting and management of the expanded volume.
Distributed File System (DFS) environments also get a boost. Administrators will gain more flexibility in DFS configuration, with faster server-to-server replication and a distributed server topography that provides failover after server failure. This feature also reflects .NET's distributed nature by enabling distributed storage environments and allowing clients to choose the nearest DFS server to connect to.
Businesses using storage area networks (SANs) will find some relief in Windows .NET Server. Microsoft has redesigned its driver and port model, which has so far focused on ATA and SCSI devices. The new version, called StorPort, incorporates better support for the architectural differences of SAN and NAS (network-attached storage) devices as well as emerging standards like InfiniBand and iSCSI.
Using the Mountvol.exe tool, administrators can override automatic volume mounting which controls how LUNs are visible to particular servers. Manual mounting allows storage administrators to better control SAN zoning as well. Up to 128 targets with 255 LUNs each are supported.
Will this change the behavior of Windows in mounting LUNs on a SAN?
Clusters can be extended onto SANs to allow all disks to be recognized as shared disks. Depending on hardware capabilities, administrators will also be able to boot a .NET Server environment remotely over a fiber-attached SAN. This obviates the need for locally-attached storage, supporting a greater degree of consolidation onto the single SAN.
Also helping administrators navigate SAN topography is a built-in Multipath I/O (MPIO) driver. MPIO allows definition of a number of different paths between storage and server, improving redundancy and availability in the event of connection interruption. It's long been supported in many Unix environments and through third-party Windows add-ons, but its incorporation into .NET Server - the basic driver will support up to 32 paths - provides a common redundancy architecture that will be a godsend to widely distributed SAN environments.
"Ultimately it's about bringing together the stream-based file system world with the RDBMS [a relational database management system, such as Oracle] world and presenting a very simple schema layer to applications," says Microsoft's Goodman.
This ease-of-use philosophy will be extended as customers increasingly adopt Windows-based storage appliances. Since Microsoft released its Windows-based server appliance kit (SAK) for OEMs last year, Windows-based devices have rapidly elbowed their way into a market dominated by boxes based on proprietary operating systems. Analyst firm IDC recently reported the Windows devices had seized 25% of the overall NAS market, with estimates predicting their share could grow to 60% by next year.
With Microsoft set to update the SAK twice a year, it won't be long before Windows-based storage systems become tightly integrated with the storage management capabilities of .NET Server. And while the Windows-based appliances may not offer the lightning-fast NFS performance of their highly optimized competitors - reports suggest Microsoft's CIFS tweaks have been more fruitful - their low cost and easy .NET-based management should make them a valuable adjunct to long-term storage strategies.
The customer approach
Storage management vendors are working closely with Microsoft's ESG to stay abreast of new .NET Server features to ensure their own platforms add value to Microsoft's core functionality. Expect .NET-aware upgrades of major backup apps at or shortly after the launch of .NET Server.
Despite Microsoft's history of making applications bigger with every upgrade, early signs suggest that .NET applications are actually smaller than their .ASP counterparts. "When you remove the source tree and leave the compiled app, it's much smaller than traditional .ASP pages," reports Dean McRobie, senior manager of technology with consulting service provider Sapient, Arlington, VA. "The code that comes out of [Visual Studio .NET] is very efficient and removes a lot of the overhead we have normally seen with COM or .ASP-based apps." This smaller footprint could conceivably pave the way for smaller .NET applications addressing a particular department's needs instead of trying to satisfy the entire company, he says.
Just because individual applications are smaller doesn't mean storage administrators will get a reprieve from growth. Although individual applications may be smaller, overall storage usage is likely to increase due to the overhead for .NET's operational data such as authentication and audit trails. Robert Sanders, practice director for global digital strategy and development at Unisys, envisions this issue alone could increase storage requirements by 5% to 10%. Of course, simply migrating existing applications to support.NET requires more testing and the other requirements of any major redevelopment.
VSS will certainly increase demands on companies' storage resources. Storage economy has never been the company's high point, and Microsoft's storage strategy is more or less built around the assumption that data is online all the time. More often than not, Microsoft views solutions through their money-is-no-object, rose-colored glasses. If data is to be available all the time, slow tape storage needs to be replaced with ATA/IDE arrays that stack hundreds of gigabytes of low-cost disks. Although they're slower than primary server drives, large numbers of slower drives do provide a cost-effective way to keep large numbers of shadow copies available to users on an ongoing basis.
Providing adequate storage infrastructure for .NET is about more than just adding disks, however. Dallas-based customer relationship management software developer Brierley & Partners, an early adopter of .NET, found that an entire storage infrastructure upgrade was called for after a customer loyalty application it built to support Sony's my.sony.com site performed over twice as fast as expected.
"We had to rearchitect the way we laid out SQL Server," says George Nemer, system architect with Brierley, which built the Sony system around a two-node Windows 2000 Datacenter cluster with 8 CPUs in each system and 1TB of available storage. "Because the performance of .NET was faster, we were hitting the database harder and the regular SCSI disk couldn't handle it. We had to split transaction logs from the data because transactions were intensive; you split things up to avoid hot spots. We are constantly learning more about increasing performance."
Brierley upgraded the system's interconnects to Fibre Channel, and boosted its backbone network from 100Mb/s Fast Ethernet to 1000Mb/s Gigabit Ethernet. The result has been a dream environment supporting more than 700,000 Sony customers using nearly 20 clustered Compaq servers support around 40,000 simultaneous users. Nemer anticipates .NET's snapshot capabilities will be a further improvement by cutting backup time.
Microsoft still has a lot to learn about enterprise storage, and it's hunkered down to climb the learning curve as quickly as possible.
One ISV - who asked not to be named - said he felt Microsoft has done a good job bringing its .NET Server platform up to speed to compete for enterprise storage business, but "it's not 100% there yet. There's just been a lot of third party development work for UNIX that isn't available for Windows," he says, adding that "it takes a while to catch up."
Looking to the future, the ISV says, Microsoft has some very aggressive plans. It's actively developing products that advance the distributed file system (DFS) capabilities. A DFS lets clients access and process data stored on the server as if it were on their own computer. When a user accesses a file on the server, the server sends the user a copy of the file, which is cached on the user's computer while the data is being processed and is then returned to the server. These efforts are a nature progression to work on global file system development.
And perhaps most disruptive, Microsoft, according to the ISV, is seeking a competitive advantage against major device vendors such as EMC, HDS and other big players, by writing APIs for .NET Server that will neutralize some of the unique functionality built into those companies' storage devices. In other words, .NET Server will become an agnostic platform and users can mix and match storage devices in their environment. Stay tuned.
Obviously, other ISVs are optimistic about Microsoft's long-term success as well, suggesting that .NET will rapidly assume a larger role within storage infrastructures as customers upgrade management software and move to .NET Server.
"Microsoft starting ESG has made us even more comfortable [with its commitment to storage]," says George Symons, vice president of product management and development with Legato Systems, which has been working closely with Microsoft to flesh out the .NET storage interfaces. Says Symons: "There really is a focus now on customers' storage requirements, what is the infrastructure we need to build, what hooks should be in there, and what it means to successfully back up Windows."