This article can also be found in the Premium Editorial Download "Storage magazine: Overview of top tape backup options for midrange systems and networking segments."
Download it now to read this article plus other related content.
Although most configurations maximize speed by directing data to locally attached libraries, NDMP's TCP/IP heritage also provides more backup flexibility by allowing aggregation of backup data from multiple NDMP-compliant filers onto libraries that aren't physically attached to them. This allows customers to put backup data where it's needed over standard networks, while still retaining full manageability over those backups from the central backup software.
Wherever backup data is ultimately stored, the NDMP-compliant tape device transmits confirmation of the write-back to the central backup server. At the same time, data is directed straight from the server to the tape library, often as quickly as the tape can take it. This reduces costs and improves the speed of NDMP solutions.
Integrated circuit maker Advanced Micro Devices (AMD) Sunnyvale, CA, boosted backup speeds to 60GB/hour, per filer using Workstation Solutions' NDMP-compliant Quick Restore to back up 15TB of Unix, Linux and Windows data from 27 NetApp filers to six tape libraries, including 35 SCSI-attached tape drives. The move has improved the robustness of the company's backup strategy, paved the way for future growth and eased management of AMD's backup environment.
Vendors jump on NDMP bandwagon
NetApp originally envisioned NDMP as a way to free itself from the need to write proprietary hooks to link its filers with every proprietary management environment
This promise was a major part of the reason Legato snapped up Intelliguard for $52 million early in 1999, then rushed NDMP-compliant versions of its products onto the market later that year.
Since then, broader support for NDMP has largely solved the major headache of NAS device interoperability. At least 39 product families from 30 vendors now support the protocol, allowing customers to choose backup software and NAS devices based on their individual characteristics, not compatibility.
NDMP is based around just eight specific operating system interfaces. Connect, configure, data, tape and SCSI provide a direct interface and control over remote tape drive and SCSI devices from NDMP-compliant vendors. Notify, file history and logging interfaces allow backup software to manage file histories of NDMP-based remote backups.
In theory, any company can easily make its products NAS-capable by implementing these interfaces (SDKs for each version of NDMP are available from http://www.ndmp.org/download). This ease of use has allowed many smaller startups to quickly bring devices such as storage appliances into the NAS market without having to first win support and certification from individual backup software vendors.
"One of the things that's key to our success is ensuring that customers don't have to change anything," says Geoff Barrall, chief technology officer of storage appliance maker BlueArc. "NDMP removes all the driver hassles customers used to have."
It certainly seems to have served this purpose: customers generally report NDMP implementation to be relatively painless, although tweaking can be necessary. For example, SAS had to increase the NDMP timeout to allow for often long delays while filers searched through hundreds of gigabytes of files to identify those that had changed. Such issues, however, are small given NDMP's ability to overcome the massive challenge of interoperability.
Lack of standards
Still, all isn't necessarily well in the world of NDMP standards. Because vendors initially installed NDMP for use with their own products, there were many interpretations of what was actually needed to ensure full compliance. And with no formal standards body to set the record straight, there have been reports of inconsistencies between implementations causing headaches for storage managers keen to take advantage of the protocol's flexibility.
Furthermore, not every vendor supports every form of NDMP transfer. One common inconsistency is support for Direct Access Recovery (DAR), a NDMP capability that allows NDMP-compliant servers to address individual files contained within volume backups. DAR support is far from ubiquitous, but its appeal to customers should make it more common in the future.
In recognition of the growing support for NDMP after the release of NDMP v.3 - now common in most implementations of the standard - its development was handed over to the Storage Networking Industry Association (SNIA), which in mid-2000 formed a NDMP working group to refine the details of NDMP v.4. This year has finally seen vendors release products based on that version, which includes a range of enhancements including improved restartability of interrupted backups and support for custom extensions.
First among these extensions is NDMP Snapshot Management, a NetApp add-on that allows NDMP-compliant devices to take and operate on instantaneous backups of their data. This should allow companies to save considerably on hardware and reduce the complexity of managing massive tape archives.
"With this technology you don't have to worry about [backup] window time anymore," says Hieu Le, senior network system administrator with programmable semiconductor manufacturer Altera, San Jose, CA, which is using NDMP-compliant products from CommVault Systems, BlueArc, and Quantum to back up 2TB of data accessed by a staff of more than 600. NDMP's seamless scalability will help Altera expand its backup strategy to meet storage demand expected to top 5TB by the end of the year.
With NDMP v.4 out the door, work has already begun on the next major upgrade, v.5. This time, the standardization is being driven under the auspices of the IETF (Internet Engineering Task Force), a move that was made to help NDMP tap into the organization's extensive knowledge base pertaining to areas such as security, authentication, operation through firewalls and congestion control. An initial specification is expected by mid-2003.
Now that NDMP is firmly entrenched within the panoply of backup vendors, it has emerged as the great unifier: a means of resolving potential incompatibilities and adding greater flexibility to relatively inexpensive and well-understood NAS environments. Coming versions will add even better features, but NDMP's already considerable growth and successful implementation confirms its value for the long run.
|NDMP roadmap: an evolving protocol|
This was first published in July 2002