Quality Awards: Backup software


This article can also be found in the Premium Editorial Download "Storage magazine: Tools for successful data migrations."

Download it now to read this article plus other related content.

Enterprise backup software: Would you buy this product again?

Product quality
Product quality and reliability were the lowest rated categories as measured by the average of all vendors. Given IT's frustration regarding B/R operations, it's not surprising these were the lowest rated areas. To assess product quality,

Requires Free Membership to View

we asked respondents to rate the products in terms of ease of use, configuration and the need for manual intervention. Only two SMB products--Retrospect and Backup Exec--received scores above 5.0 for quality, while Galaxy was the only enterprise-class product to score above 5.0 (see the "Product quality" section of the SMB backup software chart, and the "Product quality" section of the Enterprise backup software chart).

Media represents the largest single, non-personnel B/R cost. As another indicator of quality, we asked which product utilized media most efficiently. CommVault Galaxy's 5.68 led the ratings in this area, followed closely by IBM'S TSM with a 5.55. Among SMB products, Retrospect rated the highest with a 5.50.

We also asked respondents how well they felt the products delivered value for the money. They rated Galaxy as the best value (5.50); Data Protector and TSM were nearly tied for second with a 5.22 and 5.21, respectively.

Gary Wood, manager of server engineering for the County of Westchester, NY, explains why his firm switched to EMC Legato's NetWorker after using Syncsort's Backup Express for five years. "As our server base increased, and consequently the load on the product [Syncsort], we began to see some cracks appear," he says. He concedes that while implementing NetWorker "we did have a bumpy road along the way, as it wasn't configured properly by Legato and their consultants." It took about a month to iron out the problems and, as far as Wood is concerned, the configuration snafu is no longer an issue. "All in all, we're very pleased with the Legato experience."

The University of Michigan's Lewis also experienced some configuration problems when installing Veritas' NetBackup. While he's pleased with NetBackup's performance, he notes that "it's extremely cumbersome to install and configure, possibly because of the flexibility of the product."

Product reliability
Determining product reliability for B/R software is different than from most other storage components because a discussion of five-nines uptime isn't relevant. B/R rarely needs to operate on a 24/7 basis, but it must run when needed. Therefore, we examined a variety of operational characteristics to derive reliability. Many of the reasons for a B/R job failure--including tape drive failures/errors, media errors and scheduling errors--are beyond the control of B/R software. However, B/R software does control the ability to operate unattended (i.e., "lights out") without interruption due to bugs and can handle errors effectively.

The statement, "I can operate this product in a lights-out mode" generated low marks for most products. Only Galaxy (5.71) and Data Protector (5.30) stood out with scores above 5.0. The other enterprise products with favorable ratings were NetBackup (4.81) and TSM (4.79). Among SMB products, Retrospect's 4.71 narrowly nosed out Backup Exec's 4.68.

Also generating low scores across the board was, "This product experiences very few bugs." With no product breaking the 5.0 barrier for this statement, Galaxy was the highest rated with a 4.96, followed by Retrospect (4.88). Interestingly, we noted an almost direct correlation between the ratings for this statement and the overall rankings of the products. The message to vendors is consistent: Spend more time solving existing problems to create stable products rather than implementing the latest "whiz-bang" feature.

Product support
Given the low user satisfaction concerning product reliability, and especially the number of product bugs, product support gains importance as a purchase criteria (see the "Product support" category in the "SMB backup software" chart, p. 38, and the "Product support" category in the "Enterprise backup software," chart, p. 39). Because most B/R software is sold through resellers, Level-1 support is usually handled by the reselling organization. When asked to respond to the statement, "The vendor's third-party partners are knowledgeable," 11 of the 14 products in our survey rated above 5.0. Retrospect led the SMB group with a whopping 6.56; among enterprise products, IBM's TSM earned the highest mark on this question with a 5.83, followed by a 5.75 for EMC Legato NetWorker.

But support isn't the only criterion for sticking with a product. "We uncovered some shortcomings in the [Syncsort] product along the way," says Westchester County's Wood. "The product was faulty in some areas, but the support was just great."

The timeliness of problem resolution is also an important issue. Responses to the statement, "Problems are resolved in a timely manner," were consistent with the overall curve of the study. Retrospect received the highest score (5.94) among SMB products, and HP led the enterprise group with a 5.20. We also found a correlation between this statement and "Support issues rarely require escalation." When problems escalate from Level-1 support (reseller) to Level-2 support (vendor), there's a necessary hand-off between the two. Individual vendor scores for these two questions were similar in every case. Clearly, knowledgeable Level-1 support that avoids problem escalation is an important factor in overall customer satisfaction.

The Diogenes Labs-Storage magazine Quality Awards
The Diogenes Labs-Storage magazine Quality Awards is a joint project of Storage magazine and Diogenes Analytical Laboratories Inc. It's an ongoing process that surveys users to gather their perceptions related to service, reliability and quality of key storage infrastructure products. Storage reported the results for enterprise-class arrays in its August 2005 issue; other product categories will be addressed in the coming months.

Diogenes Labs and Storage magazine congratulate CommVault Galaxy and EMC/Dantz Retrospect on taking top honors among a large field of B/R products. And thanks, too, to the nearly 500 users who took the time to complete the survey and provide such insightful responses.

Rich Castagna is the Editor of Storage; Phil Goodwin is president of Diogenes Analytical Laboratories Inc.

This was first published in September 2005

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: