This article can also be found in the Premium Editorial Download "Storage magazine: The business case for solid state vs. disk storage."
Download it now to read this article plus other related content.
In for the long run
The true test of a backup application comes with the daily grind of protecting a company's data assets; first impressions and feature sets are all well and good, but long-term reliability and stability are paramount. CommVault was the only enterprise-class backup app with a 6.00-plus average for the product reliability rating category, picking up a 6.24 while leading on six of eight statements. Its strongest showings were for meeting service-level requirements (6.86) and operating system/platform support (6.84).
Second place HP (5.94 overall) also showed strength for those two reliability criteria (a 6.18 and 6.59, respectively) among its consistent marks in this category. For the two statements that CommVault didn't have the leading score, EMC's NetWorker led the pack with a 6.07 for "Requires very few unplanned patches/updates," while IBM Tivoli Storage Manager garnered a top mark of 5.87 for "Very few bugs."
Within the midrange product set, Acronis bested CA (5.38) and Symantec (5.34) for five of the eight reliability statements on its way to winning the category with a 5.68 average. This was CA's sole second-place finish; its ARCserve Backup product came out on top for meeting service-level requirements (5.94) and for requiring few unplanned patches or updates (5.83). Backup Exec finished a close third with a winning score of 5.17 for "Vendor provides comprehensive upgrade guidance" -- an area where all three vendors' ratings were relatively low, indicating that some improvement in the upgrade process would be welcomed by users.
Backup software vendors have labored diligently to make their products easier to use with improved GUIs, wizards and other interface aids. But with an application as complex as backup, users will invariably need some help from time to time, so how well a vendor supports its product will figure significantly in a user's perception of ease of use. After backup vendors as a group registered a disappointing overall average of 4.88 in the first Quality Awards survey for backup software, they have shown steady progress in the quality of support they provide, with the best numbers yet recorded this time around (a very healthy 5.89).
CommVault's 6.20 score for the technical support category was enough to nudge out HP's 6.12. IBM (5.82), EMC (5.73) and Symantec (5.58) turned up pretty good ratings as well. CommVault is apparently meeting users' support expectations, as its highest score in this category was a 6.86 for "Vendor supplies support as contractually specified." HP topped CommVault on the statements dealing with documentation (a 6.15 vs. CommVault's 6.00) and "Support issues rarely require escalation" (6.00 and 5.69, respectively). EMC was rewarded with a statement-high score of 6.27 for training its third-party partners well.
Again, we detect some grumbling among midrange users as they aren't nearly as satisfied with their vendors' support efforts. Although the overall support score average is higher than it's ever been, it's still a rather unimpressive 5.23.
Acronis topped the category with a modest 5.39 overall support score, but did manage to get the only 6.00-point rating in the category with an even 6.00 for providing support as contracted. All three vendors failed to reach the 5.00 plateau for providing adequate training, with Acronis earning a 4.80, followed by Symantec (4.74) and CA (4.59). And the three didn't fare much better for providing adequate documentation, with scores ranging from Acronis' 5.05 to Symantec's 5.08, with CA sandwiched in between at 5.07. The message to midrange backup software vendors seems clear: Provide more support and you're likely to see better all-around scores on future surveys.
As on all Quality Awards surveys, we ask respondents if, given what they now know, they would make the same purchase again. Given its overall performance on this survey, it's not too surprising that CommVault had the highest percentage of users who said they'd do it all over again (83.7%). Only 62.9% of HP's users said they would buy Data Protector again, a surprisingly low number considering how well the product was rated overall.
For the midrange applications, 69.2% of Acronis' users said they're pleased enough to buy the product again, followed by Symantec Backup Exec (66.7%) and CA ARCserve Backup (57.1%).
Although not necessarily related to user satisfaction, it's interesting to see how heavily users are taxing these backup products. Among enterprise programs, IBM's Tivoli Storage Manager is the workhorse, backing up an average of 67.1 TB weekly. For the midrange apps, Symantec's venerable Backup Exec is the heaviest lifter with an average of 13.3 TB of data backed up each week.
BIO: Rich Castagna (firstname.lastname@example.org) is editorial director of the Storage Media Group.
This was first published in June 2010