Published: 11 Aug 2008
| In our closest Quality Awards finish yet, EMC and NetApp tied for first, with Hitachi Data Systems and Hewlett-Packard only a few fractions of a point behind.
In the four years we've conducted our Diogenes Labs–Storage magazine Quality Award for enterprise arrays we've never had co-winners until EMC Corp. and NetApp tied for first place this year.
EMC has risen steadily in this survey and enters the winner's circle for the first time. Similarly, NetApp rose from prior disappointing results in the enterprise array category to join EMC at the top. Both companies posted overall scores of 6.53 on the survey's 1.0 to 8.0 scoring scale. NetApp is on a roll after winning top honors in the most recent NAS survey for its enterprise and midrange systems. Hitachi Data Systems Corp., the winner of all three previous enterprise array awards, fell to third place with a score of 6.50. And to further emphasize what a horse race this year's competition was, Hewlett-Packard (HP) Co. trailed Hitachi by a mere 0.01 of a point. IBM Corp. finished fifth with a 6.39.
The competition was close across the board (see "Products in the survey," below). Our survey includes five sections and within those sections we had four different companies post the high score (see "About the survey," below). EMC was the only company to win two categories: product features and product reliability. Surprisingly, previously dominant Hitachi Data Systems didn't win a single section. Sun Microsystems Inc. (sixth) and 3PAR Inc. (seventh) were the other two companies that didn't win a category, although Sun had a respectable 6.32 overall finish. 3PAR, in its first year as a finalist, experienced just how brutally competitive this category can be. While its 5.94 final score indicates strong user satisfaction, it was the only company to finish below 6.0, coming in last in all categories.
Average scores were slightly higher than last year, a trend with recent surveys. EMC's and NetApp's winning score of 6.53 was the highest overall tally we've seen in any of our enterprise surveys. In last year's survey, 30% of EMC respondents reported having more than 200TB of data, earning the firm the designation as the group's "heavy lifter." While 30.5% of EMC users topped the 200TB mark this year, Hitachi Data Systems nudged past EMC to claim the "heavy lifter" crown with 30.7% of its respondents having that amount of data. Across the board, large-scale sites increased significantly. All but one of the vendors in the survey had at least 20% of its users in the 200TB-plus range, the exception being 3PAR with 5% (one user).
The sales-cycle experience plays an important role and often lays the foundation for the ensuing user experience. Scores for the sales-force competence section of our survey generally correlate closely with overall results, but not this time. HP, which placed fourth overall, ranked highest for its sales force with a 6.43. NetApp wasn't far behind with a 6.39, followed by Hitachi Data Systems (6.38) and EMC (6.35). In this section, we asked users to tell us their experience regarding sales-force knowledge, flexibility and negotiability.
For many users, the sales process often involves a reseller and the product manufacturer. "We go through a reseller, but usually the reseller brings in resources from Hitachi," says Bryan Moore, storage manager for enterprise IT operations at Scripps Networks in Knoxville, TN. But even with a reseller in the picture, the vendor still plays a pivotal role. "Whenever we have anything outside of normal growth," says Moore, "we bring [Hitachi] in and they assess what our needs are and try to create the best possible fit."
James Odak, senior global systems engineer at Chicago's Wm. Wrigley Jr. Co., worked with both his reseller and HP to acquire the company's StorageWorks XP10000 and XP12000 arrays. His reseller "brought in the HP guy to help them out with the sale," he says. "They did a good job." Odak had a good idea of what Wrigley needed, but used his sales resources to confirm his choices. "I kind of tell them what I really need and they just sanity-proof it," he says.
HP's sales-force competence score was the lowest high score of any survey section. Yet no vendor knocked the cover off the ball in any particular area. Most vendors did well on the statement "My vendor's sales support team is knowledgeable," with NetApp receiving the highest score. The statement also provided EMC with its highest category score (6.58). HP received its highest score (6.63) for "My sales rep is flexible."
Wrigley's Odak has found his two HP XP systems slightly tougher to use than HP's midrange EVA line. "The EVA is extremely easy to manage--the XP takes a bit of understanding and knowledge before you can figure it well," he notes. Still, keeping the XPs up and running is hardly onerous for Odak. "Unless you're changing or adding disk constantly, there really isn't a whole lot to operation," he says.
Scores above 7.0 were hard to come by in this survey, but EMC came very close in this section with a 6.98 for "This product scales to meet my needs." EMC users apparently value the product's features in total better than its individual attributes, as it scored a 6.86 for "Overall, this product's features meet my needs." Individual responses to its snapshot, mirroring and replication features were solid, but not as high.
NetApp placed second in the section with a 6.57, and received its best section score for "This product's snapshot features meet my needs." Users also like the product's scalability, giving it a 6.73. In the reverse of EMC, NetApp users gave higher marks to individual features than to the overall system.
"It's been great," says Bates College's Bauer, referring to the overall feature set of his dual NetApp FAS3040s, but he's particularly impressed with the boxes' data protection features. "The snapshot takes five seconds at most; to start and stop the database takes three or four minutes," he notes.
One statement in this section, "This product is interoperable with other vendors," received lower scores relative to other areas across the board. EMC had the highest score of the group with a 6.73, while NetApp had its lowest section score (6.19). Only 3PAR fared worse with a 5.94.
Initial product quality
Again, how well a vendor works with its local resellers can make a big difference. "There was a NetApp engineer and one of the local engineers," says Bauer, describing his FAS3040 installations. Moore at Scripps Networks had a similar experience with the role his vendors and reseller played for the installation of his Hitachi Data Systems and 3PAR arrays. "Both are very involved in installation, and they don't leave until we're happy," he says.
NetApp apparently has the best out-of-box experience, receiving its highest section scores for "This product was installed without defects" (6.77) and "This product was easy to get up and running" (6.57). NetApp also had the highest score for "This product is easy to use" (6.49).
EMC trailed closely in most areas of this section, registering its highest score for "This product was installed without defects" (6.76). If EMC had an Achilles' heel in this section, it appears to be related to the statement "This product requires very little vendor intervention." Its 5.91 score was a rare sub-6.0 score for EMC in the entire survey. With a 6.0 or higher score on this statement, EMC would have been the outright survey winner. Of course, a higher score on any question would have pushed EMC beyond NetApp to be the outright winner, but this is the most obvious example of where the company was oh-so-close.
The statement "This product delivers good value for the money," is normally a bellwether to the overall order of finish. But this time, the results were somewhat upside down. Hitachi Data Systems was deemed the "best value" by its users with a 6.48 for this statement, and Sun was second with a 6.46. (Sun re-brands Hitachi's systems). HP, another Hitachi OEM, placed third with a 6.34. NetApp was fourth with a 6.30, followed by IBM (6.27), EMC (6.23) and 3PAR (5.88).
Despite its unimpressive ranking in this section, Moore likes 3PAR's value. "It's pretty cut and dry; when you buy it, features that are separate or unavailable in other platforms are simply built in," he notes.
EMC scored the only 7.0-plus rating in the entire survey for "This product experiences very little downtime" (7.08). Hitachi came very close with a 6.97 for the statement "This product requires very few unplanned patches." NetApp also narrowly missed a 7.0 score for "This product meets my service-level requirements" (6.96).
The statement "Patches can be applied nondisruptively" has been a key indicator in the past and was so again. EMC had the highest score on this statement (6.77), narrowly beating Hitachi's 6.75. NetApp trailed with a 6.60, but that was still good enough to be in the top three.
"The XP is rock solid," says Wrigley's Odak. "We've never had any downtime whatsoever due to the XP."
"It's never been down since it's been powered up," says Moore of one of Scripps Networks' Hitachi Lightning arrays, which was first fired up four years ago. Scripps' 3PAR InServ Storage Server S400 hasn't achieved that kind of longevity yet, but in its nine months or so of operation, Moore reports they "haven't had any problems with it."
The statement "The vendor provides comprehensive upgrade guidance" often generates below-average responses. This was true for most vendors, but EMC and NetApp bucked the trend with ratings of 6.77 and 6.76, respectively.
According to our survey, Hitachi has the most knowledgeable support group. It received a 6.89 for the statement "Vendor's support personnel are knowledgeable." EMC was second with a 6.61. EMC's only other slip in the survey was for "Problems rarely require escalation," where its 5.96 was solid but still below the 6.0-plus norm. Nevertheless, it received a 6.30 for "Problems are resolved in a timely manner."
Enhancing tech support with services like automated phone-home services helps to resolve problems and avoid bigger ones. "So far, we haven't had to have any onsite service," says Bates College's Bauer of his two NetApp arrays. Bauer says NetApp's phone-home monitoring service sent replacement disks before the suspect disks actually failed.
Odak had a similar experience with his HP XP systems' phone-home support. "As soon as something happens, they know it right away and they're on it right away," he says.
Co-winners EMC and NetApp achieved their identical scores through different means. EMC rode to the top on very strong scores in the product features and reliability sections, while NetApp was a model of consistency. NetApp was the only company to finish in the top three of every section, even though it had the high score in only one section. Nearly every vendor demonstrated some positive attributes in this survey, as well a few areas where they would probably like a do-over. That's a good reason to do it again next year.