Published: 09 Jun 2008
NetApp notched solid scores in all categories. However, IBM was a strong challenger, ranking second among enterprise and midrange systems. Hewlett-Packard (HP) Co., which was near the bottom of both categories last year, placed third among enterprise products and fourth in the midrange group. Dell Inc. improved its year-over-year placement by coming in third for midrange products. EMC Corp. took fourth place in the enterprise class and rounded out the midrange finalists with a fifth-place finish. Sun Microsystems Inc., the fifth-place finisher in the enterprise group, didn't receive enough responses to qualify for the midrange award.
BlueArc Corp., last year's enterprise winner, didn't generate enough responses to be statistically valid this year. Also missing was Hitachi Data Systems Corp., which failed for the second time to garner enough responses to be considered as a finalist since winning the enterprise category in our first NAS Quality Award.
Scores across the board were in line with last year's results. Every finalist, except Sun, received a 6.0 or higher for its overall score on our 1.0-8.0 scale, where an 8.0 is the highest score. NetApp's overall scores were 6.68 for enterprise and 6.76 for midrange. No vendor was able to crack the 7.0 barrier in any of the five categories in which they were evaluated by users.
High overall user satisfaction was evidenced by 82% of all respondents saying they'd make the same purchase decision again. On previous surveys, we've found that a 75% "Yes" rate is a more common score for the "buy again" question. (This includes all vendors in the survey, not just the finalists.)
As part of our analysis, we compare the scores a respondent provides when evaluating more than one vendor's product line. This is perhaps the best real-life comparison of systems. While Sun didn't fare as well in the overall score as other vendors, it led the group in this head-to-head comparison. When Sun StorageTek 5200/5300 systems came up against other systems, they had a higher score 56% of the time. NetApp's FAS2000/3000/6000 had higher scores 43% of the time. It's important to note that in these comparisons, scores higher than 50% are rare as there can be one winner and multiple losers per comparison.
We ask users to tell us about their experience with their vendor's sales force because it's a predictor of the likelihood that a system will be appropriately matched with a need. Vendors who keep their customers' interests foremost and have knowledgeable teams are less likely to oversell a system. Indeed, there has been a high correlation between the results of this category and the award winners. "We felt that they were working in our best interests," says Fishman, "knowing full well, of course, that we would come back for more if and when we needed it."
The sales-force competence scores provided the narrowest range of any survey section from top to bottom. NetApp led the enterprise and midrange categories but, in both cases, the scores were NetApp's lowest section scores: a 6.50 among enterprise products and a 6.48 in the midrange class. Slightly more than half of NetApp's customers purchased through a value-added reseller, while the rest bought directly from the manufacturer.
This was the one section in which IBM didn't seriously challenge NetApp. Although IBM's sales-force competence scores weren't bad, they were IBM's worst of the survey. In the enterprise group, IBM placed fourth with a 5.93, its only sub-6.0 score in any section. Among midrange products, IBM did better with a 6.16, but still ranked last.
Sometimes familiarity breeds good sales partners. "We spoke with NetApp initially and then ended up going with IBM," says Steve Baucum, manager of Unix support at Methodist Healthcare in Memphis. They already had IBM server and storage gear installed, so it was a key consideration. "Everything else that was touching the SAN was IBM, so we decided to stick with one vendor," he notes.
Having a good ongoing relationship with a vendor can help swing a deal, but the incumbent vendor also has to deliver on price. "I decided on HP--to be honest--because they had a decent price and I was very sure of their service," says Graham Husk, network services administrator at Canada Science and Technology Museum (CSTM) Corp. in Ottawa. But it wasn't just a dollar-and-cents matter. "I've never had any problems dealing with their sales staff as far as their technical capabilities," adds Husk.
NetApp received its best section score for the statement "My sales support team is knowledgeable" (6.88). Its lowest score was for "My sales rep keeps my interests foremost" (6.12). HP, which placed second for sales-force competence in both categories, received its highest mark for "My sales rep is knowledgeable" (a 6.58 in the enterprise category).
"The N Series was at first sold to us because we needed the performance," says Baucum of his firm's clustered IBM System Storage N5200 arrays, which IBM OEMs from NetApp. "But what we've gotten out of it besides the performance is the replication, snapmirror, snapshotting [and] the integration with Volume Shadow Copy Services."
With most organizations experiencing steady disk capacity growth, scalability is a key issue. Ultrasonic Precisions' Fishman is confident his firm's two NetApp FAS6000 arrays will accommodate anything he throws at them. "The maximum capacity is 420TB; that's plenty more than we'll ever need, at least in the next 10 years," he says.
Being able to grow capacity is important, but the ease with which disks can be added convinced Husk to go with a pair of HP ProLiant DL380s. "HP Systems Insight Manager allows you to actually format RAIDs on the fly after just plugging in the hot-plug drives," says Husk. "There's no rebooting required."
The rating for the other finalists fell in line with the overall rankings, except for Dell. Dell polled a steady third in all sections except this one, where it placed fifth among midrange products with a 5.93. But that's clearly not a bad score, and it was Dell's only sub-6.0 rating. IBM's enterprise users apparently like the feature set on balance most of all, giving the company a 6.89 for "Overall, this product's features meet my needs." IBM's midrange users ranked the product line's scalability as its best feature. Sun received its lowest section score (a 5.15) for product features.
Initial product quality
NetApp's midrange products received a 7.23 for "This product is easy to get up and running." This score was the highest for any statement in the survey. Eighty-three percent of respondents gave it a rating of 7.0 or 8.0. It received a 7.13 for "This product was installed without any defects," and a 7.08 for "This product requires very little user intervention."
Getting his NetApp FAS6000 arrays up and running couldn't have been easier for Ultrasonic Precisions' Fishman. "It was a three-week process, and I thought it was pretty much on target," he says, adding, "it actually took two days less than I thought it would, and that includes training." Methodist Healthcare's Baucum had a similar experience with his IBM N5200 clusters, but took a cautious approach to deployment. "We spent about two months doing some proofs of concepts on them," he says.
In this section, users are asked how well a product delivers value. Perhaps not surprisingly, NetApp's midrange products polled highest in this regard with a 6.65 for the statement "This product delivers good value for the money." NetApp's enterprise group wasn't far behind with a 6.61. IBM garnered a 6.47 for its midrange products and a 6.00 for enterprise, while HP products received a 6.24 (midrange) and a 6.18 (enterprise). Dell's midrange product received a 6.14.
IBM spoiled NetApp's quest to sweep all categories and sections. IBM's midrange product reliability score of 6.85 topped NetApp's 6.79, although NetApp won the enterprise category with a 6.70 (the highest of that group). IBM's customers awarded the company's midrange products a 7.11 for the statement "This product experiences very little downtime." IBM also received a good score for "This product requires very few unplanned patches" (6.88).
When asked about any unplanned downtime with his IBM N Series boxes, Baucum replies "So far, no." But, "I don't want to say that too loud" he adds, not wanting to jinx his good luck.
While IBM had the highest score in this section, NetApp bested IBM on two items, a score of 7.10 for "This product experiences very little downtime" and "This product requires very few unplanned patches." True to form, these questions proved a predictor of the overall winner. NetApp's undoing, however, was the statement "Patches can be applied nondisruptively." The associated 6.11 score may be decent in its own right, but it was enough to drag NetApp's section average down to second place.
"I don't think any other product compares to this one," notes Ultrasonic Precisions' Fishman about the reliability of his NetApp FAS6000s.
NetApp was the only vendor with above-7.0 scores for any statement in this section. For the statement "This vendor provides support as contractually specified," NetApp's enterprise products scored 7.00, while its midrange products received a 7.06. These scores were substantially higher than any other vendor's on that issue.
"We called them when we migrated up from Windows 2003 to Windows 2008, and they came down and consulted with us for about an hour," says Fishman of his NetApp support. "They're very good."
IBM placed second in both the enterprise and midrange categories in this section, but a 5.44 for "Problems are resolved in a timely manner" and a 5.56 for "The vendor's support personnel are knowledgeable" dragged down its average.
Methodist Healthcare's Baucum could have gone with NetApp for his NAS deployment, but decided to stick with IBM even though it was still NetApp gear. "It kind of reduces the friction if you've got something that's not compatible," he says. With a lot of IBM equipment already installed, he preferred to deal with one vendor. "You don't have to have a guessing game of visiting multiple vendors to get to the right versions of code," he explains.
Nevertheless, NetApp did have solid support among its users. In its case, 86.8% of enterprise users would make the same decision, while 82.4% of midrange users would do so. HP had solid numbers above 80%, while EMC had 78% (midrange) and 76.9% (enterprise). Sun trailed the enterprise group with a 64.3% user loyalty. Regardless, it's safe to say that while NAS users have the occasional bone to pick, they're apparently a happy group of people.