Managing and protecting all enterprise data


QUALITY AWARDS III: NetApp sweeps NAS awards

After a clean sweep of both the enterprise and midrange categories, NetApp is the undisputed NAS king. BlueArc Corp., last year's enterprise winner, didn't have a chance to challenge NetApp with too few responses to make the final cut.

Perennially strong in midrange NAS, NetApp adds the enterprise-class award in the latest Diogenes Labs-
Storage magazine Quality Award.

Get out the brooms. NetApp is the undisputed top NAS vendor after a clean sweep of the enterprise and midrange categories in this year's Diogenes Labs-Storage magazine Quality Award III for NAS survey. NetApp is synonymous with NAS and has historically done very well among the midrange products in our survey, winning the top midrange spot three years in a row. The big news this time is that NetApp also captured the top enterprise award.

NetApp notched solid scores in all categories. However, IBM was a strong challenger, ranking second among enterprise and midrange systems. Hewlett-Packard (HP) Co., which was near the bottom of both categories last year, placed third among enterprise products and fourth in the midrange group. Dell Inc. improved its year-over-year placement by coming in third for midrange products. EMC Corp. took fourth place in the enterprise class and rounded out the midrange finalists with a fifth-place finish. Sun Microsystems Inc., the fifth-place finisher in the enterprise group, didn't receive enough responses to qualify for the midrange award.

BlueArc Corp., last year's enterprise winner, didn't generate enough responses to be statistically valid this year. Also missing was Hitachi Data Systems Corp., which failed for the second time to garner enough responses to be considered as a finalist since winning the enterprise category in our first NAS Quality Award.

Scores across the board were in line with last year's results. Every finalist, except Sun, received a 6.0 or higher for its overall score on our 1.0-8.0 scale, where an 8.0 is the highest score. NetApp's overall scores were 6.68 for enterprise and 6.76 for midrange. No vendor was able to crack the 7.0 barrier in any of the five categories in which they were evaluated by users.

High overall user satisfaction was evidenced by 82% of all respondents saying they'd make the same purchase decision again. On previous surveys, we've found that a 75% "Yes" rate is a more common score for the "buy again" question. (This includes all vendors in the survey, not just the finalists.)

As part of our analysis, we compare the scores a respondent provides when evaluating more than one vendor's product line. This is perhaps the best real-life comparison of systems. While Sun didn't fare as well in the overall score as other vendors, it led the group in this head-to-head comparison. When Sun StorageTek 5200/5300 systems came up against other systems, they had a higher score 56% of the time. NetApp's FAS2000/3000/6000 had higher scores 43% of the time. It's important to note that in these comparisons, scores higher than 50% are rare as there can be one winner and multiple losers per comparison.

Click here for the Enterprise Award finialists (PDF).

Sales-force competence
"On a pain scale of one to 10, it was seamless and flawless. It was pretty much a nine," says Steven Fishman, CIO at Corona, CA-based Ultrasonic Precisions Inc., when describing his experiences working with NetApp's sales team.

We ask users to tell us about their experience with their vendor's sales force because it's a predictor of the likelihood that a system will be appropriately matched with a need. Vendors who keep their customers' interests foremost and have knowledgeable teams are less likely to oversell a system. Indeed, there has been a high correlation between the results of this category and the award winners. "We felt that they were working in our best interests," says Fishman, "knowing full well, of course, that we would come back for more if and when we needed it."

The sales-force competence scores provided the narrowest range of any survey section from top to bottom. NetApp led the enterprise and midrange categories but, in both cases, the scores were NetApp's lowest section scores: a 6.50 among enterprise products and a 6.48 in the midrange class. Slightly more than half of NetApp's customers purchased through a value-added reseller, while the rest bought directly from the manufacturer.

This was the one section in which IBM didn't seriously challenge NetApp. Although IBM's sales-force competence scores weren't bad, they were IBM's worst of the survey. In the enterprise group, IBM placed fourth with a 5.93, its only sub-6.0 score in any section. Among midrange products, IBM did better with a 6.16, but still ranked last.

Sometimes familiarity breeds good sales partners. "We spoke with NetApp initially and then ended up going with IBM," says Steve Baucum, manager of Unix support at Methodist Healthcare in Memphis. They already had IBM server and storage gear installed, so it was a key consideration. "Everything else that was touching the SAN was IBM, so we decided to stick with one vendor," he notes.

Having a good ongoing relationship with a vendor can help swing a deal, but the incumbent vendor also has to deliver on price. "I decided on HP--to be honest--because they had a decent price and I was very sure of their service," says Graham Husk, network services administrator at Canada Science and Technology Museum (CSTM) Corp. in Ottawa. But it wasn't just a dollar-and-cents matter. "I've never had any problems dealing with their sales staff as far as their technical capabilities," adds Husk.

NetApp received its best section score for the statement "My sales support team is knowledgeable" (6.88). Its lowest score was for "My sales rep keeps my interests foremost" (6.12). HP, which placed second for sales-force competence in both categories, received its highest mark for "My sales rep is knowledgeable" (a 6.58 in the enterprise category).

Product features
The product features section gauges how well a product meets users' needs and expectations. Features are broadly grouped based on functionality, including management features, scalability and data replication in particular. NetApp's enterprise and midrange scores were again nearly identical (6.82 and 6.83, respectively). Perhaps as a testimony to how consistent NetApp's product line is from top to bottom, the way the product lines arrived at their respective scores was also very similar. Both products received their top three scores for statements related to snapshot, mirroring and remote replication features. In all cases, the scores exceeded 7.0. To have such high scores over a large group of respondents is a strong statement. More than 70% of those surveyed gave the products a rating of 7.0 or 8.0 for each statement. The lowest ratings for both product lines were for "This product's management features meet my needs."

"The N Series was at first sold to us because we needed the performance," says Baucum of his firm's clustered IBM System Storage N5200 arrays, which IBM OEMs from NetApp. "But what we've gotten out of it besides the performance is the replication, snapmirror, snapshotting [and] the integration with Volume Shadow Copy Services."

With most organizations experiencing steady disk capacity growth, scalability is a key issue. Ultrasonic Precisions' Fishman is confident his firm's two NetApp FAS6000 arrays will accommodate anything he throws at them. "The maximum capacity is 420TB; that's plenty more than we'll ever need, at least in the next 10 years," he says.

Being able to grow capacity is important, but the ease with which disks can be added convinced Husk to go with a pair of HP ProLiant DL380s. "HP Systems Insight Manager allows you to actually format RAIDs on the fly after just plugging in the hot-plug drives," says Husk. "There's no rebooting required."

The rating for the other finalists fell in line with the overall rankings, except for Dell. Dell polled a steady third in all sections except this one, where it placed fifth among midrange products with a 5.93. But that's clearly not a bad score, and it was Dell's only sub-6.0 rating. IBM's enterprise users apparently like the feature set on balance most of all, giving the company a 6.89 for "Overall, this product's features meet my needs." IBM's midrange users ranked the product line's scalability as its best feature. Sun received its lowest section score (a 5.15) for product features.

Initial product quality
When asking respondents about a product's initial product quality, the survey focuses on ease of use, installation simplicity and similar issues. Based on the criteria of this survey, NetApp's midrange products topped all products in either category. Its 6.95 was the highest section score of any finalist in the survey. Its 6.77 score in the enterprise category was the highest of that group and outpaced IBM's 6.55.

NetApp's midrange products received a 7.23 for "This product is easy to get up and running." This score was the highest for any statement in the survey. Eighty-three percent of respondents gave it a rating of 7.0 or 8.0. It received a 7.13 for "This product was installed without any defects," and a 7.08 for "This product requires very little user intervention."

Getting his NetApp FAS6000 arrays up and running couldn't have been easier for Ultrasonic Precisions' Fishman. "It was a three-week process, and I thought it was pretty much on target," he says, adding, "it actually took two days less than I thought it would, and that includes training." Methodist Healthcare's Baucum had a similar experience with his IBM N5200 clusters, but took a cautious approach to deployment. "We spent about two months doing some proofs of concepts on them," he says.

In this section, users are asked how well a product delivers value. Perhaps not surprisingly, NetApp's midrange products polled highest in this regard with a 6.65 for the statement "This product delivers good value for the money." NetApp's enterprise group wasn't far behind with a 6.61. IBM garnered a 6.47 for its midrange products and a 6.00 for enterprise, while HP products received a 6.24 (midrange) and a 6.18 (enterprise). Dell's midrange product received a 6.14.

Product reliability
This survey section assesses how well a product performs over time. In particular, we ask users about downtime, the need for unplanned patches and how disruptive unplanned patches are to the operation.

IBM spoiled NetApp's quest to sweep all categories and sections. IBM's midrange product reliability score of 6.85 topped NetApp's 6.79, although NetApp won the enterprise category with a 6.70 (the highest of that group). IBM's customers awarded the company's midrange products a 7.11 for the statement "This product experiences very little downtime." IBM also received a good score for "This product requires very few unplanned patches" (6.88).

When asked about any unplanned downtime with his IBM N Series boxes, Baucum replies "So far, no." But, "I don't want to say that too loud" he adds, not wanting to jinx his good luck.

While IBM had the highest score in this section, NetApp bested IBM on two items, a score of 7.10 for "This product experiences very little downtime" and "This product requires very few unplanned patches." True to form, these questions proved a predictor of the overall winner. NetApp's undoing, however, was the statement "Patches can be applied nondisruptively." The associated 6.11 score may be decent in its own right, but it was enough to drag NetApp's section average down to second place.

"I don't think any other product compares to this one," notes Ultrasonic Precisions' Fishman about the reliability of his NetApp FAS6000s.

Technical support
It's not unusual for our survey to reveal mediocre scores for technical support. Relative to the rest of the survey, this was once again true for NAS. This was the only section in which four vendor product lines had sub-6.0 scores, although EMC did so by only a whisker with a 5.99 in the midrange category.

NetApp was the only vendor with above-7.0 scores for any statement in this section. For the statement "This vendor provides support as contractually specified," NetApp's enterprise products scored 7.00, while its midrange products received a 7.06. These scores were substantially higher than any other vendor's on that issue.

"We called them when we migrated up from Windows 2003 to Windows 2008, and they came down and consulted with us for about an hour," says Fishman of his NetApp support. "They're very good."

IBM placed second in both the enterprise and midrange categories in this section, but a 5.44 for "Problems are resolved in a timely manner" and a 5.56 for "The vendor's support personnel are knowledgeable" dragged down its average.

Methodist Healthcare's Baucum could have gone with NetApp for his NAS deployment, but decided to stick with IBM even though it was still NetApp gear. "It kind of reduces the friction if you've got something that's not compatible," he says. With a lot of IBM equipment already installed, he preferred to deal with one vendor. "You don't have to have a guessing game of visiting multiple vendors to get to the right versions of code," he explains.

Click here for the Midrange Award finialists (PDF).

Buy again?
As indicated earlier, overall user satisfaction was high, with 82% of respondents willing to make the same purchase decision again. In this important measure, IBM again topped NetApp, and this time in both categories. Its enterprise customers indicated a willingness to repurchase 90.9% of the time, and an even higher 95.5% of the time with midrange products. Previous surveys have often found high loyalty among IBM users.

Nevertheless, NetApp did have solid support among its users. In its case, 86.8% of enterprise users would make the same decision, while 82.4% of midrange users would do so. HP had solid numbers above 80%, while EMC had 78% (midrange) and 76.9% (enterprise). Sun trailed the enterprise group with a 64.3% user loyalty. Regardless, it's safe to say that while NAS users have the occasional bone to pick, they're apparently a happy group of people.

About the survey: The Diogenes Labs-Storage magazine Quality Awards are designed to identify and recognize products that have proven their quality and reliability in actual use. The results are derived from a survey of qualified Storage readers who assessed products in five main categories: sales-force competence, product features, initial product quality, product reliability and technical support. Products are rated on a 1.0-8.0 scale, where 8.0 is the most favorable score.

While our survey includes the spectrum of NAS vendors, a vendor must have at least 13 responses to be statistically valid and therefore considered as a finalist. This year, 274 respondents rated 492 systems. That's fewer respondents than 2007, but in line with the 2006 response rate. The lower response rate probably impacted smaller vendors the most, as we didn't have any of the smaller vendors crack the finalist list.

Just under half of the survey respondents (49.3%) had only one vendor, while the rest had two or more. The largest group of respondents was for the NetApp FAS2000/3000/6000 series of systems with 114. EMC Corp. Celerra had the second highest number of respondents with 67. Although the largest response groups were for enterprise systems, the survey was tilted toward the small- to medium-sized enterprise (SME). SMEs, or those organizations with less than $1 billion in revenue, comprise 62% of the survey respondents. The largest industry segment was financial with 6.3% of respondents, followed by IT services with 5.9%. Given that 14 different industries were represented, the survey population provided information from a broad spectrum of the marketplace.

PRODUCTS: The following products were included in the third Diogenes Labs-Storage magazine Quality Award III for NAS survey.

Enterprise: BlueArc Corp. Titan 2000 series*; EMC Corp. Celerra; Exanet Inc. ExaStore EX200FC/400FC/600FC*; Hewlett-Packard (HP) Co. StorageWorks Enterprise File Services (EFS); Hitachi Data Systems Corp. Lightning/NAS Blade for TagmaStore USP and Network Controller, AMS1000/500/200 and WMS100 with NAS option*; IBM Corp. System Storage N7000; NetApp FAS2000/3000/6000; Panasas Inc. ActiveStor Storage Cluster*; Sun Microsystems Inc. StorageTek 5310/5320
Midrange: Adaptec Inc. Snap Server 520/650*, Agami Systems AIS3000 series/AIS6000 series*, BlueArc Titan 1100*, Dell Inc. PowerVault NF500/600 and NX1950, EMC NS series, Exanet ExaStore EX200S/400S/600S*, NetApp FAS200/900, HP ProLiant DL storage servers, IBM System Storage N3000/5000, Isilon Systems Inc. Isilon IQ*, ONStor Inc. Pantera 2200/5000*, Sun StorageTek 5210/5220*

* Didn't receive enough responses to be included in the final results

Article 3 of 18

Dig Deeper on Storage market research reports

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Storage

Access to all of our back issues View All