We also examined head-to-head rankings when respondents owned systems from more than one vendor. Here, NetApp midrange products came out the highest, winning 20 of 34 head-to-head contests. EMC's Celerra NS series also did very well in this regard, scoring higher than other vendor's products in 17 out of 29 cases. NetApp's enterprise products ranked higher in 24 out of 47 cases. The biggest surprise was that HDS won only six of its 14 head-to-head matchups.
Clearly, head-to-head comparisons didn't track to the finish order of this survey. The reason, we believe, is that the comparisons involve all of the survey's competitors, including those with too few responses to be considered for the award. For example, products from BlueArc Corp. and Isilon Systems Inc. were rated very highly, but because of an insufficient number of responses neither was a finalist. However, statistical relevance isn't important for the head-to-head comparisons because it's a comparison of two systems by a single respondent who uses both products and therefore doesn't rely on statistical calculations.
Large field of products
The voting process for this survey differed from our prior surveys because the responses were clustered around a few major vendors. Of the 21 product lines from 15 vendors included in our survey, only eight products from five vendors generated enough responses to be statistically relevant and considered for the award (see "Products included in the survey"). Nevertheless, these vendors represent the lion's share of the market and, as such, represent a valid sample of the population. Overall, 387 respondents provided 523 system evaluations. One surprise was the number of responses for "Other": 137 respondents indicated this choice for vendor/system despite what we considered an exhaustive list of NAS products. It's likely that the "Other" responses represent a Windows-based NAS system from smaller vendors, combined SAN/NAS products, older models from the listed vendors or non-NAS systems. In any case, the results of these responses didn't figure into the award calculation. The margin of error for the survey was +/- 6% with a 95% confidence factor.
In selecting NAS systems for our survey, we attempted to be as inclusive as possible; the list of 21 products was the longest yet for any Quality Awards survey. The main criterion for inclusion was that the device support CIFS and NFS file systems simultaneously. We included NAS-only systems, as well as SAN-NAS gateways. To differentiate midrange systems from enterprise systems, we placed all Windows-based systems in the midrange category, as well as those scaling to less than 50TB. Systems that can scale to more than 50TB were categorized as enterprise NAS. However, we did also consider vendors' product positioning, as this also affects how users regard and evaluate products.
As with our previous awards, we asked respondents to rate their product experiences in five categories: sales competence, product features, initial quality, product reliability and technical support. We also asked if the system was purchased directly from the vendor or from a value-added reseller. Because all of the finalists in both categories are the manufacturers of their respective systems, original equipment manufacturer issues weren't considered.
Within each of the five categories, respondents were asked to respond to a number of positive statements on a scale of 1.0 to 8.0 (see "About the survey"), where 1.0 indicated strong disagreement and 8.0 indicated strong agreement. A rating of 6.0 was the approximate median for both the enterprise and midrange groups in this survey. The preponderance of ratings fell between 5.5 and 6.5. Rating elements above 6.5 can be regarded as exceptional strengths, while those below 5.5 can be regarded as weaknesses.
Products included in this survey
The following products were included in the Diogenes Labs-Storage magazine Quality Award NAS survey.
Dell Inc. PowerVault 7xxN storage server
EMC Corp. Celerra NS Series
Exanet Inc. ExaStore EX200S/EX400S/EX600S*
Fujitsu Limited Eternus NR1000 F200/F300*
Hewlett-Packard (HP) Co. ProLiant DL server
IBM Corp. System Storage N5000 series*
Isilon Systems Inc. Isilon IQ*
Network Appliance (NetApp) Inc. FAS200
Silicon Graphics Inc. (SGI) InfiniteStorage NAS 330*
Sun Microsystems Inc. StorEdge 5210 NAS appliance*
Winchester Systems Inc. FlashNAS FN-1130 and FN-2130*
BlueArc Corp. SiliconServer*
Fujitsu Eternus NR1000 F500*
HP StorageWorks Enterprise File Service (EFS)
Hitachi Data Systems Corp. Lightning Blade/TagmaStore Blade
Panasas Inc. ActiveScale Storage Cluster*
SGI InfiniteStorage NAS 2000/3000*
Sun StorEdge 5310 NAS appliance*
* Not included in the survey results because of an inadequate number of responses.
About the survey: The Diogenes Labs–Storage magazine Quality Award for NAS is the fifth in our series of survey-based service and reliability awards. The Quality Awards are designed to identify and recognize products that have proven their quality and reliability in actual use. The results are derived from a survey of qualified Storage readers that assesses products in five main categories: sales competence, product features, initial quality, product reliability and technical support. Our methodology incorporates statistically valid polling that eliminates market share as a factor. Indeed, our objective is to identify the most reliable products on the market regardless of vendor name, reputation or size. Products were rated on a scale from 1.0 to 8.0, where 8.0 is the most favorable possible score.
Rich Castagna is Editor-in-Chief of Storage; Phil Goodwin is president of Diogenes Analytical Laboratories Inc.
Previous Quality Award winnersHDS sweeps Quality Awards on arrays
Which backup software is best?
What are the best midrange arrays?
Top tape libraries revealed