In its first year in our survey, EqualLogic PS Series proved its mettle by unseating Sun FlexLine as the highest...
rated midrange storage array.
|Products included in the survey|
The following vendors and products were included in the 2006 Diogenes Labs-Storage magazine Quality Award survey for midrange arrays:
Apple Computer Inc.
BlueArc Corp. Titan 2000 Series*
Compellent Technologies Storage Center*
Dell Inc. CX Series
EMC Corp. Clariion CX series
EqualLogic Inc. PS Series
Hewlett-Packard (HP) Co. StorageWorks Enterprise Virtual Array (EVA) series
Hitachi Data Systems Corp. Thunder 9500 V Series
IBM Corp. FAStT, DS4000 or DS6000
LeftHand Networks Inc. SAN/iQ*
Network Appliance Inc. FAS200 or FAS900 Series
Pillar Data Systems Axiom 500*
Sun Microsystems Inc./StorageTek D-series, FlexLine FLA200 or FLA300
Sun StorEdge 6920
Xiotech Corp. Magnitude 3D*
*Didn't receive a sufficient number of responses to be included in the final survey results.
Midrange storage arrays just might be the storage hardware category offering consumers the greatest breadth of choice. This year's Diogenes Labs–Storage magazine Quality Award for midrange arrays asked Storage readers to relate their experiences using 15 different midrange product lines, ranging from Apple Computer Inc. to Xiotech Corp. Our readers provided a total of 474 evaluations of these systems, with nine products garnering enough responses to be included as award finalists (see "Products included in the survey," at right). Readers rated these systems in five different categories--sales-force competence, product features, initial product quality, product reliability and technical support--using a 1.0-8.0 scale, where 8.0 is the most favorable score.
Our survey is designed to allow smaller vendors to compete on an equal footing with larger, more established vendors. We're always curious to find out if any emerging vendor is developing a "cult" following that might indicate a "best-in-class" opportunity for IT buyers. If the results of our 2006 survey are any indication, storage groups may want to take a look at EqualLogic Inc.'s PS Series arrays, which topped a wide field of products and eclipsed last year's winner Sun Microsystems Inc./StorageTek FlexLine. EqualLogic's win is especially noteworthy because it wasn't in last year's survey.
In the 2005 Quality Awards midrange arrays survey, we were struck by the high scores products received, indicating across-the-board satisfaction. In that survey, overall product scores ranged from 6.77 down to 5.75. This year's scores fell back to earth a bit, but still demonstrate general satisfaction among all of the midrange arrays. EqualLogic's overall score of 6.00 was only .05 better that of Sun/Storage- Tek FlexLine (see "Overall rankings"). EqualLogic led the pack in four out of our five categories.
Sun/StorageTek FlexLine was a surprise winner last year, but its strong second-place showing this year proved that last year's results weren't a fluke. We separated the FlexLine and 6000 series in our survey, and the results were similar to those of last year, with the 6000 series dropping from fifth to sixth largely due to the addition of EqualLogic. The biggest positional changes occurred with Dell's CX Series (rebranded Clariion) and Network Appliance's (NetApp) Inc. FAS series. Dell fell from third place last year to ninth place this year, while NetApp dropped from second to fifth. Both vendors saw their scores decline by more than a full point. Hewlett-Packard (HP) Co.'s StorageWorks Enterprise Virtual Array (EVA) and Hitachi Data Systems (HDS) Corp.'s Thunder 9500 V improved to third and fourth, respectively. HDS' fourth-place finish was three positions higher than in 2005. While EMC Corp.'s Clariion and Dell's Clariion didn't track together in 2005, they did in 2006.
The largest number of responses was for EMC Corp.'s Clariion, with 118 evaluations; HP EVA was second with 67. The most represented industry was financial services, followed by healthcare and government/nonprofit. The distribution of respondents by company size was weighted slightly toward large organizations. The breakdown here was 28.3% from small organizations (less than $100 million in revenue), 29.4% from midsized organizations ($100 million to $1 billion) and 42.3% from large organizations (more than $1 billion). The majority of respondents are in operational roles, with only 6% from senior management. Storage administrators comprise 24.9% of the respondents, and 16.8% are storage managers.
We've found a direct correlation between a vendor's sales-force competence and its overall results. Winners of prior Quality Awards have always finished in the top two in this category; in this survey, Sun's FlexLine topped all others with a 5.88 and EqualLogic was second with a 5.57, which was its lowest score in any category (see "Sales-force competence"). Comparing Sun's FlexLine results to the results for its 6000 series is interesting because the 6000 finished sixth with a score of 5.17. In the case of FlexLine, 50% of systems were purchased through resellers, whereas 90% of the 6000s were purchased directly from Sun. Eighty percent of EqualLogic's systems were purchased through a value-added reseller (VAR).
Sun FlexLine scored a 6.07 for "My sales support personnel are knowledgeable." It also scored very well for "My sales rep is flexible" (5.97). EqualLogic had its highest ratings for "My sales rep keeps my interests foremost" and "My sales support personnel are knowledgeable." EqualLogic's weakest area was for the statement "My sales rep understands my business" (5.33).
Kern Weissman, director of network systems at Velocity Express in New York, undertook an extensive evaluation project before purchasing a midrange array. "We eventually narrowed it down to EMC, Hitachi and StorageTek," says Weissman. "We fully realized that all three would have worked."
Eventually, Velocity eliminated EMC because of less-than-satisfactory performance and Hitachi because of its complex software, and settled on a Sun FlexLine array. "We didn't have any strong bonds with any particular vendor of storage, so they weren't really able to influence our decisions," says Weissman.
Similarly, Trevor Rickards, server and storage administrator at Calgary, AB-based Compton Petroleum Corp., did a careful analysis of alternatives when his company was looking to replace its IBM DS4300. "Price was part of the initial discussions," says Rickards, "but as we got further in, realizing the differences and how they each price, we very quickly discarded that because it looked like it was going to be a moot point for the amount of storage we were looking to buy." Although the firm's experience with the IBM array was positive, it opted for an EMC Clariion.
Hard-nosed bargaining is sometimes called for. "If you have to pay list price for an [HP] EVA, I think you're paying way too much," says David Salbego, Unix and operations manager at Argonne National Laboratory in Illinois. "They're very willing to negotiate, but does that mean I know if I'm leaving money on the table or not?"
Rami Elyas, a member of the enterprise data storage and recovery services group at Lilydale Inc. in Edmonton, AB, says they put their trust in their VAR. "We worked closely with the VAR and IBM," says Elyas. "We used the reseller because they know our business pretty well and it was a nonbiased view--they didn't have much to gain whether we bought IBM or Hitachi or EMC."
Software functionality differentiates storage arrays more than the underlying hardware. Thus, our survey places an emphasis on software features, including snapshots, mirroring, remote replication and management tools. Respondents are asked to evaluate a product based on how well it meets their needs in these general areas, rather than by simply comparing product feature lists.
EqualLogic scored well in this category with a 6.26 rating--its second highest category score (see "Product features"). PS Series users particularly like its snapshot functionality, giving it a solid 6.47 for the statement "This product's snapshot features meet my needs." It also received a 6.40 for "This product scales to meet my needs." In fact, EqualLogic scored 6.0 or better for all statements in this section. This category generated the highest scores across the group, with even the low score--Dell's 5.15--still quite respectable.
Compton Petroleum's choice of an EMC Clariion was based largely on features. "The EMC had more flexibility for replication and allowed us to do better DR planning," notes the firm's Rickards. Daily operations were also a key consideration. "Their management interface seems quite intuitive," he says. "Their grouping of hardware monitoring is excellent--in a lot of ways much more detailed that what the IBM provided."
Sun's FlexLine received a 6.11 rating in the product feature category. FlexLine users responded to the statement "This product's mirroring features meet my needs" with a 6.63. This was the highest statement score for any product; even though it was higher than any score posted by EqualLogic in this category, FlexLine's overall category score was offset by four sub-6.0 ratings, the lowest of which was a 5.77 for "This product's management features meet my needs." These scores are still very solid, but substantially lower than FlexLine's 7.03 average last year.
"It's been extremely easy to expand," says Velocity Express' Weissman of his firm's Sun FlexLine. "We went from 2TB to 4TB." He also likes how he can reconfigure without disrupting production. "One of the best things about it is how you can dynamically change your settings without taking anything down," he adds.
Initial product quality
The initial product quality section brought out both the highest and lowest category scores of any section. Scores ranged from EqualLogic's 6.30 down to Dell's 4.67 (see "Initial product quality"). EqualLogic had very consistent scores across the category, ranging from a low of 6.13 for "This product was installed without defects" to a high of 6.40 for "This product offers good value for the money." Scores for Sun's FlexLine were similarly consistent across the range of statements. As with EqualLogic's PS Series, Sun FlexLine recorded its highest score for the statement "This product offers good value for the money" (6.30).
Dell prides itself as a low price "value" leader in the Windows server and PC markets, but this perception apparently hasn't translated into the storage array market. In our survey, Dell scored a 4.64 on our "value" statement, which was lower than EMC's rating of 4.77 for the same statement with identical hardware. The value perception might be influenced by Dell's other scores in this category, all of which were below 5.0 except for the statement "This product installed without defects" (5.00). In the survey, the overall perception of value very closely matched the overall order of finish.
"It's astounding to me, but I have not had one single disk failure yet," says Weismann, referring to Velocity Express' Sun FlexLine array after nearly two years of use. That level of reliability might be a stretch for most storage managers, but it underscores the faith we place in the reliable operation of installed hardware.
In contrast to the initial product quality category, which had the widest scoring spread, product reliability had the narrowest. In fact, less than one point separated EqualLogic's top finish (5.92) from EMC's 4.98 (see "Product reliability"). All other vendors finished above 5.00. EqualLogic recorded its highest ratings (6.20) in this category for two statements: "This product meets my service-level requirements" and "This product requires very few unplanned patches."
In all of our prior surveys, products that did well on the statement "Patches can be applied nondisruptively" also did well overall. This proved true once again. EqualLogic's lowest score in this section was for this statement (5.20), which might indicate a low overall score. However, the good news for EqualLogic was that the other vendors were rated even lower in this area (below 5.0) with the exception of Sun FlexLine, which had a 5.40. HP's EVA and EMC's Clariion recorded scores of 4.46 and 4.44, respectively.
Patches and firmware upgrades continue to be a thorn in the side of midrange array administrators. "For the money you're paying--you're well into six figures, if not more--you shouldn't have to take these things down or quiet all the hosts on the SAN just so that you can do a firmware upgrade," says Salbego at Argonne National Laboratories, which has two HP EVAs installed. "That's an industry-wide issue, not just HP EVA."
In most of our surveys, we've been surprised to find that product support hasn't been a big differentiator. However, this year's midrange survey provided as broad a spectrum of scores as the other categories. EqualLogic was again the winner, posting a 5.96 (see "Technical support"). Sun FlexLine finished a close second with a 5.85. Sun's 6000 series had its highest ranking in this category, placing two spots behind its FlexLine sibling with a 5.31. This scenario was exactly the same as last year's for that company's products.
Because many midrange array systems are sold through resellers, we were interested in how those partners were viewed by customers. In many cases, the reseller provides initial (Level 1) technical support. When we posed the statement "My vendor's third-party partners are knowledgeable," EqualLogic recorded a score of 6.07. Moreover, the company's partners can apparently resolve most problems themselves, as users awarded the statement "Problems rarely require escalation" with a score of 6.13.
Although EqualLogic had the best scores for partner knowledge, it was hardly a runaway. Both HP and Sun FlexLine scored 6.00 on the knowledge of their partners. "[Sun FlexLine's] onsite people are just absolutely fantastic," says Velocity Express' Weissman. "They go over and above." Even IBM, which otherwise had lackluster results, recorded a solid 5.75 in this area. "Whenever I have called their support, they've been excellent," says Compton Petroleum's Rickards. "They've set up the exact scenario that I've got in their lab and have always been able to duplicate the problems."
Elyas at Lilydale Inc., which is also an IBM DS4300 user, says, "The support has been really good. We've always been able to get the parts here within a day."
In our survey, the differentiator was the need to escalate problems. For this statement, Sun fell to a 5.53, HP to 5.30 and IBM to 4.67.
Would you buy the same system again?
When we ask respondents to tell us if they'd make the same purchase decision again, we rarely find a correlation between a positive response and the order of finish. Buying decisions are often influenced by many factors beyond the technical attributes of a product. However, this survey yielded a reasonably close correlation. Ninety-three percent of EqualLogic's customers said they'd make the same purchase decision again (see "All things considered, would you buy this product again?" at right). Most of our other vendors had positive responses that ranged between approximately 75% and 82%. Sun's FlexLine and 6000 series both received an 80% positive response rate, despite very different placements in the overall standings. IBM trailed the field significantly, with just 62.5% of its midrange users indicating a willingness to repurchase. This represents a 17.5 point drop for IBM from 2005.
In cases where respondents have two or more midrange systems, we asked them to evaluate each one. In these cases, we compare the scores to determine which system is preferred in a head-to-head match-up. EqualLogic again came out on top, winning four out of seven such comparisons (57.1%). FlexLine was preferred in 52.4% of cases (11 out of 21), while EMC Clariion was preferred 41.3% of the time (26 out of 63). Dell Clariion was preferred just 22% of the time (six out of 27). These results don't relate to future buying intentions, but rather offer the perfect vision of 20/20 hindsight.
Kudos to EqualLogic
Clearly, users of the EqualLogic PS Series feel they've purchased something special. Time will tell if the company can continue to deliver in a top-flight manner ... but the top is always a good place to start.
About our survey
The Diogenes Labs-Storage magazine Quality Awards are designed to identify and recognize products that have proven their quality and reliability in actual use. The results are derived from a survey of qualified Storage readers who assessed products in five main categories: sales-force competence, product features, initial product quality, product reliability and technical support. Our methodology incorporates statistically valid polling that eliminates market share as a factor. Our objective is to identify the most reliable product on the market regardless of vendor name, reputation or size. Products are rated on a 1.0-8.0 scale, where 8.0 is the most favorable score.
For this year's midrange arrays survey, 354 respondents provided 474 valid system evaluations. This year's response rate is about 10% higher than last year's, but carries the same +/- 5% margin of error with a 95% confidence factor.