Sun Microsystems, Inc. announced this week that the Storage Performance Council (SPC) approved the benchmark results...
for its StorEdge 6920 midrange system, awarding it first place in the midrange sector for performance and price/performance
According to the results, the StorEdge 6920 outperformed IBM's TotalStorage FAStT900 by over 150% and delivered a 35% higher price-performance ratio. Comparative results against the Hewlett-Packard (HP) StorageWorks Enterprise Virtual Array 5000 2C12D reflected more than 140% higher performance and a 50% better price-performance ratio.
The results are definitely impressive, but users should be aware that Sun may not be comparing like-for-like systems. "The comparison is not really fair since Sun tests with twice the number of drives," said Brian Garrett, Technical Director, Enterprise Strategy Group Labs. "The cost factor is supposed to level the playing field, but it doesn't always work well when comparing configurations of different sizes.".
For example in the latest published results, Garrett points out that Sun tested with 280 drives and compared the result to IBM at 108 drives and HP at 168 drives. More drives mean more IOPS (28K, 18K, 24K respectively). Sun and IBM results with the same type of 36 GB 15K RPM drives yield almost exactly the same IOPS per drive (170), Garrett said. Price per IOP of $10.73 for Sun is better than IBM's $16.78, but the difference may not be as much if IBM tested with 280 drives or if Sun compared its results to IBM's non-mirrored results ($12.63).
In addition, the 6920 uses 20 controllers but is compared with single systems from HP and IBM. Randy Kerns, senior analyst with the Evaluator Group said that this is in fact a fair comparison because to the user, the Sun box appears as one system. "From a management perspective, the user sees a single image so the cost of ownership is the same," he said.
Garrett disagrees with this analysis. "Results from a single enterprise array are in the same table and are compared to results from a bunch of little arrays …That's like comparing a 757 to a fleet of biplanes," he said.
Confusion aside, it's easy to see why Sun would be eager to spin its results. According to early feedback from Storage magazine's user purchasing survey, spending on midrange systems is exploding. Users who took the survey said that nearly 40% of their storage purchases would be for midrange systems.
Sun's midrange benchmark results
There are a couple of other factors to consider when checking out the SPC benchmarks. First, EMC doesn't submit products to the Council, which means that any comparison of systems excludes a major vendor likely to be on most users' shortlists. And second, the SPC benchmarks are not widely used by the user community.
- The SPC-1 test results showed the 20-tray Sun StorEdge 6920 system performed at 48,646.62 SPC-1 IOPS, SPC-1 price-performance of $10.73/SPC-1 IOPS and total ASU capacity of 3,022 GB (mirrored data protection level). The SPC Audit Certification Identifier is A00033.
- Results for IBM's TotalStorage FAStT900 reflected 18,447.55 SPC-1 IOPS, SPC-1 price-performance of $16.78 and total ASU capacity of 1,196.09GB (mirrored data protection level). The SPC Audit Certification Identifier is A00019.
- The HP StorageWorks Enterprise Virtual Array 5000 2C12D results clocked in at 20,096.97 SPC-1 IOPS, SPC-1 price-performance of $23.88 and total ASU capacity of 2,596.30 GB (mirrored data protection level). The SPC Audit Certification Identifier is A00007.
Full configuration details, as well as complete results of the benchmark study, can be found at Storageperformance.org