Also, look for additional test suites from the SPC to address other storage performance characteristics for a wider range of workloads. "Some of the ones that need to be rolled out are focused around testing and benchmarking NAS and file systems," says Greg Schulz, founder and senior analyst at Storage IO. Schulz suggests that future improvements in instrumentation will likely lead to more consistency between performance testing tools and better integration between tools and management consoles. "Some of that could be addressed by SMI-S," he says.
Schulz also notes that costly storage performance testing may not be an answer for every circumstance -- sometimes it's simply more expensive to test than to buy additional hardware. "Within different organizations, there's debate about how much testing should be done," he says. "How much emphasis should you put on performance testing, performance planning and performance analysis?" Groups like the Computer Measurement Group are taking a central role in determining just how much testing is necessary. Analyst Arun Taneja shares his opinions on NAS benchmarking here.
Still, other analysts believe that storage performance testing will increasingly be handled as an outsourced service rather than an in-house project. "I see it [testing] becoming more of a service-based thing, because in this fiercely competitive environment it is very difficult to keep up with all of the product updates that vendors are putting out there," says Ashish Nadkarni, senior consultant at GlassHouse Technologies Inc. While he notes that vendors are typically more honest about their performance numbers than in years past, competition makes it difficult for vendors to disclose all information. This will put the onus on third-party testing providers to report complete and unbiased results.
Go to the next part of this article: Introduction
Or skip to the section of interest: