Over the next year storage administrators will increasingly encounter the SPC-1 storage benchmark in ads, white papers and presentations. Like the TPC benchmark in transaction processing, it's important to understand what the SPC-1 is, and isn't, before relying on it in acquiring SAN or direct-attached storage equipment.
The benchmark was developed by the Storage Performance Council as the first step in an effort to provide a comprehensive series of benchmarks for storage performance. The benchmark and the process surrounding its use contain a number of steps to insure that it is fairly administered, including a requirement for validating the results by independent auditors and a 60-day comment period before an SPC benchmark ban be considered official. There is also a prohibition on testing 'benchmark specials' -- configurations and systems optimized for the test and not showing in comparable gain in real-world applications. The aim is to produce true 'apples to apples' comparisons of SAN hardware.
SPC-1 is built around two metrics: One measuring I/O of the storage system and the other the absolute time needed to complete a series of tasks. These are addressed by the I/O Per Second (IOPS) and Least Response Time (LRT) metrics respectively. To use the benchmark effectively, you need to understand which kind of performance is most important to your applications.
While SPC-1 does consider data persistence (preserving data without corruption or loss) and sustainability (the ability to maintain results over a long period of time), it does not emphasize reliability. The SPC says this will be added to future versions of the benchmark.
Rick Cook has been writing about mass storage since the days when the term meant an 80K floppy disk. The computers he learned on used ferrite cores and magnetic drums. For the last twenty years he has been a freelance writer specializing in storage and other computer issues.