How useful are storage benchmarks?


This article can also be found in the Premium Editorial Download "Storage magazine: Continuous data protection (CDP) and the future of backup."

Download it now to read this article plus other related content.

SPC roadmap
SPC-1 and SPC-2 are tailored toward objective and verifiable performance measurement of large and complex storage configurations. Their relatively high benchmarking cost, combined with configuration requirements, makes them ill-suited for testing storage components and smaller storage subsystems.

For those reasons, approximately three years ago SPC began modifying SPC-1 and SPC-2 to create derivatives, namely SPC-1C ("C" stands for components) and SPC-2C, which are a better fit for storage component benchmarking. They use the same workloads as SPC-1 and SPC-2, and their specifications and reporting requirements will be similar to those of their large storage benchmark equivalents. They'll be used to benchmark disk drives, host bus adapters/controllers, single-enclosure storage subsystems and storage software such as logical volume managers.

To add credibility and authenticity, all SPC-1C and SPC-2C benchmarks will be performed by an SPC-certified testing lab to which storage vendors will have to submit components for testing. SPC is also working on a file-system benchmark named SPC-3, which is similar to SPEC SFS. Unlike the current version of SPEC SFS, SPC-3 will be file-system agnostic. SPC-1C, SPC-2C and SPC-3 are all works in process with no release dates announced at this time.

SPEC SFS is an industry-standard benchmark used to measure NFS file-system performance of network file servers and NAS. All tests

Requires Free Membership to View

are performed by storage vendors and test results are submitted to the SPEC for peer review. After passing a SPEC audit, test results are published on the SPEC Web site.

Unlike SPC benchmarks, SPEC doesn't include cost/performance metrics and, as a result, benchmark results need to be viewed with caution. "When comparing SPEC SFS results, you have to be careful to compare like-sized and like-priced systems," cautions ESG's Garrett. "SPEC.org lists results that range from 10,000 SPEC ops/sec to 300,000 SPEC ops/sec without information about the system price." Therefore, it's crucial to look at the benchmark results in the context of the system configuration. File-system performance is highly dependent on the number of disks, disk controllers, memory (cache) and network controllers; as you add more of these, performance will rise, as will the system's price tag.

This was first published in October 2007

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: