How useful are storage benchmarks?


This article can also be found in the Premium Editorial Download "Storage magazine: Continuous data protection (CDP) and the future of backup."

Download it now to read this article plus other related content.

SPEC SFS benchmarks report two key metrics: throughput in ops/sec and response time in milliseconds. Each report includes a graph that illustrates the response time of the tested system in relation to throughput. The most relevant numbers are maximum throughput and average response time. Both are listed at the beginning of the published reports and they're typically the numbers cited and referred to by vendors and the press. Each test report also describes the tested configuration in great detail.

A strong point of SPEC SFS is its independence of NFS clients during the test and in measuring the real performance of the server. On the downside, SPEC SFS is currently limited to NFS, a shortcoming that will be addressed in a future version of SPEC SFS. "It is very likely that the next version of SPEC SFS will support CIFS," says Don Capps, chair of the SPEC SFS subcommittee.

Benchmarking tips

  • Compare similar storage systems.

  • Understand what needs to be benchmarked (IOPS, throughput, CPU utilization), as it determines the benchmarking tool to use.

  • Eliminate all factors that could impact what you benchmark. The benchmark result shouldn't be affected by some other bottleneck.

  • Run

Requires Free Membership to View

  • tests many times to ensure results are repeatable; this will also eliminate variances.

  • Monitor the environment for faults.

  • Take things like cache into consideration. Choose your workload to benchmark the performance of the storage system rather than memory performance.

  • Make sure the system has finished ramping up to ensure you measure system performance during normal operation rather than ramp-up mode.

This was first published in October 2007

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: