How useful are storage benchmarks?

Ezine

This article can also be found in the Premium Editorial Download "Storage magazine: Continuous data protection (CDP) and the future of backup."

Download it now to read this article plus other related content.

SPEC SFS benchmarks report two key metrics: throughput in ops/sec and response time in milliseconds. Each report includes a graph that illustrates the response time of the tested system in relation to throughput. The most relevant numbers are maximum throughput and average response time. Both are listed at the beginning of the published reports and they're typically the numbers cited and referred to by vendors and the press. Each test report also describes the tested configuration in great detail.

A strong point of SPEC SFS is its independence of NFS clients during the test and in measuring the real performance of the server. On the downside, SPEC SFS is currently limited to NFS, a shortcoming that will be addressed in a future version of SPEC SFS. "It is very likely that the next version of SPEC SFS will support CIFS," says Don Capps, chair of the SPEC SFS subcommittee.

Benchmarking tips

  • Compare similar storage systems.


  • Understand what needs to be benchmarked (IOPS, throughput, CPU utilization), as it determines the benchmarking tool to use.


  • Eliminate all factors that could impact what you benchmark. The benchmark result shouldn't be affected by some other bottleneck.


  • Run tests many times to ensure results are repeatable; this will also eliminate variances.

    Requires Free Membership to View

    Login



  • Monitor the environment for faults.


  • Take things like cache into consideration. Choose your workload to benchmark the performance of the storage system rather than memory performance.


  • Make sure the system has finished ramping up to ensure you measure system performance during normal operation rather than ramp-up mode.

This was first published in October 2007

Join the conversationComment

Share
Comments

    Results

    Contribute to the conversation

    All fields are required. Comments will appear at the bottom of the article.
    Back to top