Tip

How to plan for I/O-intensive environments

What you will learn from this tip: How to identify, plan for and manage high-volume storage networks.


I/O-intensive environments with large, sustained workloads of various I/O sizes have their own special management challenges. Your best bet as a manager of these environments is to anticipate high I/O workloads, and plan for them ahead of time.

While high I/O environments are commonly thought of as a large

    Requires Free Membership to View

SAN issue, it's important to realize not every large storage environment is I/O-intensive. The first step in managing a high I/O-intensive environment is to pinpoint where one exists.

In fact, network capacity and number of devices is not an indicator of performance and workload activity. Neither is the presence of a large amount of disk capacity. Small and medium-sized storage environments can be I/O-intensive, if there is a large amount of active workload and I/O processing. An environment with a relatively small number of servers and storage -- for example, an OLTP, database server or vertical application for CRM or ERP -- can be I/O intensive.

Some applications and environments that tend to be I/O-intensive:

  • Video editing and processing including post production.
  • Video on demand and video services.
  • Paging, log, journal and index files.
  • Medical sciences including healthcare and imaging.
  • Geoscience and energy exploration.
  • Aerospace, telemetry and data acquisition.
  • Still and moving video surveillance.
  • Online Transaction Processing (OLTP)
  • Weather forecasting and simulations.
  • Backup and recovery along with data migration.
Some of the previous bullet items could be storage-intensive as well as being I/O-intensive depending on the I/O workload activity. I/O-intensive environments can have large, medium or small amounts of storage, yet have active workloads with either a large number of I/Os (transactions) of various size. For larger I/O sizes, more data will be moved and more bandwidth used, while less bandwidth will be used when processing smaller I/Os. Note that an I/O intensive environment can be optimized to support many transactions, support a large amount of bandwidth, or both.

Some common I/O characteristics:

  • Small I/Os are generally 1-16 blocks (512 bytes-8 KB) in size.
  • Medium-sized I/Os are generally 8-64 blocks (8-32 KB).
  • Large I/O sizes are generally 64 blocks (32 KB) or larger.
  • Transaction-oriented environments have many I/Os per second.
  • Bandwidth environments have many megabytes per second.
  • Latency and response time requirements are tied to SLAs and SLOs.
  • Random or sequential data access occurs with reads, writes, or a mix of both.
  • Workload activity profile, i.e. steady or sustained, occurring in bursts, varying by shift or time of day, week, month, etc.
  • Block-accessed, or file-, or object-based access.
Metrics used for evaluating, tuning and configuring I/O intensive storage:
  • I/O rate per second (or the number of I/O operations performed for a given period of time, independent of the I/O size).
  • Latency or response time to perform an I/O operation. This is also be measured by service delay and queue depth, as indicators of I/O backlog and I/O congestion at the adapter.
  • Throughput, bandwidth and transfer rate.
Tips and recommendations for I/O intensive environments:
  • Load-balance storage volumes across multiple I/O paths and storage subsystems, with high performance ports attached to switches. For example, attach storage to switches using 2 Gbit and 4 Gbit ports, as they become available. Also look for switches that provide virtual output queue capabilities to prevent head of line blocking.
  • Plan for server enhancements with faster interfaces, including PCI-X V2.0 and PCI-Express, to improve I/O performance particularly for applications requiring higher throughput.
  • Utilize the applicable RAID level for specific applications, for example RAID 0+1 for I/O intensive reads and writes, including log and journal files and frequently updated database tables and files. Even though mirrored write-back cache helps RAID-5 performance, avoid using RAID-5 for write-intensive applications.
  • For storage intensive applications, Solid State Disk (SSD) can be an effective I/O and storage medium. The important metric should be dollars-per-useful-I/O, instead of dollars-per-useful–MB, as you would use for storage-intensive applications.
  • A low I/O rate and large transfer rate with large response time is normal for large, sequential I/O operations such as backup/recovery, data migration, data mining, video and imaging.
  • A low transfer rate and large number of I/Os is normal in an I/O intensive environment including OLTP, e-mail, Web and file serving.

For more information:

Tip: How to design a core/edge SAN

Tip: How to isolate and solve performance bottlenecks

Tip: How to plan for a firmware upgrade

About the author: Greg Schulz is a senior analyst with the independent storage analysis firm The Evaluator Group Inc. Greg has 25 years of IT experience as a consultant, end user, storage and storage networking vendor and industry analyst. Greg has worked with Unix, Windows, IBM Mainframe, OpenVMS and other hardware/software environments. In addition to being an analyst, Greg is also the author and illustrator of "Resilient Storage Networks." Greg holds both a computer science and software engineering degree from the University of St. Thomas.

This was first published in January 2005

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.