Problem solve Get help with specific problems with your technologies, process and projects.

iostat and sar disk analysis

This tip provides specific examples of high disk usage analysis with iostate and sar.

iostat and sar disk analysis
Imperial Technology

In previous Admin tips, we discussed the commands iostat and sar. Both commands are useful in disk and CPU analysis. The previous tips simply detailed the output of the commands in general terms. This tip will provide specific examples of high disk usage analysis.

Excessive I/O generated requests is illustrated in this example where the disk, sd15, is a magnetic disk used as a log in a high transaction rate environment.

	sdo			sd15		CPU			
rps	wps	util	rps	wps	util	us 	sy	wt	id
0	0	0	0	90	90	20	40	40	0
0	0	0	0	0	89	19	42	40	0

This is an implementation where more I/O requests are generated each second than a disk can service. The disk utilization of 90% with fully 80% of the CPU used in system mode or waiting for I/O to complete, indicates this. CPU idle time is zero, because of the excessive CPU resources being used to manage the I/O queue or waiting for the I/O to complete.

If sar was used in the same analysis on the log disk sd15 in the previous example, the results should show high avque numbers due to the high CPU resource consumption.

Because the disk speed is the actual limiting factor, adding more CPU does not help, as the additional CPU generates more unserviced I/O requests, and a larger utilization that consumes even more CPU in I/O queue management.

Replacing the log disk with a very high rate transaction device, such as a solid state disk, raises the I/O rate of the limiting factor--the disk speed--and when iostat is again run the following results would theoretically be obtained:

	sdo			sd16		CPU			
rps	wps	util	rps	wps	Util	us	sy	wt	id
0	0	0	0	322	17	40	22	5	34
0	0	0	0	300	16	44	23	8	26

Notice that the percentage of CPU waiting drops from 40% to 5% - a 35% reduction, while approximately 200 more log writes are completed each second with only 17% of the disk utilized. Because of this additional speed, I/Os are completed as they are requested, and the CPU no longer queues I/Os. Thus, the percentage of CPU spent in system mode is reduced and CPU usage doubles. In addition, 30% of the CPU is idle and available for more work. Thus, the elimination of the disk "bottleneck" at the log has now made the system capable of significantly more work, while using less CPU resource.

Database applications can vary widely in the ratio of system resources used. In general, however, high disk utilizations, very low or very high CPU idle percentages and large percentages of CPU spent in waiting or system mode, indicate significant unutilized CPU.

This tip was provided to courtesy of Imperial Technology, manufacturers of diskless servers.

What did you think of this tip? Like it? Hate it? Email and let us know.

Dig Deeper on Storage vendors