[ILUG] What process is generating disk I/O

[jez]

On Wed, Feb 14, 2007 at 12:26:19PM -0000, Kenn Humborg wrote:
> 
> Running vmstat 1 on a machine shows me a certain amount of block I/O.
> 
> But, short of strace-ing everything running on the machine, how can
> I determine which processes are generating this I/O?
> 
> lsof will show me open files, but that doesn't help much.
> 

To get a better grip of what your processes are doing over time, it
may be useful to use a combination of sar and top. sar is a part of 
the sysstat package (which also includes iostat which you might like).
You can use sar to collect per-process statistics by using the -x 
option as follows:

    sar -x ALL 1 1000 > sar-stats.txt

This will collect stats each second for 1000 seconds. The output is
ordered by PID which is useful since you can just check the bottom
of each list to see what new processes are doing. Unfortunatly the
output doesn't give process names, but that's okay because top does.

You can run top in batch mode using the -b option:

    top -d 1 -b -n 1000 > top-stats.txt

Again this will collect stats ever second for 1000 seconds. You're
really just looking to see who's using CPU at any one time, but this
output also gives you an ability to look up process names by PID
historically,

This stuff should give you plenty of information to be able to narrow
down the likely culprits. You can then use lsof and whatever, to drill
deeper.

Watch you don't fill up your disk though.