[ILUG] Extremely slow/CPU intensive SATA IO on 2 CPU HT server.

[David Wilson]

Hi folks,

We have a dual 2.8ghz Xeon server with 2gb RAM running vanilla 2.6.15.1
which is seeing some really odd IO behaviour. I've always been aware
that the machine felt slightly more sluggish than it should have, but
upon running an extremely simple UPDATE on 80,000 or so rows in
PostgreSQL last night, the machine crapped itself and jumped to a load
of 8.0.

For comparison, the same UPDATE runs on the same dataset on the same
version of PostgreSQL on a single CPU 2.8ghz P4 FreeBSD 4.10 machine in
slightly less time, and not even pushing the load past 0.2.

As a side note, the table the UPDATE ran on was freshly VACUUM FULL
ANALYSED just before running the update. The PostgreSQL fsync and wal_*
settings appear to have little effect on the load.

The disk performance feels relatively normal when doing simple
benchmarks (like sync; DD 1gb of /dev/zero to a file; sync), but in
other places, like the PostgreSQL query, and enumerating a very deeply
hashed directory tree, the machine falls to pieces.

The filesystems are ext3.

The load isn't caused by userspace, indeed PostgreSQL sits at 6%
utilisation for the most part, as the machine load climbs indefinately.

The only thing I can think of is having the hosting company pull up the
BIOS and turn off hyperthreading, something I've been meaning to do for
various other reasons anyway.

    Linux 2.6.15.1 #1 SMP Fri Jan 20 19:47:58 GMT 2006 i686
    scsi0 : ata_piix
    ATA: abnormal status 0x7F on port 0xC407
    ata1: SATA max UDMA/133 cmd 0xC800 ctl 0xC482 bmdma 0xC000 irq 17
    SCSI device sda: 586072368 512-byte hdwr sectors (300069 MB)

Does anyone have any pointers? Thanks,


David.

-- 
'tis better to be silent and be thought a fool,
than to speak and remove all doubt.
    -- Lincoln