[ILUG] Poor Bonnie results ?

[Wesley Darlington]

Hello,

On Fri, May 25, 2001 at 12:18:17PM +0100, Niall O Broin wrote:
> I was doing some testing on a big box which inspired me to run bonnie on the
> disks on my desktop machine. I tested two IDE disks and a SCSI disk with the
> following results:
> 
>               ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
> hdb    1*1500  5033 54.0  6124  6.2  3239  4.3  8571 80.8  9544  4.8   67.2  1.2
> 
>               ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
> hda    1*1500  4459 68.2  7080 21.3  3400 49.3  4363 75.6  7639 73.9   70.0  8.1

The figures for block input and output look pretty bad...

>               ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
> sda    1* 500  5263 56.1  4524  4.7  2212  2.9  3856 35.8  4292  2.4   66.3  0.8
> 
> Both IDE disks had been tuned with hdparm -c 1 -m 16. I don't know where on the

Have you got dma turned on? What about unmaskirq? (No liability accepted for lost
data as a result of turning these options on. Especially unmaskirq. ;-)

> curve the sequential I/O figures lie but the random seek figures on the IDE
> drives and the SCSI drive are atrocious, at least by comparison to the big box
> I'm testing which looks like this

Ah, now, 70 or so random seeks per second isn't *that* bad - moving the disk
head (singular!) is a slow process. It has remained pretty much constant over
the history of disk technology while everything else to do with disks has jumped 
in leaps and bounds. AIUI.

Ob-X-Thread: Compared to disk seek times, tape technology has advanced in leaps
and bounds over the last few years.

Anyway, 1000/70 ~= 14 milliseconds, average seek time, which is in the right 
ballpark for the average disk, if a tad higher than typically quoted average
seek times.

>               ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
> pogo   1*1500  6706 47.1  7297  3.9  3391  2.7  6039 43.9 79283 30.0  862.5  5.8

Ooooh, that's nice. It's raid5, innit? Without writeback. Nice block output
figure, though. :-)

<Ponders briefly>  Yeah, that random seeks/sec figure is wonderful. What sort
of raid card is that? And what sort of disks? And how much memory has this box
got? Way less than 1500MB, I hope...

> and the figures recently posted by John Ronan for a box with 2 x PIII 933,
> 1GB Ram, 2.2.19smp kernel, SmartArray 3200 with 64MB
> 
>               -------Sequential Output-------- ---Sequential Input-- --Random-Seeks
>               -Per Char- --Block--- -Rewrite-- -Per Char- --Block---
> 
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
> m1       1500  8726 65.7  9331  5.2  5740  8.4  9482 65.9 57447 30.9 785.8  7.7
>
> The big box has a bit more oomph (2 X 1G PIII, 1G RAM, 425 GB IDE RAID-5) than

...Ah, a gig. How are the tests affected if you boot with (say) mem=512M I wonder?

> my desktop (750MHz Athlon, 256M RAM) but even so, they seem like miserable
> random seek figures. I just ran the test on my notebook (700MHz PIII,  192M
> RAM) and got
>               ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
> 	      -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU   /sec %CPU
> bilbo  1* 500  4476 93.9  6281 54.1  3066 70.5  4414 86.2  6349 66.9   76.3  6.3

Ah well, that's notebooks for ya. Is dma turned on?

> which is in the same order, seek wise,  as the disks on the desktop. Is
> there something I don't know about RAID-5 subsystems which makes them
> stunning performers on random seek. They should be quite good for block
> input, as mine and John's figures show, but why random seek ?

Raid5 arrays have lots of heads to go seeking, I guess. Having said that,
O(1e3) seeks per second is pretty amazing, now that I think about it. Is
there any caching going on I wonder? (mem=256M...?)

Then there's the whole elevator sorting thing - you'd think that normal
scsi disks would do that too; perhaps not to the same extent, though.

Another consideration might be that in the single-disk cases, the filesystem
on which the seeking is done (*) is much bigger relative to the size of
the disk than in the multiple-disk cases. So in the raid5 cases, the data 
is in a much thinner slice of the [logical] disk, so much less seeking is
needed. Maybe.

Also, decent raid arrays (with plenty of cache) do optimistic readahead,
figuring that since the head is in place, it may as well read a few tracks
into the future, just in case. I'm a bit wary using this on to try to explain
why the seek times are so good, though - I have a notion that this tends to 
only be useful/done if all the disk heads are in place to read a whole slice.

To what extent does doing the tests on the bit of the disk `near the 
beginning' sweeten the results, I wonder...?

ATB,
Wesley.

(*) Assuming bonnie just seeks within its own file(s), rather than seeking
anywhere on the raw disk...?