[ILUG] Booting a broken raid array

[Michael Watterson]

paul at clubi.ie wrote:
> On Fri, 16 Jan 2009, Kenn Humborg wrote:
>
>> Interesting!  How do Sun handle the boot-from-software-RAID
>> problem?  They probably have much smarter boot loaders.
>
> Sun wrote a ZFS module for GRUB. So GRUB knows enough to pick through 
> ZFS, at least as in OpenSolaris.
>
> There may be (probably are?) constraints on the RAID levels that can 
> be used on a root-pool.
>
> This code isn't in any supported version of Solaris, yet AFAIK (but 
> might go into the next Solaris 10 update).
>
> regards,
Most OS using software RAID can only boot from Mirror as without RAID 
driver the "primary boot drive" looks like an ordinary drive. If the 
Primary BIOS/SCSI/ATA/SATA drive doesn't work, you either physically 
swap the two drives or change bios settings depending on OS, BIOS, type 
of Interface etc.

So with SW RAID the safest approach is OS on a Mirror (RAID 1) and all 
else on a single  RAID5 or RAID6. Thus a minimum of 5 drives. (or 3 if 
you partition all 3 and waste one primary partition, then part A of 
Drive 0 and 1 is Mirror with OS and part B of Drive 0, 1, 2 is RAID 5. 
However this is scary to fix when a drive fails).

Using HW RAID the CPU on the RAID controller "knows" how to boot RAID5 
or RAID6 no matter which drive has failed. I used AMI Megaraid 428 for 
years.

RAID generally needs a UPS as a sudden power fail is  VERY bad, 
especially if SW RAID. Some HW RAID have battery backup on board and 
store the status of the controller and cache during power fail. 
Allegedly with UPS and autoshut down the battery is not needed.

Traditionally for various reasons SCSI was the best candidate for HW or 
SW RAID. Certainly if you wanted a performance improvement (up to x100 
possible on multiuser access 5 drives RAID 5 SCSI compared with a Single 
IDE/SCSI) RAID5 on SCSI was needed. SW or HW. The problem was the 
inability to seek on three ATA drives while writing on another and 
reading the 5th etc.

Mirror speeds reads and slows write speed on SW RAID1 SCSI. It has 
little impact on read speed and halves write time on PATA/IDE.  RAID 5  
can  dramatically  improve read and write performance as more drives 
added, but added improvement drops as more drives added and risk of a 
drive failure increases, so 4 or 5 drives is a sensible limit. I did 
have a 7 drive RAID5 using 1G drives long ago. Never had two failures at 
once or a failure during rebuild. it can happen but on good server class 
SCSI drives very very much less likely than a  drive fail, which is 7 
times more likely than failure of a single drive system.

Some thought is needed for HW RAID configuration and MUCH thought is 
needed for SW RAID configuration.  I usually configured 2 swap files on 
SW RAID, one on BOOT (mirror) and a faster one on RAID 5 data array.

Currently I have AMI megaraid 428 with 3 drive RAID 5 for OS and 5 drive 
RAID 5 with bigger disks  for   Data. For performance these are all  10K 
rpm  Ultra Wide  & Fast  and  the  two  RAID  sets  are on  two 
separate  controller  busses/ports.  This  doubles  data  rate  and  
reduces  the  issue  of  two drives at once  problem if I had 8 same 
size drives as one RAID set.  The HW Controller is old (i960 based) and 
does not support RAID6.

You can certainly max out the Gigibit ethernet even with RANDOM read or 
Write, not block accesses. A single standalone Drive will drop maybe 
over 100x to 500x in data performance due to latency and track to track 
access time when used randomly by multiple users rather than single 
sequential big file.

So RAID5 or RAID6 serves two purposes, Resilience on a failure (parity) 
and dramatic improvement in random/multiuser performance due to stripe set

If you want simplicity, hot swap, and max performance, then HW RAID. If 
you want cheaper and not so concerned about performance or ease of 
administration SW RAID.
 



-- 
Mike