[ILUG] Xen Crisis - Help Please?[Scanned]

[Cormac McClean]

Hi,

As I have mentioned previously in postings to this list (usually when I 
am asking for help or suggestions) I teach Linux System Administration 
at Ordinary Deegree Level (BEng Computer Engineering). I also have the 
task of maintaining the related machines - Desktops in the Lab, NFS/NIS 
server and, most challenging of all, the Virtual Servers.

I am using Xen on Debian etch and, after some initial difficulty, last 
semester successfully set up 20 virtual servers which the students (and 
I) have been using for practical work. Each student gets root access to 
his/her own virtual server.

This worked perfectly until recently when a power outage in the 
Institute brought down all servers (for some reason the UPS system 
didn't kick in). When the Xen host server came back up, the virtual 
servers didn't automatically restart, so I started them manually. I 
could connect to each of the virtual servers using
	xm console n
but then found that their network setup was no longer working. (Each 
virtual server has its own IP address and is accessible using bringing 
from the host). Foolishly, perhaps, I tried to restart networking using
	/etc/init.d/networking restart
The system then produced a series of errors such as:
	"xen_net: memory squeeze in netback driver"

Under pressure to get things sorted in time for class this afternoon 
(2pm) I possibly made my second mistake - tried a system reboot, hoping 
that this would sort out the problem.

On reboot, however, the system displays a series of messages such as:
	command: Test Unit Ready 00 : 00 : 00 : 00 : 00
	mtpbase: initiating ioc0 recovery
	mptscsih: ioc(0) task abort success: (sc=c0cdb500)
	mptscsih: Attempting task abort! (sc=c0cdb500)
	sd 2:0:0:0:
and I can no longer log in.

I fully admit I am completely out of my depth here. I suspect that there 
may be a hardware fault (disk corruption) but I'm not sure.

I have 2 questions:
1. What should/can I do to get the virtual servers up and running asap?
2. Can anyone recommend someone who would be available for hire as a 
paid consultant to configure this system correctly, document it and 
provide training for the appropriate technican(s)?

I've been maintaining student servers for the last 7 years in various 
guises (last 2 years using UML) but I've finally bitten off more than I 
can chew with Xen. Any help would be gratefully accepted.

Regards,
Cormac.

The information contained in this email is confidential and is designated solely for the attention of the intended recipient(s). If you have received this email in error, please do not use or transmit it for any purpose but rather notify us immediately and delete all copies of this email from your computer system(s). Unless otherwise specifically agreed by our authorised representative, the views expressed in this email are those of the author only and shall not represent the view of or otherwise bind Athlone Institute of Technology. Contact administrator at ait.ie or telephone 090 6471858.

_____________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________