So if you’re like me you are probably somewhat lazy. I imagine most sysadmins are. Which must be why we put so much effort into scripting and automating repeated tasks. If you’ve managed a Xen VM before you are likely to notice that all your kernel images live outside the VM itself. This makes it rather difficult to patch and upgrade among other things, especially as the number and type of guests you support grows. Pvgrub can help solve this problem. It implements grub in a stub domain which in turn loads your guest domain. It works quite well and is rather easy to set up. Read on to learn more.

From the xen wiki: “PV-GRUB replaces PyGrub to boot domU images safely: it runs the regular grub inside the created domain itself and uses regular domU facilities to read the disk / fetch files from network etc.; it eventually loads the PV kernel and chain-boots it.”

This means that your domU guests can install and manage their own kernels as if they were running on regular hardware. Yum, apt-get and other package management software will let you patch your virtual machine’s kernel without the need to alter any Xen configuration. Here’s how it works.

Notes:

* Pvgrub requires Xen 3.3 or greater.
* You won’t actually need to install grub in the domU, it is implemented as a stub domain and provided with the Xen sources.

Steps:

1. Modify /etc/modprobe.conf in the guest.

We need to make sure that a couple of lines exist in /etc/modprobe.conf before we’re ready to install the kernel packages. We add these so that when the kernel install process goes to build a new initrd it will include the correct driver modules for your hardware, so we want to be certain that the Xen block and net drivers are included.

Note: If /etc/modprobe.conf doesn’t already exist you should create it.

#/etc/modprobe.conf
alias eth0 xennet
alias scsi_hostadapter xenblk

2. Update /etc/inittab and /etc/securetty in the guest.

Now we update /etc/inittab and /etc/securetty to make certain that we attach to and permit logins from the various xen consoles.

/etc/inittab

#/etc/inittab
hvc0:2345:respawn:/sbin/getty 38400 hvc0
xvc0:2345:respawn:/sbin/getty 38400 xvc0

/etc/securetty

#/etc/securetty
# hypervisor virtual consoles
hvc0
xvc0

3. Install a stub grub.conf in the guest so that the grub update scripts won’t fail.

Configuration file locations:

Depending on the Linux distribution you use you’re likely to see the grub config in either /boot/grub/grub.conf or /boot/grub/menu.lst. To keep things simple I usually use /boot/grub/grub.conf as a config file and symlink /boot/grub/menu.lst to it. This should ensure that scripts used to automatically update the grub config when a new kernel is installed will work correctly.

Reasonable stub for Debian Lenny

#/boot/grub/grub.conf
default 0
timeout 5
 
title vmlinuz-2.6.26-2-xen-686
   root (hd0)
   kernel /boot/vmlinuz-2.6.26-2-xen-686 console=hvc0 root=/dev/xvda1 ro
   initrd /boot/initrd.img-2.6.26-2-xen-686

Reasonable stub for CentOS/RHEL

#/boot/grub/grub.conf
default 0
timeout 5
 
title vmlinuz-2.6.18-128.4.1.el5xen
	root (hd0)
	kernel /boot/vmlinuz-2.6.18-128.4.1.el5xen root=LABEL=root ro
	initrd /boot/initrd-2.6.18-128.4.1.el5xen.img

4. Install kernel packages in the guest.

Debian

# apt-get install linux-image-2.6.26-1-686-bigmem

CentOS/RHEL

# yum install kernel-xen

5. Modify Xen guest config file in the dom0.

Now that the VM has been prepped for booting with pvgrub we need to update its configuration file to load the pvgrub kernel instead of a linux kernel.

Note: be certain to remove the old “root=” line, this seems to confuse pvgrub. Also, make certain that the “extra=” line points to the location of your grub config file inside the guest. My example does not use partitions, if you do use partitions in your guests you’ll need so use something like (hd0,0), etc.

name    = 'superchunk'
 
vcpus =2;
 
memory 	= 512;
maxmem  = 4096;
 
#pvgrub here
kernel = "/usr/lib/xen/boot/pv-grub-x86_32.gz"
extra = "(hd0)/boot/grub/grub.conf"
#end pvgrub
 
disk = [ 'phy:/dev/vg0/vm_superchunk-root,xvda1,w', \
         'phy:/dev/vg0/vm_superchunk-swap,xvda2,w' ];
 
vif = [ 'bridge=eth1, vifname=superchunk, mac=B0:00:00:00:B1:35' ];

6. Shutdown and then create the domain.

My Xen guest is named superchunk. You should be familiar with shutdown and create at this point. Here’s how, just in case…

xm shutdown superchunk
#(wait until it has shut down)
xm create -c /path/to/superchunk-config

At this point you should see some kernel output and then either a grub menu or the grub command line. If you get the command line, no worries. This usually means that it couldn’t find a grub configuration file. Make sure that the “extra=” line of the Xen guest domU configuration file references the right location and that you have pointed grub at the correct disk, i.e. (hd0) vs. (hd0,0), etc…

Good luck! And feel free to drop me a line at [email protected], I always appreciate feedback and do my best to help.