Xen on Ubuntu 12.04 Notes

The recently released Ubuntu 12.04 has great support for Xen, simply use "sudo apt-get install xcp-xapi" to install it. I did run in to just a couple issues though:

1.  There is an installer option to use "openvswitch" rather than typical Xen network bridging.  I initially chose openvswitch so I could play with it, but decided to temporarily revert back to traditional Xen scripts until everything was setup.  To revert all you have to do is add back "(network-script network-bridge)" to /etc/xen/xend-config.sxp.  This doesn't remove or even disable openvswitch, it just has Xen automatically create the bridges for you.

2.  While Xen was added to grub, it didn't become the default on the bootup menu.  To fix this you could change /etc/default/grub to point to the correct list entry, but this could cause issues down the line as you update kernels.  Instead, I renamed the "20_linux_xen" to be "06_linux_xen" in /etc/grub.d/, then ran "sudo update-grub".  Per bjmuccolloch, below, the best way to do this is "dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen".  This will put the Xen options at the top of the grub list and make the latest xen/linux kernel pair the default.  Be sure to make it "06" and not just "6", otherwise it won't work...

3.  While there appears to be a new xapi xe toolset that can be used to manage/create VMs (and comes with the xcp-xapi package we already installed), I'm more familiar with the existing xen-tools package.  This is still available in apt, so just do a simple "sudo apt-get install xen-tools" to get it.

Oddly, by default it doesn't allow you to use xen-create-image to make Ubuntu 12.04 or even 11.10 images.  To fix this simply create a new symlink in the /usr/lib/xen-tools/ directory by doing something like "sudo ln -s debian.d oneiric.d; sudo ln -s debian.d precise.d;".

I had some console issues using xen-tools (which probably aren't a problem if you use the xe tools).  To fix this you just have to add:

echo "start on stopped rc RUNLEVEL=[2345] stop on runlevel [!2345] respawn exec /sbin/getty -8 38400 ${serial_device}" > ${prefix}/etc/init/${serial_device}.conf

to /usr/lib/xen-tools/debian.d/30-disable-gettys.  This will make VMs spawn a tty on the console device.

Of course, as usual, you probably want to setup your defaults in /etc/xen-tools/xen-tools.conf

4.  Finally, I had some issues with the Xen bridge not picking up a custom MAC in /etc/network/interfaces file.  This is important, since the bridge and the Dom0 interface have to share a MAC address on my network (since that is how VLANs are assigned).  The cause of this looked to be that the init.d network script was being run after some of the xen init.d scripts, but I could be wrong.  The simple fix is simply to add a "ifconfig peth0 hw ether xx:xx:xx:xx:xx:xx" to the end of /etc/init.d/xendomains, before the "rc_exit".  This is a bit cludgy, but it worked.

That's about it, everything else worked great!  Maybe I'll do an update after I get a chance to play with the xe tools or openvswitch.