Openstack compute node cleanup

I’ve never used openstack before, which I imagine is similar to many other people out there. Its actually pretty cool, although I encountered a problem the other day that I think is worthy of some more documentation. Openstack runs virtual machines for users, in much the same manner as Amazon’s EC2 system. These instances are started with a base image, and then copy on write is used to write differences for the instance as it changes stuff. This makes sense in a world where a given machine might be running more than one copy of the instance.

However, I encountered a compute node which was running low on disk. This is because there is currently nothing which cleans up these base images, so even if none of the instances on a machine require that image, and even if the machine is experiencing disk stress, the images still hang around. There are a few blog posts out there about this, but nothing really definitive that I could find. I’ve filed a bug asking for the Ubuntu package to include some sort of cleanup script, and interestingly that led me to learn that there are plans for a pretty comprehensive image management system. Unfortunately, it doesn’t seem that anyone is working on this at the moment. I would offer to lend a hand, but its not clear to me as an openstack n00b where I should start. If you read this and have some pointers, feel free to contact me.

Anyways, we still need to cleanup that node experiencing disk stress. It turns out that nova uses qemu for its copy on write disk images. We can therefore ask qemu which are in use. It goes something like this:

    $ cd /var/lib/nova/instances
    $ find -name "disk*" | xargs -n1 qemu-img info | grep backing | \
      sed -e's/.*file: //' -e 's/ .*//' | sort | uniq > /tmp/inuse

/tmp/inuse will now contain a list of the images in _base that are in use at the moment. Now you can change to the base directory, which defaults to /var/lib/nova/instances/_base and do some cleanup. What I do is I look for large image files which are several days old. I then check if they appear in that temporary file I created, and if they don’t I delete them.

I’m sure that this could be better automated by a simple python script, but I haven’t gotten around to it yet. If I do, I will be sure to mention it here.

I got MythBurn working!

It took a few hours, but I have MythBurn (a web interface for creating DVDs with MythTV) finally working. The problem was that the mjpegtools shipped with Ubuntu Dapper crashes with an invalid free(). Compiling from source fixed that.

Update: I might have lied. I now have troubles getting Xine to playback the DVD when I mount the ISO image through a loop back. I don’t have any DVD media handy, so I will have to wait a couple of days until I can do more testing. The symptom I see is that the intro MythTV screen plays, and then Xine freezes before displaying the menu.

Compiling MythTV 0.19 for Ubuntu Dapper

I’ve been finding that MythTV is really unreliable after dist-upgrading from Breezy to Dapper. In fact, it’s been crashing six or seven times a day, which has been really annoying. I figured I though therefore bight the bullet and upgrade to the latest stable version of MythTV. I was previously running a SVN snapshot because it fixed a couple of bugs I was having when I first installed Myth.

Installation was pretty easy. First, grab the one or two libraries you’re missing and that the configure script doesn’t necessarily detect the dependency for properly:

    sudo apt-get install mesa-common-dev libgl1-mesa-dev libglu1-mesa-dev \
    libqt3-mt-dev libqt4-dev mesa-common-dev qt3-apps-dev libglu1-mesa fftw3 \
    fftw3-dev libsdl1.2-dev festival festival-dev libdvdread3 libdvdread3-dev

And then the usual configure, make, make install in the mythtv and mythplugins directories.