The Ghost Brigades (2)


The second time around I think my opinion has changed a little. I found the plot a little hard to believe (perhaps I am scarred by other book’s twee explorations of the motivations of alien species), and overall the book not as good as Old Man’s War. Then again, its far from the worst book I have read this year.

Original post about this book.

[award: nominee prometheus 2007]
[isbn: 0765354063]


Old Man’s War (2)


I’ve been in a rut recently where I haven’t really been enjoying the books I’ve been reading. The number of books I read has also dropped off a lot since I moved back to Australia. Some of the drop off is associated with living in a house instead of an apartment — there is constant maintenance work to be done, and I might never finish painting this place. However, I was worried that perhaps I simply wasn’t as into reading as I was a couple of years ago. So, I decided to go back and read a book I enjoyed before, and see if I still liked it. This was that book.

The answer is hells yes. This book is still fantastic, and I really enjoyed it. I also knocked it over in a time similar to when I was in the US. So, its not me that’s broken — its the books I’m reading. I need to find more books to be enthused about, instead of letting reading be a chore.

Original post about this book.

[awards: nominee hugo 2006]
[isbn: 0765348276]


Red Mars


This is another book on colonization. To be totally honest I enjoyed the first half of the book more than the second, and I rather thought the book dragged on and could have done with a more vigorous editing. There are sections which are deeply descriptive, but it doesn’t progress the story. Overall, I was a little disappointed.

[isbn: 0553560735]
[awards: hugo nominee 1993; nebula winner 1993]


Openstack compute node cleanup


I’ve never used openstack before, which I imagine is similar to many other people out there. Its actually pretty cool, although I encountered a problem the other day that I think is worthy of some more documentation. Openstack runs virtual machines for users, in much the same manner as Amazon’s EC2 system. These instances are started with a base image, and then copy on write is used to write differences for the instance as it changes stuff. This makes sense in a world where a given machine might be running more than one copy of the instance.

However, I encountered a compute node which was running low on disk. This is because there is currently nothing which cleans up these base images, so even if none of the instances on a machine require that image, and even if the machine is experiencing disk stress, the images still hang around. There are a few blog posts out there about this, but nothing really definitive that I could find. I’ve filed a bug asking for the Ubuntu package to include some sort of cleanup script, and interestingly that led me to learn that there are plans for a pretty comprehensive image management system. Unfortunately, it doesn’t seem that anyone is working on this at the moment. I would offer to lend a hand, but its not clear to me as an openstack n00b where I should start. If you read this and have some pointers, feel free to contact me.

Anyways, we still need to cleanup that node experiencing disk stress. It turns out that nova uses qemu for its copy on write disk images. We can therefore ask qemu which are in use. It goes something like this:

    $ cd /var/lib/nova/instances
    $ find -name "disk*" | xargs -n1 qemu-img info | grep backing | \
      sed -e's/.*file: //' -e 's/ .*//' | sort | uniq > /tmp/inuse

/tmp/inuse will now contain a list of the images in _base that are in use at the moment. Now you can change to the base directory, which defaults to /var/lib/nova/instances/_base and do some cleanup. What I do is I look for large image files which are several days old. I then check if they appear in that temporary file I created, and if they don’t I delete them.

I’m sure that this could be better automated by a simple python script, but I haven’t gotten around to it yet. If I do, I will be sure to mention it here.


On syncing with Google Contacts


So, I started with a new company a few weeks ago, and one of the things I missed from my previous company was having the entire corporate directory synced onto my phone. Its really handy as an on caller to be able to give people a call when something goes wrong, without having to dig around and find their details.

Back in the good old days at Google the way you got this sort of data onto your phone was to run a script written by one of the guys on the gmail team. The script grabbed the LDAP directory, and pushed it into Google contacts, which you could then sync with your phone. Now I wanted something very similar — especially as the contacts sync stuff with Android is pretty reasonable.

However, I’d never coded with the Google public APIs before, and that turned out to be the hardest part of the problem.

First off I wrote a little script which dumped the corporate directory into a text file. I mostly did this because I wanted other people to be able to run the script in as light weight a manner as possible — for example, if we wanted to roll this out for hundreds of people, then you wouldn’t want to run the LDAP query hundreds of times. The format for my text file is kinda lame to be honest:

    Michael Still: {'telephoneNumber': ['+61 123 123 123'], 'ID': ['mikalstill'], 'mail': ['']}

So, you get the user’s name, then a python dictionary with three keys in it. There isn’t any particular reason for having just three keys, it was just the three fields I thought were most interesting at the time. Note that each field is an array. A simple human readable format like this means that I can also grep through the file if I ever quickly want a user’s details, which is a nice side effect.

The most important thing I learnt here is that the ID field is really important. If you don’t have something you feel you can use there, then you might need to synthesize something — perhaps an ascii representation of the user’s name or something. This is important because I discovered that Google rewrites Unicode characters you ask it to store, so if you do a simple text comparison against the user’s name, then you might get a false negative and end up creating more than one entry for that user. That was particularly a problem for me because there are a fair few people in the company with European accented characters in their names.

The docs for the Google contacts API are ok, although I did have to spend some time randomly searching for examples of some of the things I wanted to do. For example, the docs didn’t have an example of how to store a phone number that I could find. Also, I am a little shocked to discover there is no query interface in contacts for contact name. This seems like a pretty massive oversight to me, but here’s what the docs have to say on the issue:

For more information about query parameters, see the Contacts Data API Reference Guide and the Google Data APIs Reference Guide. In particular, there is no support for full-text queries or locating a contact by email address.

Whatever intern wrote the API should have his ball pit rights revoked until he fixes that. After that it was all gravy. Here’s the code:

I note that there is an enterprise shared contacts API (see here), but you have to be a premiere customer for it to work.