python

Rejected talk proposal: Design at scale: OpenStack versus Kubernetes

This proposal was submitted for pyconau 2018. It wasn’t accepted, but given I’d put the effort into writing up the proposal I’ll post it here in case its useful some other time. The oblique references to OpensStack are because pycon had an “anonymous” review system in 2018, and I was avoiding saying things which directly identified me as the author.

OpenStack and Kubernetes solve very similar problems. Yet they approach those problems in very different ways. What can we learn from the different approaches taken? The differences aren’t just technical though, there are some interesting social differences too. (more…)

Continue ReadingRejected talk proposal: Design at scale: OpenStack versus Kubernetes

Accepted talk proposal: Learning from the mistakes that even big projects make

  • Post author:
  • Post category:OpenStack

This proposal was submitted for pyconau 2018. It was accepted, but hasn’t been presented yet. The oblique references to OpensStack are because pycon had an “anonymous” review system in 2018, and I was avoiding saying things which directly identified me as the author.

Since 2011, I’ve worked on a large Open Source project in python. It kind of got out of hand – 1000s of developers and millions of lines of code. Yet despite being well resourced, we made the same mistakes that those tiny scripts you whip up to solve a small problem make. Come learn from our fail.

(more…)

Continue ReadingAccepted talk proposal: Learning from the mistakes that even big projects make
Read more about the article pyconau 2018 call for proposals now open
hdr

pyconau 2018 call for proposals now open

The pyconau call for proposals is now open, and runs until 28 May. I took my teenagers to pyconau last year and they greatly enjoyed it. I hadn't been to a pyconau in ages, and ended up really enjoying thinking about things from topic areas I don't normally need to think about. I think expanding one's horizons is generally a good idea. Should I propose something for this year? I am unsure. Some random ideas that immediately spring to mind: something about privsep: I think a generalised way to make privileged calls in unprivileged code is quite interesting, especially in a language which is often used for systems management and integration tasks. That said, perhaps its too OpenStacky given how disinterested in OpenStack talks most python people seem to be. nova-warts: for a long time my hobby has been cleaning up historical mistakes made in OpenStack Nova that wont ever rate as a major feature change. What lessons can other projects learn from a well funded and heavily staffed project that still thought that exec() was a great way to do important work? There's definitely an overlap with the privsep talk above, but this would be more general. a talk…

Continue Readingpyconau 2018 call for proposals now open

A pythonic example of recording metrics about ephemeral scripts with prometheus

  • Post author:
  • Post category:Prometheus

In my previous post we talked about how to record information from short lived scripts (I call them ephemeral scripts by the way) with prometheus. The example there was a script which checked the SMART status of each of the disks in a machine and reported that via pushgateway. I now want to work through a slightly more complicated example. I think you hit the limits of reporting simple values in shell scripts via curl requests fairly quickly. For example with the SMART monitoring script, SMART is capable of returning a whole heap of metrics about the performance of a disk, but we boiled that down to a single "health" value. This is largely because writing a parser for all the other values that smartctl returns would be inefficient and fragile in shell. So for this post, we're going to work through an example of how to report a variety of values from a python script. Those values could be the parsed output of smartctl, but to mix things up a bit, I'm going to use a different script I wrote recently. This new script uses the Weather Underground API to lookup weather stations near my house, and then generate…

Continue ReadingA pythonic example of recording metrics about ephemeral scripts with prometheus

Terrible pong

The kids at coding club have decided that we should write an implementation of pong in python. I took a look at some options, and decided tkinter was the way to go. Thus, I present a pong game broken up into stages which are hopefully understandable to an 11 year old: Operation Terrible Pong.

Continue ReadingTerrible pong

Getting started with OpenStack development

  • Post author:
  • Post category:OpenStack

I just gave my presentation at the Havana Conference about how to get started with OpenStack development. A few people asked for my slide deck, so I am posting it here. The talk was taped, and I am sure some more formal release will happen in the future, but I wanted to get this out there for the people who had asked for it.

Continue ReadingGetting started with OpenStack development

On syncing with Google Contacts

  • Post author:
  • Post category:Google

So, I started with a new company a few weeks ago, and one of the things I missed from my previous company was having the entire corporate directory synced onto my phone. Its really handy as an on caller to be able to give people a call when something goes wrong, without having to dig around and find their details. Back in the good old days at Google the way you got this sort of data onto your phone was to run a script written by one of the guys on the gmail team. The script grabbed the LDAP directory, and pushed it into Google contacts, which you could then sync with your phone. Now I wanted something very similar -- especially as the contacts sync stuff with Android is pretty reasonable. However, I'd never coded with the Google public APIs before, and that turned out to be the hardest part of the problem. First off I wrote a little script which dumped the corporate directory into a text file. I mostly did this because I wanted other people to be able to run the script in as light weight a manner as possible -- for example, if we wanted…

Continue ReadingOn syncing with Google Contacts

Building a symlink tree for MythTV recordings

  • Post author:
  • Post category:Mythtv

I wanted to build a directory of symlinks that pointed to my MythTV recordings, so I wrote a little python script to do it for me. I figure someone else might find this useful too... #!/usr/bin/python # Copyright (C) Michael Still (mikal@stillhq.com) 2007 # Released under the terms of the GNU GPL import MySQLdb import os import re from socket import gethostname # Connect to the MythTV database based on the MythTV config config_values = {} home = os.environ.get('HOME') config = open(home + '/.mythtv/mysql.txt') for line in config.readlines(): if not line.startswith('#') and len(line) > 5: (key, value) = line.rstrip('\n').split('=') config_values[key] = value db_connection = MySQLdb.connect(host = config_values['DBHostName'], user = config_values['DBUserName'], passwd = config_values['DBPassword'], db = config_values['DBName']) cursor = db_connection.cursor(MySQLdb.cursors.DictCursor) # Regexp for what is allowed in the symlink name unsafe = re.compile('[^a-zA-Z0-9\-\:_]+') # Find the recordings directory -- this assumes you haven't used an # identifier string for this machine... cursor.execute('select * from settings where value="RecordFilePrefix" and ' 'hostname="%s";' % gethostname()) row = cursor.fetchone() basedir = row['data'] # Now find all the recordings we have at the moment cursor.execute('select title, subtitle, starttime, basename from recorded;') for i in range(cursor.rowcount): row = cursor.fetchone() title = row['title'] subtitle = row['subtitle'] if subtitle…

Continue ReadingBuilding a symlink tree for MythTV recordings

End of content

No more pages to load

Newer Posts
Older Posts