OpenStack – Page 6 – Made by Mikal

Thoughts from the PTL

Post author:mikal
Post published:April 15, 2014
Post category:OpenStack

I sent this through to the openstack-dev mailing list (you can see the thread here), but I want to put it here as well for people who don't actively follow the mailing list. First off, thanks for electing me as the Nova PTL for Juno. I find the outcome of the election both flattering and daunting. I'd like to thank Dan and John for running as PTL candidates as well -- I strongly believe that a solid democratic process is part of what makes OpenStack so successful, and that isn't possible without people being will to stand up during the election cycle. I'm hoping to send out regular emails to this list with my thoughts about our current position in the release process. Its early in the cycle, so the ideas here aren't fully formed yet -- however I'd rather get feedback early and often, in case I'm off on the wrong path. What am I thinking about at the moment? The following things: * a mid cycle meetup. I think the Icehouse meetup was a great success, and I'd like to see us do this again in Juno. I'd also like to get the location and venue nailed down…

Juno Nova PTL Candidacy

Post author:mikal
Post published:March 30, 2014
Post category:OpenStack

This is a repost of an email to the openstack-dev list, which is mostly here for historical reasons. Hi. I would like to run for the OpenStack Compute PTL position as well. I have been an active nova developer since late 2011, and have been a core reviewer for quite a while. I am currently serving on the Technical Committee, where I have recently been spending my time liaising with the board about how to define what software should be able to use the OpenStack trade mark. I've also served on the vulnerability management team, and as nova bug czar in the past. I have extensive experience running Open Source community groups, having served on the TC, been the Director for linux.conf.au 2013, as well as serving on the boards of various community groups over the years. In Icehouse I hired a team of nine software engineers who are all working 100% on OpenStack at Rackspace Australia, developed and deployed the turbo hipster third party CI system along with Joshua Hesketh, as well as writing nova code. I recognize that if I am successful I will need to rearrange my work responsibilities, and my management is supportive of that. The…

Comparing alembic with sqlalchemy migrate

Post author:mikal
Post published:November 5, 2013
Post category:OpenStack

In the last few days there has been a discussion on the openstack-dev mailing list about converting nova to alembic. Nova currently uses sqlalchemy migrate for its schema migrations. I would consider myself a sceptic of this change, but I want to be a well educated sceptic so I thought I should take a look at an existing alembic user, in this case neutron. There is also at least one session on database changes at the Icehouse summit this coming week, and I wanted to feel prepared for those conversations. I should start off by saying that I'm not particularly opposed to alembic. We definitely have problems with migrate, but I am not sure that these problems are addressed by alembic in the way that we'd hope. I think we need to dig deeper into the issues we face with migrate to understand if alembic is a good choice. sqlalchemy migrate There are two problems with migrate that I see us suffering from at the moment. The first is that migrate is no longer maintained by upstream. I can see why this is bad, although there are other nova dependencies that the OpenStack team maintains internally. For example, the various…

On Continuous Integration testing for Nova DB

Post author:mikal
Post published:November 3, 2013
Post category:OpenStack

To quote Homer Simpson: "All my life I've had one dream, to achieve my many goals.". One of my more recent goals is a desire to have real continuous integration testing for database migrations in Nova. You see, at the moment, database migrations can easily make upgrades painful for deployers, normally by taking a very long time to run. This is partially because we test on trivial datasets on our laptops, but it is also because it is hard to predict the scale of the various dimensions in the database -- for example: perhaps one deployment has lots of instances; whilst another might have a smaller number of instances but a very large number of IP addresses. The team I work with at Rackspace Australia has therefore been cooking up a scheme to try and fix this. For example, Josh Hesketh has been working on what we call Turbo Hipster, which he has blogged about. We've started off with a prototype to prove we can get meaningful testing results, which is running now. Since we finished the prototype we've been working on a real implementation, which is known as Turbo Hipster. I know it's an odd name, but we couldn't…

Exploring a single database migration

Post author:mikal
Post published:August 3, 2013
Post category:OpenStack

Yesterday I was having some troubles with a database migration download step, and a Joshua Hesketh suggested I step through the migrations one at a time and see what they were doing to my sqlite test database. That's a great idea, but it wasn't immediately obvious to me how to do it. Now that I've figured out the steps required, I thought I'd document them here. First off we need a test environment. I'm hacking on nova at the moment, and tend to build throw away test environments in the cloud because its cheap and easy. So, I created a new Ubuntu 12.04 server instance in Rackspace's Sydney data center, and then configured it like this: $ sudo apt-get update $ sudo apt-get install -y git python-pip git-review libxml2-dev libxml2-utils libxslt-dev libmysqlclient-dev pep8 postgresql-server-dev-9.1 python2.7-dev python-coverage python-netaddr python-mysqldb python-git virtualenvwrapper python-numpy virtualenvwrapper sqlite3 $ source /etc/bash_completion.d/virtualenvwrapper $ mkvirtualenv migrate_204 $ toggleglobalsitepackages Simple! I should note here that we probably don't need the virtualenv because this machine is disposable, but its still a good habit to be in. Now I need to fetch the code I am testing. In this case its from my personal fork of nova, and the git…

Nova database continuous integration

Post author:mikal
Post published:July 4, 2013
Post category:OpenStack

I've had some opportunity recently to spend a little quality time off line, and I spent some of that time working on a side project I've wanted to do for a while -- continuous integration testing of nova database migrations. Now, the code isn't perfect at the moment, but I think its an interesting direction to take and I will keep pursuing it. One of the problems nova developers have is that we don't have a good way of determining whether a database migration will be painful for deployers. We can eyeball code reviews, but whether code looks reasonable or not, its still hard to predict how it will perform on real data. Continuous integration is the obvious solution -- if we could test patch sets on real databases as part of the code review process, then reviewers would have more data about whether to approve a patch set or not. So I did that. At the moment the CI implementation I've built isn't posting to code reviews, but that's because I want to be confident that the information it gathers is accurate before wasting other reviewers' time. You can see results at openstack.stillhq.com/ci. For now, I am keeping an…

Merged in Havana: fixed ip listing for single hosts

Post author:mikal
Post published:April 27, 2013
Post category:OpenStack

Nova has supported listing the fixed ips for a single host for a while. Well, except for that time we broke it by removing the database call it used and not noticing. My change to fix that situation has just landed, so this should now work again. To list the fixed ips used on a host, do something like: nova-manage fixed list hostname I will propose a backport to grizzly for this now.

Michael’s surprisingly unreliable predictions for the Havana Nova release

Post author:mikal
Post published:April 21, 2013
Post category:OpenStack

I should start out by saying that because OpenStack is an open source project, it is hard to know exactly what will land in Havana -- the developers are volunteers, and sometimes things get in the way of them doing the work they intended. However, these are the notes I wrote up on the high points of the summit for me -- I didn't see all the same sessions as other nova developers, so hopefully others will pitch in with their notes as well. Scheduler The scheduler seems to be a point of planned work for a lot of people in this release, with talk about having more scheduling code in the common library, and of adding new filter types. There is definite interest in being able to schedule by methods we don't currently support -- things like rack or PDU diversity, or trying to collocate a tenants machines together. HP is also interested in being able to sell dedicated machines to tenants -- in other words, they would guarantee that only one tenants instances appeared on a machine in return for a fee. At the moment this requires setting up a host aggregate for the tenant. Feeding additional data…

Getting started with OpenStack development

Post author:mikal
Post published:April 17, 2013
Post category:OpenStack

I just gave my presentation at the Havana Conference about how to get started with OpenStack development. A few people asked for my slide deck, so I am posting it here. The talk was taped, and I am sure some more formal release will happen in the future, but I wanted to get this out there for the people who had asked for it.

Faster pip installs

Post author:mikal
Post published:April 9, 2013
Post category:OpenStack

Last week with the help of the lovely openstack-infra people, I discovered that you can have a local cache of pip downloads. This speeds up rebuilding test environments when you need to jump between branches with different dependencies. Its as simple as chucking something like: export PIP_DOWNLOAD_CACHE=~/cache/pip ...into your .bashrc or equivalent.