simpler_cloud_thoughts – Made by Mikal

The Mythical Man-Month

I expect everyone (well, almost everyone) involved in some way in software engineering has heard of this book. I decided that it was time to finally read it, largely prompted by this excellent blog post by apenwarr which discusses second systems effect among other things. Now, you can buy this book for a surprisingly large amount of money, but as Michael Carden pointed out, the PDF is also made available for free by the Internet Archive. I’d recommend going that route.

The book is composed of a series of essays, which discuss the trials of the OS/360 team in the mid-1960s, and uses those experiences to attempt to form a series of more general observations on the art of software development and systems engineering.

Continue reading “The Mythical Man-Month”

Title: The Mythical Man-month
Author: Frederick Phillips Brooks, Frederick P. Brooks, Jr.,
Genre: Computer programming
Publisher: Reading, Mass. ; Don Mills, Ont. : Addison-Wesley Publishing Company
Release Date: 1975
Pages: 195

Deciding when to filter out large scale refactorings from code analysis

I want to be able to see the level of change between OpenStack releases. However, there are a relatively small number of changes with simply huge amounts of delta in them — they’re generally large refactors or the delete which happens when part of a repository is spun out into its own project.

I therefore wanted to explore what was a reasonable size for a change in OpenStack so that I could decide what maximum size to filter away as likely to be a refactor. After playing with a couple of approaches, including just randomly picking a number, it seems the logical way to decide is to simply plot a histogram of the various sizes, and then pick a reasonable place on the curve as the cutoff. Due to the large range of values (from zero lines of change to over a million!), I ended up deciding a logarithmic axis was the way to go.

For the projects listed in the OpenStack compute starter kit reference set, that produces the following histogram:I feel that filtering out commits over 10,000 lines of delta feels justified based on that graph. For reference, the raw histogram buckets are:

Commit size	Count
< 2	25747
< 11	237436
< 101	326314
< 1001	148865
< 10001	16928
< 100001	3277
< 1000001	522
< 10000001	13

A quick summary of OpenStack release tags

I wanted a quick summary of OpenStack git release tags for a talk I am working on, and it turned out to be way more complicated than I expected. I ended up having to compile a table, and then turn that into a code snippet. In case its useful to anyone else, here it is:

Release	Release date	Final release tag
Austin	October 2010	2010.1
Bexar	February 2011	2011.1
Cactus	April 2011	2011.2
Diablo	September 2011	2011.3
Essex	April 2012	2012.1.3
Folsom	September 2012	2012.2.4
Grizzly	April 2013	2013.1.5
Havana	October 2013	2013.2.4
Icehouse	April 2014	2014.1.5
Juno	October 2014	2014.2.4
Kilo	April 2015	2015.1.4
Liberty	October 2015	Glance: 11.0.2 Keystone: 8.1.2 Neutron: 7.2.0 Nova: 12.0.6
Mitaka	April 2016	Glance: 12.0.0 Keystone: 9.3.0 Neutron: 8.4.0 Nova: 13.1.4
Newton	October 2016	Glance: 13.0.0 Keystone: 10.0.3 Neutron: 9.4.1 Nova: 14.1.0
Ocata	February 2017	Glance: 14.0.1 Keystone: 11.0.4 Neutron: 10.0.7 Nova: 15.1.5
Pike	August 2017	Glance: 15.0.2 Keystone: 12.0.3 Neutron: 11.0.8 Nova: 16.1.8
Queens	February 2018	Glance: 16.0.1 Keystone: 13.0.4 Neutron: 12.1.1 Nova: 17.0.13
Rocky	August 2018	Glance: 17.0.1 Keystone: 14.2.0 Neutron: 13.0.7 Nova: 18.3.0
Stein	April 2019	Glance: 18.0.1 Keystone: 15.0.1 Neutron: 14.4.2 Nova: 19.3.2
Train	October 2019	Glance: 19.0.4 Keystone: 16.0.1 Neutron: 15.3.0 Nova: 20.4.1
Ussuri	May 2020	Glance: 20.0.1 Keystone: 17.0.0 Neutron: 16.2.0 Nova: 21.1.1
Victoria	October 2020	Glance: 21.0.0 Keystone: 18.0.0 Neutron: 17.0.0 Nova: 22.0.1

Or in python form for those so inclined:

RELEASE_TAGS = {
    'austin': {'all': '2010.1'},
    'bexar': {'all': '2011.1'},
    'cactus': {'all': '2011.2'},
    'diablo': {'all': '2011.3'},
    'essex': {'all': '2012.1.3'},
    'folsom': {'all': '2012.2.4'},
    'grizzly': {'all': '2013.1.5'},
    'havana': {'all': '2013.2.4'},
    'icehouse': {'all': '2014.1.5'},
    'juno': {'all': '2014.2.4'},
    'kilo': {'all': '2015.1.4'},
    'liberty': {
        'glance': '11.0.2',
        'keystone': '8.1.2',
        'neutron': '7.2.0',
        'nova': '12.0.6'
    },
    'mitaka': {
        'glance': '12.0.0',
        'keystone': '9.3.0',
        'neutron': '8.4.0',
        'nova': '13.1.4'
    },
    'newton': {
        'glance': '13.0.0',
        'keystone': '10.0.3',
        'neutron': '9.4.1',
        'nova': '14.1.0'
    },
    'ocata': {
        'glance': '14.0.1',
        'keystone': '11.0.4',
        'neutron': '10.0.7',
        'nova': '15.1.5'
    },
    'pike': {
        'glance': '15.0.2',
        'keystone': '12.0.3',
        'neutron': '11.0.8',
        'nova': '16.1.8'
    },
    'queens': {
        'glance': '16.0.1',
        'keystone': '13.0.4',
        'neutron': '12.1.1',
        'nova': '17.0.13'
    },
    'rocky': {
        'glance': '17.0.1',
        'keystone': '14.2.0',
        'neutron': '13.0.7',
        'nova': '18.3.0'
    },
    'stein': {
        'glance': '18.0.1',
        'keystone': '15.0.1',
        'neutron': '14.4.2',
        'nova': '19.3.2'
    },
    'train': {
        'glance': '19.0.4',
        'keystone': '16.0.1',
        'neutron': '15.3.0',
        'nova': '20.4.1'
    },
    'ussuri': {
        'glance': '20.0.1',
        'keystone': '17.0.0',
        'neutron': '16.2.0',
        'nova': '21.1.1'
    },
    'victoria': {
        'glance': '21.0.0',
        'keystone': '18.0.0',
        'neutron': '17.0.0',
        'nova': '22.0.1'
    }
}

Rejected talk proposal: Shaken Fist, thought experiments in simpler IaaS clouds

This proposal was submitted for FOSDEM 2021. Given that acceptances were meant to be sent out on 25 December and its basically a week later I think we can assume that its been rejected. I’ve recently been writing up my rejected proposals, partially because I’ve put in the effort to write them and they might be useful elsewhere, but also because I think its important to demonstrate that its not unusual for experienced speakers to be rejected from these events.

OpenStack today is a complicated beast — not only does it try to perform well for large clusters, but it also embraces a diverse set of possible implementations from hypervisors, storage, networking, and more. This was a deliberate tactical choice made by the OpenStack community years ago, forming a so called “Big Tent” for vendors to collaborate in to build Open Source cloud options. It made a lot of sense at the time to be honest. However, OpenStack today finds itself constrained by the large number of permutations it must support, ten years of software and backwards compatability legacy, and a decreasing investment from those same vendors that OpenStack courted so actively.

Shaken Fist makes a series of simplifying assumptions that allow it to achieve a surprisingly large amount in not a lot of code. For example, it supports only one hypervisor, one hypervisor OS, one networking implementation, and lacks an image service. It tries hard to be respectful of compute resources while idle, and as fast as possible to deploy resources when requested — its entirely possible to deploy a new VM and start it booting in less than a second for example (if the boot image is already held in cache). Shaken Fist is likely a good choice for small deployments such as home labs and telco edge applications. It is unlikely to be a good choice for large scale compute however.