Made by Mikal – Page 16

A breadmaker loaf my kids will actually eat

Post author:mikal
Post published:May 10, 2020
Post category:Cooking

My dad asked me to document some of my baking experiments from the recent natural disasters, which I wanted to do anyway so that I could remember the recipes. Its taken me a while to get around to though, because animated GIFs on reddit are a terrible medium for recipe storage, and because I've been distracted with other shiney objects. That said, let's start with the basics -- a breadmaker bread that my kids will actually eat. This recipe took a bunch of iterations to get right over the last year or so, but I'll spare you the long boring details. However, I suspect part of the problem is that the receipe varies by bread maker. Oh, and the salt is really important -- don't skip the salt! Wet ingredients (add first) 1.5 cups of warm water (we have an instantaneous gas hot water system, so I pick 42 degrees) 0.25 cups of oil (I use bran oil) Dry ingredients (add second) I just kind of chuck these in, although I tend to put the non-flour ingredients in a corner together for reasons that I can't explain. 3.5 cups of bakers flour (must be bakers flour, not plain flour) 2…

Exporting volumes from Cinder and re-creating COW layers

Post author:mikal
Post published:April 14, 2020
Post category:Google Google Compute Engine OpenStack

Today I wandered into a bit of a rat hole discovering how to export data from OpenStack Cinder volumes when you don't have admin permissions, and I thought it was worth documenting here so I remember it for next time. Let's assume that you have a Cinder volume named "child1", which is a 64gb volume originally cloned from "parent1". parent1 is a 7.9gb VMDK, but the only way I can find to extract child1 is to convert it to a glance image and then download the entire volume as a raw. Something like this: $ cinder upload-to-image $child1 "extract:$child1" Where $child1 is the UUID of the Cinder volume. You then need to find the UUID of the image in Glance, which the Cinder upload-to-image command will have told you, but you can also find by searching Glance for your image named "extract:$child1": $ glance image-list | grep "extract:$cinder_uuid" You now need to watch that Glance image until the status of the image is "active". It will go through a series of steps with names like "queued", and "uploading" first. Now you can download the image from Glance: $ glance image-download --file images/$child1.raw --progress $glance_uuid And then delete the intermediate glance…

The Calculating Stars

Post author:mikal
Post published:April 6, 2020
Post category:Book

Winner of both a Hugo, Locus and a Nebula, this book is about a mathematical prodigy battling her way into a career as an astronaut in a post-apolocalyptic 1950s America. Along the way she has to take on the embedded sexism of America in the 50s, as well as her own mild racism. Worse, she suffers from an anxiety condition. The book is engaging and well written, with an alternative history plot line which believable and interesting. In fact, its quite topical for our current time. I really enjoyed this book and I will definitely be reading the sequel.

Configuring load balancing and location headers on Google Cloud

Post author:mikal
Post published:March 4, 2020
Post category:Google Google Compute Engine

I have a need at the moment to know where my users are in the world. This helps me to identify what compute resources to serve their request with in order to reduce the latency they experience. So how do you do that thing with Google Cloud? The first step is to setup a series of test backends to send traffic to. I built three regions: Sydney; London; and Los Angeles. It turns out in hindsight that wasn't actually nessesary though -- this would work with a single backend just as well. For my backends I chose a minimal Ubuntu install, running this simple backend HTTP service. I had some initial trouble finding a single page which walked through the setup of the Google Cloud load balancer to do what I wanted, which is the main reason for writing this post. The steps are: Create your test instances and configure the backend on them. I ended up with a setup like this: Next setup instance groups to contain these instances. I chose unmanaged instance groups (that is, I don't want autoscaling). You need to create one per region. But wait! There's one more layer of abstraction. We need a backend…

Interviewing hints (or, so you’ve been laid off…)

Post author:mikal
Post published:February 1, 2020
Post category:Uncategorized

This post is an attempt to collect a set of general hints and tips for resumes and interviews. It is not concrete truth though, like all things this process is subjective and will differ from place to place. It originally started as a Google doc shared around a previous workplace during some layoffs, but it seems more useful than that so I am publishing it publicly. I'd welcome comments if you think it will help others. So something bad happened I have the distinction of having been through layoffs three times now. I think there are some important first steps: Take a deep breath. Hug your loved ones and then go and sweat on something -- take a walk, go to the gym, whatever works for you. Research shows that exercise is a powerful mood stabiliser. Make a plan. Who are you going to apply with? Who could refer you? What do you want to do employment wise? Updating your resume is probably a good first step in that plan. Treat finding a job as your job. You probably can’t do it for eight hours a day, but it should be your primary goal for each “workday”. Have a todo…

Writing a terraform remote state server

Post author:mikal
Post published:January 14, 2020
Post category:Terraform

Terraform is a useful tool for deploying cloud resources. This post isn't an introduction to terraform, so I'll assume you already know and love it. If you want more, then this getting started guide would be a sensible start. At its most basic level, terraform deploys cloud resources and stores information about those resources in a file on local disk called terraform.tfstate -- it needs that state information so it can make later changes to the deployment, be those modifying resources in use or tearing the whole deployment down. If you had an operations team working on an environment, then you could store the tfstate file in git or a shared filesystem so that the entire team could manage the deployment. However, there is nothing with that approach that stops two members of the team making overlapping changes. That's where terraform state servers come in. State servers can implement optional locking, which stops overlapping operations from happening. The protocol that these servers talk isn't well documented (that I could find). I wanted to explore that more, so I wrote a simple terraform HTTP state server in python. To use this state server, configure your terraform file as per demo.tf. The…

Setting up VXLAN between nested virt VMs on Google Compute Engine

Post author:mikal
Post published:January 6, 2020
Post category:Google Google Compute Engine

I wanted to play with a VXLAN mesh between VMs on more than one hypervisor node, but the setup for VXLAN ended up being a separate post because it was a bit long. Read that post first if you want to follow the instructions here. Now that we have a working VXLAN mesh between our two nodes we can move on to installing libvirt (which is called libvirt-daemon-system on Debian, not libvirt-bin as on Ubuntu): sudo apt-get install -y qemu-kvm libvirt-daemon-system sudo virsh net-start default sudo virsh net-autostart --network default I'm going to use a little python helper to launch my VMs, so I need some other dependancies as well: sudo apt-get install -y python3-pip pkg-config libvirt-dev git git clone https://github.com/mikalstill/shakenfist cd shakenfist git checkout 6bfac153d249752b27d224ad9d079095b640498e sudo mkdir /srv/shakenfist sudo cp template.debian.xml /srv/shakenfist/template.xml sudo pip3 install -r requirements.txt Let's launch a quick test VM to make sure the helper works: sudo python3 daemon.py sudo virsh list You can destroy that VM for now, it was just testing the install. sudo virsh destroy ...name... Next we need to tweak the template that shakenfist is using to start instances so that it uses the bridge for networking (that template is the one…

Setting up VXLAN on Google Compute Engine

Post author:mikal
Post published:January 5, 2020
Post category:Google Google Compute Engine

So my ultimate goal here is to try out VXLAN between some VMs on instances in Google compute engine, but today I'm just going to get VXLAN working because that took a fair bit longer than I expected. First off, boot your instances -- because I will need nested virt later I chose two instances on Google Cloud. Please note that you need to do a bit of a dance to turn on nested virt there. I also chose to use Debian for this experiment: gcloud compute instances create vx-1 --zone us-central1-b --min-cpu-platform "Intel Haswell" --image nested-vm-image Now do those standard things you do to all new instances: sudo apt-get update sudo apt-get dist-upgrade -y Now let's setup VXLAN between the two nodes, with a big nod to this web page. First create a VXLAN interface on each machine (if you care about the port your VXLAN traffic is on being to IANA standards, see the postscript at the end of this): sudo ip link add vxlan0 type vxlan id 42 dev eth0 dstport 0 Now we need to put the two nodes into a mesh, where 34.70.161.180 is the IP of the node we are not running this command on…

Playing with the python prometheus query API

Post author:mikal
Post published:January 3, 2020
Post category:Prometheus Python

The last few days have been a bit icky around here, with my house apparently proudly residing in the major city with the dirtiest air in the world. So, I needed a distraction... It has also been quite hot, so I wondered how my energy usage was going. I have prometheus monitoring of my power draw, so now seemed as good a time as any to learn how to do some historical querying over the API. I ended up with a python script which can output things like this: "Yesterday had a maximum temperature of 38 and we used 28.36 kwh. The average for similar days is 25.56 kwh." The code is on github if it is of interest to others. I am sure I could push more of this processing down into the prometheus engine, but I couldn't see how to do it today. Hints welcome!

Coming to grips with Kubernetes in 2020: online training

Post author:mikal
Post published:December 27, 2019
Post category:Kubernetes

There are a few online training resources I've had a play with while learning Kubernetes, so I figure that's worth a quick write up. This is a follow on from my post about Kubernetes podcasts I've tried. I've tried three training providers so far: The Linux Foundation Kubernetes course (LFS258 Kubernetes Fundamentals) is probably the "go to" resource for many people, and is often sold bundled with the certification exams. Unfortunately, it is really terrible. It is by far the worst course I've seen so far. On the other hand, the Linux Academy Kubernetes course is really good. It is flaw is that you have to sign up to Linux Academy, which provides you with all you can eat courses for a rather steep annual fee. Finally, I discovered Mumshad Mannambeth's Udemy courses, and frankly they're excellent. He's put a huge amount of effort into them and it really shows. Even better, with Udemy's regular sales you can pick up his three Kubernetes courses (intro, admin certification, and developer certification) for under $50 AUD. There are even plenty of online quizzes. If I was going to pick a course to try, I'd definitely go with Mumshad.