linux.conf.au should rethink how they present hardware content at linux.conf.au

Share

I was originally going to pursue this as a bit of a project in 2019, but that’s no longer possible as I resigned from the linux.conf.au papers committee late on Thursday to spend more time with my family. So instead I’ll just throw this out there and maybe something will come of it, but also maybe not.

Should hardware events such as the Open Hardware miniconf or this year’s Home Automation workshop be part of linux.conf.au? Certainly they’re very popular, often selling out and having to maintain waiting lists in case someone cancels.

Let me take you on an journey and explain why is hard to take hands on hardware content for the conference… I feel qualified to have an opinion on all this having served on the LCA papers committee since 2004, and having been on a conference core team twice.

Complexity

Hardware assembly sessions are much more complicated to run well than a conference talk. While we expect conference speakers to be well researched and have depth in their field, we don’t expect them to perform flawlessly in an environment with a lot of variables. A static slide deck or maybe a demo or two is all we ask of them.

On the other hand, we do expect hardware assembly sessions to take a group of people with varying skill levels, and get them to assemble non-trivial hardware and have it work at the end. These devices have now reached a level of complexity where they have surface mount components, microprocessors equivalent to to the one in your phone, and software stacks often as large as that on your laptop.

Under recognition of the number of people involved

I haven’t asked the Open Hardware miniconf how many people hours were spent preparing for this year, but I am sure it was a lot. I know that Alastair D’Silva spent a truly huge amount of time on his preparation — think all his spare time for several months while having a small child.

How do we reward that preparation effort? With a single conference ticket. This discourages there from being a team contributing to the project, because at the end of the day only one person can be rewarded with a discount to LCA. This happens because the conference has budget concerns, and can’t give away 20 free tickets to a miniconf without running the risk of failing to break even. It is also because the assumption in the past has been that miniconfs are “conference light” and require less preparation than a main conference talk, and therefore the reduced subsidy for the event was justified.

Lead time

The conference papers review process is fairly complicated these days, with a lot of effort put into promoting the event so that we get the widest range of speakers possible, as well as a diverse review team attempting to ensure that we have good representation of the topic areas you expect to have covered at LCA, as well as diversity in our speaker population. This means that while we normally advertise our call for papers in July, we don’t normally inform speakers that they have been accepted until September.

So, we have complicated hardware projects with a lot of variables, and we give the presenters three months runway to prepare.

One of the obvious ways to lengthen the runway is to just assume your proposal will be accepted, and start preparing earlier. However, that brings me to my next concern…

Out of pocket expenses

Let’s assume you start preparing for your hardware event earlier than September. Let’s also assume that because your hardware is a little complicated and involves surface mount components, that you’ll go through a few prototype rounds and get the final board partially fabricated professionally. This is how things work at the moment.

Guess who gets to pay for that? The organisers of the event. One of those organisers this year was personally out of pocket over $10,000. If the event doesn’t go ahead, or if no one registers for it, then that money is gone. That doesn’t strike me as fair when Linux Australia has $800,000 in the bank at the moment.

The registration process is a nightmare

I don’t know if you noticed, but the registration process for hardware events is also a mess. Again, its up to the individuals running the event to collect money from participants and ensure that there is a fair registration process. The first indications of the level of interest from conference attendees are in the few weeks before the conference itself, when its way too late to course correct if that’s needed.

This is harder than it looks, and comes at a time when the team is flat out ensuring the event runs at all. Worse, then people get annoyed and complain.

Finally on registration, the events are too expensive. In an attempt to simplify registration, there’s only one option — buy the hardware. However, if you already own a Raspberry Pi, why would you buy another one? That’s an extra $60 or so that you’re charged for hardware that you probably don’t really want. If you do want another Raspberry Pi great, but it should be an add on to the event registration.

In summary

Conference organisers generally phrase things in terms of risk versus reward when trying to run a conference. So, the hardware events we have at LCA are high reward for those who attend them, but hugely risky for the conference. Failure of such an event would reflect poorly on LCA, which has worked for 20 years to have a reputation as one of the best technical conferences in the world.

So this is why we shouldn’t run these hardware events — they’re risky, as well as being a super hard slog for the people running them.

But…

But… They’re also hugely popular and an important part of what LCA has become. In a way, LCA is what it is because it embraces a diversity of interests. Without the hardware events, LCA would be diminished from what we have today.

So instead…

If we can’t accept not having these hardware events, we’re only really left with one other option. Let’s blow up the status quo and find a better way of running them.

That’s what originally wrote this post to do — propose something concrete that minimises the impact on the conference organising team while reducing the risk of these events failing and hopefully making life easier for the event organisers.

So here’s Michael’s simple plan for hardware events at LCA:

There should be a separate call for hardware content from now on, for a specific small number of hardware slots at the conference. This should be done really really early. Like stupidly early. As in, opening next month for 2020 and closing in in March. One of the things which holds the call for papers for the main conference up is having the conference web site ready. So let’s not do that — we can use a Google form or something instead. The number of proposals is going to be small, and we can use a “higher touch” process for the review and approval process.

Linux Australia should provide fiscal sponsorship for seed funding to accepted events, based on a proposed bill of materials that is submitted as part of the proposal, on similar terms as is provided for LCA itself and other conference. In other words, Linux Australia should bear the financial risk of failure. Linux Australia should also directly fund conference tickets for hardware event organisers — obviously, we need to somehow validate how many people actually did some work and there need to be limits, but that can be worked through without too much pain.

The LCA website itself should offer registration for hardware events. I see this as relatively trivial in that we can just treat these events as new types of t-shirts and the code to handle that has been around for a really long time.

And finally, we as a community should recognise better the huge amount of effort that has gone into these events in the past (and hopefully in the future) and make a point of thanking the organisers as often as possible. Sometimes there might be warts, but these are human beings like most of us working hard to do something complicated and exciting while still holding down day jobs.

As I said at the start, I had intended to have a go at fixing these issues this year, but events have overtaken me and that will no longer be possible. Hopefully someone else will have a go.

Share

Support for Raspberry Pi and Orange Pi GPIOs in Home Assistant

Share

So, I’ve been off in the GPIO library salt mines for a while, but am now ready to circle back and document how to get GPIO inputs and outputs working in Home Assistant. This now works on both Raspberry Pi and OrangePi, assuming that my patch gets merged.

First off, let’s talk about GPIO outputs. This is something which has been working for a while on both platforms (a while being a week or so, assuming you’ve patched Home Assistant with my pull request, but you’re all doing that right?).

To configure an output in Home Assistant, you would add the following to configuration.yaml:

rpi_gpio:
  board_family: orange_pi

switch:
 - platform: rpi_gpio
   ports:
     PA7: LED

Where board_family can be either “raspberry_pi” or “orange_pi”. Note that for Raspberry Pis, the pin numbers are always numbers whereas for OrangePi we are using “SUNXI” numbering, which is of the form “PA7”.

The circuit for this LED is really simple:

A simple LED circuit

Now we have a switch we can control in Home Assistant:

Raspberry Pi LED switch in Home Assistant

GPIO inputs are similar. The configuration looks like this:

rpi_gpio:
  board_family: orange_pi

binary_sensor:
 - platform: rpi_gpio
   invert_logic: true
   ports:
     PA7: PUSHYBUTTON

With a circuit like this:

A circuit with a button in it

invert_logic set to true is required because our circuit sends the value of PA7 to ground when the button is pressed.

A push button being pressed in Home AssistantNoting that sensors look different to switches in Home Assistant, you can see the binary sensor at the top right of the image, with its history being displayed in the dialog box in the foreground.

Share

Updated examples for OrangePi GPIOs

Share

As part of working through adding OrangePi support to Home Assistant, Alastair and I decided to change to a different GPIO library for OrangePi to avoid the requirement for Home Assistant to have access to /dev/mem.

I just realised that I hadn’t posted updated examples of how to do GPIO output with the new library. So here’s a quick post about that.

Assuming that we have an LED on GPIO PA7, which is pin 29, then the code to blink the LED would look like this with the new library:

import OPi.GPIO as GPIO
import time


# Note that we use SUNXI mappings here because its way less confusing than
# board mappsings. For example, these are all the same pin:
# sunxi: PA7 (the label on the board)
# board: 29
# gpio:  7

GPIO.setmode(GPIO.SUNXI)
GPIO.setwarnings(False)
GPIO.setup('PA7', GPIO.OUT)

while True:
    GPIO.output('PA7', GPIO.HIGH)
    time.sleep(1)
    GPIO.output('PA7', GPIO.LOW)
    time.sleep(1)

The most important thing there is the note about SUNXI pin mappings. I find the whole mapping scheme hugely confusing, unless you use SUNXI and then its all fine. So learn from my fail people!

What about input? Well, that’s not too bad either. Let’s assume that you have a button in a circuit like this:

A circuit with a button in it

The to read the button the polling way, you’d just do this:

import OPi.GPIO as GPIO
import time

GPIO.setmode(GPIO.SUNXI)
GPIO.setwarnings(False)
GPIO.setup('PA7', GPIO.IN, pull_up_down=GPIO.PUD_DOWN)

while True:
    print('Reading...')
    if GPIO.input('PA7') == GPIO.LOW:
        print('Pressed')
    else:
        print('Released')
    time.sleep(1)

Let’s pretend it didn’t take me ages to get that to work right because I had the circuit wrong, ok?

Now, we have self respect, so you wouldn’t actually poll like that. Instead you’d use edge detection, and end up with code like this:

import OPi.GPIO as GPIO
import time

GPIO.setmode(GPIO.SUNXI)
GPIO.setwarnings(False)
GPIO.setup('PA7', GPIO.IN, pull_up_down=GPIO.PUD_DOWN)

def event_callback(channel):
    print('Event detected: %s' % GPIO.input('PA7'))
    
GPIO.add_event_detect('PA7', GPIO.BOTH, callback=event_callback, bouncetime=50)

while True:
    time.sleep(1)

So there ya go.

Share

GPIO inputs on Raspberry Pi

Share

Now that I have GPIO outputs working nicely for Home Assistant using either a Raspberry Pi or an Orange Pi, I want to get GPIO inputs working as well. Naively, that’s pretty easy to do in python on the Raspberry Pi:

import RPi.GPIO as GPIO
import time

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(17, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)

while True:
    print('Reading...')
    if GPIO.input(17) == GPIO.HIGH:
        print('Pressed')
    else:
        print('Released')
    time.sleep(1)

That code is of course horrid. Its horrid because its polling the state of the button, and its quite likely that I can sneak a button press in during one of those sleeps and it will never be noticed. Instead we can use edge detection callbacks to be informed of button presses as they happen:

import RPi.GPIO as GPIO
import time

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(17, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)

def event_callback(channel):
    print('Event detected: %s' % GPIO.input(17))
    
GPIO.add_event_detect(17, GPIO.BOTH, callback=event_callback, bouncetime=50)

while True:
    time.sleep(1)

This second program provides helpful output like this:

pi@raspberrypi:~ $ python gpio_button_edge_detect.py 
Event detected: 1
Event detected: 0

Which is me pressing the button once (it go high when pressed, and then low again when released). This is of course with a button wired to GPIO17 with a current limiting resistor between the button and the 3.3v rail.

Share

Pull Requests for the LCA2019 Home Automation tutorial

Share

A quick list of things I did for the LCA2019 Home Automation tutorial. Of course Alistair did a lot more, but I still want to track these.

Share

Further adventures in Home Assistant OrangePi GPIO

Share

Its funny how a single sentence can change your course. In the last post about this work, I said:

We also need to run hass as root,  because OrangePi GPIO support requires access to /dev/mem for reasons I haven’t dug into just yet.

That’s turned out to be (reasonably) a pretty big sticking point upstream. Access to /dev/mem gives you a whole bunch of access to the machine that Home Assistant probably shouldn’t have.

Alastair went off spelunking because he’s more patient than me and found yet another OrangePi GPIO library. I think we’re up to three or four of these at the moment, but this is the first one we’ve found which supports the sysfs interface to GPIO pins. That’s exciting because it removes our run-as-root requirement. Its unexciting in that the sysfs interface has been deprecated by the kernel, but will remain supported for a while.

I think people would be within their rights to conclude that the state of GPIO libraries for OrangePi is a bit of a dumpster fire right now.

Anyways, the point of this post is mostly to write down how to use the sysfs interface to GPIO pins so that I can remember it later, I’ll take more about this new library and if it meets our needs in a later post.

The first step is to determine what pin number the GPIO pin is. On the OrangePi these are labelled with names like “PA7”, which is the 7th bit in the “A” GPIO register. To convert that into a pin number as used by sysfs you do this:

def pin_number(letter, digit):
    return (ord(letter) - ord('A')) * 32 + digit

So, pin_number(‘A’, 7) for PA7 is just … 7.

Note that I now recommend that people use SUNXI pin mapping, as its much less confusing. You can read more about alternative pin mappings in this post of worked OrangePi GPIO examples.

Now we can enable the pin, set it to output, and then blink the LED to prove it works:

# cd /sys/class/gpio
# echo 7 > export
# cd gpio7
# echo "out" > direction
# echo 1 > value
# sleep 1
# echo 0 > value

The next step? To make sure that the new GPIO library supports sysfs access to GPIOs on the OrangePi Prime.

Share

Adventures in Home Assistant Raspberry Pi GPIO

Share

Alastair D’Silva is running what looks to be a very well prepared home automation tutorial at LCA2019 based on Home Assistant. I offered to have a hack on the support for GPIO pins on OrangePi boards in Home Assistant because it sounded interesting for a vacation week. The only catch being that I’d never done anything with GPIO pins at all on either Raspberry Pi or Orange Pi.

A simple LED circuit for a Raspberry PiThe first step seemed to be to get GPIO working at all on a Raspberry Pi (which is currently supported out of the box with Home Assistant). This online tutorial has a simple example of a circuit and the associated python code to blink a LED on a Raspberry Pi, so off I went to build that circuit.

The circuit has a LED with a 330 ohm pull up resistor on GPIO pin 18 on the board. The sample python code on the page above just blinks that LED, which I used to make sure that the circuit as working as intended.

To configure the GPIO pin as a switch in Home Assistant, I added the following to configuration.yaml (noting that the empty rpi_gpio entry isn’t strictly required, but will be later):

rpi_gpio:

switch:
 - platform: rpi_gpio
   ports:
     18: LED

Which leaves me with something like this in the web UI:

Raspberry Pi LED switch in Home Assistant

It even works!

I’ve lied to you a little bit above, for which I apologise. I’ve also been working on helping Alastair with adding Orange Pi to the rpi_gpio component in Home Assistant, as the tutorial is based on Orange Pi Primes, with a custom home automation shield installed. Now that I have a sample configuration that works for Raspberry Pi and a test circuit, its time to make sure that Orange Pi works correctly too.

Home Assistant doesn’t currently have any support for Orange Pi GPIOs. The first approach I took was to forward port this ancient patch which adds Orange Pis as a new component beside Raspberry Pis. That port is available here, but in the end I decided it would be nicer to just have the existing Raspberry Pi component also support Orange Pis, instead of duplicating a whole bunch of code and adding some confusion.

(It should be noted that there are downsides to this new approach — the code is more complicated this way, and Raspberry Pi owners need to download the Orange Pi GPIO library even though they’ll never use it. That said, I see these downsides as relatively minor).

GPIO pin mapping for an Orange PiA small hitch however. Orange Pi names the GPIO ports in a quite different way from how Raspberry Pi does, and this took some time to get used to. The mapping of GPIO pins was a little hard to find, so I’ll include it here (the image to the left). A second hitch was that I needed a linux image for the board. I’ve used Armbian Stretch, as the hass.io image is quite locked down (no ssh to the base OS for example).

Based on the pin image instead of the Pin 18 from the previous example, I moved to what is labelled on the tutorial shield as “PA7”, and which is referred to in code as Pin 29.

The code for blinking is a bit different from the example linked above, so here is a tweaked version:

import OPi.GPIO as GPIO
import time


GPIO.setboard(GPIO.PRIME)
GPIO.setmode(GPIO.BOARD)
GPIO.setwarnings(False)
GPIO.setup(29, GPIO.OUT)

while True:
    GPIO.output(29, GPIO.HIGH)
    time.sleep(1)
    GPIO.output(29, GPIO.LOW)
    time.sleep(1)

Note here that we need to specify what board we’re on (in this case a Prime), and we set the mode differently than the linked example.

So now let’s be over achievers and get things working in Home Assistant too! We need a configuration.yaml which includes something like this:

rpi_gpio:
  board_family: orange_pi
  board: prime

switch:
 - platform: rpi_gpio
   ports:
     29: LED

Note the additional config in the rpi_gpio entry. We also need to run hass as root,  because OrangePi GPIO support requires access to /dev/mem for reasons I haven’t dug into just yet.

OrangePi GPIO support currently requires a patch to Home Assistant, which you can find at a github branch.

Share

Learning from the mistakes that even big projects make

Share

The following is a blog post version of a talk presented at pyconau 2018. Slides for the presentation can be found here (as Microsoft powerpoint, or as PDF), and a video of the talk (thanks NextDayVideo!) is below:

 

OpenStack is an orchestration system for setting up virtual machines and associated other virtual resources such as networks and storage on clusters of computers. At a high level, OpenStack is just configuring existing facilities of the host operating system — there isn’t really a lot of difference between OpenStack and a room full of system admins frantically resolving tickets requesting virtual machines be setup. The only real difference is scale and predictability.

To do its job, OpenStack needs to be able to manipulate parts of the operating system which are normally reserved for administrative users. This talk is the story of how OpenStack has done that thing over time, what we learnt along the way, and what I’d do differently if I had my time again. Lots of systems need to do these things, so even if you never use OpenStack hopefully there are things to be learnt here.

Continue reading Learning from the mistakes that even big projects make

Share

The last week for linux.conf.au 2019 proposals!

Share
Dear humans of the Internet — there is ONE WEEK LEFT to propose talks for linux.conf.au 2019. LCA is one of the world’s best open source conferences, and we’d love to hear you speak!
 
Unsure what to propose? Not sure if your talk is what the conference would normally take? Just want a chat? You’re welcome to reach out to papers-chair@linux.org.au to talk things through.
 
Share

pyconau 2018 call for proposals now open

Share

The pyconau call for proposals is now open, and runs until 28 May. I took my teenagers to pyconau last year and they greatly enjoyed it. I hadn’t been to a pyconau in ages, and ended up really enjoying thinking about things from topic areas I don’t normally need to think about. I think expanding one’s horizons is generally a good idea.

Should I propose something for this year? I am unsure. Some random ideas that immediately spring to mind:

  • something about privsep: I think a generalised way to make privileged calls in unprivileged code is quite interesting, especially in a language which is often used for systems management and integration tasks. That said, perhaps its too OpenStacky given how disinterested in OpenStack talks most python people seem to be.
  • nova-warts: for a long time my hobby has been cleaning up historical mistakes made in OpenStack Nova that wont ever rate as a major feature change. What lessons can other projects learn from a well funded and heavily staffed project that still thought that exec() was a great way to do important work? There’s definitely an overlap with the privsep talk above, but this would be more general.
  • a talk about how I had to manage some code which only worked in python2, and some other code that only worked in python3 and in the end gave up on venvs and decided that Docker containers are like the ultimate venvs. That said, I suspect this is old hat and was obvious to everyone except me.
  • something else I haven’t though of.

Anyways, I’m undecided. Comments welcome.

Also, here’s an image for this post. Its the stone henge we found at Guerilla Bay last weekend. I assume its in frequent use for tiny tiny druids.

Share