Getting Digitech / Bresser 6in1 weather stations working with rtl_433, MQTT, and Home Assistant

Please note: this post is a re-write of a previous post about my Vevor weather station that failed after a few months because of ineffective weather seals on the outdoor unit. Given the poor warranty service from Vevor, I instead went and bought a weather station from a local retailer. In my case a Jaycar Digitech XC-0434, which also appears to be sold online as a Bressser 6in1 weather station.

Now that we’ve defeated QNAP’s slightly broken udev, we can run a Docker container with rtl_433 in it to wire up our Digitech / Bresser 6in1 weather station to Home Assistant via MQTT. First off, we need a Docker container running rtl_433, which assumes you’ve already setup the udev rule mentioned in the previous post, even if you’re not using a QNAP!

(more…)

Continue ReadingGetting Digitech / Bresser 6in1 weather stations working with rtl_433, MQTT, and Home Assistant

FastCDC, puzzlefs, and de-duplicating container and VM images

Since about 2017, a group at Cisco has been working on an “OCI native operating system” under the title “project machine”, which is a terrible project name. I note that most of the people publicly involved in the project according to github commits no longer work at Cisco, so I cannot vouch for the health of the overall project. That said, they did come up with some interesting ideas along the way and given its a quiet time of year I figured I could do some reading.

(more…)

Continue ReadingFastCDC, puzzlefs, and de-duplicating container and VM images

Getting Vevor 7in1 weather stations working with rtl_433, MQTT, and Home Assistant

Please note: my Vevor weather station failed after a few months because of ineffective weather seals on the outdoor unit. Vevor is refusing to provide any sort of warranty on the item, so I would strongly encourage you to consider literally any other vendor’s product… In the end I bought a weather station from Jaycar which works just fine.

Now that we’ve defeated QNAP’s slightly broken udev, we can run a Docker container with rtl_433 in it to wire up our Vevor 7in1 weather station to Home Assistant via MQTT. First off, we need a Docker container running rtl_433, which assumes you’ve already setup the udev rule mentioned in the previous post, even if you’re not using a QNAP!

(more…)

Continue ReadingGetting Vevor 7in1 weather stations working with rtl_433, MQTT, and Home Assistant

Issues building Kolla images with recent versions of Python requests

  • Post author:
  • Post category:OpenStack

If you find yourself having issues building Kolla docker container images with errors like this: INFO:kolla.common.utils:Using engine: docker INFO:kolla.common.utils:Found the container image folder at /home/mikal/src/kerbside-patches/venv-stable/2023.1-patches/share/kolla/docker ERROR:kolla.common.utils:Unable to connect to container engine daemon, exiting INFO:kolla.common.utils:Exception caught: Error while fetching server API version: Not supported URL scheme http+docker It is likely that you're running a recent version of requests which has an unresolved issue. Try downgrading requests manually, like this: $ ...venvpath.../bin/pip install requests==2.31.0 Collecting requests==2.31.0 Downloading requests-2.31.0-py3-none-any.whl (62 kB) ...snip... Successfully installed requests-2.31.0 Thanks for this stack overflow post for the pointer.  

Continue ReadingIssues building Kolla images with recent versions of Python requests

On-demand Container Loading in AWS Lambda

  • Post author:
  • Post category:AWS

My team at work now has a daily personal learning time called “egg time” — its a slightly silly story involving a manager who was good at taking some time to learn things each day, and an egg shaped chair.

Today I decided that I should read this paper about container image loading in AWS lambda, as recommended by Robert Collins on LinkedIn. The paper details the work they had to do to transition from all Lambda functions being packaged as relatively small zip files (250mb), to relatively large Docker containers (10gb+) while maintaining their aggressive target cold-start time of 50ms.

(more…)

Continue ReadingOn-demand Container Loading in AWS Lambda

Interpreting whiteout files in Docker image layers

I've been playing again with Docker images and their internal layers a little more over the last week -- you can see some of my previous adventures at Manipulating Docker images without Docker installed. The general thrust of these adventures is understanding the format and how to manipulate it by building a tool called Occy Strap which can manipulate the format in useful ways. My eventual goal there is to be able to build OCI compliant image bundles and then have a container runtime like runc execute them, and I must say I am getting a lot closer. This time I was interested in the exact mechanisms used by whiteout files in those layers and how that interacts with Linux kernel overlay filesystem types. Firstly, what is a whiteout file? Well, when you delete a file or directory from a lower layer in the Docker image, it doesn't actually get removed from that lower layer, as layers are immutable. Instead, the uppermost layer records that the file or directory has been removed, and it is therefore no longer visible in the Docker image that the container sees. This has obvious security implications if you delete a file like a password you…

Continue ReadingInterpreting whiteout files in Docker image layers

Manipulating Docker images without Docker installed

Recently I've been playing a bit more with Docker images and Docker image repositories. I had in the past written a quick hack to let me extract files from a Docker image, but I wanted to do something a little more mature than that. For example, sometimes you want to download an image from a Docker image repository without using Docker. Naively if you had Docker, you'd do something like this: docker pull busybox docker save busybox However, that assumes that you have Docker installed on the machine downloading the images, and that's sometimes not possible for security reasons. The most obvious example I can think of is airgapped secure environments where you need to walk the data between two networks, and the unclassified network machine doesn't allow administrator access to install Docker. So I wrote a little tool to do image manipulation for me. The tool is called Occy Strap, is written in python, and is available on pypi. That means installing it is relatively simple: python3 -m venv ~/virtualenvs/occystrap . ~/virtualenvs/occystrap/bin/activate pip install occystrap Which doesn't require administrator permissions. There are then a few things we can do with Occy Strap. Downloading an image from a repository and…

Continue ReadingManipulating Docker images without Docker installed

Quick hack: extracting the contents of a Docker image to disk

Hello! Please note I've written a little python tool called Occy Strap which makes this a bit easier, and can do some fancy things around importing and exporting multiple images. You might want to read about it? For various reasons, I wanted to inspect the contents of a Docker image without starting a container. Docker makes it easy to get an image as a tar file, like this: docker save -o foo.tar image But if you extract that tar file you'll find a configuration file and manifest as JSON files, and then a series of tar files, one per image layer. You use the manifest to determine in what order you extract the tar files to build the container filesystem. That's fiddly and annoying. So I wrote this quick python hack to extract an image tarball into a directory on disk that I could inspect: #!/usr/bin/python3 # Call me like this: # docker-image-extract tarfile.tar extracted import tarfile import json import os import sys image_path = sys.argv[1] extracted_path = sys.argv[2] image = tarfile.open(image_path) manifest = json.loads(image.extractfile('manifest.json').read()) for layer in manifest[0]['Layers']: print('Found layer: %s' % layer) layer_tar = tarfile.open(fileobj=image.extractfile(layer)) for tarinfo in layer_tar: print(' ... %s' % tarinfo.name) if tarinfo.isdev(): print(' -->…

Continue ReadingQuick hack: extracting the contents of a Docker image to disk

Juno nova mid-cycle meetup summary: containers

  • Post author:
  • Post category:OpenStack

This is the second in my set of posts discussing the outcomes from the OpenStack nova juno mid-cycle meetup. I want to focus in this post on things related to container technologies. Nova has had container support for a while in the form of libvirt LXC. While it can be argued that this support isn't feature complete and needs more testing, its certainly been around for a while. There is renewed interest in testing libvirt LXC in the gate, and a team at Rackspace appears to be working on this as I write this. We have already seen patches from this team as they fix issues they find on the way. There are no plans to remove libvirt LXC from nova at this time. The plan going forward for LXC tempest testing is to add it as an experimental job, so that people reviewing libvirt changes can request the CI system to test LXC by using "check experimental". This hasn't been implemented yet, but will be advertised when it is ready. Once we've seen good stable results from this experimental check we will talk about promoting it to be a full blown check job in our CI system. We have…

Continue ReadingJuno nova mid-cycle meetup summary: containers

End of content

No more pages to load