Welcome to my blog.

Have a look at the most recent posts below, or browse the tag cloud on the right. Archives of all posts and all comments are also available.

At Collabora one of the many things we do is build Debian derivatives/overlays for customers on a variety of architectures including 32 bit and 64 bit ARM systems. And just as Debian does, our OBS system builds on native systems rather than emulators.

Luckily with the advent of ARM server systems some years ago building natively for those systems has been a lot less painful than it used to be. For 32 bit ARM we've been relying on Calxeda blade servers, however Calxeda unfortunately tanked ages ago and the hardware is starting to show its age (though luckily Debian Stretch does support it properly, so at least the software is still fresh).

On the 64 bit ARM side, we're running on Gigabyte MP30-AR1 based servers which can run 32 bit arm code (As opposed to e.g. ThunderX based servers which can only run 64 bit code). As such running armhf VMs on them to act as build slaves seems a good choice, but setting that up is a bit more involved than it might appear.

The first pitfall is that there is no standard bootloader or boot firmware available in Debian to boot on the "virt" machine emulated by qemu (I didn't want to use an emulation of a real machine). That also means there is nothing to pick the kernel inside the guest at boot nor something which can e.g. have the guest network boot, which means direct kernel booting needs to be used.

The second pitfall was that the current Debian Stretch armhf kernel isn't built with support for the generic PCI host controller which the qemu virtual machine exposes, which means no storage and no network shows up in the guest. Hopefully that will get solved soonish (Debian bug 864726) and can be in a Stretch update, until then a custom kernel package is required using the patch attach to the bug report is required but I won't go into that any further in this post.

So on the happy assumption that we have a kernel that works, the challenge left is to nicely manage direct kernel loading. Or more specifically, how ensure the hosts boots the kernel the guest has installed via the standard apt tools without having to copy kernels around between guest/host, which essentially comes down to exposing /boot from the guest to the host. The solution we picked is to use qemu's 9pfs support to share a folder from the host and use that as /boot of the guest. For the 9p folder the "mapped" security mode seems needed as the "none" mode seems to get confused by dpkg (Debian bug 864718).

As we're using libvirt as our virtual machine manager the remainder of how to glue it all together will be mostly specific to that.

First step is to install the system, mostly as normal. One can directly boot into the vmlinuz and initrd.gz provided by normal Stretch armhf netboot installer (downloaded into e.g /tmp). The setup overall is straight-forward with a few small tweaks:

  • /srv/armhf-vm-boot is setup to be the 9p shared folder (this should exist and owned by the libvirt-qemu user) that will be used for sharing /boot later
  • the kernel args are setup to setup root= with root partition intended to be used in the VM, adjust for your usage.
  • The image file to use the virtio bus, which doesn't seem the default.

Apart from those tweaks the resulting example command is similar to the one that can be found in the virt-install man-page:

virt-install --name armhf-vm --arch armv7l --memory 512 \
  --disk /srv/armhf-vm.img,bus=virtio
  --filesystem /srv/armhf-vm-boot,virtio-boot,mode=mapped \
  --boot=kernel=/tmp/vmlinuz,initrd=/tmp/initrd.gz,kernel_args="console=ttyAMA0,root=/dev/vda1"

Run through the install as you'd normally would. Towards the end the installer will likely complain it can't figure out how to install a bootloader, which is fine. Just before ending the install/reboot, switch to the shell and copy the /boot/vmlinuz and /boot/initrd.img from the target system to the host in some fashion (e.g. chroot into /target and use scp from the installed system). This is required as the installer doesn't support 9p, but to boot the system an initramfs will be needed with the modules needed to mount the root fs, which is provided by the installed initramfs :). Once that's all moved around, the installer can be finished.

Next, booting the installed system. For that adjust the libvirt config (e.g. using virsh edit and tuning the xml) to use the kernel and initramfs copied from the installer rather then the installer ones. Spool up the VM again and it should happily boot into a freshly installed Debian system.

To finalize on the guest side /boot should be moved onto the shared 9pfs, the fstab entry for the new /boot should look something like:

virtio-boot /boot  9p trans=virtio,version=9p2000.L,x-systemd.automount 0 0

With that setup, it's just a matter of shuffling the files in /boot around to the new filesystem and the guest is done (make sure vmlinuz/initrd.img stay symlinks). Kernel upgrades will work as normal and visible to the host.

Now on the host side there is one extra hoop to jump through, as the guest uses the 9p mapped security model symlinks in the guest will be normal files on the host containing the symlink target. To resolve that one, we've used libvirt's qemu hook support to setup a proper symlink before the guest is started. Below is the script we ended up using as an example (/etc/libvirt/hooks/qemu):

vm=$1
action=$2
bootdir=/srv/${vm}-boot

if [ ${action} != "prepare" ] ; then
  exit 0
fi

if [ ! -d ${bootdir} ] ; then
  exit 0
fi

ln -sf $(basename $(cat ${bootdir}/vmlinuz))  ${bootdir}/virtio-vmlinuz
ln -sf $(basename $(cat ${bootdir}/initrd.img))  ${bootdir}/virtio-initrd.img

With that in place, we can simply point the libvirt definition to use /srv/${vm}-boot/virtio-{vmlinuz,initrd.img} as the kernel/initramfs for the machine and it'll automatically get the latest kernel/initramfs as installed by the guest when the VM is started.

Just one final rough edge remains, when doing reboot from the VM libvirt leaves qemu to handle that rather than restarting qemu. This unfortunately means a reboot won't pick up a new kernel if any, for now we've solved this by configuring libvirt to stop the VM on reboot instead. As we typically only reboot VMs on kernel (security) upgrades, while a bit tedious, this avoid rebooting with an older kernel/initramfs than intended.

Posted Wed Jun 14 15:08:05 2017 Tags:

Apart from being somewhat slow, one of the downsides of the original Raspberry Pi SoC was that it had an old ARM11 core which implements the ARMv6 architecture. This was particularly unfortunate as most common distributions (Debian, Ubuntu, Fedora, etc) standardized on the ARMv7-A architecture as a minimum for their ARM hardfloat ports. Which is one of the reasons for Raspbian and the various other RPI specific distributions.

Happily, with the new Raspberry Pi 2 using Cortex-A7 Cores (which implement the ARMv7-A architecture) this issue is out of the way, which means that a a standard Debian hardfloat userland will run just fine. So the obvious first thing to do when an RPI 2 appeared on my desk was to put together a quick Debian Jessie image for it.

The result of which can be found at: https://images.collabora.co.uk/rpi2/

Login as root with password debian (Obviously do change the password and create a normal user after booting). The image is 3G, so should fit on any SD card marketed as 4G or bigger. Using bmap-tools for flashing is recommended, otherwise you'll be waiting for 2.5G of zeros to be written to the card, which tends to be rather boring. Note that the image is really basic and will just get you to a login prompt on either serial or hdmi, batteries are very much not included, but can be apt-getted :).

Technically, this image is simply a Debian Jessie debootstrap with a extra packages for hardware support. Unlike Raspbian the first partition (which contains the firmware & kernel files to boot the system) is mounted on /boot/firmware rather then on /boot. This is because the VideoCore expects the first partition to be a FAT filesystem, but mounting FAT on /boot really doesn't work right on Debian systems as it contains files managed by dpkg (e.g. the kernel package) which requires a POSIX compatible filesystem. Essentially the same reason why Debian is using /boot/efi for the ESP partition on Intel systems rather the mounting it on /boot directly.

For reference, the RPI2 specific packages in this image are from https://repositories.collabora.co.uk/debian/ in the jessie distribution and rpi2 component (this repository is enabled by default on the image). The relevant packages there are:

  • linux: Current 3.18 based package from Debian experimental (3.18.5-1~exp1 at the time of this writing) with a stack of patches on top from the raspberrypi github repository and tweaked to build an rpi2 flavour as the patchset isn't multiplatform capable :(
  • raspberrypi-firmware-nokernel: Firmware package and misc libraries packages taken from Raspbian, with a slight tweak to install in /boot/firmware rather then /boot.
  • flash-kernel: Current flash-kernel package from debian experimental, with a small addition to detect the RPI 2 and "flash" the kernel to /boot/firmware/kernel7.img (which is what the GPU will try to boot on this board).

For the future, it would be nice to see the Raspberry Pi 2 support out of the box on Debian. For that to happen, the most important thing would be to have some mainline kernel support for this board (supporting multiplatform!) so it can be build as part of debians armmp kernel flavour. And ideally, having the firmware load a bootloader (such as u-boot) rather than a kernel directly to allow for a much more flexible boot sequence and support for using an initramfs (u-boot has some support for the original Raspberry Pi, so adding Raspberry Pi 2 support should hopefully not be too tricky)

Update: An updated image (20150705) is available with the latest packages from Jessie and a GPG key that's not expired :).

Posted Tue Feb 3 10:47:08 2015 Tags:

It appears I didn't get around to make any blogposts this year, so to fix that and given that that the internet can always use more pictures of cats:

Hacker cat

What Zappa was working on will likely always remain a mystery.

Posted Wed Jul 11 21:55:41 2012 Tags:

One of the more exciting upcoming features of Farsight is the ability to change the amount of required bandwidth depending on network conditions. Which means Telepathy based VoIP clients will be able to use higher resolutions and better audio/video quality if the network connectivity allows. And ofcourse, when the available bandwidth is low, the bandwidth usage can be lowered such that calls are still possible with lower but hopefully still acceptable quality.

One side of the full story is measuring the available bandwidth, which hopefully Olivier will tell at some point. The other part is actually using less bandwidth. In case technical details are not that interesting to you, probably best to stop reading at this point..

An important way to control the bandwidth we're using in a call is setting the target bitrate on the encoding GStreamer element. While previously it was enough for us to set the bitrate before encoding started, we now need to be able to change it at runtime. And not only that, the codec implementation should ideally respond within a few frames and have a reasonable constant per-frame size.

All in all, a lot of requirements on codecs we didn't really have before. Which usually means we need to retest the various codecs we often use and make sure we configure them correctly for the behaviour we'd like to see (and fix them if necessary). To do this I've written a small test program, which simply changes the target bitrate over time and outputs some numbers suitable to feed into gnuplot to turn into graphs.

For example, when using x264enc with the current default farsight settings a part of the resulting graph looks something like this (full graph here):

x264enc bitrate scaling

The blue impulses is the per-frame size (scaled), the green line the bitrate average over 10 frames and the red line the target bitrate. Clearly this isn't an ideal picture, we will need to do some more tweaks to our default settings to make x264 behave the way we want. The same will be true for all other codecs and codec elements, at the moment the GStreamer VP8 element doesn't handle run-time bitrate changes at all and the Theora element acts it its very own interesting way (graph here)

Posted Mon Apr 18 17:47:01 2011 Tags:

While trying the new Meego Tablet preview today the cat of some of my friends was nice enough to provide support for the ExoPC during the initial installation..

Meego cat support

I can recommend some feline assistance whenever installing things on an ExoPC.

Posted Sat Apr 2 00:05:20 2011 Tags:

Sometimes life has its own way of telling you that you should have just stayed in bed. For instance the following might happen when you're cycling back after doing some sunday afternoon groceries.

Rear tire piercing

Oh well, guess it's time for a nice walk to get some fresh new inner tubes.

Posted Sun Feb 13 17:08:28 2011 Tags:

A long time ago, 2006 according to git, I started helping out on the Pulseaudio packaging team. Didn't have much time back then, but that was fine as there was a quite active maintainer involved in the Team. Unfortunately for various reasons he no longer had time for his Debian work, since, oh around end of 2008. With myself not gaining more time (quite the contrary), that means the pulseaudio packaging hasn't had a lot of love for quite some time...

Updating the packaging for the lastest Pulseaudio release last weekend reminded me (again) that some fresh blood in pkg-pulseaudio would really be quite welcome. So for anyone that's looking for some nice new challenges, is interested in how pulseaudio and audio on linux in general works, and would like to help integrating pulseaudio in Debian even better please let us know and come join the team :)

Posted Tue Dec 7 03:30:27 2010 Tags:

Since some time the good people at NlNet have been sponsoring Collabora to add multi-party audio/video calling to XMPP and do an implementation of this in Telepathy. This project is otherwise known as Muji.

This work has resulted in:

  • an experimental XMPP extension called Muji
  • a new and more modern telepathy API for doing calls (currently still in draft), which allows multi-party calls as well as adding the possibility to support call forking (ringing multiple remote devices, call goes to the one that's picked up first).
  • An implementation of both of these in telepathy-gabble
  • A small telepathy client for testing and demo purposes.

To proof it actually works, a nice screenshot of the demo client in a 4-way call:

4-way Muji Call

The screenshot is featuring myself in Collabora's UK office (top-left), Robert McQueen from a hotelroom in Taipei (top-right), Will Thompson and Magical trevor also from the Collabora UK office (bottom-left) and Mike Ruprecht from somewhere in Missouri. So we were quite spread out over our little planet. Both audio and video quality was quite good, even though the network in robs hotel was a bit flaky from time to time.

In case you want to try it out have a look at the MujiDemoClient page.

Posted Mon Jun 7 14:38:08 2010 Tags:

Yesterday Google announced the WebM Project, giving the world a new open and royalty-free video codec called VP8. Thanks to Collabora Multimedia and Entropy Wave support for it landed in gstreamer git shortly after the announcement.

In Telepathy we quite like video conferencing, so as a logical next step i quickly put together some proof of concept RTP payloading elements for Gstreamer with the obvious result: A call with tux

The screenshot isn't very exciting, which is exactly how it should be. In the world i would like to live in people don't have to care about video codecs, whether it is for video calls, playing online media or whatever else they want to do with video. Looking at the list of supporters for the WebM project this may actually happen and that in my opinion is much is actually the most important aspect of it all.

The most important aspect that is missing at this point to bring VP8 as the video calling codec to the world is the lack of a standard RTP specification. Which is something we will hopefully start working on with various other parties in the WebM project reasonably soon. For now if you want to feel cutting edge get Empathy from git master and my rtp elements. To give things a first try :)

A Belorussian translation of this article is available here

Posted Thu May 20 20:13:31 2010 Tags:

After various people have nagged me for ages in one way or another , i've finally setup my blog :)

Based on ikiwiki with some bits and pieces of setup stolen from smcv.

Hopefully some actually interesting stuff will appear here at some point...

Posted Sun Jan 17 12:39:39 2010 Tags:

This blog is powered by ikiwiki.