At Gatewatcher1, we put efforts in making our building system reproducible and working offline, so that we can reduce the risk of supply chain attacks. Some efforts are also made so that our building system run with as few privileges as possible.
One of the few things we were still running as a privileged user recently was the build of our initial Debian root filesystem, for our base system and for our containers.
Indeed, the official Debian Docker container from Docker Hub was not generated
in a way that we can consider secure for our need. It basically downloads a blob
from a web server, does no verification whatsoever of that blob and ships it as
the root filesystem2. Even though the root filesystem they are using
can be rebuilt in a reproducible way, downloading the result from Internet
without verifying it against the expected hash is sort of missing the point of
reproducible builds. Also, debuerreotype uses
debootstrap, which is
problematic in itself, as explained hereafter.
To create such root filesystem, multiple tools are provided by the Debian team,
Multistrap has not been updated in many years3, and suffers from some limitations that were show-stoppers for us, but it is capable to create root filesystems from an unprivileged user without hacks.
On the other hand, deboostrap is not really friendly with the idea of building a system from an unprivileged user.
First of, there is a check to ensure we are running it with UID 04.
This can be bypassed in several documented ways, including using
which overloads some libc calls, using
LD_PRELOAD. An other, less hacky, way
is to run the program in a user namespace.
Unfortunately, this is not sufficient to run
debootstrap, since it performs
another check consisting of trying to create a “/dev/null” node5.
This is more problematic since nodes cannot be created from a user namespace, as
this would create a easy way of escaping the namespace.
As it seems, though, there is a way to build an unprivileged Debian root
filesystem that is even built into deboostrap, using the installation variant
“fakechroot”. Alternate code paths exist in deboostrap when this variant is
selected that side-step some checks, and fake some calls. This variant also adds
a check to ensure this variant is run only if the
fakechroot utility is in
use. Therefore, you are expected to run debootstap as followed, as documented in
# apt update && apt install -y debootstrap fakeroot fakechroot $ fakechroot fakeroot debootstrap --variant=fakechroot bullseye $HOME/rootfs
fakechroot works by overloading some functions with
LD_PRELOAD, and has some
documented limitations regarding symlinks. As it happens, these limitations
include rewrites of absolute symlinks, by prefixing them with the path of the
faked chroot. As a result, within the chroot, you will find links that are
broken when actually chrooting, such as when you would use that directory
hierarchy as a root filesystem on a container or a virtual machine.
$ readlink /path/to/my/chroot/usr/sbin/telinit /path/to/my/chroot/bin/systemctl
fakechroot (this is what you want to see, in a normal system):
$ readlink /path/to/my/chroot/usr/sbin/telinit /bin/systemctl
After some verifications, we decided that it was safe to fake the use of
fakechroot, while using the “fakechroot” installation variant. For this, we
set the environment variable
true, which fakechroot is
supposed to set and which is controlled by
debootstrap to authorize the use of
the “fakechroot” variant. And it worked.
So to build a working root filesystem from an unprivileged user, we are now doing the following:
$ podman unshare # FAKECHROOT=true debootstrap --variant=fakechroot bullseye chroot/ # tar -C chroot/ --exclude=dev/* -czf ./chroot.tgz . # exit $ cat <<EOF > Containerfile FROM scratch ADD chroot.tgz . CMD ["/bin/bash"] EOF $ podman build -f Containerfile
This series of commands builds a Debian container from an unprivileged user. More work needs to be done to achieve offline reproducible builds, of course, but none require hacks like this, thankfully.