How do I make multi-arch Docker images?

I've created a Dockerfile for a CLI utility, and it works great on my Linux laptop. I tried running the container on a Raspberry Pi with Docker installed, but it won't work.

I know myriad companies create images to support multiple architectures, how are they doing this?

I want to know, because I want the container for the CLI to be deployable to a Raspberry Pi, as well as my Linux laptop.

> I know other companies create images to support multiple architectures, how are they doing this?

The quick answer is that it is NOT possible to create a single image that will support multiple architectures. However, it appears as if one image will satisfy multiple architectures because you are actually interacting with a manifest file. A manifest file acts as a pointer to the correct image based on the Docker client that requested the image.

For example, you can run docker build on an x86_64 platform (e.g. your Linux laptop) and again on an armv7l platform (e.g. a Raspberry Pi), and two completely different images will be created. The resulting images should be given platform specific names and pushed to your Docker image repository (e.g. hub.docker.com).

Once that has been completed, then you will need to enable experimental mode in the Docker CLI, and create and upload a manifest file with a platform neutral name that points to the corresponding platform specific images.

TL;DR These three commands will do the trick (assuming you are working from the x86_64 machine)

docker manifest create <new-list-ref-name> <image-ref> [<image-ref>...]
docker manifest annotate <new-list-ref-name> <image-ref> --os linux --arch arm
docker manifest push <new-list-ref-name> [--purge]

NOTE: Depending on whether or not you are using a multi-arch base image, you may not need the docker manifest annotate step.

NOTE: When pushing a manifest file (i.e. docker manifest push ...), be sure to use the --purge option; it is not discussed in the linked articles. Not using --purge, will cause you heartache if you plan on using a tag that will be reused at a later time (e.g. latest).

buildx can be used to generate a multi-arch manifest list and images that are pushed to a registry. When you pull that from the registry, the appropriate manifest for your platform is retrieved.

This is available in 19.03 as a CLI plugin that is available when the CLI is in experimental mode. You can enable this by:

export DOCKER_CLI_EXPERIMENTAL=enabled

For a multi-node builder environment, configure your builder instance with docker buildx create, using the --append option to add more nodes to the builder. Then docker buildx use on that builder to use the instance for your commands.

For building on a single node with emulation, that can be done on debian based systems by installing the latest qemu-user-static (package name may vary for other Linux distributions), e.g.:

apt-get install -t testing qemu-user-static

For an explanation of why a newer release is needed, see:

• https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=868030
• https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1815100

Once you have qemu installed with binfmt_misc setup with the F flag in /proc/sys/fs/binfmt_misc/qemu-*, then you can cross compile on your local host using:

docker buildx create --driver docker-container --name local \
  --use unix:///var/run/docker.sock
docker buildx inspect --bootstrap local

You then build the multi-architecture image with:

docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 \
  -t $registry_img_and_tag --output type=registry .

Include each platform you want to support with --platform.

I've got a presentation covering this. Use P to see the presenter notes. Using FROM --platform=$TARGETPLATFROM is useful to avoid needing to pull fresh images for other platforms.