Docker build fio with librbd

Question

I need to build a docker image of fio with librbd so that I can set the ioengine=rbd. I am building for arch aarch64. When I run fio with ioengine=rbd I get an error saying unable to load librbd. Any help here really appreciated.

Here is my Dockerfile:

# syntax=docker/dockerfile:1
From alpine

RUN apk add --update-cache fio librbd --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community

I build and push my docker image:

$ sudo docker build . -t manjo8/fio
Sending build context to Docker daemon  2.048kB
Step 1/2 : From alpine
 ---> bb3de5531c18
Step 2/2 : RUN apk add --update-cache fio librbd --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community
 ---> Running in 68e44788feb0
fetch http://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/19) Installing libaio (0.3.112-r1)
(2/19) Installing numactl (2.0.14-r0)
(3/19) Installing fio (3.28-r1)
(4/19) Installing libblkid (2.37.2-r0)
(5/19) Installing libbz2 (1.0.8-r1)
(6/19) Installing libgcc (10.3.1_git20210424-r2)
(7/19) Installing xz-libs (5.2.5-r0)
(8/19) Installing boost1.76-iostreams (1.76.0-r0)
(9/19) Installing boost1.76-thread (1.76.0-r0)
(10/19) Installing libstdc++ (10.3.1_git20210424-r2)
(11/19) Installing fmt (7.1.3-r1)
(12/19) Installing eudev-libs (3.2.10-r0)
(13/19) Installing librados (16.2.6-r1)
(14/19) Installing argon2-libs (20190702-r1)
(15/19) Installing device-mapper-libs (2.02.187-r1)
(16/19) Installing json-c (0.15-r1)
(17/19) Installing libuuid (2.37.2-r0)
(18/19) Installing cryptsetup-libs (2.3.6-r0)
(19/19) Installing librbd (16.2.6-r6)
Executing busybox-1.33.1-r3.trigger
OK: 26 MiB in 33 packages
Removing intermediate container 68e44788feb0
 ---> 1534fb8d9308
Successfully built 1534fb8d9308
Successfully tagged manjo8/fio:latest

When I run the docker image I get an error saying its unable to load librdb.

fio-5c6dc65894-bh7dt
fio: file:ioengines.c:127, func=dlopen, error=Error loading shared library rbd: No such file or directory
fio: engine rbd not loadable
fio: failed to load engine

user_530160 · Answer

If you look at the build log for fio, it says that it is building fio without support for Rados Block Device engine:

Rados Block Device engine     no

So, the only thing to do is to build it from source. Using the build script from alpine, plus some googling for how to build fio with rbd support, I came up with this:

From alpine

ENV fio_version=3.28

RUN set -vuo pipefail \
 && apk add --no-cache libaio-dev librados librbd \
 && apk add --no-cache --virtual .build-deps linux-headers zlib-dev coreutils gcc make musl-dev ceph-dev \
 && cd /tmp \
 && wget https://github.com/axboe/fio/archive/fio-$fio_version.tar.gz \
 && tar -xzf fio-$fio_version.tar.gz \
 && rm /tmp/fio-$fio_version.tar.gz \
 && cd /tmp/fio-fio-$fio_version \
 && ./configure  \
     --prefix=/usr \
     --disable-native \
 && make -j \
 && make install \
 && rm -rf /tmp/fio-fio-$fio_version \
 && apk del --no-network .build-deps

I'm only able to test it in a limited way, given that I don't have a ceph cluster, but I'm able to get past the error you were having.

I also spent some effort optimizing the build so it wouldn't waste space by including build-time dependencies into the image. (This is the --virtual .build-deps and apk del .build-deps stages you see above.)

Docker build fio with librbd

1 Answers