Can you run DPDK in a non-privileged container as as non-root user?

user_7292105 0
Viewed 1254

I am trying to run DPDK in a non-privileged docker container. While I can limit the container's privileges and specify the container as non-privileged, I still need to run a dpdk application (say testpmd) as root. I can also run the container as non-root and use sudo to start testpmd.

I was wondering if anyone is able to run dpdk (without the --no-huge option) as non-root user, inside a docker container. If so, are there certain privileges or permissions that need to be granted?

UPDATED:

I'm using DPDK 20.02. I think I've narrowed down the problem to a ulimit -l setting.

From testpmd: EAL: cannot set up DMA remapping, error 12 (Cannot allocate memory)

From Dmesg dmesg: [ 5697911.199003] vfio_pin_pages_remote: RLIMIT_MEMLOCK (65536) exceeded.

In response to Vipin:

Did you need to adjust the limits for the container? if so how? I am using helm to deploy the pods so I'm not sure if I can modify the docker run command, it looks like I would need to edit /etc/security/limits.conf on the host and redeploy.

Also, what did you use to give ownership of the fs? Doesn't having a non-privilieged container prevent you? For testing, I just sudo it, but ultimately I want to be able to drop SETUID/SETGID.

dpdk docker
1 Answers

We can run DPDK on Host or inside docker with non root user.

To run DPDK as non-root user

  1. Create or choose the user without root privelleges
  2. set access to RUNTIME directory value as export XDG_RUNTIME_DIR=/tmp (since all users has access to tmp folder and on certain distros /var/run might not be accessible
  3. Mount the huge pages to similar folder mkdir -p /tmp/mnt/huge; mount -t hugetlbfs nodev /tmp/mnt/huge
  4. assign ownership to user to access the huge page as chown -R [non-root user]:[non-root user] /tmp/mnt/huge
  5. If access to devices are required check the same with either iommu or no-iommu drivers using lsmod | grep vfio
  6. change the ownership of the device chown -R [non-root user]:[non-root user] /dev/vfio/[device id]
  7. user DPDK rte_eal_init option --huge-dir o point to /tmp/mnt/huge
  8. Certian PMD might fail even after step 7, for those use option --legacy-mem this resolves the issue.

In order to run the DPDK application inside a docker, couple more things need to addressed

  1. Use DPDK either 19.11 LTS or greater (there are patches related to docker, namespace, memory limit)
  2. certain SE policy (Linux) does not allow sharing of huge Pages, so use option --in-memory to disable sharing of MMAP to huge pages (this should avoid most of the issues).

Note: assumption made

  1. there is only one single application to run on docker
  2. as mentioned in the comments --privileged -v /sys/bus/pci/drivers:/sys/bus/pci/drivers -v /sys/kernel/mm/hugepages:/sys/kernel/mm/hugepages -v /sys/devices/system/node:/sys/devices/system/node -v /dev:/dev are used to run DPDk in docker with sudo privelleges.
  3. I assume based on the question ulimit -c unlimited cannot be executed also.
  4. If there are multiple dockers running dpdk application always use --file-prefix to distinguish.
  5. I have not tried this with DPDK 21.02, 21.05, 21.08

[EDIT-1] the earlier question that got removed is running DPDK as non root

user_76659720
Related