AF-XDP: Implement Shared Umem sockets

user_2826018 0
Viewed 917

I want to implement XDP_SHARED_UMEM: https://www.kernel.org/doc/html/latest/networking/af_xdp.html#xdp-shared-umem-bind-flag

The libbpf library function xsk_socket__create (https://github.com/libbpf/libbpf/blob/master/src/xsk.c) checks the xsk_umem->refcount value. In case it is greater than 1, the XDP_SHARED_UMEM option of a struct sockaddr_xdp is set.

So as far as I understand correctly, I "just" need to pass the original umem struct of the socket I want to share the umem with and the rest is done by libbpf.

The way I tried to do it was to let the first process copy its umem-struct into a shared-memory area where the second process could load it from. But because struct xsk_umem is defined in xsk.c it is hidden from the user and I am not able to do something like this:

memcpy(shdm_ptr, umem, sizeof(struct xsk_umem))

I don't know how they expect someone to use the shared umem feature?

xdp-bpf linux c
1 Answers

So this was discussed on the xdp-newbies mailing list. Reporting here for the record.

It is not recommended to go with a multi-process setup as you are trying to do. Björn says:

Note that if you'd like to do a multi-process setup with shared umem, you: need to have a control process that manages the fill/completion rings, and synchronize between the processes, OR re-mmap the fill/completetion ring from the socket owning the umem in multiple processes and synchronize the access to them. Neither is pleasant.

Honestly, not a setup I'd recommend.

And he adds:

Just for completeness; To setup shared umem:

  1. create socket 0 and register the umem to this.
  2. mmap the fr/cr using socket 0
  3. create socket 1, 2, n and refer to socket 0 for the umem.

So, in a multiprocess solution step 3 would be done in separate processes, and step 2 depending on your application. You'd need to pass socket 0 to the other processes and share the umem memory from the process where socket 0 was created. This is pretty much a threaded solution, given all the shared state.

I advice not taking this path.

(Credits to Björn.)

user_37165520
Related