How to reliably detect dropped netlink requests or responses

user_1505451 0
Viewed 993

I'm interested in using netlink for a straightforward application (reading cgroup stats at high frequency).

The man page cautions that the protocol is not reliable, hinting that the application needs to be prepared to handle dropped packets:

However, reliable transmissions from kernel to user are impossible in any case. The kernel can't send a netlink message if the socket buffer is full: the message will be dropped and the kernel and the user-space process will no longer have the same view of kernel state. It is up to the application to detect when this happens (via the ENOBUFS error returned by recvmsg(2)) and resynchronize.

Since my requirements are simple, I'm fine with just destroying the socket and creating a new one whenever anything unexpected happens. But I can't find any documentation on what the expectations are on my program—the man page for recvmsg(2) doesn't even mention ENOBUFS for example.

What all do I need to worry about in order to make sure I can tell that a request from my application or a response from the kernel has been dropped, so that I can reset everything and start over? It's clear to me that I could do so whenever I receive an error from any of the syscalls involved, but for example what happens if my request is dropped on the way to the kernel? Will I just never receive a response? Do I need to build a timeout mechanism where I wait only so long for a response?

netlink linux-kernel linux
2 Answers

I found the following in Communicating between the kernel and user-space in Linux using Netlink sockets by Ayuso, Gasca, and Lefevre:

If Netlink fails to deliver a message that goes from kernel to user-space, the recvmsg() function returns the No buffer space available (ENOBUFS) error. Thus, the user-space process knows that it is losing messages [...]

On the other hand, buffer overruns cannot occur in communications from user to kernel-space since sendmsg() synchronously passes the Netlink message to the kernel subsystem. If blocking sockets are used, Netlink is completely reliable in communications from user to kernel-space since memory allocations would wait, so no memory exhaustion is possible.

Regarding acks, it looks like worrying about them is optional:

NLM_F_ACK: the user-space application requested a confirmation message from kernel-space to make sure that a given request was successfully performed. If this flag is not set, the kernel-space reports the error synchronously via sendmsg() as errno value.

So it sounds like for my simplistic use case I can just use sendmsg and recvmsg naively, reacting to any error (except for EINTR) by starting the whole thing over, perhaps with backoff. My guess it that since I only get one response per request and the responses are tiny, I should never even see ENOBUFS as long as I have only one request in flight at at a time.

user_15054510

As a side note, we can see the dropped netlink packets in the Drops column of /proc/net/netlink. For example:

# cat /proc/net/netlink
sk       Eth Pid    Groups   Rmem     Wmem     Dump     Locks     Drops     Inode
76f0e0ed 0   1966   00080551 0        0        0 2        0        36968
36a83ab1 0   1431   00000001 0        0        0 2        0        30297
d7d5db8e 0   563    00000440 0        0        0 2        0        19572
a10eb5c0 0   795    00000515 704      0        0 2        0        23584
c52bbce9 0   474    00000001 0        0        0 2        0        17511
3c5a89a5 0   989856248 00000001 0        0        0 2        0        31686
051108c1 0   0      00000000 0        0        0 2        0        25
ed401538 0   562    00000440 0        0        0 2        0        19576
38699987 0   469    00000557 0        0        0 2        0        19806
d7bbb203 0   728    00000113 0        0        0 2        0        22988
4d31126f 2   795    40000000 0        0        0 2        0        31092
febb9674 2   2100   00000001 0        0        0 2        0        37904
8c18eb5b 2   728    40000000 0        0        0 2        0        22989
922a7fcf 4   0      00000000 0        0        0 2        0        8681
16cfa740 7   0      00000000 0        0        0 2        0        7680
4e55a095 9   395    00000000 0        0        0 2        0        15142
0b2c5994 9   1      00000000 0        0        0 2        0        10840
94fe571b 9   0      00000000 0        0        0 2        0        7673
a7a1d82c 9   396    00000000 0        0        0 2        0        14484
b6a3f183 10  0      00000000 0        0        0 2        0        7517
1c2dc7e3 11  0      00000000 0        0        0 2        0        640
6dafb596 12  469    00000007 0        0        0 2        0        19810
2fe2c14c 12  3676872482 00000007 0        0        0 2        0        19811
3f245567 12  0      00000000 0        0        0 2        0        8682
0da2ddc4 12  3578344684 00000000 0        0        0 2        0        43683
f9720247 15  489    00000001 0        0        0 2        11       17781
e84b6d30 15  519    00000002 0        0        0 2        0        19071
c7d75154 15  1550   ffffffff 0        0        0 2        0        31970
02c1c3db 15  4070855316 00000001 0        0        0 2        0        10852
e0d7b09a 15  1      00000002 0        0        0 2        0        10821
78649432 15  0      00000000 0        0        0 2        0        30
8182eaf3 15  504    00000002 0        0        0 2        0        22047
40263df1 15  1858   ffffffff 0        0        0 2        0        34001
49283e31 16  0      00000000 0        0        0 2        0        696
user_143937390
Related