How to set ndo_start_xmit cpu affinity in ethernet driver?

user_19749523 0
Viewed 81

On the receiving side we have the opportunity to choose the CPU that will process the receive queue (RPS). On the sender side XPS selects the send queue for the given CPU, but there is no way to select the CPU on which ndo_start_xmit() will be launched. Taskset is able to bind user task, but in ndo_start_xmit() binding differs.

In my case CPU0 reserved for polling kthread, because our NIC have no interrupts, therefore it is necessary. I want nothing else to run on this CPU.

Setting CPU1 for RPS on both nodes:

echo 0x2 > /sys/class/net/<dev>/queues/rx-0/rps_cpus

Then run iperf on two nodes:

host1: taskset -c 1 iperf -s
host2: taskset -c 1 iperf -c host1

After adding pr_info("cpu%d\n", smp_processor_id()); in my ndo_start_xmit() method, see in dmesg:

host1: dmesg | grep cpu0 | wc -l
0
host2: dmesg | grep cpu0 | wc -l
6512

Is it possible to choose the CPU on which ndo_start_xmit() will be launched on the sender side?

linux-device-driver linux
1 Answers

Found the answer to my question. Doubt that anyone will ever need it, but nevertheless. The kernel network stack initializes the next ndo_start_xmit() on the cpu where the dev_kfree_skb() function was called. In my case, i moved out dev_kfree_skb() from the polling kthread to a function that makes enqueuing.

user_197495230
Related