Can Hard and Soft-IRQ for the same network-packet be executed on different CPU-Cores when network packets are processed on Linux x86_64 kernel 3.16?
Hard-IRQ raises Soft-IRQ: http://she-devel.com/Chaiken_ELCE2016.pdf
Soft IRQs may run directly after the hard IRQ that raises them, or at a later time in ksoftirqd.
RSS raises Soft-IRQ on the same CPU-Core as Hard-IRQ: http://balodeamit.blogspot.ru/2013/10/receive-side-scaling-and-receive-packet.html
In case of multi queues (RSS), hardware interrupt will go to matching CPU processor, and that processor will also be responsible for softIRQ processing.
In case of monoqueues, without RPS the Soft-IRQ processed on the same CPU-Core as Hard-IRQ, but with RPS the Soft-IRQ processed on another CPU-Core than Hard-IRQ: http://balodeamit.blogspot.ru/2013/10/receive-side-scaling-and-receive-packet.html
In case of monoqueues, the hardware interrupt generated is from single queue and same CPU is also responisble for processing softIRQ. If RPS is enabled on mono queue, the incoming packets are hashed, load is distributed across multiple CPU processors.
RPS raises Soft-IRQ on the other CPU-Core than Hard-IRQ and it increases the rate of inter-processor interrupts (IPIs): https://en.wikipedia.org/wiki/Interrupt#Performance_issues
As a downside, RPS increases the rate of inter-processor interrupts (IPIs).
Does it mean that:
Ethernet-adapter (NIC) copy packet to the socket-buffer (mbuf) by using NIC-DMA and then NIC invoke Hard-IRQ on CPU Core-N
Then code of this Hard-IRQ do nothing more than invoke Soft-IRQ on the different CPU Core-M by using RPS
On this CPU Core-M the Soft-IRQ processes this packet by using TCP/IP-stack in kernel-space
Then our Application by using
read(,buf,)get the data from the socket-buffer (mbuf) to the our buffer in user-space on CPU Core-K
Is that correct?
Can Hard and Soft-IRQ for the same network-packet be executed on different cores when used RPS?