RSS: A method to optimize network packet processing using multicore CPU

Introduction

Laymen explanation

If you remember the time when you plug LAN cable to LAN port of your Personal computer (PC), you had internet connection. This cable is internally connected to NIC card. This NIC card processes received packets. Now replace your laptop with a router. Router will have lot many packet receiving rate compared to your PC. To respond faster, router uses multi-core CPUs. But the question is that which CPU will process which received packets? In order word, how to share load among CPUs. RSS (Receive side scaling) approach helps in this regard.

Technical explanation

Receive side scaling (RSS) is a network driver technology that enables the efficient distribution of network receive processing across multiple CPUs in multiprocessor systems. The RSS design ensures that the processing that is associated with a given connection stays on an assigned CPU.

How it works

Contemporary NICs support multiple receive and transmit descriptor queues (multi-queue). On reception, a NIC can send different packets to different queues to distribute processing among CPUs. The NIC distributes packets by applying a filter to each packet that assigns it to one of a small number of logical flows. Packets for each flow are steered to a separate receive queue, which in turn can be processed by separate CPUs. This mechanism is generally known as “Receive-side Scaling” (RSS). The goal of RSS and the other scaling techniques is to increase performance uniformly. Multi-queue distribution can also be used for traffic prioritization, but that is not the focus of these techniques. The filter used in RSS is typically a hash function over the network and/or transport layer headers-- for example, a 4-tuple hash over IP addresses and TCP ports of a packet. The most common hardware implementation of RSS uses a 128-entry indirection table where each entry stores a queue number. The receive queue for a packet is determined by masking out the low order seven bits of the computed hash for the packet (usually a Toeplitz hash), taking this number as a key into the indirection table and reading the corresponding value.

The following figure illustrates the RSS mechanism for determining a CPU.

Hardware support

Many NIC vendors supports RSS.

RSS runs the interrupt service routine (ISR) on a CPU. The following figure illustrates the levels of hardware support for RSS.

Reference

https://msdn.microsoft.com/en-us/library/windows/hardware/ff556942(v=vs.85).aspx

http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000006703.html

https://communities.intel.com/community/wired/blog/2009/12/11/enabling-receive-side-scaling-in-windows

https://www.kernel.org/doc/Documentation/networking/scaling.txt