User-Space Network Stacks On Linux




User-Space Network Stacks on Linux

4 June 2019

New York

Added 01-Jan-1970

Historically we've been relying on the Kernel to take care of our networking needs. It is well tested, supports all kinds of protocols and hardware, yet we are looking to re-implement - part of - it in user-space. Why would we want to do that? That sounds crazy!

10 Gigabit Ethernet (10GbE) technology is more than 15 years old, there is a good chance that we skip 40GbE and jump straight to 100GbE. Out-of-the-box you can only expect a fraction of the 10GbE throughput on a general purpose Linux. Luckily there are plenty of parameters to tune, so you can easily double the baseline performance, but even with those, you will not be able to sustain 10Gbit throughput (unless the packets are large enough)... and we have not even looked at 40GbE or 100GbE yet! And this should answer the previous question: the reason we want to build a network stack in userland is performance.

We will see that moving the data-plane portion of the network stack to user-space, offloading certain work to the network card (NIC), tuning OS parameters, we can achieve 10Gbit throughput with all packet sizes, while maintaining single digit microsecond latency - without sacrificing security.

In this talk I'll briefly introduce the Linux network subsystem, standard network stack tuning options, then show a few user-space network stack implementation - both proprietary and open-source. By the end of the talk you'll have a much better understanding where to look when you experience packet loss and what direction to take when you reached the tipping point and want to make use of a user-space networking stack.