Linux kernel scalability and tradeoffs

The 2.6 kernel introduced a number of new performance and scalability features for high-end SMP and NUMA systems. For example, the 2.6 kernel introduces finer-grain locking in the scheduler, the block I/O layer, hardware and software interrupts, memory management, and the VFS layer. In addition, 2.6 brings new primitives such as RCU and per-cpu data, as well as scalable user-level APIs including sys_epoll() and futexes. These features bring with them some interesting issues that demand careful consideration. For example:

1. The benefits of fine-grain locking can be overshadowed by the overhead of the additional lock acquisitions in critical paths.

2. RCU-based lock-free caches must avoid excessive memory consumption, even when faced with excessive numbers of updates, such as can occur during a denial-of-service attack (e.g. route cache in a router).

3. Increased use of softirq handlers can degrade realtime response, since such handlers are non-preemptible.

4. Although a number of scalability improvements introduced very little complexity, now is a good time to think about possible simplifications that still achieve good performance.

In this BOF session, we discuss examples of these issues and future courses of action. We will also discuss some new ideas, including lock-free VM, ARC/CAR/CART disk replacement algorithms, lock-free file descriptor management using RCU with struct file (for which a patch already exists), lock-free TCP connection table, lock-free conntrack, and scalable NFS. Each idea will be vetted for potential performance and scalability gains, and for ways of making it as simple as possible (but no simpler!).