Linux as a Hypervisor - An Update

Through its history, the Linux kernel has had increasing demands placed on it as it supported new applications and new workloads. A relatively new demand is to act as a hypervisor, as virtualization has become increasingly popular. In the past, there were many weaknesses in the ability of Linux to be a hypervisor. Today, there are noticably fewer, but they still exist.

Not all virtualization technologies stress the capabilities of the kernel in new ways. User-mode Linux (UML) is the only prominent example of a virtualization technology which uses the capabilities of a stock Linux kernel. As such, UML has been the main impetus for improving the ability of Linux to be a hypervisor.

A number of new capabilities have resulted in part from this, some of which have been merged and some of which haven't. Many of these capabilities have utility beyond virtualization, as they have also been pushed by people who are interested in applications that are unrelated to virtualization.

An early problem was the inability of ptrace of Linux/i386 to nullify intercepted system calls. This was fixed very early, as it is essential in order to virtualize system calls.

Another ptrace weakness was its requirement that both system call entry and exit must be intercepted. A ptrace extension, PTRACE_SYSEMU addresses this. It causes only system call entrances to be intercepted, causing a noticable performance improvement in UML, even on workloads that aren't system call-dependent.

UML wasn't one of the main drivers behind AIO and O_DIRECT but it benefits from them. These allow UML to behave more like the host kernel by allowing multiple outstanding I/O requests and to be more fully in charge of its own memory by bypassing the host's caching.

Another I/O improvement that improves the virtualization capabilities of the kernel is the ability to poke a hole in a file. Proposals for a sys_punch system call had circulated for years. MADVISE_REMOVE, which was the first to be merged, removes a range of pages from a tmpfs file. This allows Linux to support memory hotplug in its guests.

FUSE (Filesystems in Userspace) is another recent addition of interest. It doesn't contribute to the ability to host virtual machine, but it does contribute to the ability to manage them. UML is using FUSE export its filesystem to the host, allowing some guest system management to be done from the host.

There are a number of other capabilities which are not merged.

The large number of virtual memory areas (VMAs) that UML creates on the host is a noticable performance problem. Ingo Molnar implemented a new system call, remap_file_pages, to fix this problem. This allows pages within a mapping to be rearranged, reducing the number of VMAs for a UML process from nearly one per mapped page to one.

PTRACE_SYSEMU notwithstanding, system call interception is still a performance problem. Ingo has another patch, VCPU (Virtual CPU), which improves this. In effect, it allows a process to trace itself, eliminating the context switching that ptrace currently requires.