Probing the Guts of Kprobes

Kernel Probes (Kprobes) can insert probes into a running kernel for purposes of debugging, tracing, performance evaluation, fault injection, etc. A user-defined handler is run when a probepoint is hit. From the barebones implementation in Linux 2.6.9, kprobes has undergone a number of improvements - support for colocated probes, function return probes, reentrant probes and the like. Handlers are now executed without any locks held, leading to lower overhead compared to the earlier "single spinlock serialization" method. Other enhancements are on the anvil - the kprobe "booster" series, userspace probes and watchpoint probes, to name a few. This paper will trace the developments in kprobes and also touch upon the current state of the aforementioned enhancements.