How it works

The code reference generated by rustdoc is published at /rustdoc/ and the repository's docs/ARCHITECTURE.md goes deeper. This page is the map.

Process model

proot (supervisor)
 ├─ main thread ........ ptrace tracer: seccomp-trace stops for execve, chdir,
 │                        chroot, bind, connect, inotify_add_watch, O_PATH opens
 └─ notify-N threads ... leader/followers pool answering user notifications for
                          every other path syscall
tracee tree ............ one seccomp filter, installed by the first child

The first child installs the filter with the new-listener flag, passes the listener descriptor back over a socketpair and waits for a byte. The parent seizes it with ptrace, starts the pool and sends the byte. From then on every traced syscall of every descendant has a supervisor.

Without ptrace

A hardened container profile that filters the ptrace syscall still allows the listener and memory access, and proot-next notices at start-up (or is told with PROOT_NO_PTRACE=1). The handful of syscalls the tracer used to rewrite are then served through notifications too: the working directory becomes virtual, unix sockets are connected or bound by the supervisor and swapped into the tracee's descriptor, and an exec goes through a tiny static trampoline that the supervisor installs under $XDG_RUNTIME_DIR and names in the tracee's path string in place, with the real target in a record file beside it. ptrace stays the first choice where it works: it costs nothing it does not have to, and it is what Steam is tested against.

Hosts that refuse access to a child

Both modes need the kernel's ptrace access check to pass towards every tracee. Yama's kernel.yama.ptrace_scope of 2 grants that only to a holder of CAP_SYS_PTRACE in the tracee's user namespace, so on such a host the first tracee enters a user namespace of its own before installing the filter. The supervisor stays outside and owns it, which is enough. The namespace maps your uid and gid onto themselves and nothing else, so other users' files read as nobody through calls the kernel answers directly (fstat), supplementary groups read as nogroup, and setgroups is refused. Path lookups served by the supervisor keep the real owners.

The choice is made at start-up by trying the accesses on a throwaway child. Where neither works (Yama 3, or a seccomp profile that also refuses unshare, as the default Docker and Kubernetes RuntimeDefault profiles do for a user without CAP_SYS_ADMIN), proot-next falls back to the guest-stub mode on x86_64 and aarch64. Elsewhere, or with PROOT_GUEST_STUB=0, it stops with a message instead of starting a guest it cannot serve.

The guest-stub mode

The fallback where nothing else reaches the guest's memory, on x86_64 and aarch64 hosts. The filter sends the syscalls proot-next serves to a small handler inside each guest process, which copies the paths and buffers they use into memory shared with the supervisor, has the supervisor answer from that copy, and copies the results back, so the supervisor never needs to reach the guest's own memory. The handler lives in the exec trampoline, which loads every program in place instead of exec'ing it so the handler survives, the way the kernel would load it. 32-bit programs get a 32-bit build of the handler and trampoline, where the kernel runs 32-bit programs at all.

No namespaces inside the guest

Creating a user, mount or other namespace inside the guest is refused with EPERM, as a default container refuses it. A sandbox helper that got its own mount namespace could not be served by a supervisor doing the filesystem work in its own, and every such helper already handles the refusal: bubblewrap, which GTK's glycin image loaders and WebKit use, reports that namespaces are unavailable and the application carries on without the sandbox; Firefox's content sandbox degrades the same way.

Nested proot and bwrap

A proot started inside the guest cannot be a second supervisor: the kernel allows one tracer per process and one seccomp listener per filter chain. It does not need to be one. An execve of proot-next itself, or of any program named proot or bwrap (a distribution's bubblewrap included), is served by the supervisor already there: the command line is parsed on the caller's behalf, every path in it is composed through the caller's view, and the process that called proot becomes the command, in a view of its own that its children inherit. The caller's child, exit status and signals are the kernel's own, and --info-fd, --json-status-fd, --seccomp, --die-with-parent and --new-session behave as at top level. A nested view can see no more than the view it was asked from, and nesting composes to any depth. This is what lets proot-apps' graphical installer run proot -R app /install from inside its own guest, and GTK's glycin run its image loaders inside their sandbox root.

Two options cannot be served nested unless the outer launch already has them, because the seccomp filter is fixed at launch: -0 and -k are refused with a message. PROOT_NEST=0 turns off the match by name.

Why a small warm pool

The kernel hands a notification to the longest-waiting receiver. On an idle host that is a cold core, and the wake-up costs more than the work. One thread waits at a time and spins briefly first; whoever grabs the token next is already warm. The spin budget decays to zero when requests stop arriving.

Path translation

A guest path is walked one component at a time, on descriptors: the root and every binding are held open, each component is opened relative to the directory walked so far, and a symlink is read and its target re-entered as a guest path, so a link inside the rootfs never leads the walk out of it. The syscall is then made through the directory the walk ended in, on the final name with no symlink following, rather than on a host path the kernel would walk again: a guest that renames a directory of its own and puts a symlink in its place while its call is in flight gets the directory it had, or the link resolved inside the guest on a retry, never the host directory the link names. The syscalls the tracee must run itself (an exec; under ptrace also chdir, connect, bind, inotify_add_watch and O_PATH opens) still walk a host path. proot-next confines paths, not privileges, and is no boundary against a guest working against it. Relative paths are resolved against the tracee's directory descriptor by reading its path from /proc. A cache of directory descriptors skips the open of a component already walked.

execve

The kernel resolves PT_INTERP itself, which would escape the guest, so the exec is retargeted at the host path of the guest's dynamic loader with the program as its argument. Both glibc and musl loaders accept --argv0, so the program sees the name it was called by. 32-bit programs go to the 32-bit loader. The program the tracee asked for is recorded so /proc/<pid>/exe answers truthfully.

Staying invisible

A ptrace supervisor is indistinguishable from a debugger unless it works at it. /proc/<pid>/status is served with TracerPid zeroed; an int3 breakpoint is resumed past as a debugger would, while a real crash sequence (int3; ud2) still dies on the ud2; /proc/<pid>/fd/N links are left for the kernel to re-open, since they name memfds, pipes and deleted files that have no path. Tracees are seized so group-stops are distinguishable from signals and job control keeps working.