Reading runc Source Code (Part 2): Container Filesystem in SPEC.md

Introduction ​

In the previous article, we explored the namespace section of SPEC.md and learned how containers achieve resource isolation through 7 types of Linux Namespaces. This article continues into the Filesystem chapter, examining how a container’s filesystem is built from scratch.

Filesystem initialization is one of the most complex stages of container startup. It involves providing an isolated directory tree, mounting virtual filesystems, creating device nodes, configuring terminals, and setting up I/O redirection — each step with strict ordering and security requirements.

Rootfs: The Container’s Isolated World ​

SPEC.md establishes the first principle of container filesystems:

A root filesystem must be provided to a container for execution. The container will use this root filesystem (rootfs) to jail and spawn processes inside where the binaries and system libraries are local to that directory. Any binaries to be executed must be contained within this rootfs.

Every binary and library the container needs must exist within its rootfs.

OS context: The rootfs typically comes from a container image — when you run docker pull ubuntu, you’re downloading a minimal Ubuntu directory structure (/bin, /etc, /usr, etc.). runc uses pivot_root or chroot to jail the container process inside this directory, preventing access to the host filesystem.

SPEC.md also highlights an important automatic cleanup mechanism:

Mounts that happen inside the container are automatically cleaned up when the container exits as the mount namespace is destroyed and the kernel will unmount all the mounts that were setup within that namespace.

All mount operations inside the container occur within an isolated Mount Namespace. When the container exits and the namespace is destroyed, the kernel automatically unmounts everything, leaving no residue on the host.

Virtual Filesystem Mounts ​

A rootfs with only static files isn’t enough. Linux programs need to interact with the kernel, and this interaction relies on virtual filesystems. SPEC.md defines 6 filesystems that must be mounted at container startup:

Purpose of Each Mount Point ​

• /proc (procfs): Window into process information and kernel state. Without it, commands like ps and top won’t work. Each process’s state is exposed as a /proc/<pid>/ directory
• /dev (tmpfs): Device file directory, mounted as an in-memory temporary filesystem. It starts empty and is populated by the runtime with device nodes
• /dev/shm (tmpfs): POSIX shared memory for high-speed inter-process data exchange. size=65536k caps shared memory at 64MB
• /dev/mqueue (mqueue): POSIX message queue filesystem for asynchronous inter-process communication
• /dev/pts (devpts): Pseudo-terminal slave device directory. newinstance ensures the container has its own pts namespace — docker run -it interactive terminals depend on this
• /sys (sysfs): Export interface for kernel objects and hardware device information. Note it’s mounted read-only (MS_RDONLY) to prevent containers from modifying kernel parameters

Security Implications of Mount Flags ​

These flags form the security perimeter of the container filesystem:

Device Nodes: Hardware Abstractions for Containers ​

After virtual filesystems are mounted, the /dev directory is still empty. SPEC.md requires the runtime to create a set of essential device nodes:

It is expected that a rootfs does not need to have any device nodes specified for /dev within the rootfs as the container will setup the correct devices that are required for executing a container’s process.

Image builders don’t need to pre-populate these device files in the rootfs — the runtime creates them automatically.

The Six Essential Devices ​

• /dev/null: Data black hole. Writes are discarded, reads return EOF. command > /dev/null is the most common usage
• /dev/zero: Zero-byte source. Reads return infinite \0 bytes, commonly used for file initialization or creating blank block devices
• /dev/full: Always-full device. Writes return ENOSPC (No space left on device), useful for testing disk-full error handling
• /dev/tty: The current process’s controlling terminal, allowing programs to interact directly with the user
• /dev/random and /dev/urandom: Random number generators. Cryptographic operations, UUID generation, TLS handshakes, and more depend on these

Permission Design ​

• Mode 0666 (rw-rw-rw-): Readable and writable by all users. This is reasonable since unprivileged programs also need /dev/null and /dev/urandom
• Access rwm: These are cgroup device controller permissions — r (read), w (write), m (mknod, allowing device node creation). The runtime grants these specific devices a pass at the cgroup level as well

I/O and Pseudo-Terminals ​

With device nodes in place, SPEC.md turns to the core I/O problem: terminals and standard input/output.

ptmx and Pseudo-Terminals ​

/dev/ptmx will need to be a symlink to the host’s /dev/ptmx within the container.

/dev/ptmx (Pseudo-Terminal Master Multiplexer) is the "factory entrance" for pseudo-terminals. When a program opens it, the kernel allocates a Master/Slave pseudo-terminal pair:

• Master end: Held by the host (Docker engine), connected to your real keyboard and screen
• Slave end: Given to the process inside the container, behaving like a real terminal device

Since containers don’t have their own kernel, /dev/ptmx must link to the host’s counterpart to request new pseudo-terminals from the kernel.

Bind Mounting /dev/console ​

The use of a pseudo TTY is optional within a container and it should support both.

Containers support two modes:

• Background mode (e.g., docker run -d nginx): No terminal needed
• Interactive mode (e.g., docker run -it ubuntu): Requires a pseudo-terminal

In interactive mode, SPEC.md requires bind mounting the host’s PTY slave end to the container’s /dev/console:

The 0600 permission means only root can read/write the console, preventing unprivileged users inside the container from eavesdropping on terminal data.

OS context: Think of this as connecting an external monitor to a headless machine — the host calls pty.Open() to request a pseudo-terminal pair from the kernel, obtains the slave path (e.g., /dev/pts/3), and runc bind-mounts it to the container’s /dev/console. When bash inside the container writes to /dev/console, data flows through Slave → Master back to the host and onto your screen.

I/O Safety: dup2 Leak Prevention ​

If the container’s io is pointing to /dev/null outside the container we close and dup2 the /dev/null that is local to the container’s rootfs.

If the container’s standard I/O points to the host’s /dev/null (e.g., docker run ubuntu > /dev/null), runc detects this cross-namespace file descriptor reference, closes it, and uses dup2 to redirect it to the container’s own internal /dev/null. This ensures the container holds no file handles pointing to the host filesystem, maintaining sandbox isolation integrity.

Standard I/O Symlinks ​

After the container has /proc mounted a few standard symlinks are setup within /dev/ for the io.

These symlinks must be created after /proc is mounted, since their targets reside within /proc.

Why create them manually? On a regular Linux system, udev or systemd creates these links at boot. But the container’s /dev is a fresh tmpfs with no device manager running, so the runtime must create them explicitly. Many programs (especially logging frameworks) hardcode paths like /dev/stdout for log output — without these symlinks, they’d fail with "No such file or directory".

The elegance of /proc/self/fd/1 lies in self being a dynamic pointer — whichever process accesses it, it resolves to that process’s file descriptor table. This means different processes writing to /dev/stdout each have their data routed to the correct output target.

pivot_root: Switching the Root Directory ​

Once the filesystem is ready, runc uses pivot_root to switch the container process’s root directory to the rootfs:

put_old = mkdir(...);
pivot_root(rootfs, put_old);
chdir("/");
unmount(put_old, MS_DETACH);
rmdir(put_old);

Here’s what each step does:

1. Create a temporary directory put_old inside the rootfs
2. pivot_root switches the current root to the rootfs and mounts the old root at put_old
3. chdir("/") ensures the working directory points to the new root
4. Unmount and remove put_old, completely severing the connection to the host filesystem

OS context: The difference between pivot_root and chroot is that pivot_root truly changes the root mount point of the entire Mount Namespace, while chroot only changes the current process’s root directory view — which is less secure (it can be escaped through repeated chroot calls).

For containers running with a rootfs inside ramfs, pivot_root is not supported, so MS_MOVE + chroot is used as a fallback:

mount(rootfs, "/", NULL, MS_MOVE, NULL);
chroot(".");
chdir("/");

Finally, umask is reset to 0022, restoring the standard file creation permission mask.

The Complete Filesystem Initialization Flow ​

Putting all the steps together, runc’s complete filesystem initialization flow is:

1. Mount virtual filesystems: Mount /proc, /dev, /dev/shm, /dev/mqueue, /dev/pts, /sys per the spec table
2. Create device nodes: Populate the empty /dev with /dev/null, /dev/zero, and the other 4 essential devices
3. Configure pseudo-terminal: Create the /dev/ptmx symlink; if interactive mode is requested, bind mount /dev/console
4. I/O safety check: Detect and fix cross-namespace file descriptor references (dup2)
5. Create I/O symlinks: Establish /dev/stdin, /dev/stdout, /dev/stderr, and /dev/fd standard links
6. Switch root directory: Use pivot_root to jail the process inside the rootfs
7. Reset umask: Restore to 0022

Each step has strict ordering dependencies — /proc must precede I/O symlinks, /dev’s tmpfs must precede device node creation, and all mounts must precede pivot_root.

Summary ​

The Filesystem chapter of SPEC.md defines the entire process of building a container filesystem from nothing to fully functional. From mounting virtual filesystems to creating device nodes, from configuring pseudo-terminals to establishing I/O symlinks, and finally the pivot_root switch — each step strikes a precise balance between security and functionality.

In the next article, we’ll continue exploring the Resources (cgroup controls) section of SPEC.md.