avatar🌌
TangentTangent Blog

技术博客 | Go & Kubernetes & Linux

技术博客 | Go & Kubernetes & Linux

Reading runc Source Code (Part 9): Core OCI Spec Data Structures (Part 3) — LinuxResources and LinuxSeccomp

Preface ​

In the previous article, we completed the "Part 2" deconstruction of the Linux struct, diving deep into namespaces and identity mapping, kernel parameter tuning, filesystem masking, and the platform execution domain.

As promised at the end of that article, this final installment of the OCI Spec data structure series focuses on the two most complex and voluminous subsystems in all of config.go:

  1. LinuxResources — The complete data mapping of Linux Cgroups v2 resource controllers
  2. LinuxSeccomp — The intricate configuration structure for Linux system call filtering

These two subsystems form the bedrock of production container isolation. Understanding their data structure design is a prerequisite for understanding container resource guarantees and security hardening.

1. Resource Control Overview: LinuxResources ​

In vendor/github.com/opencontainers/runtime-spec/specs-go/config.go, the full definition of LinuxResources is:

go
type LinuxResources struct {
    // Device allowlist controller
    Devices []LinuxDeviceCgroup `json:"devices,omitempty"`
    // Memory limits
    Memory *LinuxMemory `json:"memory,omitempty"`
    // CPU resource limits
    CPU *LinuxCPU `json:"cpu,omitempty"`
    // PID count limits
    Pids *LinuxPids `json:"pids,omitempty"`
    // Block I/O limits
    BlockIO *LinuxBlockIO `json:"blockIO,omitempty"`
    // Hugepage limits
    HugepageLimits []LinuxHugepageLimit `json:"hugepageLimits,omitempty"`
    // Network priority and traffic shaping
    Network *LinuxNetwork `json:"network,omitempty"`
    // RDMA resource limits (keyed by device name)
    Rdma map[string]LinuxRdma `json:"rdma,omitempty"`
    // Unified cgroup v2 tree resources (forward compatibility)
    Unified map[string]string `json:"unified,omitempty"`
}

As you can see, the OCI spec provides comprehensive coverage of Linux resource controllers. Let’s break each one down.

2. Memory Subsystem: LinuxMemory ​

go
type LinuxMemory struct {
    Limit            *int64  `json:"limit,omitempty"`              // Memory limit (bytes)
    Reservation      *int64  `json:"reservation,omitempty"`         // Memory soft limit (bytes)
    Swap             *int64  `json:"swap,omitempty"`                // Memory + Swap total limit (bytes)
    Kernel           *int64  `json:"kernel,omitempty"`              // Kernel memory limit (deprecated)
    KernelTCP        *int64  `json:"kernelTCP,omitempty"`           // TCP kernel memory limit (bytes)
    Swappiness       *uint64 `json:"swappiness,omitempty"`           // Swap tendency (0-100)
    DisableOOMKiller *bool   `json:"disableOOMKiller,omitempty"`    // Disable OOM Killer
    UseHierarchy     *bool   `json:"useHierarchy,omitempty"`        // Enable hierarchical memory accounting
    CheckBeforeUpdate *bool  `json:"checkBeforeUpdate,omitempty"`   // Check limit < current usage before update
}

Key Field Breakdown ​

Limit is the most commonly used field, directly mapping to cgroups v2’s memory.max. Once set, the container process group’s memory usage is strictly capped at the specified byte count. Exceeding it triggers OOM — either killed by the OOM Killer, or if memory.swap.max is enabled, pages are swapped out.

Reservation (soft limit) maps to cgroups v2’s memory.low. It does not hard-block memory allocations, but when system memory is tight, the kernel prioritizes reclaiming memory from cgroups that exceed their soft limit. Unlike hard limits which are absolute, soft limits are a cooperative resource guarantee mechanism, well-suited for multi-tenant scenarios with QoS requirements.

Swap maps to memory.swap.max. It’s important to note that in cgroups v2, Swap limits are independently configurable from memory limits — even with memory.limit set to 1GB, you can set memory.swap.max to 2GB, allowing the container up to 1GB physical memory + 1GB Swap. This is extremely useful for containing unexpected memory spikes in containerized environments.

DisableOOMKiller is a safety switch. When set to true, exhausting the container’s memory will not trigger the Linux OOM Killer — instead, subsequent memory allocation requests will return ENOMEM errors (NULL or -1). The application must handle this gracefully. In availability-focused financial or compute-intensive scenarios, explicitly disabling the OOM Killer + having the application handle ENOMEM is a more elegant degradation strategy.

⚠️ Deprecated field Kernel: The kernel-memory limit was used in the cgroups v1 era to cap kernel-side memory usage (e.g., slab allocator). It has been removed since Linux kernel 5.4, and the runc runtime does not actually apply this field. The spec’s comment explicitly documents the deprecation reason and kernel commit reference — a clear example of the OCI spec keeping pace with kernel evolution.

3. CPU Subsystem: LinuxCPU ​

go
type LinuxCPU struct {
    Shares         *uint64 `json:"shares,omitempty"`          // CPU weight (relative ratio)
    Quota          *int64  `json:"quota,omitempty"`            // Usable CPU time per period (microseconds)
    Burst          *uint64 `json:"burst,omitempty"`             // Burstable CPU time (microseconds)
    Period         *uint64 `json:"period,omitempty"`            // CPU period (microseconds, default 100000 = 100ms)
    RealtimeRuntime *int64 `json:"realtimeRuntime,omitempty"`  // Max realtime scheduling time (microseconds)
    RealtimePeriod *uint64 `json:"realtimePeriod,omitempty"`    // Realtime scheduling period (microseconds)
    Cpus           string  `json:"cpus,omitempty"`              // Available CPU cores (cpuset)
    Mems           string  `json:"mems,omitempty"`              // Available memory nodes (nodelist)
    Idle           *int64  `json:"idle,omitempty"`              // Scheduler idle weight (0=normal, 1=SCHED_IDLE)
}

CPU Weight vs. CPU Caps ​

The Linux CPU scheduler has two fundamentally different mechanisms for handling resource contention: relative weight and absolute cap. Shares is the former; Quota+Period is the latter.

Shares (maps to cgroups v2’s cpu.weight) is a relative weight, defaulting to 1024. When only containers A and B are running on the host, with Shares of 1024 and 2048 respectively, they receive CPU time in a 1:2 ratio — but this does not mean B can saturate 2 CPUs. It means that when contention occurs, B gets 2/3 of the CPU cycles. If only one container is running, it can still use all available CPU. Shares means "proportionally distribute whatever resources remain."

Quota + Period provides an absolute ceiling. The quota is the maximum CPU time the container can use per period microseconds. Setting quota=50000, period=100000 means the container can use at most 50ms of CPU time in each 100ms period — capping it at 50% of a single core. This is commonly used for:

  • Cloud multi-tenant isolation: Ensuring each tenant’s container doesn’t exceed its agreed-upon CPU quota
  • Latency-sensitive services: Guaranteeing each container a deterministic CPU time budget

Burst (cgroups v2 cpu.burst) is a more recent field supporting bursty resource usage. In the earlier example, quota=50000, period=100000 implies a 50% steady-state cap. But in real applications, many services have pronounced traffic spikes — a web service needs more CPU during sudden traffic surges. Burst allows a container to temporarily exceed its Quota (up to the accumulated Burst value) for higher CPU throughput. This is safer than directly raising Quota because Burst naturally recedes after accumulation, preventing sustained resource hogging.

Cpus / Mems map to cgroups v2’s cpuset.cpus and cpuset.mems, binding the container to specific CPU cores and NUMA memory nodes. In HPC scenarios, pinning a container to cores within a single NUMA node can significantly reduce memory access latency (avoiding cross-NUMA access).

Idle scheduler is a special option. When set to 1, the container’s processes use SCHED_IDLE scheduling — they are only dispatched when the system is completely idle. This is suitable for non-critical background batch tasks, preventing them from affecting interactive user workloads.

4. Block I/O Subsystem: LinuxBlockIO ​

go
type LinuxBlockIO struct {
    Weight                  *uint16 `json:"weight,omitempty"`
    LeafWeight              *uint16 `json:"leafWeight,omitempty"`
    WeightDevice            []LinuxWeightDevice `json:"weightDevice,omitempty"`
    ThrottleReadBpsDevice   []LinuxThrottleDevice `json:"throttleReadBpsDevice,omitempty"`
    ThrottleWriteBpsDevice  []LinuxThrottleDevice `json:"throttleWriteBpsDevice,omitempty"`
    ThrottleReadIOPSDevice  []LinuxThrottleDevice `json:"throttleReadIOPSDevice,omitempty"`
    ThrottleWriteIOPSDevice []LinuxThrottleDevice `json:"throttleWriteIOPSDevice,omitempty"`
}

The Block I/O controller enables fine-grained control over a container’s disk I/O. Unlike CPU’s "time slicing," the problem with disk I/O is that disks are physical rotating devices — concurrent access dramatically reduces throughput. Therefore cgroups provide two control strategies:

Weight control (Weight / WeightDevice): Sets relative weights via iosched.weight (cgroups v2). The scheduler proportionally allocates I/O bandwidth when multiple processes compete. Note that weights only take effect when the I/O dispatch queue has backlog — if the disk itself has no I/O buildup, weights produce no actual effect.

Ceiling control (Throttle* series): A more direct approach, setting absolute upper bounds via io.max (cgroups v2) for specific devices. ThrottleReadBpsDevice caps bytes per second for reads, ThrottleReadIOPSDevice caps operations per second for reads. This is commonly used for databases and other I/O-latency-sensitive services to prevent them from saturating disk bandwidth and affecting other containers.

IOPS vs Bps: For SSDs with no seek latency, limiting IOPS is more sensible than limiting Bps, because large-block sequential reads and small-block random reads can have wildly different Bps while IOPS remains relatively stable. For mechanical drives, limiting Bps better reflects actual disk load.

5. PID Subsystem: LinuxPids ​

go
type LinuxPids struct {
    Limit *int64 `json:"limit,omitempty"` // Maximum PID count inside the container
}

This is the simplest subsystem, mapping to cgroups v2’s pids.max. It prevents fork bombs inside the container from exhausting the system’s PID resources. The default is "unlimited." Setting a reasonable value (e.g., 1024 or 4096) significantly improves system robustness.

One important detail: the limit counts the total PIDs across the entire container process tree, not independent processes. If you have a Python process spawning 10 threads, each thread is a distinct PID in Linux (via clone(2) with CLONE_THREAD), and all 10 count against the same pids.max.

6. Network Subsystem: LinuxNetwork ​

go
type LinuxNetwork struct {
    ClassID    *uint32 `json:"classID,omitempty"`             // Network traffic class ID (for tc qdisc)
    Priorities []LinuxInterfacePriority `json:"priorities,omitempty"` // Per-interface traffic priority
}
go
type LinuxInterfacePriority struct {
    Name     string `json:"name"`      // Network interface name (e.g., eth0)
    Priority uint32 `json:"priority"`  // Priority (0-15 or higher)
}

ClassID is an elegant design. Mapping to cgroups v2’s net_cls.classid, it allows tagging container-generated network traffic with a specific tc (traffic control) marker, enabling fine-grained traffic shaping and rate limiting at the host’s tc layer. This is a decoupled design — the OCI spec only handles "tagging," while the specific traffic shaping policy is configured by the operator on the host via tc commands.

Priorities directly sets IEEE 802.1p VLAN priority at the interface level, for container-to-host network QoS.

7. RDMA Subsystem: LinuxRdma ​

go
type LinuxRdma struct {
    HcaHandles *uint32 `json:"hcaHandles,omitempty"` // Max HCA handles that can be opened
    HcaObjects *uint32 `json:"hcaObjects,omitempty"`  // Max RDMA objects that can be created
}

RDMA (Remote Direct Memory Access) is a core technology for HPC and low-latency networking (RDMA over Converged Ethernet, RoCE). Linux exposes RDMA resources via /sys/class/infiniband/. The LinuxRdma struct allows capping a container’s usage of InfiniBand or RoCE card resources, preventing containers from over-consuming scarce hardware resources like RDMA Queue Pairs and memory registration regions.

In Kubernetes environments, RDMA devices typically provide high-performance data transfer channels for AI/ML training workloads via GPUDirect RDMA or RoCE adapters. At this point, RDMA resource limits become critical for ensuring multi-tenant fairness.

8. Unified Cgroup Tree: Unified ​

go
Unified map[string]string `json:"unified,omitempty"`

This is a key-value dictionary that directly maps to any configuration file under /sys/fs/cgroup/ in the cgroups v2 unified tree. This is a forward compatibility design — when the kernel introduces a new cgroup controller or parameter before the OCI spec has defined a dedicated data structure for it, users can pass the kernel interface directly through the Unified field, without waiting for a spec update.

Common uses include setting pids.max, cpu.stat (read-only), io.stat (read-only), and various newer cgroups v2 controller configurations.

9. Device Allowlist: LinuxDeviceCgroup ​

go
type LinuxDeviceCgroup struct {
    Allow  bool   `json:"allow"`               // Allow or deny
    Type   string `json:"type,omitempty"`      // Device type: block, char, fifo
    Major  *int64 `json:"major,omitempty"`     // Major device number
    Minor  *int64 `json:"minor,omitempty"`     // Minor device number
    Access string `json:"access,omitempty"`    // Access permissions: r, w, m (mknod)
}

This is the core of container "device visibility" control. Both docker run --device and runc’s device management are backed by this struct.

The m in the Access field is a security point that is easy to overlook: the mknod permission allows the container to create device node files under /dev. Combined with wildcard rules where Major/Minor are nil (e.g., {"allow": true, "type": "block", "access": "rwm"}), you can precisely control which devices the container can see. By default, containers only see the device list generated by runc spec (null, zero, random, urandom, tty, console) — they cannot access other block or character devices on the host.

10. System Call Filtering: LinuxSeccomp ​

We now arrive at one of the most powerful weapons in the container security arsenal — Seccomp (Secure Computing Mode).

go
type LinuxSeccomp struct {
    DefaultAction    LinuxSeccompAction `json:"defaultAction"`      // Default action
    DefaultErrnoRet  *uint              `json:"defaultErrnoRet,omitempty"` // Default errno return value
    Architectures    []Arch             `json:"architectures,omitempty"`    // Target architecture list
    Flags            []LinuxSeccompFlag `json:"flags,omitempty"`            // Seccomp filter flags
    ListenerPath     string             `json:"listenerPath,omitempty"`     // seccomp NOTIFY_FD path
    ListenerMetadata string             `json:"listenerMetadata,omitempty"` // Metadata (for listener use)
    Syscalls         []LinuxSyscall     `json:"syscalls,omitempty"`        // System call rules
}

Default Action and Syscall Matching ​

DefaultAction is the fallback policy for the entire Seccomp configuration — the action taken when no rule matches. Common options:

ActionBehavior
SCMP_ACT_KILLKill the process that triggered the syscall (SIGSYS)
SCMP_ACT_KILL_PROCESSKill the entire process group
SCMP_ACT_KILL_THREADKill only the thread that triggered the call
SCMP_ACT_TRAPSend SIGSYS and陷入内核, process can handle it
SCMP_ACT_ERRNOReturn specified errno to userspace (via DefaultErrnoRet)
SCMP_ACT_TRACETrigger ptrace(PTRACE_SECCOMP), hand control to tracer
SCMP_ACT_LOGLog but allow the call (for audit mode)
SCMP_ACT_NOTIFYTrigger seccomp NOTIFY, hand decision to userspace listener

Production best practice: Start with DefaultAction: SCMP_ACT_LOG, observe which syscalls are triggered (/proc/sys/kernel/seccomp/actions_logged), then set allowed calls to SCMP_ACT_ALLOW and keep the rest as SCMP_ACT_KILL or SCMP_ACT_ERRNO. Setting SCMP_ACT_KILL directly without auditing may cause applications to crash because they use unexpected syscalls.

DefaultErrnoRet allows changing the default action from "kill" to "return a specific error code." For example, setting it to 1 (EPERM) makes blocked syscalls appear to the application as "permission denied," allowing the application to degrade gracefully rather than crash.

Architecture Support: Architectures ​

This field specifies which CPU architectures the rules apply to. In containers supporting multiple architectures (e.g., running an i386 container on an x86_64 host), you can set different syscall filtering rules for each architecture. By default, only the host’s native architecture is permitted.

Supported architectures include: X86, X86_64, X32, ARM, AARCH64, MIPS, MIPS64, PPC, PPC64, PPC64LE, S390, S390X, RISCV64, LOONGARCH64, M68K, SH, SHEB — 17 architectures in total.

Seccomp Flags: Flags ​

SECCOMP_FILTER_FLAG_LOG is extremely useful in production — it logs all non-ALLOW actions to actions_logged without blocking them. This is the low-level implementation of the "audit mode" mentioned earlier.

SECCOMP_FILTER_FLAG_SPEC_ALLOW relates to CPU hardware security vulnerabilities (Speculative Store Bypass, SSB). Enabling it allows applications to use syscalls that might be speculatively executed maliciously, while disabling hardware-level SSB mitigation — a performance vs. security tradeoff. In high-security multi-tenant SaaS environments, this flag should be enabled.

Core Rule Definition: Syscalls + Args ​

go
type LinuxSyscall struct {
    Names    []string           `json:"names"`              // Syscall name list (one rule can match multiple)
    Action   LinuxSeccompAction `json:"action"`              // Action on match
    ErrnoRet *uint              `json:"errnoRet,omitempty"` // Override default errno
    Args     []LinuxSeccompArg  `json:"args,omitempty"`     // Argument matching conditions
}

The core of each rule is a syscall name list + action on match. Via Names, you can group related syscalls (e.g., ["open", "openat", "openat2"]) under the same security policy.

The Args array enables fine-grained filtering of specific syscall arguments — one of Seccomp’s most powerful capabilities:

go
type LinuxSeccompArg struct {
    Index    uint                 `json:"index"`     // Argument index (0 = first argument)
    Value    uint64               `json:"value"`     // Expected value
    ValueTwo uint64               `json:"valueTwo,omitempty"` // Second value for mask comparison
    Op       LinuxSeccompOperator `json:"op"`        // Comparison operator
}

Supported comparison operators:

OperatorMeaningTypical Use Case
SCMP_CMP_EQEqual toOnly allow open("/path", O_RDONLY)
SCMP_CMP_NENot equal toBlock writes to specific fd
SCMP_CMP_LTLess thanLimit nice value range for setpriority
SCMP_CMP_LELess than or equal—
SCMP_CMP_GEGreater than or equal—
SCMP_CMP_GTGreater than—
SCMP_CMP_MASKED_EQMasked equalityarg[0] & O_ACCMODE == O_RDONLY (check read-only flag)

A practical example: To only allow a container process to use the O_RDONLY (read-only) flag when opening files:

json
{
  "names": ["open", "openat"],
  "action": "SCMP_ACT_ERRNO",
  "errnoRet": 1,
  "args": [
    {
      "index": 1,
      "value": 0,
      "valueTwo": 3,
      "op": "SCMP_CMP_MASKED_EQ"
    }
  ]
}

This uses O_ACCMODE = 3 (binary 11) mask compared against O_RDONLY = 0, ensuring the low two bits of the second argument (flags) are 00 — forcing read-only. This is a strong barrier against malicious filesystem writes from within a container.

Seccomp NOTIFY: ListenerPath ​

This is the most modern Seccomp collaboration mode, introduced in Linux 5.0. The limitation of traditional Seccomp is: once the BPF filter is loaded, the kernel takes over and there is no way to dynamically modify rules or interact with userspace programs.

ListenerPath registers a userspace process (via user_notification_addfd) as the Seccomp event handler. When a process inside the container executes a filtered syscall, the kernel sends a notification message through a pre-opened fd, and the userspace listener can:

  • Inspect the call arguments
  • Make allow/deny decisions
  • Even modify the return value

This mechanism is the low-level foundation for collaboration between container runtimes (like containerd-shim) and security policy engines (like Falco, Sysdig) — they can dynamically decide at runtime whether to allow a "dangerous" syscall (like mount, ptrace) rather than making a static decision at startup.

Conclusion ​

From LinuxResources to LinuxSeccomp, we have seen how the OCI spec translates Linux kernel’s two major isolation mechanisms — resource control (Cgroups) and system call filtering (Seccomp) — into clean, structured data.

Across this OCI Spec data structure series, we traveled from the top-level Spec all the way down to Seccomp’s argument-level filtering rules. This knowledge chain reveals a core design philosophy: the OCI spec does not simply expose kernel interfaces directly, but provides semantic abstraction through Go data structures, making container configuration both precise and portable.

Understanding the details of these data structures is irreplaceable for troubleshooting container performance issues, designing multi-tenant isolation strategies, and hardening container security boundaries. This is precisely the value of reading source code — finding that clear line of abstraction through the maze of syscalls and kernel parameters.

Linux Capabilities 详解:进程权限控制的艺术
runc 源码解读(九):OCI Spec 核心数据结构解析 (下) —— 资源控制与系统调用过滤
由 Valaxy v1.2.2 驱动|主题-Yunv1.2.2