Kubernetes v1.36 Introduces Pod-Level Resource Managers for Enhanced Performance

By • min read

Introduction

With the release of Kubernetes v1.36, a significant alpha feature called Pod-Level Resource Managers arrives, designed to provide a more flexible and powerful resource management model for performance-sensitive workloads. This enhancement extends the kubelet's existing Topology, CPU, and Memory Managers to support pod-level resource specifications (.spec.resources), evolving from a strictly per-container allocation model to a pod-centric approach. The change addresses long-standing challenges in balancing performance and resource efficiency, particularly for pods that include auxiliary sidecar containers.

Why Pod-Level Resource Managers?

Running performance-critical workloads—such as machine learning (ML) training, high-frequency trading applications, or low-latency databases—often requires exclusive, NUMA-aligned resources for primary application containers to ensure predictable performance. However, modern Kubernetes pods rarely consist of a single container; they frequently include sidecars for logging, monitoring, service meshes, or data ingestion.

Before this feature, a trade-off existed: to obtain NUMA-aligned, exclusive resources for the main application, you had to allocate exclusive, integer-based CPU resources to every container in the pod, including lightweight sidecars. This led to wasteful allocation if sidecars needed minimal resources. Conversely, if you avoided allocating integer CPUs to sidecars, the pod could not achieve a Guaranteed Quality of Service (QoS) class, losing critical performance benefits such as CPU pinning and memory locality.

How Pod-Level Resource Managers Work

Enabling pod-level resources support for the resource managers—via the PodLevelResourceManagers and PodLevelResources feature gates—allows the kubelet to create hybrid resource allocation models. The system now recognizes a pod-level resource specification that defines an overall budget (requests and limits) for the entire pod, rather than requiring each container to individually declare exclusive resources.

With this model, the Topology, CPU, and Memory Managers can perform NUMA alignment based on the pod's entire budget, then assign exclusive resources to the primary container while allowing sidecar containers to share a pod shared pool from the remaining budget. This eliminates the previous trade-off, enabling both NUMA alignment and efficient resource usage. The shared pool provides isolation among auxiliary containers but does not interfere with the exclusive slices allocated to the main workload.

Real-World Use Cases

The feature is especially valuable in scenarios where a pod runs a latency-sensitive main container alongside multiple auxiliary sidecars. Below are practical examples, depending on the configured Topology Manager scope.

Tightly-Coupled Database (Topology Manager Pod Scope)

Consider a latency-sensitive database pod that includes a main database container, a local metrics exporter, and a backup agent sidecar. When configured with the pod Topology Manager scope, the kubelet performs a single NUMA alignment based on the entire pod's budget. The database container receives its exclusive CPU and memory slices from that NUMA node. The remaining resources from the pod's budget form a new pod shared pool. The metrics exporter and backup agent run within this pod shared pool, sharing resources with each other but strictly isolated from the database's exclusive slices and the rest of the node.

This approach allows safe co-location of auxiliary containers on the same NUMA node as the primary workload without wasting dedicated cores on them. The pod specification would look like:

apiVersion: v1
kind: Pod
metadata:
  name: tightly-coupled-database
spec:
  # Pod-level resources establish the overall budget and NUMA alignment size.
  resources:
    requests:
      cpu: "8"
      memory: "16Gi"
    limits:
      cpu: "8"
      memory: "16Gi"
  initContainers:
    - name: metrics-exporter
      image: metrics-exporter:v1
      restartPolicy: Always
    - name: backup-agent
      image: backup-agent:v1
      # Additional sidecar specifications omitted for brevity

In this example, the pod-level resources section defines the total budget. The kubelet uses this to allocate exclusive resources to the main database container (not shown), while the init containers (or sidecars) use the shared pool.

Enabling the Feature

To try Pod-Level Resource Managers, you must enable the alpha feature gates in the kubelet configuration:

• PodLevelResourceManagers
• PodLevelResources

Additionally, setting the Topology Manager scope to pod is recommended for workloads that require NUMA alignment. The feature is available in Kubernetes v1.36 as an alpha and may undergo changes in future releases. It is not enabled by default, so cluster administrators must explicitly configure it.

Conclusion

Pod-Level Resource Managers represent a major step forward in Kubernetes resource management for high-performance workloads. By allowing pod-level resource specifications and hybrid allocation models, the feature eliminates the previous trade-off between NUMA alignment and efficient sidecar resource usage. This enables more predictable performance for applications like databases, ML training, and low-latency services, while still accommodating the convenience of sidecar patterns—all without wasting dedicated cores on auxiliary containers. As the feature matures, it promises to become a cornerstone for performance-critical deployments in Kubernetes.