GPU Accelerated Pods (NVIDIA & AMD)

Berkelium provides a diverse fleet of enterprise NVIDIA and AMD accelerators (see the live Cluster Status page for current node inventory and availability):

• NVIDIA B300 SXM6 AC (Blackwell): 275 GB HBM3e memory per GPU, NVLink, and CUDA 13.0
• NVIDIA H200 NVL (Hopper): 141 GB HBM3e memory per GPU and dual 4-way NVLink domains
• NVIDIA H100 80GB HBM3 (Hopper): 80 GB HBM3 memory per GPU and NVLink
• NVIDIA A100 80GB PCIe (Ampere): 80 GB HBM2e memory per GPU
• NVIDIA A40 48GB (Ampere): 48 GB GDDR6 memory per GPU
• AMD Instinct MI100 (CDNA): 32 GB HBM2 memory per GPU with ROCm support

Strictly Prohibited: Holding GPUs with sleep infinity

GPU workloads submitted as batch Jobs must execute active computation and exit automatically as soon as the task completes. Running sleep infinity, tail -f /dev/null, or any equivalent command in a Job or Deployment to reserve GPUs or keep a completed GPU pod alive is strictly prohibited and will result in immediate termination. Save your checkpoints to a ceph-rbd or cephfs PVC so your job can exit cleanly when finished.

---

GPU Scheduling Mechanisms

Berkelium supports two methods for allocating accelerators:

1. Dynamic Resource Allocation (DRA - Recommended): The Kubernetes standard (resource.k8s.io/v1) active across all NVIDIA and AMD nodes. DRA enables precise model selection via cluster-wide DeviceClass objects (e.g. deviceClassName: nvidia-b300, nvidia-h200, nvidia-h100, nvidia-a100, or nvidia-a40), NVLink domain/topology isolation via CEL selectors, and on-demand Dynamic Multi-Instance GPU (Dynamic MIG) partitioning without brittle node selectors.
2. Extended Resource Limits (nvidia.com/<model>, nvidia.com/gpu, and amd.com/gpu): Integer count declarations placed in container resources.limits (e.g. nvidia.com/b300, nvidia.com/h200, nvidia.com/h100, nvidia.com/a100, nvidia.com/a40, or default nvidia.com/gpu for A40). Berkelium’s DeviceClass objects map these extended resource requests directly to DRA drivers under the hood.

---

NVIDIA Dynamic Resource Allocation (DRA)

With DRA, your workload claims devices using a ResourceClaimTemplate and binds that claim directly inside the container under resources.claims.

1. Target NVIDIA B300 SXM6 AC (Blackwell)

Ideal for extreme-scale foundation models, FP4/FP8 training, and memory-intensive scientific simulations:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: b300-gpu-claim
  namespace: my-project
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-b300
            count: 1
---
apiVersion: v1
kind: Pod
metadata:
  name: blackwell-training-pod
  namespace: my-project
spec:
  restartPolicy: OnFailure
  resourceClaims:
    - name: gpu
      resourceClaimTemplateName: b300-gpu-claim
  containers:
    - name: pytorch
      image: nvcr.io/nvidia/pytorch:25.01-py3
      command: ["python", "-c", "import torch; print(f'CUDA: {torch.cuda.is_available()}, Device: {torch.cuda.get_device_name(0)}')"]
      resources:
        requests:
          cpu: "16"
          memory: "64Gi"
        limits:
          cpu: "32"
          memory: "128Gi"
        claims:
          - name: gpu

2. Target NVIDIA H200 NVL (141GB HBM3e) & Dual NVLink Domains

NVIDIA H200 NVL nodes host 8x NVIDIA H200 NVL GPUs (141 GB HBM3e each). Unlike 8-way NVSwitch HGX baseboards, the 8 H200 NVL cards on a node are interconnected via two independent 4-GPU NVLink bridge domains aligned with PCIe root complexes:

• Domain 0 (pci0000): GPUs 0–3 (device.attributes["gpu.nvidia.com"].pcieRoot == "pci0000") — full 4-way peer-to-peer NVLink (NV6)
• Domain 1 (pci0001): GPUs 4–7 (device.attributes["gpu.nvidia.com"].pcieRoot == "pci0001") — full 4-way peer-to-peer NVLink (NV6)

Cross-domain traffic between GPUs 0–3 and GPUs 4–7 traverses host PCIe/NUMA (SYS).

Single or General H200 NVL Allocation (nvidia-h200)

To request 1 or more H200 NVL GPUs from either domain:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: h200-gpu-claim
  namespace: my-project
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-h200
            count: 1

Automatic 4-GPU NVLink Domain Selection (firstAvailable)

When running a 4-GPU distributed training or inference job on an H200 NVL node, you want all 4 GPUs to reside within the same 4-way NVLink domain (either 0–3 or 4–7), without hardcoding which domain is currently free. Use DRA’s firstAvailable subrequests with CEL pcieRoot selectors to automatically pick whichever 4-GPU NVLink domain has all 4 GPUs available:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: h200-4gpu-nvlink-domain
  namespace: my-project
spec:
  spec:
    devices:
      requests:
        - name: gpu
          firstAvailable:
            - name: domain0
              deviceClassName: nvidia-h200
              allocationMode: ExactCount
              count: 4
              selectors:
                - cel:
                    expression: device.attributes["gpu.nvidia.com"].pcieRoot == "pci0000"
            - name: domain1
              deviceClassName: nvidia-h200
              allocationMode: ExactCount
              count: 4
              selectors:
                - cel:
                    expression: device.attributes["gpu.nvidia.com"].pcieRoot == "pci0001"
---
apiVersion: v1
kind: Pod
metadata:
  name: h200-nvlink-pod
  namespace: my-project
spec:
  restartPolicy: OnFailure
  resourceClaims:
    - name: gpu
      resourceClaimTemplateName: h200-4gpu-nvlink-domain
  containers:
    - name: trainer
      image: nvidia/cuda:12.8.0-base-ubuntu24.04
      command: ["nvidia-smi", "topo", "-m"]
      resources:
        requests:
          cpu: "48"
          memory: "256Gi"
        limits:
          cpu: "96"
          memory: "512Gi"
        claims:
          - name: gpu

Targeting a Specific NVLink Domain

You can also target a specific 4-GPU NVLink bridge domain directly (for example, requesting 2 GPUs guaranteed to be within Domain 0 or Domain 1) by adding the device.attributes["gpu.nvidia.com"].pcieRoot == "pci0000" (or "pci0001") CEL selector under exactly with deviceClassName: nvidia-h200.

3. Target NVIDIA H100 80GB HBM3 (Hopper)

Ideal for Transformer Engine, FP8 deep learning, Megatron-LM, and NVLink-interconnected workloads:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: h100-gpu-claim
  namespace: my-project
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-h100
            count: 2
---
apiVersion: v1
kind: Pod
metadata:
  name: h100-distributed-pod
  namespace: my-project
spec:
  restartPolicy: OnFailure
  resourceClaims:
    - name: gpu
      resourceClaimTemplateName: h100-gpu-claim
  containers:
    - name: deepspeed
      image: nvcr.io/nvidia/pytorch:24.04-py3
      command: ["nvidia-smi", "-L"]
      resources:
        requests:
          cpu: "32"
          memory: "128Gi"
        limits:
          cpu: "64"
          memory: "256Gi"
        claims:
          - name: gpu

4. Target NVIDIA A100 (80GB PCIe) or Default NVIDIA A40 (48GB)

Use deviceClassName: nvidia-a100 for 80 GB A100 PCIe GPUs, or deviceClassName: nvidia-a40 (or the default gpu.nvidia.com class) for general-purpose NVIDIA A40 48 GB GPUs:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: a40-nvidia-claim
  namespace: my-project
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-a40 # Or gpu.nvidia.com (A40 default) / nvidia-a100
            count: 1
---
apiVersion: v1
kind: Pod
metadata:
  name: general-gpu-pod
  namespace: my-project
spec:
  restartPolicy: OnFailure
  resourceClaims:
    - name: gpu
      resourceClaimTemplateName: a40-nvidia-claim
  containers:
    - name: worker
      image: nvcr.io/nvidia/pytorch:24.04-py3
      command: ["nvidia-smi", "-L"]
      resources:
        requests:
          cpu: "8"
          memory: "32Gi"
        limits:
          cpu: "16"
          memory: "64Gi"
        claims:
          - name: gpu

---

Dynamic Multi-Instance GPU (Dynamic MIG) with DRA

Berkelium runs nvidia-dra-driver-gpu (v25.12.0) with Dynamic MIG (DynamicMIG: true) enabled on all MIG-capable GPU models (NVIDIA B300, H200 NVL, H100, and A100).

Using Kubernetes DRA Partitionable Devices (sharedCounters and consumesCounters), physical GPUs do not need to be statically pre-partitioned ahead of time:

• When a pod requests a mig.nvidia.com device, the Kubernetes scheduler tracks the underlying GPU’s hardware memory and compute slice counters and selects a compatible physical GPU.
• The DRA kubelet plugin dynamically creates the hardware-isolated MIG instance before the container starts and automatically destroys the MIG slice when the ResourceClaim is released.
• Full-GPU workloads (type == "gpu") and fractional MIG workloads (type == "mig") coexist seamlessly on the same nodes.

Available Dynamic MIG Profiles by GPU Model

GPU Model	1-Slice Profiles (Up to 7/GPU)	2-Slice Profiles (Up to 3/GPU)	3- or 4-Slice Profiles (Up to 2/GPU)	Full-GPU MIG Profile (1/GPU)
NVIDIA B300 SXM6 AC	1g.34gb, 1g.34gb+me, 1g.67gb	2g.67gb	3g.135gb, 4g.135gb	7g.269gb
NVIDIA H200 NVL	1g.18gb, 1g.18gb+me, 1g.35gb	2g.35gb	3g.71gb, 4g.71gb	7g.141gb
NVIDIA H100 80GB HBM3	1g.10gb, 1g.10gb+me, 1g.20gb	2g.20gb	3g.40gb, 4g.40gb	7g.80gb
NVIDIA A100 80GB PCIe	1g.10gb, 1g.10gb+me, 1g.20gb	2g.20gb	3g.40gb, 4g.40gb	7g.80gb

Requesting a Dynamic MIG Partition

Request deviceClassName: mig.nvidia.com under exactly and filter by the desired profile (and optionally productName) using a CEL selector:

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: h200-mig-1g-18gb-claim
  namespace: my-project
spec:
  spec:
    devices:
      requests:
        - name: mig-gpu
          exactly:
            deviceClassName: mig.nvidia.com
            allocationMode: ExactCount
            count: 1
            selectors:
              - cel:
                  expression: >-
                    device.attributes["gpu.nvidia.com"].profile == "1g.18gb" &&
                    device.attributes["gpu.nvidia.com"].productName == "NVIDIA H200 NVL"
---
apiVersion: v1
kind: Pod
metadata:
  name: mig-inference-pod
  namespace: my-project
spec:
  restartPolicy: OnFailure
  resourceClaims:
    - name: mig-gpu
      resourceClaimTemplateName: h200-mig-1g-18gb-claim
  containers:
    - name: server
      image: nvidia/cuda:12.8.0-base-ubuntu24.04
      command: ["nvidia-smi", "-L"]
      resources:
        requests:
          cpu: "4"
          memory: "16Gi"
        limits:
          cpu: "8"
          memory: "32Gi"
        claims:
          - name: mig-gpu

Cluster Administrator Reference: How Dynamic MIG is Enabled

Dynamic MIG is enabled in the nvidia-dra-driver-gpu Helm chart via featureGates.DynamicMIG: true in nvidia-dra-driver-gpu-values.yaml:

featureGates:
  DynamicMIG: true

Because DynamicMIG publishes both type: "gpu" and type: "mig" devices under the gpu.nvidia.com driver in each node’s ResourceSlice, all full-GPU DeviceClass objects on Berkelium explicitly include device.attributes["gpu.nvidia.com"].type == "gpu" in their CEL selectors so full-GPU claims never accidentally match a fractional MIG slice.

---

NVIDIA Extended Resource Limits (nvidia.com/<model> & nvidia.com/gpu)

If you prefer classic Kubernetes resource declarations in resources.limits, Berkelium maps extended resource names directly to DRA DeviceClasses:

• nvidia.com/b300: "1" — NVIDIA B300 SXM6 AC (275 GB HBM3e)
• nvidia.com/h200: "1" — NVIDIA H200 NVL (141 GB HBM3e)
• nvidia.com/h100: "1" — NVIDIA H100 80GB HBM3
• nvidia.com/a100: "1" — NVIDIA A100 80GB PCIe
• nvidia.com/a40: "1" — NVIDIA A40 48GB
• nvidia.com/gpu: "1" — NVIDIA A40 48GB (default general-purpose GPU resource)

1. Default General-Purpose GPU (nvidia.com/gpu / NVIDIA A40)

apiVersion: v1
kind: Pod
metadata:
  name: simple-gpu-pod
  namespace: my-project
spec:
  restartPolicy: OnFailure
  containers:
    - name: cuda-app
      image: nvcr.io/nvidia/pytorch:24.04-py3
      command: ["nvidia-smi", "-L"]
      resources:
        requests:
          cpu: "8"
          memory: "32Gi"
        limits:
          cpu: "16"
          memory: "64Gi"
          nvidia.com/gpu: "1"

2. Targeting Specific GPU Models via Extended Resources

To schedule on A100, H100, H200, or B300 nodes using extended resources, specify the model’s resource key in resources.limits (and optionally resources.requests):

apiVersion: v1
kind: Pod
metadata:
  name: h100-extended-resource-pod
  namespace: my-project
spec:
  restartPolicy: OnFailure
  containers:
    - name: cuda-app
      image: nvcr.io/nvidia/pytorch:24.04-py3
      command: ["nvidia-smi", "-L"]
      resources:
        requests:
          cpu: "16"
          memory: "64Gi"
          nvidia.com/h100: "1"
        limits:
          cpu: "32"
          memory: "128Gi"
          nvidia.com/h100: "1" # Or nvidia.com/b300, nvidia.com/h200, nvidia.com/a100, nvidia.com/a40

---

Requesting AMD Instinct MI100 GPUs (ROCm)

Berkelium features AMD Instinct MI100 accelerators (32GB HBM2 each) managed by the AMD GPU Operator with ROCm support.

Method 1: Using Standard Resource Limits (amd.com/gpu)

The cluster-wide gpu.amd.com DeviceClass maps requests for amd.com/gpu directly to DRA under the hood:

apiVersion: v1
kind: Pod
metadata:
  name: rocm-pytorch-job
  namespace: my-project
spec:
  restartPolicy: OnFailure
  containers:
    - name: rocm-container
      image: rocm/pytorch:latest
      command: ["python3", "-c", "import torch; print(f'ROCm available: {torch.cuda.is_available()}, Devices: {torch.cuda.device_count()}, Name: {torch.cuda.get_device_name(0)}')"]
      resources:
        requests:
          cpu: "8"
          memory: "32Gi"
        limits:
          cpu: "8"
          memory: "32Gi"
          amd.com/gpu: "1"

Method 2: Using Dynamic Resource Allocation (ResourceClaimTemplate)

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: amd-gpu-claim
  namespace: my-project
spec:
  spec:
    devices:
      requests:
        - name: amd-gpu
          exactly:
            deviceClassName: gpu.amd.com
            count: 1
---
apiVersion: v1
kind: Pod
metadata:
  name: rocm-dra-job
  namespace: my-project
spec:
  restartPolicy: OnFailure
  resourceClaims:
    - name: gpu-claim
      resourceClaimTemplateName: amd-gpu-claim
  containers:
    - name: rocminfo
      image: rocm/rocm-terminal:latest
      command: ["rocminfo"]
      resources:
        requests:
          cpu: "4"
          memory: "16Gi"
        limits:
          cpu: "4"
          memory: "16Gi"
        claims:
          - name: gpu-claim

---

Supported DeviceClasses Reference

DeviceClass	Driver	Extended Resource	Hardware Match & Characteristics
nvidia-b300	gpu.nvidia.com	nvidia.com/b300	NVIDIA B300 SXM6 AC (275 GB HBM3e, Blackwell, NVLink)
nvidia-h200	gpu.nvidia.com	nvidia.com/h200	NVIDIA H200 NVL (141 GB HBM3e, Hopper)
nvidia-h100	gpu.nvidia.com	nvidia.com/h100	NVIDIA H100 80GB HBM3 (80 GB HBM3, Hopper, NVLink)
nvidia-a100	gpu.nvidia.com	nvidia.com/a100	NVIDIA A100 80GB PCIe (80 GB HBM2e, Ampere)
nvidia-a40	gpu.nvidia.com	nvidia.com/a40	NVIDIA A40 48GB (48 GB GDDR6, Ampere)
gpu.nvidia.com	gpu.nvidia.com	nvidia.com/gpu	NVIDIA A40 48GB (Default general-purpose NVIDIA GPU class)
mig.nvidia.com	gpu.nvidia.com	N/A	Dynamic NVIDIA Multi-Instance GPU (MIG) partitions (1g, 2g, 3g, 4g, 7g)
gpu.amd.com	gpu.amd.com	amd.com/gpu	AMD Instinct MI100 (32 GB HBM2, ROCm)

Cluster Resource Policies

Remember that Berkelium admission policies require explicit CPU and memory specifications for both requests and limits in every container to avoid fallback defaults.