Interactive Pods & Long-Idle Guidelines

Interactive pods enable live development, notebook prototyping, and terminal access to GPU hardware.

Strictly Prohibited: sleep infinity in Batch Jobs or GPU Workloads

Interactive pods (kind: Pod without a controller) are automatically terminated after 6 hours.

• Batch Jobs: Using sleep infinity, tail -f /dev/null, or any equivalent command in a Job is strictly prohibited—all jobs must execute an unattended task and exit by themselves when finished.
• Holding GPUs: Never use a Job or Deployment with sleep infinity to hold a GPU or reserve large CPU/memory requests.
• Lightweight CPU Utility Deployments: A Deployment running sleep infinity is acceptable only for lightweight CPU-only utility pods that do not reserve large resources (e.g., a small CPU/memory request with a larger limit, and no GPUs).

Launching an Interactive GPU Pod

Because GPU scheduling requires specifying resource limits (nvidia.com/gpu), you can launch an interactive session using either a declarative YAML manifest (recommended) or a one-liner with --overrides.

Option A: Using a YAML Manifest (Recommended)

Save the following as dev-session.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: dev-session
  namespace: <your-namespace>
spec:
  restartPolicy: Never
  containers:
    - name: dev-container
      image: pytorch/pytorch:2.3.0-cuda12.1-cudnn8-runtime
      stdin: true
      tty: true
      command: ["/bin/bash"]
      resources:
        limits:
          nvidia.com/gpu: "1"
          cpu: "8"
          memory: "32Gi"

Apply and attach to the session:

# Launch the pod
kubectl apply -f dev-session.yaml

# Attach to the running shell
kubectl attach -it dev-session -n <your-namespace>

# Or exec an independent interactive bash shell
kubectl exec -it dev-session -n <your-namespace> -- /bin/bash

Direct GitHub & Package Repository Access via NAT64

Berkelium provides native IPv6 routing along with automatic DNS64/NAT64 translation (2620:83:8000:64::/96). You can run git clone https://github.com/..., pip install, uv pip install, conda, and apt-get directly inside your interactive terminal without configuring any HTTP proxy variables. See Internet Egress & NAT64 for details.

Option B: One-Liner with kubectl run and --overrides

Since kubectl run does not have a native --limits flag, pass container resource limits using --overrides:

kubectl run dev-session \
  --namespace <your-namespace> \
  --image=pytorch/pytorch:2.3.0-cuda12.1-cudnn8-runtime \
  --restart=Never \
  --rm -it \
  --overrides='{
    "spec": {
      "containers": [{
        "name": "dev-session",
        "image": "pytorch/pytorch:2.3.0-cuda12.1-cudnn8-runtime",
        "stdin": true,
        "tty": true,
        "resources": {
          "limits": {
            "nvidia.com/gpu": "1",
            "cpu": "8",
            "memory": "32Gi"
          }
        }
      }]
    }
  }' -- /bin/bash

Option C: Quick CPU-Only Interactive Shell

For lightweight debugging or scripting that doesn’t require GPUs:

kubectl run dev-session \
  --namespace <your-namespace> \
  --image=ubuntu:22.04 \
  --restart=Never \
  --rm -it -- /bin/bash

---

Best Practices for Interactive Work

1. Clean up when done: Do not leave idle GPU pods running overnight without computations. When finished, exit and delete the pod:

   kubectl delete pod dev-session -n <your-namespace>

2. Mount Persistent Storage: Always mount a CephFS PVC so your notebooks, code edits, and conda environments persist across pod restarts.
3. Port Forwarding for Web UIs: If your pod runs TensorBoard, Streamlit, or JupyterLab on port 8888:

   kubectl port-forward dev-session 8888:8888 -n <your-namespace>

   Open http://localhost:8888 on your local laptop to access the interactive web interface securely via encrypted SSH/tunnel.