Introduction to Kubernetes for HPC
If you have experience submitting Slurm batch scripts (sbatch, srun, squeue) on cluster systems, this guide provides a direct conceptual mapping to Kubernetes (kubectl, Pods, Jobs).
Conceptual Mapping: Slurm vs. Kubernetes
Section titled “Conceptual Mapping: Slurm vs. Kubernetes”| Slurm Concept | Kubernetes Equivalent | Purpose |
|---|---|---|
sbatch job.sh | kubectl apply -f job.yaml | Submits a batch computation to the scheduler |
squeue -u $USER | kubectl get pods -n <namespace> | Checks the status of active and queued tasks |
scancel <jobid> | kubectl delete job <job-name> | Cancels and terminates running execution |
#SBATCH --gpus=2 | resources.limits: {"nvidia.com/gpu": 2} | Requests hardware GPU accelerators |
#SBATCH --mem=64G | resources.limits: {memory: "64Gi"} | Enforces RAM limits |
Module Load (module load cuda) | Docker Image (image: pytorch/pytorch:2.3.0-cuda12.1) | Fully encapsulated runtime environment |
Cluster Scratch (/global/scratch) | PersistentVolumeClaim (CephFS/NVMe) | Shared high-throughput scientific storage |
Why Kubernetes for Science?
Section titled “Why Kubernetes for Science?”- Complete Reproducibility: No missing shared libraries or conflicting Python virtualenvs. Everything is packaged in standard OCI containers.
- Instant Microservices: Host interactive Dash/Streamlit visualization portals, FastAPI model servers, and notebooks with automatic URL routing.
- Resilience & Auto-Healing: If a worker node suffers a hardware fault, Kubernetes automatically restarts your pod on a healthy node.
- Dynamic Elastic Scaling: Auto-scale Ray or Dask workers based on real-time task queue depth.