Skip to content
Berkelium Docs

Frequently Asked Questions (FAQ) & Support

General Questions

Section titled “General Questions”

Who can use Berkelium?

Section titled “Who can use Berkelium?”

Any researcher, staff scientist, postdoc, or visiting affiliate at Lawrence Berkeley National Laboratory with an active @lbl.gov OneID account. External collaborators can be added as namespace members by an LBNL project PI.

How is Berkelium different from Lawrencium?

Section titled “How is Berkelium different from Lawrencium?”
  • Lawrencium (Science IT HPC): Traditional Slurm-scheduled institutional HPC cluster designed for batch-queued compute jobs, multi-node MPI simulations, and shared environment modules (module load).
  • Berkelium (Science IT Kubernetes): Agile, cloud-native Kubernetes cluster built for containerized workloads, interactive Coder and Jupyter environments, persistent web services and APIs, KubeVirt VMs, and distributed AI/ML frameworks like Ray and PyTorch.

Are resources free for LBNL scientists?

Section titled “Are resources free for LBNL scientists?”

Standard development and scientific compute tiers are subsidized by Science IT baseline infrastructure. Dedicated multi-GPU allocations (e.g. 8x H100 dedicated nodes) or massive multi-petabyte Ceph storage allocations can be sponsored through division recharge accounts or grant projects.

Technical Questions

Section titled “Technical Questions”

Can I run a Job with sleep infinity (or tail -f /dev/null) to keep a container or GPU reserved?

Section titled “Can I run a Job with sleep infinity (or tail -f /dev/null) to keep a container or GPU reserved?”

No — sleep infinity in batch Jobs or to hold GPUs is strictly prohibited.

  • Batch Jobs (kind: Job): Must run active computations without human intervention and exit by themselves as soon as their task finishes. Save your outputs and checkpoints to a persistent ceph-rbd or cephfs PVC so the job can exit cleanly.
  • GPUs & Large Resource Reservations: Never use sleep infinity in any workload (Job or Deployment) to hold a GPU or reserve large CPU/memory requests. For interactive GPU work, use a bare Interactive Pod (6-hour timeout) or Coder.
  • When is sleep infinity allowed?: It is acceptable only in a CPU-only Deployment that does not reserve large resources (for example, a lightweight utility/file-management container configured with a small CPU/memory request, a larger burst limit, and no GPUs).

Why is my pod in Pending state?

Section titled “Why is my pod in Pending state?”

Common causes:

  1. Resource exhaustion: Your namespace has hit its ResourceQuota (check kubectl describe quota -n <namespace>).
  2. GPU constraints: You requested a GPU model (nvidia.com/gpu: 4) when all requested nodes are actively running other jobs.
  3. PVC Binding: Your PersistentVolumeClaim hasn’t bound or has an invalid storage class name.

How do I attach persistent storage to my containers?

Section titled “How do I attach persistent storage to my containers?”

Use ceph-rbd (ReadWriteOnce block storage — recommended for single-pod workloads, conda/pip environments, and small files) or cephfs (ReadWriteMany shared POSIX filesystem for multi-pod access) in your PersistentVolumeClaim manifest. Refer to our CephFS & RBD Storage Guide and Storage & PVC Tutorial.

How do I access GitHub or other IPv4 resources from inside my pod?

Section titled “How do I access GitHub or other IPv4 resources from inside my pod?”

Berkelium is an IPv6-native cluster with transparent DNS64 and NAT64 (2620:83:8000:64::/96) routing. Outbound connections to both IPv6 and IPv4-only services—including GitHub (github.com), PyPI, Hugging Face, and OS package repositories—work automatically inside running containers without configuring any HTTP proxy.

  • Always connect using DNS hostnames (e.g., https://github.com) so CoreDNS can synthesize NAT64 AAAA records automatically.
  • For architecture details and best practices, see the Internet Egress & NAT64 Guide.

How do I troubleshoot kubelogin errors (remote SSH, WSL, port 8000 in use, keyring errors)?

Section titled “How do I troubleshoot kubelogin errors (remote SSH, WSL, port 8000 in use, keyring errors)?”

If you experience authentication errors with kubectl and the kubelogin helper:

  • Headless / Remote SSH: If you cannot open a browser, add --grant-type=device-code and --skip-open-browser to your kubeconfig to log in via a printed terminal link.
  • Windows Subsystem for Linux (WSL): If the browser does not open, configure --browser-command="/mnt/c/Program Files/Google/Chrome/Application/chrome.exe".
  • Port Conflict: If port 8000 is taken, configure --listen-address=127.0.0.1:18000.
  • Keyring Error (/run/user/1000/bus not found): On headless Linux, configure --token-cache-storage=disk.
  • Stale Permissions / Token Refresh: Run kubectl oidc-login clean to force a new login.

For step-by-step examples, see the Cross-Platform Kubelogin Fixes.

Getting Technical Support

Section titled “Getting Technical Support”