AI Agent Workloads & MCP Endpoint

Berkelium provides a native Model Context Protocol (MCP) server endpoint that enables client AI agents—such as Claude Desktop, Cursor, Antigravity, or custom autonomous research agents—to interact directly with the cluster.

Through this endpoint, AI agents can inspect cluster hardware specifications and admission rules, as well as validate Kubernetes manifests against strict cluster policies before submitting them to the cluster.

---

Endpoint Details

Berkelium supports both the current Streamable HTTP transport (the official MCP standard since specification 2025-03-26) and legacy HTTP+SSE for backward compatibility:

Parameter	Streamable HTTP (Standard)	Legacy HTTP+SSE
Status	Recommended (MCP 2025-03-26+)	Deprecated in MCP spec; maintained for compatibility
Endpoint URL	https://berkelium.scs.lbl.gov/mcp	https://berkelium.scs.lbl.gov/mcp/sse
Message Endpoint	Same (https://berkelium.scs.lbl.gov/mcp)	https://berkelium.scs.lbl.gov/mcp/messages
Local Dev URL	http://localhost:8080/mcp	http://localhost:8080/mcp/sse
Authentication	None required (Open Access)	None required (Open Access)

Why Streamable HTTP?

The original MCP specification required two separate endpoints (/sse for streaming events and /messages for POST requests) with stateful server sessions. The MCP 2025-03-26 specification officially deprecated this split in favor of Streamable HTTP, which unifies JSON-RPC communication over a single HTTP endpoint with standard GET/POST methods and optional streaming responses.

---

Cluster Admission Policies Enforced by MCP

To prevent cluster destabilization and resource starvation, the MCP endpoint includes a built-in gatekeeper that validates manifests against admission rules. Manifests that fail any rule are rejected with clear, actionable diagnostics:

1. Mandatory Requests AND Limits:
   • Every container (including init containers) MUST define both resources.requests (cpu, memory) and resources.limits (cpu, memory).
   • If requesting GPUs via extended resources, nvidia.com/gpu (or amd.com/gpu, nvidia.com/b300, etc.) must be specified in limits.
   • Alternatively, when using Dynamic Resource Allocation (DRA), devices are declared via ResourceClaimTemplate and bound under container resources.claims.
2. Prohibition of sleep infinity in Batch Jobs & GPU Workloads:
   • Batch Jobs must execute active computation and exit automatically when finished.
   • Using sleep infinity, tail -f /dev/null, or indefinite sleep loops in a Job or to hold a GPU / large resource reservation is strictly rejected. (sleep infinity is only acceptable in lightweight CPU-only Deployments with small resource requests and no GPUs.)
   • For interactive GPU or debugging sessions, use an interactive Pod (6-hour timeout) or Coder.
3. StorageClasses ceph-rbd and cephfs:
   • PersistentVolumeClaims must specify storageClassName: ceph-rbd (ReadWriteOnce block storage) or storageClassName: cephfs (ReadWriteMany shared filesystem).
   • Legacy names like cephfs-sc, cephfs-replicated, or local-nvme-sc do not exist.
4. Local NVMe Scratch is Ephemeral (emptyDir):
   • High-throughput node-local NVMe scratch is strictly ephemeral. It is cleared whenever a pod stops or restarts.
   • Configure local scratch volumes using emptyDir: {} (with optional sizeLimit), never via PVCs.
5. Protected System Namespaces:
   • Manifests targeting system namespaces (kube-system, authentik, cert-manager, monitoring, ingress-nginx) are rejected. Workloads must target designated research namespaces.

---

Agent Configuration

1. Claude Desktop Setup

Add the Berkelium MCP server to your Claude Desktop configuration file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "berkelium": {
      "url": "https://berkelium.scs.lbl.gov/mcp"
    }
  }
}

(If using an older Claude Desktop build that only supports legacy SSE, use "url": "https://berkelium.scs.lbl.gov/mcp/sse").

Restart Claude Desktop. You will see the Berkelium cluster tools (with a hammer icon) available in the chat interface.

---

2. Cursor IDE Setup

1. Open Cursor Settings (Cmd + , or Ctrl + ,).
2. Navigate to Features > MCP Servers.
3. Click Add New MCP Server:
   • Name: berkelium
   • Type: http (or sse for legacy fallback)
   • URL: https://berkelium.scs.lbl.gov/mcp (or https://berkelium.scs.lbl.gov/mcp/sse for SSE)
4. The status indicator will turn green once connected.

---

3. Custom Python Agent (MCP SDK)

You can connect directly to the Berkelium MCP server from your own automated research scripts using the official mcp Python SDK:

import asyncio
from mcp import ClientSession
from mcp.client.streamable_http import streamable_http_client

async def run_scientific_agent():
    async with streamable_http_client("https://berkelium.scs.lbl.gov/mcp") as (read_stream, write_stream):
        async with ClientSession(read_stream, write_stream) as session:
            # Initialize session
            await session.initialize()

            # 1. Inspect cluster guidelines and available GPUs
            specs = await session.call_tool("get_cluster_rules_and_specs", {})
            print("Cluster Specs:\n", specs.content[0].text)

            # 2. Validate a manifest against cluster admission rules
            manifest = """
apiVersion: batch/v1
kind: Job
metadata:
  name: lbnl-agent-sim
  namespace: materials-sim
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
      - name: simulation
        image: python:3.11-slim
        command: ["python", "-c", "print('Running simulation on Berkelium!')"]
        resources:
          requests:
            cpu: "2"
            memory: "4Gi"
          limits:
            cpu: "4"
            memory: "8Gi"
"""
            validation = await session.call_tool("validate_manifest", {"yaml": manifest})
            print("Validation Result:\n", validation.content[0].text)

if __name__ == "__main__":
    asyncio.run(run_scientific_agent())

---

Exposed MCP Tools Reference

Tool Name	Parameters	Description
get_cluster_rules_and_specs	None	Returns admission policies, mandatory resource requirements, storage classes, GPU fleet specifications, and DRA DeviceClasses (B300, H200 NVL + NVLink domains, H100, A100, A40, Dynamic MIG, MI100).
validate_manifest	yaml (string)	Dry-run gatekeeper check. Parses multi-document YAML and returns validation status and specific remedy instructions.

MCP Resources

Resource URI	MIME Type	Description
berkelium://cluster-guidelines	text/markdown	Full cluster admission policies, resource guidelines, DRA DeviceClasses, and GPU hardware specifications.

---

Example Compliant GPU Workloads

Here are complete multi-document manifests conforming to all Berkelium rules:

• Mandatory CPU/Memory requests & limits
• POSIX shared storage via storageClassName: cephfs
• High-speed ephemeral scratch via emptyDir
• Validated accelerator allocation (via DRA or extended resource limits)

Method 1: Modern Dynamic Resource Allocation (DRA - Recommended)

This manifest allocates a dedicated NVIDIA B300 SXM6 AC Blackwell GPU via native Kubernetes DRA (resource.k8s.io/v1):

apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: b300-blackwell-claim
  namespace: materials-sim
spec:
  spec:
    devices:
      requests:
        - name: gpu
          exactly:
            deviceClassName: nvidia-b300  # Also available: nvidia-h200, nvidia-h100, nvidia-a100, nvidia-a40, gpu.nvidia.com (A40), mig.nvidia.com, gpu.amd.com
            count: 1
---
apiVersion: batch/v1
kind: Job
metadata:
  name: blackwell-training-job
  namespace: materials-sim
spec:
  backoffLimit: 2
  template:
    spec:
      restartPolicy: OnFailure
      resourceClaims:
        - name: gpu-claim
          resourceClaimTemplateName: b300-blackwell-claim
      containers:
        - name: trainer
          image: nvcr.io/nvidia/pytorch:25.01-py3
          command:
            - python
            - -c
            - "import torch; print(f'Training on {torch.cuda.get_device_name(0)}');"
          resources:
            requests:
              cpu: "16"
              memory: "64Gi"
            limits:
              cpu: "32"
              memory: "128Gi"
            claims:
              - name: gpu-claim
          volumeMounts:
            - name: shared-data
              mountPath: /data
            - name: local-scratch
              mountPath: /scratch
      volumes:
        - name: shared-data
          persistentVolumeClaim:
            claimName: project-shared-pvc # PVC must use storageClassName: cephfs
        - name: local-scratch
          emptyDir:
            sizeLimit: 100Gi # Node-local NVMe scratch (wiped on restart)

Method 2: Extended Resource Limits (nvidia.com/<model> & nvidia.com/gpu)

This manifest requests an NVIDIA H100 80GB HBM3 GPU via model-specific extended resource limits (nvidia.com/h100):

apiVersion: batch/v1
kind: Job
metadata:
  name: pytorch-training-job
  namespace: materials-sim
spec:
  backoffLimit: 2
  template:
    spec:
      restartPolicy: OnFailure
      containers:
        - name: trainer
          image: nvcr.io/nvidia/pytorch:24.04-py3
          command:
            - python
            - -c
            - "import torch; print(f'Training on {torch.cuda.get_device_name(0)}');"
          resources:
            requests:
              cpu: "16"
              memory: "64Gi"
              nvidia.com/h100: "1"
            limits:
              cpu: "32"
              memory: "128Gi"
              nvidia.com/h100: "1"
          volumeMounts:
            - name: shared-data
              mountPath: /data
            - name: local-scratch
              mountPath: /scratch
      volumes:
        - name: shared-data
          persistentVolumeClaim:
            claimName: project-shared-pvc # PVC must use storageClassName: cephfs
        - name: local-scratch
          emptyDir:
            sizeLimit: 100Gi # Node-local NVMe scratch (wiped on restart)