Production Hardening Guide¶

This guide details the steps and configurations necessary to harden the Edge Computing LLM platform for production environments.

1. Network Security and Isolation¶

By default, Kubernetes clusters have a flat network. We enforce zero-trust networking within the cluster using Kubernetes NetworkPolicy.

Example NetworkPolicy¶

The following policy ensures that only Open WebUI can communicate with the Ollama API, preventing unauthorized internal pods from accessing the LLM directly.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-webui-to-ollama
  namespace: llm-observability
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: ollama
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: default
      podSelector:
        matchLabels:
          app.kubernetes.io/name: open-webui
    ports:
    - protocol: TCP
      port: 11434

2. Role-Based Access Control (RBAC)¶

Ensure that services operate with the principle of least privilege.

• The GPU Operator requires elevated permissions but its service accounts are tightly scoped to the gpu-operator namespace.
• Observability tools (Prometheus) have read-only cluster role access specifically tailored for scraping metrics endpoints.

[!CAUTION] Never bind the cluster-admin role to application service accounts.

3. TLS Termination¶

All external access to the Open WebUI and Grafana dashboards must be secured via HTTPS.

1. Deploy cert-manager (handled by edge-cli apply --profile heavy optionally).
2. Configure an Ingress resource with TLS:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: open-webui-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - llm.yourdomain.com
    secretName: llm-tls-secret
  rules:
  - host: llm.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: open-webui
            port:
              number: 8080

4. Image Pinning and Provenance¶

To prevent supply chain attacks, do not use latest tags. All Helm values and manifests must use specific image digests.

Bad: image: ollama/ollama:latest

Good: image: ollama/ollama@sha256:4a3b1...

5. Pod Security Standards (PSS)¶

Enforce Pod Security Standards at the namespace level to prevent privileged containers or host path mounts by rogue workloads.

apiVersion: v1
kind: Namespace
metadata:
  name: default
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

Note: The gpu-operator namespace requires the privileged profile due to hardware interaction requirements.

6. Runtime Security¶

Consider deploying a runtime security tool like Falco to monitor kernel system calls for anomalous behavior, such as: - Unexpected spawned shells within the Ollama container. - Modifications to critical files in /etc/. - Unauthorized network connections originating from the edge node.