Prerequisites¶

The supported reference path for this project is a local Ubuntu-family host utilizing systemd, k3s, Helm, a working NVIDIA driver, and sufficient storage for container images and GGUF models.

Attempting to bypass these prerequisites will likely result in failure during the deployment of Layer 1 (the GPU Operator) or Layer 2 (Ollama scheduling).

Host Checklist¶

OS: 64-bit Ubuntu (22.04 LTS or newer) or Xubuntu with sudo access.
GPU: An NVIDIA GPU visible to the host system.
Kubernetes: k3s server active and kubectl correctly configured for the intended cluster.
Helm: Version 3.12 or newer.
Tools: Git and Go (1.21+) available for cloning and repository validation tools.
Python: Python 3.11+ (only required if building this documentation locally).
Storage: Free disk space for Helm charts, container images, Persistent Volume Claims (PVCs), and explicitly supplied model weights.
Model: A local GGUF model path or internet access to pull from the Ollama library.

Verification Commands¶

Run these exact commands to verify your environment without modifying the host state.

1. Verify NVIDIA Driver¶

nvidia-smi

Expected Output: A table showing the driver version (e.g., 535+), CUDA version, and active GPU devices. If this fails, install the proprietary NVIDIA drivers via ubuntu-drivers autoinstall.

2. Verify k3s and kubectl¶

sudo systemctl is-active k3s
kubectl config current-context
kubectl get nodes -o wide

Expected Output: systemctl should return active. The context should match your k3s instance (usually default). get nodes should show your node in the Ready state.

[!IMPORTANT] Confirm that the context and node are the ones you intend to use. Do not proceed just because a command returns a cluster: a stale ~/.kube/config can point to a cloud cluster instead of your local edge device!

3. Verify Helm, Go, and Git¶

helm version --short
go version
git --version

Expected Output: Versions that meet the minimum requirements (e.g., Helm v3.12+, Go 1.21+).

Capacity Planning Table¶

A 1 GiB-class GPU can accelerate a compact quantized model, but it generally cannot hold every layer. Treat num_gpu as a tuning input, using observed VRAM as the absolute guardrail. Start low, measure, and increase one step at a time.

Component / Task	Min RAM Required	Recommended RAM	Min Disk Space	Min GPU VRAM
OS + k3s baseline	2 GB	4 GB	10 GB	N/A
Telemetry Stack	1 GB	2 GB	5 GB	N/A
Ollama (7B Q4 model)	8 GB	16 GB	10 GB (for weights)	4 GB - 8 GB
Ollama (13B Q4 model)	16 GB	32 GB	20 GB (for weights)	8 GB - 12 GB

Memory requests across monitoring components can exceed a small edge node's capacity if every optional dashboard and collector is enabled. Begin with the verified minimal edge profile, then enable additional components only after measuring steady-state usage.

Inputs You Must Supply¶

The project repositories do not include model weights, secret credentials, TLS private keys, or kubeconfig data.

Keep GGUF files entirely outside Git.
Use absolute host paths only in a private, uncommitted values.local.yaml file.
Public examples and defaults use placeholder paths like /path/to/model.gguf.

[!CAUTION] Do not expose development defaults to the internet. Local profiles may relax authentication and network isolation for loopback (localhost) access convenience. Never bind these services to an untrusted external network interface without completing the hardening checklist.