Ownership boundaries¶

The Edge Computing LLM project is composed of several independent layers. Clear ownership boundaries ensure that a GPU driver failure is not confused with a model-serving failure, and that validation tools do not unexpectedly mutate cluster state.

[!IMPORTANT] The golden rule of troubleshooting this stack: Route a failure to the lowest layer that cannot satisfy its contract.

Layer Ownership Matrix¶

Layer	Component	Owner / Maintainer	Primary Responsibility
Layer 0	Host OS, Kernel, Drivers	Host Administrator	Provide a healthy Linux environment, storage, and raw hardware access (`nvidia-smi`).
Layer 1	k3s, NVIDIA Runtime	`k3s-nvidia-edge`	Provide a Kubernetes cluster capable of scheduling GPU workloads (`RuntimeClass`).
Layer 2	Ollama, Open WebUI, Stack	`llm-observability-stack`	Provide model serving, API endpoints, user interfaces, and workload scheduling.
Layer 3	Telemetry, Metrics, Alerts	`llm-observability-stack`	Provide scraping, data storage, and visualization dashboards.
Validation	Contract Tests, Diagnostics	`edge-llm-tests`, `gguf-observability`	Read-only verification of contracts without mutating state.
Orchestration	CLI Tools	`edge-cli`	Automate deployment, dependency gating, and diagnostics.

Failure Routing Decision Tree¶

Use this flowchart to determine which repository or component is responsible when an issue occurs.

flowchart TD
    Start[Issue Detected] --> S1{Does nvidia-smi work on host?}

    S1 -- No --> FailHost[Host Admin: Fix NVIDIA Driver]
    S1 -- Yes --> S2{Is nvidia.com/gpu allocatable in k3s?}

    S2 -- No --> FailL1[k3s-nvidia-edge: Fix GPU Operator/Runtime]
    S2 -- Yes --> S3{Is Ollama Pod Running?}

    S3 -- No --> FailL2A[llm-observability-stack: Check Helm values/scheduling]
    S3 -- Yes --> S4{Is Model loaded in ollama ps?}

    S4 -- No --> FailL2B[llm-observability-stack: Check Model registration]
    S4 -- Yes --> S5{Is Inference fast but metrics are missing?}

    S5 -- Yes --> FailL3[llm-observability-stack: Fix Prometheus/ServiceMonitors]
    S5 -- No --> Diagnostics[gguf-observability: Profile served model contract]

Detailed Troubleshooting Scenarios¶

1. `nvidia-smi` fails on the host¶

Owner: Host administrator Reason: The Kubernetes stack cannot repair a missing host driver. If the kernel module is missing or mismatched with the library, Layer 0 is broken. Action: Reinstall the NVIDIA driver natively on Ubuntu.

2. NVIDIA RuntimeClass is missing or Node has no allocatable GPU¶

Owner: k3s-nvidia-edge (Layer 1) Reason: Container runtime integration and device discovery belong to Layer 1. Action: Check the gpu-operator namespace. Ensure the nvidia-container-toolkit daemonset is running successfully.

3. Ollama pod is pending after GPU readiness¶

Owner: llm-observability-stack (Layer 2) Reason: Workload scheduling, resource requests, and Helm values belong to Layer 2. Action: Run kubectl describe pod -l app.kubernetes.io/name=ollama -n llm-observability. Check if requested memory/CPU exceeds node capacity.

4. Model serves but violates processor/profile contract¶

Owner: gguf-observability (for diagnosis) Reason: The model serves, but it might be running on CPU instead of GPU, or using too much RAM. Action: Use gguf-observability tools to validate the served-model contract. It validates without mutating runtime.

Rules that Prevent Drift¶

To maintain these strict ownership boundaries, all contributors and operators must adhere to these rules:

Do not copy Helm templates into the CLI. edge-cli must orchestrate Helm, not replace it.
Do not put application services in the substrate chart. Keep k3s-nvidia-edge strictly for infrastructure.
Do not make validation tools install or repair resources. Diagnostics must remain read-only.
Do not commit model weights or secret material to any repository. Use .gitignore and persistent volumes.
Keep generated evidence schema-stable and sanitized. Redact sensitive prompts from logs.
Prefer versioned profiles over hand-edited live objects. Avoid kubectl edit; update the Helm values.yaml instead.

Change Routing Examples¶

When updating the system, understand which components need changes:

Adding a New Model¶

A new model normally touches the Layer 2 model profile (values.yaml), the GGUF observer catalog, the test matrix, and this documentation. It should not require a Layer 1 change unless the accelerator/runtime contract itself changes (e.g., needing a new CUDA version).

Upgrading Prometheus¶

Updating the monitoring stack is a Layer 2/3 change. Update the llm-observability-stack Helm chart dependencies. No changes to edge-cli or k3s-nvidia-edge are required.

Fixing a GPU Driver Issue¶

If a kernel update breaks the GPU driver, this is a Layer 0 issue. The host administrator must resolve it via apt or DKMS. The Kubernetes resources should automatically recover once the node advertises nvidia.com/gpu again.