Skip to content

NVIDIA Layer Installation Guide

This guide details the deployment of the NVIDIA GPU Operator on your k3s cluster, which automates the lifecycle of software required to provision GPUs.

1. Overview of the GPU Operator

The NVIDIA GPU Operator uses the operator framework within Kubernetes to automate the management of all NVIDIA software components needed to provision GPUs. These components include: - NVIDIA Container Toolkit - NVIDIA Device Plugin - NVIDIA DCGM Exporter - Node Feature Discovery (NFD)

2. Deployment via edge-cli

The easiest way to deploy the NVIDIA layer is using our custom CLI tool.

cd /path/to/edge-cli
go run main.go install --profile gpu

This command runs through the preparation steps, rendering the Helm templates, and applying them.

3. Deployment via Helm (Manual Method)

If you prefer to install directly via Helm, follow these steps.

Add the NVIDIA Helm Repository

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
helm repo update

Install the GPU Operator

We create a dedicated namespace and deploy the operator with specific flags optimized for our edge k3s environment. Crucially, we use containerd as the CRI and tell the operator not to install drivers (since they are on the host).

kubectl create namespace gpu-operator

helm install -n gpu-operator gpu-operator nvidia/gpu-operator \
  --set driver.enabled=false \
  --set toolkit.enabled=true \
  --set toolkit.env[0].name=CONTAINERD_CONFIG \
  --set toolkit.env[0].value=/var/lib/rancher/k3s/agent/etc/containerd/config.toml \
  --set toolkit.env[1].name=CONTAINERD_SOCKET \
  --set toolkit.env[1].value=/run/k3s/containerd/containerd.sock \
  --set toolkit.env[2].name=CONTAINERD_RUNTIME_CLASS \
  --set toolkit.env[2].value=nvidia \
  --set toolkit.env[3].name=CONTAINERD_SET_AS_DEFAULT \
  --set toolkit.env[3].value=true

4. Verification

It's critical to ensure all components are running correctly.

Check Pods in gpu-operator Namespace

Run the following command to see all operator pods:

kubectl get pods -n gpu-operator

Expected Output: 

NAME                                       READY   STATUS      RESTARTS   AGE
gpu-operator-6b797b5f79-q2f2m              1/1     Running     0          5m
nfd-gc-68d87b9985-5h2r5                    1/1     Running     0          5m
nfd-master-7f6b9c9c8-4zt8f                 1/1     Running     0          5m
nfd-worker-p2v8c                           1/1     Running     0          5m
nvidia-container-toolkit-daemonset-29k8f   1/1     Running     0          4m
nvidia-cuda-validator-x7v9h                0/1     Completed   0          4m
nvidia-dcgm-exporter-l8m4d                 1/1     Running     0          4m
nvidia-device-plugin-daemonset-5j2n7       1/1     Running     0          4m
nvidia-operator-validator-2c4f8            1/1     Running     0          4m

[!IMPORTANT] The nvidia-cuda-validator should be in the Completed state. If it is CrashLoopBackOff, it indicates a problem with the container toolkit configuration or driver mismatch.

Verify RuntimeClass

The operator should create an nvidia RuntimeClass.

kubectl get runtimeclass

Expected Output: 

NAME     HANDLER   AGE
nvidia   nvidia    5m

Verify Node GPU Capacity

The device plugin advertises the GPU capacity to the k3s node.

kubectl describe node | grep -A 5 "Capacity:"

Expected Output: 

Capacity:
  cpu:                12
  ephemeral-storage:  239029984Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             32688056Ki
  nvidia.com/gpu:     1
  pods:               110

Notice the nvidia.com/gpu: 1 entry.

5. Troubleshooting the NVIDIA Layer

If you do not see the nvidia.com/gpu capacity:

  1. Check NFD logs: kubectl logs -n gpu-operator -l app=nfd-worker
  2. Check Device Plugin logs: kubectl logs -n gpu-operator -l app=nvidia-device-plugin-daemonset
  3. Verify k3s config: Ensure k3s containerd config at /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl was updated if manual templating was required (the operator usually handles this via the toolkit daemonset).

6. Architecture Flow

graph TD
    A[k3s Node] --> B(GPU Operator)
    B --> C(NFD)
    C -->|Detects GPU| B
    B --> D(Container Toolkit)
    D -->|Configures containerd| A
    B --> E(Device Plugin)
    E -->|Advertises nvidia.com/gpu| A
    B --> F(DCGM Exporter)
    F -->|Exposes GPU Metrics| G(Prometheus)