Week 2 moves the stack onto Kubernetes. The first obstacle is the cluster itself. Standard distributions such as kubeadm or OpenShift can take 4 to 8 GB of RAM for the control plane before a single workload runs, which is most of a 16 GB laptop.

K3s and k3d

K3s is a CNCF-certified lightweight Kubernetes. It removes legacy cloud-provider drivers, in-tree storage plugins and alpha feature gates, and for a single server it can use an embedded SQLite datastore instead of etcd. It keeps full Kubernetes API compatibility, so the manifests you write here work on bigger clusters later.

k3d runs K3s inside Docker containers: one container per node.

Diagram: FIG 1 · A full Kubernetes control plane can consume several GB of RAM before it runs a workload. K3s is far lighter. Figures are approximate, so check yours with docker stats. See the animated version.

Diagram: FIG 2 · k3d runs each Kubernetes node as a Docker container, so a multi-node cluster fits on a laptop. A proxy container exposes ports 80 and 443. See the animated version.

Describing the cluster

A config file makes the cluster reproducible.

# k3d-config.yaml
apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
  name: ai-cluster
servers: 1
agents: 2
image: rancher/k3s:v1.30.0-k3s1
ports:
  - port: 80:80
    nodeFilters: [loadbalancer]
  - port: 443:443
    nodeFilters: [loadbalancer]
options:
  k3s:
    extraArgs:
      - arg: --disable=traefik          # we install our own ingress later (Envoy or Cilium)
        nodeFilters: [server:0]
      - arg: --disable=servicelb
        nodeFilters: [server:0]
  kubeconfig:
    updateDefaultKubeconfig: true
    switchCurrentContext: true

Diagram: FIG 3 · A declarative config pins the K3s version, sets one server and two agents, publishes ports through the proxy and disables the bundled ingress and service load balancer. See the animated version.

k3d cluster create --config k3d-config.yaml
kubectl get nodes -o wide
docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}"

The first command creates one server and two agents, and the cluster is usually ready in well under a minute. The second shows the three nodes as Ready, and the third shows how much memory each node container uses, so you can budget the rest of the machine for workloads. Remove everything with k3d cluster delete ai-cluster.

Disabling Traefik and the service load balancer is a choice for this series, because later lessons install Envoy and Cilium. If you just want an ingress working quickly, leave Traefik enabled, as the Week 2 capstone does. Also note that a plain k3d cluster does not expose a GPU to its nodes. GPU workloads need extra setup, so on a laptop it is normal to use CPU-friendly engines here.

Why real nodes matter

Because k3d creates actual multiple nodes, you get true multi-node scheduling on one machine: taints, tolerations and anti-affinity all behave as they would in production. You can keep stateful vector databases and stateless inference pods on separate nodes, so a noisy model pod cannot starve the database.

Diagram: FIG 4 · Because k3d creates real nodes, you can practise taints, tolerations and anti-affinity locally, so stateful vector databases and stateless inference pods stay on separate nodes. See the animated version.

A cluster that boots in seconds and disappears with one command makes experiments cheap, which is exactly what you want while learning.

Run it yourself

This is the exact example from the video. This really creates a K3d cluster from the file and deletes it again. I used host port 18080 to avoid clashes on my laptop.

apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
  name: ai-demo
servers: 1
agents: 1
image: rancher/k3s:v1.35.5-k3s1
ports:
  - port: 18080:80
    nodeFilters: [loadbalancer]
options:
  k3s:
    extraArgs:
      - arg: --disable=traefik
        nodeFilters: [server:0]

Run it:

k3d cluster create --config k3d-config.yaml

Output I got while recording the video:

ai-demo 1/1 1/1 true
k3d-ai-demo-agent-0 Ready
k3d-ai-demo-server-0 Ready
containers: 3
80/tcp -> 0.0.0.0:18080
traefik pods: 0

Coming Up Next

Day 9: Kubernetes core for machine learning: pods, Deployments and headless services.

Kubernetes #K3d #K3s #DevOps #CloudNative #PlatformEngineering #LocalDev #MLOps

Originally published at https://sureshpallapothu.in/blog/ai-int-day-8-k3d-lightweight-kubernetes, where this post includes animated diagrams.