Learning Kubernetes in Docker Desktop

I recently built a comprehensive hands-on training course for Kubernetes on top of Docker Desktop — https://github.com/jasonumiker/kubernetes-training.

I was interested in seeing how far I could take that given how easy an entry-point it is for people. And found it to be a very viable solution to the “local Kubernetes” problem for a few reasons:

1. It allows you to learn for ‘free’ with nearly any machine without needing to pay for cloud services like EKS, GKE or AKS — or buy a new Raspberry Pi cluster etc. — to get started. That ‘free’ is assuming that you are not a business that would need to pay for Docker Desktop (a company of more than 250 employees OR more than $10 million in annual revenue) and has not done so. Though, you should still be able to use it for free on a personal laptop for learning like this in my understanding…
2. It works nearly the same on the Mac as well as on Windows — so the training/instructions would be useful for people using either OS.
3. It is actually built on a fairly standard kubeadm-based Kubernetes under-the-hood — which is both easy to work out how to automate it and also makes it a good candidate to learn (i.e. it isn’t something ‘strange’ — it is normal Kubernetes). Kubeadm is the Kubernetes-type that is used in the CKA and CKS exams I took — so understanding how that works will help you a bit in your certification journey too.
4. It is perhaps the simplest way to get started with Kubernetes (assuming that you are already using Docker Desktop on your Mac or Windows machine) — you just turn on one switch in its settings and are good to go.
5. It has a built-in LoadBalancer service controller that auto-magically wires any LoadBalancer services in to localhost on your host machine— so you can work with ‘standard’ LoadBalancer Services easily as part of your learning.
6. It also has the fastest and simplest fix for any issues — if you ‘break’ anything you can just click a button in the UI to reset the Kubernetes or even the whole VM back to a fresh default state within minutes. Which, I’d argue, is a key part of making it a safe and easy place to learn!

I also learned a few things along the way that I thought might be useful to others using the Kubernetes that is part of Docker Desktop that I wanted to share.

1. The Kubernetes is installed right in the docker-desktop VM. And you can ‘break into’ that with this command. nsenter is a tool that allows you to switch Linux namespaces and — privileged and — pid=host allows you the access via docker for your container for it to switch to the host’s.

docker run -it --rm --privileged --pid=host debian:12.8 nsenter -t 1 -m -u -n -i

1. You can reconfigure the Kubernetes that is part of Docker Desktop to be able to have its metrics collected by kube-prometheus-stack using commands like that one by to switch various services to listen on 0.0.0.0 instead of 127.0.0.1. I want to thank SpoddyCoder on GitHub for this script which got me most of the way there (with a little tweaking) — https://github.com/jasonumiker/kubernetes-training/blob/main/monitoring/docker-desktop-update.sh
2. The one thing that was missing from the Prometheus was that the cAdvisor container-level labels on the metrics scraped via the kubelet. It had the Pod-level ones but not the container-level ones. I reconfigured the Prometheus stack and adapter around that and got everything working — but the missing labels did still leave many of the built-in Grafana dashboards of kube-prometheus-stack which expect them broken unfortunately. See how I configured it all in the end here — https://github.com/jasonumiker/kubernetes-training/tree/main/monitoring. My theory on why they are missing is that Docker Desktop is using the non-standard CRI cri-docker (to co-exist with Docker in Desktop) https://github.com/kubernetes/kubernetes/issues/122182 — but I am not sure in the end. Does anybody know more?
3. You can even inject multiple users in into Docker Desktop to try out assigning different users different Roles/ClusterRoles in different kubectl contexts to explore Kubernetes API authorization and security. See how I did that via token-based authentication here — https://github.com/jasonumiker/kubernetes-training/tree/main/k8s-authz
4. You can run Istio on Docker Desktop to learn about that as well— though without their CNI (since Docker Desktop doesn’t have a normal CNI to chain it off of from what I can tell). The only downside to that is that it is the ‘full’ sidecar-based Istio and that it will require the privileged init containers be injected into each Pod as well. See how I did that here — https://github.com/jasonumiker/kubernetes-training/tree/main/istio. Note that I did try briefly to get Istio’s Ambient Mesh to work and that I was unable to — I think it would need a more real/complete CNI for that.
5. You can explore workloads that need PersistentVolumes with a ‘mock’ Storage Controller that lets you experience dynamic/automatic provisioning but just maps back to host volumes (which is okay for our single-Node setup and learning). I borrowed this one from microk8s’s built-in hostpath-storage add-on — https://github.com/jasonumiker/kubernetes-training/blob/main/pvs-and-statefulsets/hostpath-provisioner.yaml

The further on this journey, the more impressed I got with just how much of Kubernetes and the ecosystem I could get running on a MacBook with only the default 8GB of RAM dedicated to its Docker VM. The topics I was able to cover here ultimately included:

    Kubernetes Training
        Table of Contents
        Prerequisites
        Pods, Probes, Services, ReplicaSets, Deployments and StatefulSets
            A Pod
            A Service
            Probes
            ReplicaSets
            Deployments
            Quarantine a Pod (by removing the label from it that the operators are selecting it on)?
            Sidecar and Init containers within a Pod
            PersistentVolumes, PersistentVolumeClaims and StorageClasses
            StatefulSets
            ConfigMaps and Secrets
            DaemonSets
        Requests, Limits and Scaling Pods
            First let's install Prometheus for Metrics/Monitoring
            The Horizontal Pod Autoscaler (HPA)
            CPU and Memory Requests
            CPU and Memory Limits - and how misconfiguring them can really hurt your performance and availability
            Kubernetes Event-driven Autoscaling (KEDA)
        Jobs and CronJobs
        Kubernetes Namespaces and API Authorization (via Roles/ClusterRoles)
        Ingress
            What is 'wrong' with Ingress for it to need to be eventually replaced (by Gateway)?
        Gateway
        Istio
            Carefully consider whether you need a Service Mesh like Istio
        Kustomize and Helm
            Kustomize
            Helm
            What is a Custom Resource Definition (CRD)?
        Controllers/Operators
            Admission Controllers / OPA Gatekeeper
        GitOps with Argo CD
        Progressive Delivery with Argo Rollouts
        Kubernetes Pod Security / Multi-tenancy Considerations
            Why is this a concern?
            How to strengthen pod-level security?
        Other topics that we didn't cover because Docker Desktop's K8s is not suitable for exploring them

Please have a look at the training — I hope you find it interesting/useful. And I hope that some of the tips above and in the repo on getting various things working with Docker Desktop’s Kubernetes (many of which took me hours of tinkering and Googling to get to) save you some time if you are doing something similar.