Microk8s vs k3s: Lightweight Kubernetes distribution showdown
What is Microk8s?
Microk8s is another lightweight Kubernetes distribution from Canonical (makers of Ubuntu) aimed at developers looking to set up and manage a Kubernetes cluster quickly.
Snap Package Installation
Microk8s leverages snap packages, allowing a seamless installation process. If you need to install snap, it is installed using the command:
sudo apt update
sudo apt install snapdOnce you have snap installed, you can install Microk8s using the following command:
sudo snap install microk8s classicThis enables rapid deployment on various Linux distributions.
To add nodes to your Microk8s cluster, you run the following command that will display the join token.
sudo microk8s add-nodeOn your other nodes, you will run the command:
sudo microk8s join <join token information>Portainer Microk8s installation
You may not know the latest versions of Portainer allow automated deployments of Microk8s. Simply point Portainer to vanilla Linux nodes you want to use for your Microk8s nodes, give it authentication information, customizations you want to implement, and then it will do the rest.
It gives Portainer a very Rancher-like configuration experience where it takes over and configures the nodes and brings up the Microk8s cluster. I covered this in a blog post here: Mikrok8s Automated Kubernetes Install with new Portainer Feature.
Development and Testing Purposes
With features designed for easy configuration, Microk8s is ideal for development, testing, and even full-fledged production environments.
With the easy snap installation and also the great new feature in Portainer allowing you to spin up a new Microk8s cluster in an automated way, it is a great Kubernetes distribution for the home lab.
High Availability and Storage Options
Microk8s offers high availability through multiple master nodes and provides various storage options. With Microk8s, you can use the following storage options:
NFS persistent storage
MicroCeph/Ceph storage
Automatic Updates
Through its snap package system, Microk8s can be set for automatic updates, ensuring users can always access the latest features and security enhancements.
Canonical mentions that Microk8s is a “zero-ops infrastructure”. With the easy snap updates, Canonical has automated updates for your Kubernetes cluster nodes, taking a lot of the heavy lifting out of the process.
Support for Cloud Providers and Networking
Microk8s is designed to integrate smoothly with different cloud providers, and its networking capabilities enable robust and scalable services.
Installing Microk8s on Different Linux Distributions
Microk8s is a production-ready Kubernetes distribution designed to run Kubernetes on various Linux distribution releases. Through a single command like snap install microk8s classic, developers can install and configure it, making it ideal for testing purposes or full-fledged operations.
Exploring Snap Packages and Configuration Options in Microk8s
Snap packages in microk8s simplify the installation process. Microk8s supports multiple master nodes, and automatic updates, and is configured to utilize hostpath storage efficiently. This production-ready Kubernetes distribution is suitable for different cloud providers, and its networking services are robust and scalable.
Microk8s and High Availability
Microk8s ensures high availability by supporting multiple master nodes. Its default storage options and control plane are designed to create production-grade clusters that suit various system resources and developers’ needs.
What is k3s?
K3s is a lightweight Kubernetes distribution developed to offer a simplified and efficient way to deploy Kubernetes, particularly in environments with limited resources. It’s a production-ready Kubernetes distribution designed with the following features:
Simplified Installation
K3s offers a single command for installation, making it a user-friendly option for both seasoned developers and those new to Kubernetes.
To install the master node(s), use the following:
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=v1.27.3+k3s1 sh -To install k3s worker nodes, use the following command:
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=v1.27.3+k3s1 K3S_URL=https://10.1.149.123:6443 K3S_TOKEN=<your join token> sh -Optimized for Edge Computing and IoT Devices
The minimal memory usage and GPU acceleration options make k3s suitable for running Kubernetes on edge computing platforms and IoT devices.
Built-in Container Runtimes
K3s comes with container runtimes configured by default, eliminating the need for additional configuration.
High Availability
Through multiple master nodes, k3s ensures high availability, making it a reliable option for critical applications. With k3s, you can spin up a multi-master cluster with embedded etc. You can also use the Kubevip project, which is an excellent way to share a virtual IP address between your multi-master nodes to ensure the IP address of your cluster API is always up and available.
Check out my write-up covering Kubevip here: Kube-VIP configuration for K3s control plane HA.
Also, check out my video of Kube-VIP and k3sup here:
Support for Various Linux Distributions
K3s is compatible with various Linux distributions, enhancing its appeal across different systems and user preferences.
It stands out for its minimal set, designed to run Kubernetes on IoT devices and edge computing environments. The configuration options in k3s, including single command installation, GPU acceleration, and lightweight Kubernetes clusters, make it versatile.
Memory Usage and Docker Containers in K3s
K3s’s minimal memory usage and compatibility with Docker containers make it ideal for various Linux distributions. Its high availability features and container runtimes allow for creating robust Kubernetes clusters.
Networking and Storage in K3s
K3s’s networking services are designed for high availability and low resource consumption. Its storage options, including hostpath storage, are configurable, providing flexible solutions for IoT devices and lightweight applications.
Issues monitoring?
Apparently, there are issues that can present themselves with attempting to monitor k3s effectively. Take a look at the post here on GitHub: Expose kube-scheduler, kube-proxy and kube-controller metrics endpoints · Issue #3619 · k3s-io/k3s (github.com).
K3s consolidates all metrics (apiserver, kubelet, kube-proxy, kube-scheduler, kube-controller) at each metrics endpoint, unlike the separate metric for the embedded etcd database on port 2831, if utilized. If you heed the guidance provided in this issue and establish scrape jobs for each individual component, you will end up collecting the same metrics five times over, thereby unnecessarily consuming Prometheus resources.
Comparing Microk8s and K3s
Kubernetes Configuration: Microk8s vs K3s
K3s and microk8s may seem to offer essentially the same basic thing (simple Kubernetes), but they offer different approaches to Kubernetes configuration. While microk8s uses snap packages and is designed for various Linux distributions, k3s focuses on minimal resources for running Kubernetes on IoT devices and edge computing.
Ultimately, both are much easier to configure than vanilla K8s standard Kubernetes cluster configurations and have certainly lowered the barrier of entry in spinning up a Kubernetes cluster.
Understanding Kubernetes Clusters: Single Node vs Multiple Master Nodes
Both distributions offer single-node and multi-master cluster options. Microk8s and k3s can create clusters with multiple master nodes. These configurations ensure high availability. Although comparing the two, Microk8s is easier to configure.
In fact, with Microk8s if you build a cluster that has three or more nodes enabled, high-availability is automatically enabled.
K3s offers an embedded etcd cluster that allows building out an odd number of nodes for etcd and quorum. For a cluster with n servers, quorum is (n/2)+1. One thing to note is Rancher details that embedded etcd (HA) may have performance issues on slower disks such as Raspberry Pis running with SD cards.
Storage and GPU Acceleration: Comparing Features
From host path storage to GPU acceleration, microk8s and k3s provide different solutions. Microk8s offers more configuration options, while k3s’s lightweight design ensures efficient memory usage and quick operations.
The Installation Experience: Snap Packages and Command Line Tools
Microk8s: Snap Installation and Configuration
Microk8s offers the snap install microk8s classic command, a single-line tool to install the distribution on Ubuntu and other Linux distributions. Snap packages streamline the installation process, enabling developers to create Kubernetes clusters efficiently.
K3s: Simplified Installation with a Single Command
K3s offers a similar streamlined experience with a single command for installation. GPU acceleration and other configuration options are accessible through basic command line tools, without the need for complex setup procedures.
All in all, I think Microk8s makes installation a tad easier with snap and the join process to add nodes doesn’t require you to cat files to find join tokens, etc. So, for ones just getting started Microk8s is a bit easier.
Performance Evaluation: Memory Usage and Cluster Resources
Analyzing Memory Usage in Microk8s and K3s
Both microk8s and k3s focus on lightweight performance, minimizing memory usage. K3s is particularly designed to run Kubernetes on edge and IoT devices, where resource constraints are common. Either distribution will work out well in the home lab for those looking for a lightweight Kubernetes distribution to learn about the inner workings of K8s without many resources.
Cluster Resources and High Availability
High availability, achieved through multiple master nodes, ensures uninterrupted service. Both microk8s and k3s provide this feature. For all practical purposes, I think both these distributions are a tie in this category as they offer great features.
Deployment Scenarios: IoT Devices, Edge Computing, and Virtual Machines
Kubernetes on IoT Devices: Microk8s and K3s Compared
IoT devices benefit from lightweight Kubernetes distributions. While microk8s and k3s are suitable, k3s’s minimal features might be more suitable for devices with limited resources. However, with multi-master Kubernetes nodes, it is noted the embedded etcd will wear out SD cards in devices like Raspberry Pis.
Edge Computing with Kubernetes
Edge computing requires lightweight, responsive solutions. Both microk8s and k3s have been developed with this in mind, offering GPU acceleration and other features tailored to edge environments.
Virtual Machines and Containers
Both distributions can be run inside a virtual machine. K3s can be run inside Docker containers using K3D, giving developers flexibility in their development and testing environments. Check out my post on using K3D with k3s here: Install K3s on Ubuntu with K3D in Docker.
Security and Automatic Updates
Ensuring Security in Kubernetes Distributions
Security is vital in choosing a Kubernetes distribution. Both microk8s and k3s come with default configurations that prioritize security. With Microk8s over the air updates and easy updating with snap, you can easily keep your cluster up to date. Both of them with their minimalistic designs helps to reduce the attack surface of the distributions.
Automatic Updates: Keeping Your System Current
Microk8s, through its snap package system, offers automatic updates. This feature ensures developers always work with the latest, most secure version without manual intervention. K3s isn’t quite as easy to update as Microk8s with the snap update procedure, but it is a single binary, making updates very easy regardless.
Monitoring
As mentioned in the k3s section, there are challenges when monitoring k3s with tools like Prometheus and Grafana. Microk8s doesn’t have the same known issues as Microk8s with monitoring, so this may definitely be a win on the side of Microk8s.
Wrapping up
For me, when comparing Microk8s vs k3s, they are both awesome distributions. I would recommend either distribution in the home lab. I think Microk8s is a tad easier to get started with as Canonical has made it super easy to get up and running using the snap installation method and enabling and disabling components in your Kubernetes cluster.
However, k3s is equally powerful, and I think it offers an experience more aligned with what you will see in most other Kubernetes environments and management workflows. There are also a TON of projects out there that make use of and integrate with k3s. We have already mentioned Kube-Vip, but there is also Kubespray, K3D, and countless other projects that take advantage of k3s.
There are challenges with K3s and monitoring with issues ones have seen with k3s and Prometheus/Grafana stacks. So, keep this in mind as well.
With their lightweight nature and extensive configuration options, both Microk8s and k3s are strong contenders for any developer, operation team, or home lab enthusiast looking to harness the power of Kubernetes.
