Nomad vs. Kubernetes: A complete comparison

Let’s look at the differences between nomad vs kubernetes and details about each. Take a look at the comparison below.

What is HashiCorp Nomad?

Nomad comes from the well-known Hashicorp corporation. Recently bought by IBM, Hashicorp is known for its IaC solutions like Terraform and Packer. Nomad is an open source solution that provides scheduling of workloads.

It is a platform-agnostic tool that can manage many different workloads, including Linux containers, virtual machines, and also even standard apps you may run.

Many are starting to take notice of Nomad as a great alternative to Kubernetes for running highly-available workloads in the enterprise.

Below, you can see after downloading the nomad executable, running the command without any parameters we can see the available commands with the solution.

Learn more about Hashicorp Nomad here: Nomad by HashiCorp (nomadproject.io)

Nomad Architecture

Nomad is made up of Nomad servers and Nomad clients. The Nomad server schedules workloads and manages the cluster. The Nomad clients are like Kubernetes workers, these are responsible for running workloads. They also let the server know of statuses and other data. Nomad also has a web-based user interface. It includes a command-line tool for managing the cluster.

For a UI for a redis-job as an example, you can use the command below:

nomad ui redis-job

It supports several types of workloads. These include Docker containers. However, as mentioned earlier, it also can orchestrate VMs, and even standalone applications.

Nomad Features

Nomad provides several features. These include:

• Multi-cluster management

• You can do rolling updates for containerized applications using Nomad

• It supports external services. These include services such as databases and message queues

• Nomad supports virtual machines and containers both

• It supports role-based access using ACL lists to control access to the cluster and manage the permissions of users and applications

What is Kubernetes?

Kubernetes is a container orchestration tool. It can orchestrate containers (called pods). It automates deployment and scaling of your containerized apps. It is also designed for large and very complex deployments as you can imagine from Google’s infrastructure.

Learn more about Kubernetes here: Kubernetes.io

Kubernetes Architecture

Kubernetes has a complex architecture with several components and pieces. These include the Kubernetes control plane and worker nodes. The control plane has components like the API server, the etcd database, kube-scheduler, the kube-controller-manager, and the kubelet service. The worker nodes run workloads and report their status to the Kubernetes control plane.

Kubernetes Features

Kubernetes provides several features that include:

• Can manage large environments

• Can configure built-in load balancing

• Kubernetes provides built-in service discovery support

• Kubernetes provides built-in support for storage orchestration

• Kubernetes enables rolling updates for containerized applications

• Secrets management is built in

Cluster Management

Nomad uses a centralized management plane to manage clusters. The Nomad server is responsible for scheduling workloads, managing worker nodes, and providing a unified workflow for application deployment.

Nomad has a lightweight resource manager that can run on any platform. It is a portable binary agent that can be installed on data centers or public clouds.

On the other hand, Kubernetes uses a centralized control plane to manage clusters. The Kubernetes control plane includes several components. These include the API server, etcd, kube-scheduler, kube-controller-manager, and kubelet.

The control plane components are responsible for scheduling workloads, managing worker nodes, and providing a unified workflow. Kubernetes has a more complex installation process but is designed to support large-scale, highly scalable clusters.

Container Orchestration

Both Nomad vs. Kubernetes are container orchestration tools. It means they natively handle Linux containers. They support container storage interface (CSI) for efficient storage orchestration. They also support service discovery and load balancing.

Nomad enables rolling updates for containerized applications. It means that a new version of the application can be deployed without any downtime. Nomad ensures self-healing by automatically rolling back to the previous version if an update fails.

You can add the update section in the job specification:

job "geo-api-server" {
  # ...

  group "api-server" {
    count = 6

    # Add an update stanza to enable rolling updates of the service
    update {
      max_parallel     = 2
      min_healthy_time = "30s"
      healthy_deadline = "10m"
    }

    task "server" {
      driver = "docker"

      config {
        image = "geo-api-server:0.1"
      }

      # ...
    }
  }
}

Kubernetes also supports running multiple versions of an application simultaneously. However, it requires more configuration to achieve rolling updates. Kubernetes has a more fragmented distribution compared to Nomad. It has different distributions, including the upstream version and third-party distributions such as OpenShift, Rancher, and Docker Enterprise.

External Services

Nomad vs. Kubernetes support external services, such as databases and message queues, that are not containerized. Nomad has built-in support for external services and can manage their lifecycle alongside containerized applications. Kubernetes, on the other hand, requires third-party tools such as Helm and Operators to manage external services.

Virtual Machines

Nomad supports virtualized workloads alongside containerized applications, while Kubernetes is primarily designed for containerized workloads.

However, Kubernetes has a project called KubeVirt that can be used to run virtual machines and many are seeing this as a viable option for running virtual machines. Rancher Harvester is a solution that allows running VMs on Kubernetes.

• Kubevirt Running VMs on a Kubernetes Cluster

Nomad Advantages

Nomad has several advantages over Kubernetes. First, many note it is a single binary that can be installed on any platform, providing a portable and flexible workload support. However, there are variants of Kubernetes like k3s that is a single binary as well.

Second, it has a simpler installation process and a lightweight resource manager, which makes it a good choice for small to medium-sized deployments. Third, Nomad natively handles multi-datacenter deployments and can span multiple clusters, providing a unified workflow for application deployment.

Kubernetes Advantages

Kubernetes has a strong community effort and support from major cloud providers such as Google Cloud, Microsoft Azure, and IBM Cloud. It supports running stateful workloads like databases and provides efficient storage orchestration.

It has a lot of third-party tools that can greatly extend its functionality and what you can do with it. Additionally, Kubernetes has a centralized control plane, which provides a unified interface for cluster management and workload scheduling.

Multiple clusters

Nomad is designed to manage multiple clusters, including clusters that span multiple data centers and regions. It provides a unified workflow for deploying applications across clusters. Kubernetes has a means for multiple clusters but each one must be managed independently.

Resource Requirements

Nomad provides a more flexible platform for deploying workloads. It supports virtualized workloads, standalone applications, and external services. Nomad can manage various resource requirements, from small to large, from simple to complex.

Self-Healing Nodes

Nomad can roll back to the previous version of an application if an update fails. It can also enable self-healing nodes. This means that if a worker node fails, Nomad can automatically spin up the failed workload on a healthy machine. Kubernetes also supports self-healing nodes, but it requires additional configuration.

Community Support and learning

Community support is an extremely important aspect of learning any new technology. Both Nomad and Kubernetes have strong community support. Nomad has a smaller community than the Kubernetes community but is multiplying.

Nomad is supported by HashiCorp however is gaining more traction in the community with a broader knowledge of the product. Kubernetes has a large and active community with many contributors and users, and it isn’t hard to find blog posts, YouTube videos, and many other resources describing and teaching K8s.

Steps to create a Nomad cluster

Setting up a Nomad cluster involves configuring Nomad servers and Nomad clients and deploying workloads to the cluster.

Install Nomad

The first step in setting up a Nomad cluster is to install Nomad on each server and client machine. Nomad is installed using a single binary. You can download pre-built packages for popular Linux distributions.

Configure Nomad Servers, Agents, and Workloads

After installing Nomad, the next step is to configure Nomad servers.

The Nomad Server is determined by a special configuration file found in the /etc/nomad.d/nomad.hcl file. The file determines if a Nomad node is a server or an agent.

Here is an example configuration file:

# /etc/nomad.d/nomad.hcl

# General configurations
datacenter = "dc1"
data_dir = "/opt/nomad/data"
log_level = "INFO"
region = "global"

# Network configurations
bind_addr = "0.0.0.0" # The address to bind for network services

# Advertise address
advertise {
  http = "192.168.1.100:4646" # Replace with the IP of the server or agent
  rpc  = "192.168.1.100:4647"
  serf = "192.168.1.100:4648"
}

# Server-specific configurations
server {
  enabled = true
  bootstrap_expect = 3
}

# Client-specific configurations
client {
  enabled = true
  servers = ["192.168.1.100:4647", "192.168.1.101:4647", "192.168.1.102:4647"] # List of server addresses
}

# ACL configurations (if needed)
acl {
  enabled = true
  token_ttl = "30m"
  policy_ttl = "30m"
  replication_token = "your_replication_token_here"
}

# Consul integration (if needed)
consul {
  address = "127.0.0.1:8500" # Consul agent address
}

# Telemetry configurations
telemetry {
  collection_interval = "1m"
  publish_allocation_metrics = true
  publish_node_metrics = true
}

You can see the server status by issuing the command:

nomad server members

Note the information you see below:

The next step is to configure Nomad agents. Nomad agents are the nodes that run workloads and take directions from the Nomad serveryou set up. You use the same configuration file shown above, found at /etc/nomad.d/nomad.hcl.

After configuring the Nomad server and client, the next step is to deploy workloads to the cluster. Workloads can be defined in a job file, which specifies the container image, resource constraints, and network configuration. Nomad supports several types of workloads, including Docker containers, virtual machines, and standalone applications.

Steps to create a Kubernetes cluster

Kubernetes installation involves configuring the Kubernetes control plane and worker nodes, and deploying workloads to the cluster.

Kubernetes Distribution

The first thing you will need to do is choose a Kubernetes distribution. Many are available, including distributions like OpenShift, Rancher, VMware Tanzu, K3s, etc. Each distribution has its installation process and set of features.

Kubernetes Control Plane

The control plane components should be installed on dedicated servers or virtual machines. The components should be configured with a unique cluster name and security policies.

Install Kubernetes Worker Nodes

Worker nodes run workloads and report their status to the Kubernetes control plane. Kubernetes worker nodes should be installed on dedicated servers or virtual machines. The worker nodes should be configured to communicate with the Kubernetes control plane through a shared network interface.

Deploy Workloads

After installing the Kubernetes control plane and worker nodes, the next step is to deploy workloads to the cluster. Workloads can be stood up in a Kubernetes manifest file. Here, you can specify the container image, resource constraints, and network configuration. Kubernetes supports several workloads. These include Deployments, StatefulSets, DaemonSets, and Jobs.

Wrapping Up

Hopefully, this comparison between Nomad vs. Kubernetes helps to showcase the similarities between the two technologies, but also their distinct differences. Nomad is a lightweight, portable, and flexible platform that can manage various workloads. It is a good choice for small SMB deployments and natively handles multi-datacenter deployments and workloads spanning multiple clusters. Kubernetes is the 400 lbs gorilla in this space and is known and trusted by many. However, depending on your use case, Nomad may be a viable option.