Automating Your Home Lab with Ansible, Terraform, and CI/CD Pipelines

There has never been a better time to automate your home lab with tools like Ansible, Terraform, and CI/CD pipelines. With the AI tools and help we have today and all the resources on available on the web, automation is something that can be achieved much easier than ever before. Automation with these tools helps to transform your processes for deploying and managing infrastructure from being tedious, to error-free and easy. We will look at how we can use Ansible for configuration management, Terraform for provisioning, and CI/CD pipelines for continuous automation.

Why Automate Your Home Lab?

First of all, is automation really worth it? In a word, yes! It might seem like the time it takes to automate a process could just as easily be spent in doing it manually. That may be true with processes that you do only once. But what about processes that you need to do daily, weekly, monthly? This is where automation can really reap huge rewards in time savings and just overall stability of your network.

Note the following benefits:

• Consistency – You can easily make sure your configurations and infrastructure is configured and provisioned in the same way every single time
• Efficiency – It helps eliminate manual efforts during provisioning and configuring your infrastructure
• Scalability – Infrastructure can be expanded and scaled over time using automated processes and procedures
• Version Control – Using automation you can then start taking advantage of things like source control using Git repositories
• Disaster Recovery – The more you have automated, the easier things are to recover if you have to rebuild things during a failure or disaster

Tool talk: Ansible, Terraform, and CI/CD Pipelines

There are no end to automation and DevOps tools available out there. However, there are three that I think are front and center tools that you need to start learning and using in the home lab. These are Ansible, Terraform, and CI/CD pipelines. Let’s take these one by one and see what benefits they each bring to the table and how to use them.

Terraform – Infrastructure as Code (IaC)

Terraform is an Infrastructure as Code (IaC) tool. It allows you to describe what you want the infrastructure to look like in code and then Terraform makes the infrastructure look like that. It is declarative in nature. Terraform uses a concept called providers. These allow Terraform to interact with different types of environments, whether these are on-premises or in the cloud.

There are providers for things like VMware vSphere and Proxmox, and then providers for AWS, GCP, Azure, etc. Terraform can even do configuration management, but unlike Ansible, that really isn’t its bread and butter.

It is ok to do a few light configuration tasks with Terraform, but this is where Ansible shines. And, conversely, I think Terraform is a better provisioning tool than Ansible.

Benefits of Terraform:

• Declarative syntax that allows you to describe in code what the infrastructure needs to be
• Both on-premises and cloud environments are supported with the same tool
• It handles the dependencies
• You can use Terraform with GitOps workflows

Ansible – Configuration Management

Ansible is the tool that is widely used to automate system configuration. You can use it to deploy apps, perform network configurations, and other types of configurations. With Ansible, you get YAML formatted playbooks that allow you to define desired states. This allows you to manage multiple machines simultaneously.

I currently use Ansible for many things in the home lab. One project I have is using Ansible in conjunction with CI/CD in GitLab to get a list of all Linux virtual machines in my VM inventory and then using that list to target for an Ansible playbook to run Linux updates.

Key Benefits of Ansible:

• Ansible is one of the only configuration management tools that is agentless. It means you only need SSH or WinRM to configure your hosts.
• Simple YAML syntax for playbooks (simple is relative here, but with AI troubleshooting, and other tools, it is getting easier)
• Multi-platform deployments.
• You can use Ansible with Terraform for some really complex and powerful configurations

CI/CD Pipelines – Automating Workflows

I have gotten more into CI/CD in the past few years in the home lab. It is a great way to securely run scripts and other code without having scheduled tasks or other scripts spread out through the environment.

CI/CD (Continuous Integration and Continuous Deployment) pipelines can be used for automated testing, validation, and deployment of infrastructure changes. Popular CI/CD tools for home labs include tools like GitLab CI, Jenkins, GitHub Actions, and Drone CI.

Key Benefits of CI/CD Pipelines:

• Automates testing and deployment of infrastructure
• Integrates with Git repositories for change tracking
• Ensures rollback capabilities

Automating Infrastructure with Terraform

Terraform enables you to define home lab infrastructure as code. Here’s a basic example of provisioning a Proxmox VM using Terraform:

provider "proxmox" {
  pm_api_url = "https://proxmox.example.com:8006/api2/json"
  pm_user    = "root@pam"
  pm_password = "your_password"
  pm_tls_insecure = true
}

resource "proxmox_vm_qemu" "home_lab_vm" {
  name        = "ansible-managed-vm"
  target_node = "proxmox-node01"
  clone       = "ubuntu-template"
  cores       = 4
  memory      = 8192
  network {
    model   = "virtio"
    bridge  = "vmbr0"
  }
  disk {
    size    = "50G"
    type    = "virtio"
  }
}

Applying Terraform and running your code

Run the following commands to provision the VM:

terraform init  # Initialize Terraform
terraform plan  # Preview changes
terraform apply  # Deploy the infrastructure

Once deployed, the VM is ready for configuration using Ansible. Take a look also at the post I created showing how to clone an LXC container template using terraform automation here:

• Clone Proxmox LXC Containers with Terraform Automation

Configuring VMs with Ansible

After you use Terraform to deploy the virtual machine, you can pick things up with Ansible and use the example playbook below to install Docker on your Ubuntu Server virtual machine.

- name: Install Docker on Ubuntu
  hosts: home_lab
  become: yes
  tasks:
    - name: Update apt packages
      apt:
        update_cache: yes

    - name: Install dependencies
      apt:
        name: ["apt-transport-https", "ca-certificates", "curl", "software-properties-common"]
        state: present

    - name: Add Docker’s GPG key
      shell: curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

    - name: Add Docker repository
      shell: add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

    - name: Install Docker
      apt:
        name: docker-ce
        state: present

    - name: Start and enable Docker service
      systemd:
        name: docker
        enabled: yes
        state: started

Running the Ansible Playbook

Take a look at the code below. here we are running the ansible-playbook command, passing in the inventory file and then the actual playbook we want to run.

ansible-playbook -i inventory.ini install_docker.yml

This ensures that Docker is installed and configured consistently across all VMs.

Automating Deployments with CI/CD Pipelines

CI/CD pipelines ensure that infrastructure and configurations are applied consistently. Here’s an example GitLab CI/CD pipeline that applies Terraform changes and runs an Ansible playbook:

stages:
  - terraform
  - ansible

terraform:
  stage: terraform
  script:
    - terraform init
    - terraform apply -auto-approve
  only:
    - main

ansible:
  stage: ansible
  script:
    - ansible-playbook -i inventory.ini install_docker.yml
  only:
    - main

How It Works:

1. Terraform Stage – Provisions infrastructure
2. Ansible Stage – Configures the provisioned infrastructure
3. Triggered by Git Push – Automatically runs when changes are committed to the main branch

Video on a very helpful terminal

Take a look at the video I recently released covering the Warp terminal. This is one of the best tools for exploring automation and getting real-time help with your Terraform and Ansible code.

Best Practices for Home Lab Automation

These include my list of best practices for home lab automation. You may have other best practices you have found that work well for you. These are just a few that I have benefited from.

• Pick a real challenge or use case you have for automation – Create a project customized to your use case and work through getting the automation created to solve your specific challenge. This type of problem/solution approach has been something I have fond to be helpful.
• Use Git for version control – Keep Terraform and Ansible configurations in a Git repository. You will not be disappointed at the benefits of using Git and having the version control that you have using it
• Secure Secrets – Use HashiCorp Vault, Ansible Vault, or environment variables in your CI/CD platform of choice to store credentials securely.
• Test changes locally – Test Terraform plans and Ansible playbooks before committing changes to your repo, this will just save you time committing, running pipelines, etc if you can work out the bugs locally first
• Monitor and log automation – Use Prometheus, Grafana, or Netdata to monitor home lab resources will help with your automated efforts as it allows keeping an eye on resources even when these are ephemeral or stateless
• Backup your configurations – Regularly back up Terraform state files and Ansible playbooks as this makes sure your declarative state is safe

Wrapping up

Automating your home lab with Ansible, Terraform, and CI/CD pipelines will reap huge rewards in terms of efficiency, learning, and overall satisfaction with your home lab. Automation is awesome and it leads to learning core DevOps skills that will help you not only in the home lab environment but also production.

I found that once I started automating and then integrating with CI/CD solutions and keeping my code in a Git repository, things just started to click with learning other core DevOps skills. One thing leads to another and then another and then another.