NOTE: This article was initialy written for the SemaphoreCi community page.
Introduction
In a previous article, we wrote about testing Ansible roles directly against Semaphore. This is great when you are targeting the same operating system as your continuous integration server, but what to do if you you need to target other operating systems as well? This is where Docker comes into play. Docker allows you to easily create and run containers in any operating system.
In this article, we will use the Ansible roles and tests we developed in the previous article, and walk you through the necessary steps to set up a Docker environment and test your Ansible roles against multiple operating systems on Semaphore.
Docker Containers
Docker is all about making it easier to create, deploy, and run applications using containers. Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that your code will always run the same way, regardless of the environment in which it is running.
Docker allows you to run all major Linux distributions with support for every infrastructure.
If you’re not familiar with Docker and would like to read more on the subject, head over to the Docker website, and dig into the documentation.
Ansible and Ansible Roles
Ansible is a configuration management tool written in Python that allows you to manage nodes over SSH or PowerShell. Ansible then uses YAML to express reusable descriptions of systems.
The simplest usage is sending ad-hoc commands or one big playbook to (multiple) nodes. However, that’s not where its true power lies. As a developer or system engineer, you want to be able to re-use your code. This is where Ansible roles come in.
An Ansible role is a collection of tasks that need to be performed to achieve a certain outcome.
As a best practice, you want your role to focus on only one thing. As the Unix philosophy states: do one thing, and do it well.
Targeting Multiple OSes
An Ansible role can target multiple OSes. Take, for instance, our example role for installing the Erlang language. Erlang Solutions offer a variety of packages, targeting multiple OSes.
You don’t want to write OS-specific roles. Instead, you can create one role and target multiple OSes.
Our Ansible role will have the following structure:
# Role structure:
ansible-role-erlang/
defaults/
main.yml
tasks/
debian.yml
main.yml
redhat.yml
templates/
erlang-solutions.j2
tests/
test.yml
inventory
Role Default Variables
The defaults/main.yml file is used for configuring the default variables that will be used throughout the execution of the role. By adding them to the defaults/main.yml file, you can easily overwrite values later if you need to. Overwriting variables is usually done on host or group level.
Our default values look as follows:
---
erlang_ppa_repo: 'deb http://packages.erlang-solutions.com/{{ ansible_distribution | lower }} {{ ansible_distribution_release | lower }} contrib'
erlang_ppa_key: 'http://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc'
erlang_ppa_key_id: 'A14F4FCA'
erlang_rpm_key: 'http://packages.erlang-solutions.com/rpm/erlang_solutions.asc'
erlang_yum_repo_path: '/etc/yum.repos.d'
erlang_yum_repo_name: 'Centos $releasever - $basearch - Erlang Solutions'
erlang_yum_repo_baseurl: 'http://packages.erlang-solutions.com/rpm/centos/$releasever/$basearch'
erlang_yum_repo_gpgcheck: '1'
erlang_yum_repo_enabled: '1'
erlang_packages:
- erlang
Let’s break down the variables:
erlang_ppa_repo: this is the repository address used by the Aptitude package manager (for Debian based OSes)erlang_ppa_key: the location of the GPG key for verifying package integrity (for Debian-based OSes)erlang_ppa_key_id: the ID of the PPA key (for Debian-based OSes)erlang_rpm_key: the location of the GPG key for verifying package integrity (for RedHat-based OSes)erlang_yum_repo_path: the location on the file system where new repository configuration files are stored (for RedHat-based OSes)erlang_yum_repo_name: the name you want to give the new YUM repository (RedHat-based OSes)erlang_yum_repo_baseurl: the URL of the YUM repository (RedHat-based OSes)erlang_yum_repo_gpgcheck: enable repository GPG check (RedHat-based OSes)erlang_yum_repo_enabled: enable/disable the added repository (RedHat based OSes)erlang_packages: contains a list of packages you want to install (Debian and RedHat-based OSes)
Role Tasks
You can store all the tasks that need to be executed on the nodes in the tasks directory. Ansible always starts off by parsing the main.yml file. Since our role is designed to target both Debian and RedHat-based distros, our main.yml looks as follows:
--- - include: debian.yml when: ansible_os_family == 'Debian' tags: package - include: redhat.yml when: ansible_os_family == 'RedHat' tags: package
We told Ansible to include the debian.yml file, and only execute the tasks when the ansible_os_family variable equals Debian. It also needs to include the redhat.yml file and execute these specific tasks, but only when ansible_os_family equals RedHat.
ansible_os_family is an Ansible-provided variable that tells you which OS you are currently targeting. You can find more information on variables and facts in the Ansible documentation. You can also access a list of all available facts in a playbook.
This means that the following tasks will only be executed when we target a Debian-based OS. These tasks are defined in the debian.yml file:
---
- name: Debian | repository - add the GPG key
apt_key:
url: '{{ erlang_ppa_key }}'
id: '{{ erlang_ppa_key_id }}'
state: present
when: erlang_ppa_key != None
- name: Debian | add repository to install Erlang from
apt_repository:
repo: '{{ erlang_ppa_repo }}'
update_cache: yes
when: erlang_ppa_repo != None
- name: Debian | install packages
apt:
pkg: '{{ item }}'
state: installed
update_cache: yes
cache_valid_time: 3600
with_items: erlang_packages
The first task adds the GPG key to Aptitude, and the second task adds the repository. Both tasks have a conditional. This is to check if the erlang_ppa_key and erlang_ppa_repo are filled in. If those variables are not filled in, Ansible will skip those tasks.
For instance, you might not wish to install Erlang for another PPA, but just want to install the package from the default distro repositories.
The last task installs all the packages defined in the erlang_packages variable. In our case, the Erlang package.
If the OS is RedHat-based, Ansible will start executing the tasks from redhat.yml.
---
- name: RedHat | repository - add the GPG key
rpm_key:
state: present
key: '{{ erlang_rpm_key }}'
when: erlang_rpm_key != None
- name: RedHat | add Erlang repository
template:
src: erlang-solutions.j2
dest: '{{ erlang_yum_repo_path }}/erlang-solutions.repo'
owner: root
group: root
mode: 0644
- name: RedHat | install packages
yum:
name: '{{ item }}'
state: installed
update_cache: yes
with_items: erlang_packages
These three tasks are the RedHat counterparts of the Debian tasks we just saw. In the first task, we added the RPM GPG key of the Erlang repository.
Next, we added the repository details through a template, and set the proper permissions.
In the last task, we installed the needed packages.
Role Templates
In the RedHat-specific tasks, you’ll notice that we install the new repository by copying a repo template to the proper location. The template is nothing more than a text file containing the necessary variables it needs to fill in. Ansible parses these template files through the Jinja2 templating engine.
Our erlang-solutions.j2 contains:
[erlang-solutions]
name={{ erlang_yum_repo_name }}
baseurl={{ erlang_yum_repo_baseurl }}
gpgcheck={{ erlang_yum_repo_gpgcheck }}
gpgkey={{ erlang_rpm_key }}
enabled={{ erlang_yum_repo_enabled }}
How to Determine What to Test
Before we continue working on the content of the test directory, let’s take some time to define what exactly we want to test.
There are 3 points you need to consider:
- Is the YAML syntax of my role correct? Even though writing YAML is easy, everyone makes typos once in a while.
- Does your role run through all tasks without failing?
- Is your role built in an idempotent way? This means that a second run cannot create new changes.
Now that we have a clear view on what we will be testing, it’s time to put things into practice.
Setting Up the Test Environment
Installing Docker and Setting Up the Containers
Semaphore currently offers Docker support in a beta platform. You can drop them an e-mail to enable it on your account.
Once you have Docker support, don’t forget to run all Docker commands through sudo to avoid permission problems.
Docker support is only one part of the puzzle. Next, you’ll need proper images. These images need to contain the following:
- the correct OS you want to target
- Ansible pre-installed
- Git or other VCS, depending on where you store your Ansible roles, so you can easily pull the latest version of your role into your Docker container.
You can create an image by writing a Dockerfile that suits your needs. You can create one image With one Dockerfile. Docker uses the instructions from your Dockerfile to build the image.
If, for instance, we wanted to create a Docker image to test our role against CentOS 6, the Dockerfile could look like this:
FROM centos:centos6 MAINTAINER Michaël Rigart <michael@netronix.be> RUN echo "====> Installing EPEL... <====" && \ yum -y install epel-release && \ \ \ echo "====> Installing sudo... <====" && \ yum -y install sudo && \ \ \ echo "====> Installing Ansible... <====" && \ yum -y install ansible && \ \ \ echo "====> Installing Git... <====" && \ yum -y install git && \ \ \ echo "====> Removing unused YUM resources... <====" && \ yum -y remove epel-release && \ yum clean all CMD [ "ansible-playbook", "--version" ]
In a Dockerfile, you start off by defining the base image. In this case, we used the official CentOS 6 image from the CentOS repository. Next, we defined the maintainer of the Dockerfile, a handy reference when someone is in need of contacting them.
We proceeded with installing the needed software and dependencies so that our image is pre-loaded with everything we need.
Here, we need to install Ansible, sudo and Git. We need Ansible to run our test playbook, and Git so we can clone our role from our Git server later. Sudo is needed because we’ll run our role as root.
We’ll display the Ansible version as the default command.
We also need to test our role against Ubuntu 14.04. The Dockerfile looks similar to the one for CentOS. Of course, we’ll use Ubuntu-specific packages to manage our dependencies.
FROM ubuntu:14.04 MAINTAINER Michaël Rigart <michael@netronix.be> ENV DEBIAN_FRONTEND noninteractive RUN echo "====> Adding Ansible's PPA... <====" && \ echo "deb http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main" | tee /etc/apt/sources.list.d/ansible.list && \ echo "deb-src http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main" | tee -a /etc/apt/sources.list.d/ansible.list && \ apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 7BB9C367 && \ apt-get update && \ \ \ echo "====> Installing Git... <====" && \ apt-get install -y git && \ \ \ echo "====> Installing Ansible... <====" && \ apt-get install -y ansible && \ \ \ echo "====> Removing Ansible PPA... <====" && \ rm -rf /var/lib/apt/lists/* /etc/apt/sources.list.d/ansible.list CMD [ "ansible-playbook", "--version" ]
Here, we pulled our base image from the official Ubuntu repository, and then install Git and Ansible.
You can find more Dockerfile examples in this Github repository with Ansible images.
To speed up the build process when testing your Ansible roles, it is best not to build your images directly on the CI server every time you push a new commit.
Instead, build your images locally, and push them to the Docker Hub. You can find pre-configured Ansible images for Ubuntu, Debian, CentOS and Fedora here. If these images are not to your liking, you can always create your own.
Playbook Inventory
Now that we have everything we need on the Docker side, we can start preparing our Ansible role.
For convenience, we’ll store all of our test scripts in the tests directory.
The inventory file contains the nodes you want to access. Since we will be testing our role from inside the Docker container, we will be adding localhost to the file:
localhost
We will eventually use the command’s --connection=local option to tell Ansible to run the test playbook on the local machine, and in our case, the Docker containers.
Playbook Play
The tests.yml file is where we define our actual play.
- hosts: all
roles:
- { role: ../../ansible-role-erlang, sudo: Yes }
Here, we told Ansible to run this play on all nodes: hosts: all. In the roles list, we defined the Erlang role that needs to be executed. With the sudo: Yes variable, we told Ansible to run the tasks as a super user.
Configuring the Test Tasks
Next, we need to define the build settings inside the Semaphore interface. We can do this by going to the Project Settings and selecting Build Settings.
Inside the build settings, we can start by adding the build commands you need. You can choose when a certain command needs to be executed for each build.
The Free plan provides you with 2 threads, but more threads are available on paid plans, so the actual number you are seeing depends on your account. The more threads you have access to, the smaller your build time will be. You can check out the available plans here. We’ll use the two threads available on the Free plan against two Docker containers.
We will test if our Erlang role runs on both CentOS 6 and Ubuntu 14.04. Let’s start off by configuring Semaphore to run our tests against CentOS 6.
First, we need to load the CentOS 6 image.
sudo docker run --name 'centos6' -d michaelrigart/ansible:centos6 /bin/sh -c "while true; do sleep 1; done"
With docker run, we told Docker to run an image in the background as a daemon (-d), and name that container centos6. If you are familiar with Docker, you know that Docker will only run a single command. Once that command stops running, so does your container. Since we want to run multiple commands, we need to keep it running. That’s why we added an infinite loop to it — /bin/sh -c "while true; do sleep 1; done". This loop will keep the container up and running.
This is in conflict with proper Docker usage, but it’s a good idea to overlook it in this case. This is because firstly, running a new container for every command would take up more build time and secondly, if we ran every command on a new container, we would be unable to run the idempotency test, unless we choose to execute both our first run and idempotency test with a single command. This would result in even longer build time. So, to keep build time as minimal as possible, we’ll use just one single container per distro. michaelrigart/ansible:centos6defines the image stored in the michaelrigart user account from the ansible repository containing the centos6 tag. If Docker doesn’t find that image locally, it will automatically pull the image from Docker Hub.
Once the Docker container is running, we need to load our Ansible role within our container. In this example, our Ansible role is published on Github, so we’ll pull it from there, and store it in the /tmp directory.
sudo docker exec centos6 git clone https://github.com/michaelrigart/ansible-role-erlang.git /tmp/ansible-role-erlang
The docker exec command tells Docker to execute git clone git clone https://github.com/michaelrigart/ansible-role-erlang.git /tmp/ansible-role-erlang in the container named centos6. Keep in mind that your container needs to run in order to perform the docker exec command. You can always pass the -t option (docker exec -t) to allocate a pseudo-TTY. The pseudo-TTY will then directly display any output, so you don’t have to wait for the full command to end.
Assign these two commands to Thread #1.
Next, we will apply the same principle to our Ubuntu 14.04 image.
Same as above, we start by naming the Ubuntu 14.04 image ubuntu14.04 and running an infinite loop against it.
sudo docker run --name 'ubuntu14.04' -d michaelrigart/ansible:ubuntu14.04 /bin/sh -c "while true; do sleep 1; done"
Then, we clone the Ansible role inside our Ubuntu 14.04 image into the /tmp directory.
sudo docker exec ubuntu14.04 git clone https://github.com/michaelrigart/ansible-role-erlang.git /tmp/ansible-role-erlang
Assign the two build tasks for Ubuntu 14.04 to Thread #2. This way, both builds can run in parallel.
Do not assign these tasks to the Setup thread. We’ll explain later why this is a bad idea.
1. Testing the Role Syntax
The first item in our list is testing the role syntax. The ansible-playbook command has a built-in option that will check a playbook’s syntax and all files included in a role.
For our CentOS6 container, we’ll be using the following build command:
sudo docker exec centos6 ansible-playbook -i /tmp/ansible-role-erlang/tests/inventory /tmp/ansible-role-erlang/tests/test.yml --syntax-check
For Ubuntu 14.04, you can copy this statement and change the container name from centos6 to ubuntu14.04.
sudo docker exec ubuntu14.04 ansible-playbook -i /tmp/ansible-role-erlang/tests/inventory /tmp/ansible-role-erlang/tests/test.yml --syntax-check
Assign the command for centos6 to Thread #1, and the one for ubuntu14.04 to Thread #2.
2. Testing the First Role Run
After we have checked the syntax, we’ll want to make sure that our role runs correctly. Let’s run the ansible-playbook command against the test.yml playbook on the Docker containers. In the ansible-playbook command, we will specify the --sudo option to run the command as a super user and specify our test inventory file.
So, for our CentOS 6 container, we will use:
sudo docker exec centos6 ansible-playbook -i /tmp/ansible-role-erlang/tests/inventory /tmp/ansible-role-erlang/tests/test.yml --connection=local --sudo
As previously shown, you can use the same statement and change the name of the centos6 container to ubuntu14.04 to target the Ubuntu 14.04 container.
sudo docker exec ubuntu14.04 ansible-playbook -i /tmp/ansible-role-erlang/tests/inventory /tmp/ansible-role-erlang/tests/test.yml --connection=local --sudo
If the playbook fails, Ansible will return a non-zero exit, which Docker will in turn pick up and return the same value to Semaphore. This way, Semaphore knows whether the Ansible playbook has run successfully in your Docker container.
Assign the command for centos6 to Thread #1, and the one for ubuntu14.04 to Thread #2.
3. Testing the Role Idempotency
Now that our first test has run successfully, it’s time to test the role idempotency. This basically means testing if your role changes anything if it runs a second time. This should not be the case, since all the tasks you perform via Ansible should be idempotent.
Use this final test command for CentOS 6:
sudo docker exec centos6 ansible-playbook -i /tmp/ansible-role-erlang/tests/inventory /tmp/ansible-role-erlang/tests/test.yml --connection=local --sudo | tee /tmp/output.txt; grep -q 'changed=0.*failed=0' /tmp/output.txt && (echo 'Idempotence test: pass' && exit 0) || (echo 'Idempotence test: fail' && exit 1)
While replacing the centos6 container name to ubuntu14.04 to target the Ubuntu container:
sudo docker exec ubuntu14.04 ansible-playbook -i /tmp/ansible-role-erlang/tests/inventory /tmp/ansible-role-erlang/tests/test.yml --connection=local --sudo | tee /tmp/output.txt; grep -q 'changed=0.*failed=0' /tmp/output.txt && (echo 'Idempotence test: pass' && exit 0) || (echo 'Idempotence test: fail' && exit 1)
As you can see, this is the exact same Ansible command as before. The only difference here is that we piped the Ansible output to the tee utility. Tee does 2 things — it pipes output to the /tmp/output.txt file, and shows it at the same time. Then, we grep the /tmp/output.txt file to make sure that both the changed and failed output report 0. If they do, then the idempotency test passes, and we exit with 0 (OK). Otherwise, it fails, and we exit with 1. This way, Docker and Semaphore know if our test has passed or failed.
Assign the command for centos6 to Thread #1 and the one for for ubuntu14.04 to Thread #2.
Ready… Set… Test
Now that your build settings are configured, you are all set. Once you start committing changes to your repository, Semaphore will start running your tests.
The only thing left to do is to get your Semaphore build badge, and show the world that your role is fully functional.
