Ansible is a great config management tool. Unfortunately I lacked the time of writing a bit more about minding, except of my 2 previous posts (Getting started with configuration management: Ansible and Configuration management with Ansible: Playbooks & Execution )
Not that I haven’t done anything new on that front. I’m currently using / maintaining some roles I released open-source on Ansible Galaxy.
During the development of these roles, it is hard to track whether your role is still functioning correctly. You could run it manually again to check if everything is still fine, but hey, why not automate this process.
Using a continuous integration tool like Travis CI for automated testing can help you a great way along. Travis allows me to run tests against my Ansible role with every commit I push to the repository. This way, you’ll know if something breaks down with every push you make.
What to test in your Ansible roles?
Good question. Like every testing strategy, you need to know what to test.
- Make sure the role syntax is correct.
- Does your role run through all tasks without failing
- Is your role build in an idempotent way? ( a second run cannot create new changes )
- Did the role everything you wanted
Points 1 through 3 are relatively easy to obtain. But I have to admit, I don’t alway test against point 4. During my tests, I don’t check if all packages are correctly installed and / or software is configured properly. Perhaps I should do this in the future, but doing so, would take up a certain amount of time and I’m not sure the benefit is there.
In the end, if your role doesn’t do the things you expect it to do, wouldn’t you immediately notice it in your environment?
Setup your Ansible roles for Travis CI
Preparing your roles for tests is pretty easy. First, lets add a “tests” directory in your Ansible role. You are freely to chose the name of your testing folder, but if you follow good convention, “tests” is the most suitable.
Inside your “tests” directory, you’ll need 2 files:
- inventory
- test.yml
# Directory structure:
ansible-role-mysql/
tests/
test.yml
inventory
Inside your inventory file, you just add:
localhost
This is the inventory file we’ll use for our testing. We will eventually use the –connection=local option to tell Ansible to run the test playbook on the local machine.
Inside your test.yml playbook, add:
- hosts: all
roles:
- { role: ../../ansible-role-mysql, sudo: Yes }
The test.yml playbook is fairly simple. We tell Ansible to run the playbook on all hosts and your playbook contains only one role. In this case it is the ansible-role-mysql. We also provide the sudo option, so the role gets executed as a super user.
So substitute your own role for ansible-role-mysql. You can add other vars if you want, but keep your playbook as simple as possible.
.travis.yml
The next step is to add the .travis.yml file, so Travis CI picks up our tests. Add this file to the root level of your role and add the following parameters:
--- language: python python: "2.7" before_install: - sudo apt-get update -qq - sudo apt-get install -qq python-apt install: - pip install ansible==1.9.1 script:
Lets break it down a bit. First we tell Travis CI we’ll be using python version 2.7.
Next, we run some custom commands before the installation step (before_install). Here we update apt-get and install the python-apt package.
Next, during the installation step, we install Ansible through pip. Here, I explicitly define Ansible version 1.9.1 to be installed. You can omit the version number to install the latest one available.
In the script step, we’ll add our test commands.
1. Role syntax
Testing your role syntax is the easiest test there is. The ansible-playbook command has a build in command for this, that will check a playbook’s syntax (and all files included in a role). So add this as the first command in the script section of your .travis.yml file:
- ansible-playbook -i tests/inventory tests/test.yml --syntax-check
If you have any syntax errors, the Travis build will fail and output errors in the build log.
2. First role run
After we checked the syntax, we want to make sure our role runs correctly. So we run the ansible-playbook command against the test.yml playbook against the local host. In the ansible-playbook command we specify the –sudo option to run the command as super user and specify our test inventory file:
- ansible-playbook -i tests/inventory tests/test.yml --connection=local --sudo
Ansible returns a non-zero exit if the playbook fails, so Travis knows whether the command succeeds or not.
3. Role idempotency
If your first run is successful, you want to run an idempotency test. This basically means, does your role change anything if it runs a second time? This should not be the case, since all tasks you perform via Ansible should be idempotent.
So add the following new command to the Travis script section:
- "ansible-playbook -i tests/inventory tests/test.yml --connection=local --sudo | tee /tmp/output.txt; grep -q 'changed=0.*failed=0' /tmp/output.txt && (echo 'Idempotence test: pass' && exit 0) || (echo 'Idempotence test: fail' && exit 1)"
As you see, this is the exact same Ansible command as before. The only difference here is that we pipe the Ansible output to the tee utility. Tee does 2 things, it pipes it to the file /tmp/output.txt and shows it at the same time. We then grep the output.txt file to make sure the changed and failed output both report 0. If it does, then the idempotency test passes and we exit with 0 (ok). Otherwise, it fails and we exit with 1. This way, Travis knows if our test passes or fails.
You can find similar command on the internet without the use of tee. The reason I use tee, is so I still can see the output of my second run. When the test fails, I know exactly on which task, which makes debugging a lot easier.
4. Role result
Testing that the role you wrote does exactly what you want is something that I don’t actually do. Take for instance a mysql role. At the end, you could specifically check if the MySQL server is up and running by checking the output of
# service mysqld status
Grep the output and exit with 0 or 1 according to the status.
But for most of my roles, if I need a service to run, I tell Ansible to make sure a service is started:
- name: start/stop mysql service
service:
name: mysql<
state: 'started'
enabled: 'yes'
when: mysql_server_install
tags: service
This will tell Ansible to start and enable the MySQL service. On your first run, it will start the service and set the status on change. If for instance the service does not come up due to a config problem, the idempotency test will pick this up. Because when you run it for a second time, the service task will again try to start MySQL and set the status on changed again. Since this happens during your idempotency test, your test will fail and you know something is up.
There can be cases where testing some functionality might be in place, but I think there are better solutions out there.
Caveats
Travis CI is an easy way to test your roles. But there are some things to keep in mind:
- Travis CI is still working on Ubuntu 12.04. So roles that target specific RedHat based OS’es can’t be tested here. Also keep in mind that package versions can differ on newer version, which can have implications on your roles. For instance, newer versions of Apache and Fail2ban, which are available on Ubuntu 14.04 have some minor tweaks (other naming conventions or subfolders) to store config data in.
- Travis CI comes with some preinstalled software: MySQL, Apache, Ruby, etc . If you are writing a role that can have conflicts with these preinstalled packages, you’ll need to remove them first. You can to this by using apt-get remove –purge [package] in the .travis.yml before_install section. More info on the Travis CI build environemnt
- For some roles it might be hard to test them against idempotency. Take for instance NewRelic. In my NewRelic role, I specifically tell the service to start up. The problem is, if you don’t provide an official license key, the service will automatically stop. Meaning that during your idempotency test, if will again try and start the service, which will make the idempotency test fail.
Note: article inspired by Guy Geerling
