Unusual Unit testing (part 1) — Bash scripts with Bats

6 min readAug 17, 2020

Introduction

In my work I often see well-intended implementations of CI/CD pipelines that might have functional tests but lack unit testing. This impacts the reliability of the functional testing: since the major difference is white-box vs black-box testing, a functional test might succeed even if some components are failing (for instance due to some internal side effect).

Developers in the Java, Python or Ruby world might be familiar with unit testing using tools like JUnit, PyTest and TestUnit. But what about system administrators that maintain Bash or Powershell scripts? These scripts can contain a lot of functionality and are often used as a dependency in automation.

This challenge inspired me to experiment with some unit testing tools for scripting. So, if you are a System Admin, maintain important Bash and/or Powershell scripts and want to increase reliability through (automated) testing, this might be interesting to you.

Unit testing Bash scripts with Bats

In this blog we will dive into unit testing Bash scripts. I do think that Bash is challenging as a programming language, especially when it comes to variable scope and isolation or how functions are implemented, so we’ll need our scripts to be testable.

We’re going to use Bats for unit testing Bash scripts. Bats stands for BASH Automated Testing System. It’s a TAP-compliant testing framework for Bash and it provides a simple way to verify that the UNIX programs you write behave as expected.

Testable code anyone?

As you might already know, a unit test is a method that instantiates a small portion of an application and verifies its behavior independently from other parts. A good approach for Bash scripts is to define functions to isolate code and make it testable.

Often there is internal/external interaction between code parts, and we’ll need setup a test double to control the behavior.

Let’s get started — preparation

For creating and executing our unit tests we’ll use a docker container which will give us a consistent and ephemeral environment. The only thing you’ll need is to have docker engine installed.

The example code for this exercise can be found here. First we need to build our own container image and add additional Bats helpers for testing.

This is our Dockerfile:

FROM bats/bats:latest 
COPY ./temp_clone_dir /opt/bats-test-helpers 
WORKDIR /code/

With that we can use this script to clone the bats-helpers code and build our container:

[ -d "temp_clone_dir" ] && rm -rf "temp_clone_dir"
mkdir temp_clone_dir git clone https://github.com/ztombol/bats-support temp_clone_dir/bats-support
git clone https://github.com/ztombol/bats-assert temp_clone_dir/bats-assert
git clone https://github.com/lox/bats-mock temp_clone_dir/lox-bats-mockdocker build . -t bats-with-helpers:latest

Now let’s write some tests!

Test 1 — assert the return of a function

Consider this very basic function:

# example1.shfunction func1() {
  return 1
}

We want to make sure that this function always returns 1, and we can easily do this with Bats. We’re going to create a seperate test script and define a positive and a negative test:

# test_example1.bats
#!/usr/bin/env bats@test "func1 function should return 1" {
  source /code/example1.sh
  run func1
  [ "$status" -eq 1 ]
}@test "func1 function should return 2" {
  source /code/example1.sh
  run func1
  [ "$status" -eq 2]
}

With ‘@test’ we define a test stanza with a description of our test. The first thing we do is source the function defined in example1.sh. Be aware that it actually executes the commands in the sourced script, as a best practice only define functions in there.

The run helper executes the function and saves the result and variables $status and $output which can be used for assertion.

Let’s execute the tests:

The docker container has entrypoint of the bats command, so we simply point to the bats test file as an argument. Bats will then discover available tests (in our case 2) and execute the tests. As expected, the positive test succeeds and the negative test fails. Bats also shows what exactly failed.

Test 2 — test a condition and a file mutation

function func2() {
  if [ ${ENV_GRASS} = "green" ]
  then
    touch colors.conf
    echo 'grass="green"' > colors.conf
  fi
}

This function assesses the variable ENV_GRASS and will create a file when the value is ‘green’. Our basic test will assert the condition and the side effect of the file creation:

# test_example1.bats
#!/usr/bin/env bats
load '/opt/bats-test-helpers/bats-support/load.bash'
load '/opt/bats-test-helpers/bats-assert/load.bash'@test "func2 function should create config file is grass is green" {
  source example1.sh
  ENV_GRASS="green"
  run func2
  assert [ -e 'colors.conf' ]
}@test "func2 function should not create config file if grass is not green" {
  source example1.sh
  ENV_GRASS="red"
  run func2
  assert [ ! -e 'colors.conf' ]
}

For this test we’ll use the bats-helper bats-assert which has a lot of testing functionality. Again we’re sourcing our function, declaring the ENV_GRASS variable, executing the function and assert that the file is created/not created.

Test 3 — test a terraform wrapper script

When I use Terraform I’d like to use a wrapper script that sets things up. For instance, this function makes sure the .terraform directory does not exist before we do a terraform init:

init_tf() {
  if [ -d ".terraform" ]; then
    echo "unclean working dir, .terraform dir still exists. removing .terraform"
    rm -rf .terraform
  fi  terraform init -input=false
}

This gives us ample testing angles:

We do not want terraform init to actually be called, so we’ll need to stub the terraform command which we can do with the bats-helper bats-mock
The init_tf function should remove the directory .terraform if exists
The init_tf function should print message if the directory .terraform exists

Our test code looks like this:

# test_example_tf_plan.bats
#!/usr/bin/env bats
load '/opt/bats-test-helpers/bats-support/load.bash'
load '/opt/bats-test-helpers/bats-assert/load.bash'
load '/opt/bats-test-helpers/lox-bats-mock/stub.bash'setup() {
  stub terraform \
  "'init -input=false' : echo 'I am stubbed for terraform init!'"
  if [ -d .terraform ] ; then rm -rf .terraform ; fi
}@test "init_tf function should remove dir .terraform if exists" {
  source example_tf_plan.sh
  mkdir .terraform
  run init_tf
  assert [ ! -d '.terraform' ]
}@test "init_tf function should print message if dir. .terraform exists" {
  source example_tf_plan.sh
  mkdir .terraform
  run init_tf
  assert_output "unclean working dir, .terraform dir still exists. removing .terraform"
}

This time we’re loading the bat-mock helper and we have another section ‘setup’. In setup you can set things up that are needed for our tests. Firstly we are stubbing the terraform command with specific arguments, so that it doesn’t hit the actual terraform binary. Secondly we need the .terraform directory not to exist for our tests. Now that we have a setup we can define the tests.

The first test asserts that the .terraform directory is deleted if it exists. We’re sourcing the function, deliberately creating the .terraform directory, run the function and asserting the directory.

The second test is doing the same, but then asserting the message output.

Conclusion

As you can see, it’s quite easy to increase test coverage for your Bash scripts which gives you more confident in the system. Whenever someone breaks a script, the unit tests in the pipeline should fail.

Next up is Powershell, stay tuned!