Unit testing bash with assert.sh and stub.sh

I’ve been having a lot of problems getting Docker Compose to link to data-only containers. I want to be able to set up a Dockerized WordPress/MySQL combo and be able to back up volume data and easily move sites between domains. After struggling with Compose, I finally decided to do this manually. I also want to be able to repeat this process with other Dockerized WordPress sites. Writing a bash script is the way to go, but how do I test it?

Context

Supposing I configure a docker-compose.yml file like this:

1
2
3
4
cd ~
mkdir mysite.com
cd mysite.com
vim docker-compose.yml

with contents that look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
wordpress:
  image: wordpress
  restart: always
  links:
    - mysql
  environment:
    - WORDPRESS_DB_PASSWORD=secretp@ssword
    - VIRTUAL_HOST=mysite.com
  expose:
    - 80
mysql:
  image: mysql
  restart: always
  environment:
    - MYSQL_ROOT_PASSWORD=secretp@ssword
    - MYSQL_DATABASE=wordpress

This will create two containers named mysitecom_wordpress_1 and mysitecom_mysql_1. Both of these write data to the host file system that needs to be backed up and restored if I ever want to move from mysite.com to someothersite.com.

Manual backup and recovery

Backup

MySQL

1
docker run --rm --volumes-from mysitecom_mysql_1 -v $(pwd):/backup mysql tar cvf /backup/mysitecom_mysql.tar /var/lib/mysql

WordPress

1
docker run --rm --volumes-from mysitecom_wordpress_1 -v $(pwd):/backup wordpress tar cvf /backup/mysitecom_wordpress.tar /var/www/html

Create new WordPress/MySQL data-only containers

MySQL

1
docker create --name someothersitecom_mysql_data -v /var/lib/mysql mysql

WordPress

1
docker create --name someothersitecom_wordpress_data -v /var/www/html wordpress

Write the data to the new data-only containers

MySQL

1
docker run --rm --volumes-from someothersitecom_mysql_data -v $(pwd):/backup mysql tar xvf /backup/mysitecom_mysql.tar

WordPress

1
docker run --rm --volumes-from someothersitecom_wordpress_data -v $(pwd):/backup wordpress tar xvf /backup/mysitecom_wordpress.tar /var/www/html -C /

Deploy new WordPress/MySQL

MySQL

1
docker run -d --restart always --volumes-from someothersitecom_mysql_data -e MYSQL_ROOT_PASSWORD=secretp@ssword -e MYSQL_DATABASE=wordpress --name someothersitecom_mysql_image mysql

WordPress

1
docker run -d --restart always --volumes-from someothersitecom_wordpress_data --link someothersitecom_mysql_image:mysql -e WORDPRESS_DB_PASSWORD=secretp@ssword -e VIRTUAL_HOST=someothersite.com -p 80 --name someothersitecom_wordpress_image wordpress

Yikes! Imagine typing all that in over and over again.

The test utilities

I picked the assert.sh/stub.sh combo because I saw no other easy way to stub out Docker commands. If there is an easy way that I missed, I’d love to hear about it.

assert.sh

I’ll make project and dependency directories and install the assert.sh dependency there:

1
2
3
4
cd ~
mkdir -p docker-wordpress-mysql-utils/deps
cd docker-wordpress-mysql-utils/deps
wget https://raw.github.com/lehmannro/assert.sh/v1.1/assert.sh

stub.sh

From the project deps/ directory:

1
wget https://raw.githubusercontent.com/jimeh/stub.sh/master/stub.sh

Now, I simply need to source these in my test scripts.

TDD

The backup and redeploy procedure will be broken down into a series of bash scripts, each reflecting the individual steps. A test script will be written for each corresponding script.

Host data backup

Tests

The most important thing to test is that the Docker commands are formatted correctly given the command line options. At this point, I’m not ambitious enough to ensure that Docker itself is executing properly. I’ll just have to trust that it works as intended.

Suppose I have two containers (mysitecom_wordpress_1 and mysitecom_mysql_1), I want to be able to provide the mysite.com domain and have their data tarred to a directory that I specify. The command will look like this:

1
./backup_wordpress_and_mysql.sh mysite.com ~/backups

With this in mind, I need to be careful about how I structure my backup_wordpress_and_mysql.sh script. First I’ll create a tests directory in my project’s directory:

1
2
3
4
5
6
cd ~/docker-wordpress-mysql-utils
mkdir tests
cd tests
touch backup_wordpress_and_mysql_tests.sh
chmod 775 backup_wordpress_and_mysql_tests.sh 
vim backup_wordpress_and_mysql_tests.sh

The tests look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/bin/bash

## Source the test utilities
. ../deps/assert.sh
. ../deps/stub.sh

#
# Stub the docker command
#
stub docker

#
# Exit if not provided with exactly two command line arguments
#
assert_raises "bash ../backup_wordpress_and_mysql.sh" 1
assert_raises "bash ../backup_wordpress_and_mysql.sh mysite.com" 1
assert_raises "bash ../backup_wordpress_and_mysql.sh mysite.com backup extra" 1

# Make sure the Docker commands were called with the correct parameters
. ../backup_wordpress_and_mysql.sh mysite.com backup_dir

assert "stub_called_times docker" 2

assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from mysitecom_mysql_1 -v backup_dir:/backup mysql tar cvf /backup/mysitecom_mysql.tar /var/lib/mysql"
assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from mysitecom_wordpress_1 -v backup_dir:/backup wordpress tar cvf /backup/mysitecom_wordpress.tar /var/www/html"

assert_end backup_wordpress_and_mysql

Script

To get these tests to pass, I first need to create the script:

1
2
3
4
cd ~/docker-wordpress-mysql-utils
touch backup_wordpress_and_mysql.sh
chmod 775 backup_wordpress_and_mysql.sh 
vim backup_wordpress_and_mysql.sh

The backup_wordpress_and_mysql.sh looks like this:

1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash

if [ $# -ne 2 ]
  then
    echo "$0 source_domain destination_dir"
    exit 1
fi
# Remove periods from domain
prefix=${1//.}
docker run --rm --volumes-from "$prefix"_mysql_1 -v $2:/backup mysql tar cvf /backup/"$prefix"_mysql.tar /var/lib/mysql
docker run --rm --volumes-from "$prefix"_wordpress_1 -v $2:/backup wordpress tar cvf /backup/"$prefix"_wordpress.tar /var/www/html

To run these tests, I execute the following from the project’s test directory:

1
2
cd test
./backup_wordpress_and_mysql_tests.sh

The remaining steps in the backup/recovery process follow a similar pattern…

Create data-only containers

Tests

As before, the only thing I’m really testing is that the Docker commands are formatted correctly given the command line option.

I follow the Compose-imposed naming conventions, even though I’m not using it to redeploy my site on a new domain. Here, assuming I want to move mysite.com to someothersite.com, I’ll provide the someothersite.com parameter to the new containers. The command will look like this:

1
./create_wordpress_and_mysql_data_only_containers.sh someothersite.com

Here are the tests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/bin/bash

## Source the test utilities
. ../deps/assert.sh
. ../deps/stub.sh

#
# Stub the docker command
#
stub docker

#
# Exit if not provided with exactly one command line arguments
#
assert_raises "bash ../create_wordpress_and_mysql_data_only_containers.sh" 1
assert_raises "bash ../create_wordpress_and_mysql_data_only_containers.sh someothersite.com extra" 1

# Make sure the Docker commands were called with the correct parameters
. ../create_wordpress_and_mysql_data_only_containers.sh someothersite.com

assert "stub_called_times docker" 2

assert_raises "stub_called_with_exactly_times docker 1 create --name someothersitecom_mysql_data -v /var/lib/mysql mysql"
assert_raises "stub_called_with_exactly_times docker 1 create --name someothersitecom_wordpress_data -v /var/www/html wordpress"

assert_end create_wordpress_and_mysql_data_only_containers

Script

1
2
3
4
5
6
7
8
9
10
#!/bin/bash

if [ $# -ne 1 ]
  then
    echo "$0 domain"
    exit 1
fi
prefix=${1//.}
docker create --name "$prefix"_mysql_data -v /var/lib/mysql mysql
docker create --name "$prefix"_wordpress_data -v /var/www/html wordpress

To run these tests, I do as before and execute the following from the project’s test directory:

1
./create_wordpress_and_mysql_data_only_containers_tests.sh

Write tar files to data-only containers

Tests

This time I need to provide the mysite.com domain, the new someothersite.com domain, and the directory in which the tar files are contained. The command will look like this:

1
./write_wordpress_and_mysql_data_only_containers.sh mysite.com someothersite.com backup_source_dir

The tests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#!/bin/bash

## Source the test utilities
. ../deps/assert.sh
. ../deps/stub.sh

#
# Stub the docker command
#
stub docker

#
# Exit if not provided with exactly three command line arguments
#
assert_raises "bash ../write_wordpress_and_mysql_data_only_containers.sh" 1
assert_raises "bash ../write_wordpress_and_mysql_data_only_containers.sh mysite.com" 1
assert_raises "bash ../write_wordpress_and_mysql_data_only_containers.sh mysite.com someothersite.com" 1
assert_raises "bash ../write_wordpress_and_mysql_data_only_containers.sh mysite.com someothersite.com backup_source_dir extra" 1

# Make sure the Docker commands were called with the correct parameters
. ../write_wordpress_and_mysql_data_only_containers.sh mysitecom someothersite.com backup_source_dir

assert "stub_called_times docker" 2

assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from someothersitecom_mysql_data -v backup_source_dir:/backup mysql tar xvf /backup/mysitecom_mysql.tar"
assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from someothersitecom_wordpress_data -v backup_source_dir:/backup wordpress tar xvf /backup/mysitecom_wordpress.tar /var/www/html -C /"

assert_end write_wordpress_and_mysql_data_only_containers

Script

1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash

if [ $# -ne 3 ]
  then
    echo "$0 old_domain new_domain backup_source_dir"
    exit 1
fi
old_prefix=${1//.}
new_prefix=${2//.}
docker run --rm --volumes-from "$new_prefix"_mysql_data -v $3:/backup mysql tar xvf /backup/"$old_prefix"_mysql.tar
docker run --rm --volumes-from "$new_prefix"_wordpress_data -v $3:/backup wordpress tar xvf /backup/"$old_prefix"_wordpress.tar /var/www/html -C /

To run these tests, execute the following from the project’s test directory:

1
./write_wordpress_and_mysql_data_only_containers_tests.sh

Deploy WordPress and MySQL containers

Tests

This time I need to provide the destination site’s domain. The command will look like this:

1
./deploy_new_wordpress_and_mysql_containers.sh someothersite.com

The tests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#!/bin/bash

## Source the test utilities
. ../deps/assert.sh
. ../deps/stub.sh

#
# Stub the docker command
#
stub docker

#
# Exit if not provided with exactly one command line argument
#
assert_raises "bash ../deploy_new_wordpress_and_mysql_containers.sh" 1
assert_raises "bash ../deploy_new_wordpress_and_mysql_containers.sh someothersite.com extra" 1

# Make sure the Docker commands were called with the correct parameters
. ../deploy_new_wordpress_and_mysql_containers.sh someothersite.com

assert "stub_called_times docker" 2

assert_raises "stub_called_with_exactly_times docker 1 run -d --restart always --volumes-from someothersitecom_mysql_data -e MYSQL_ROOT_PASSWORD=secretp@ssword -e MYSQL_DATABASE=wordpress --name someothersitecom_mysql_image mysql"
assert_raises "stub_called_with_exactly_times docker 1 run -d --restart always --volumes-from someothersitecom_wordpress_data --link someothersitecom_mysql_image:mysql -e WORDPRESS_DB_PASSWORD=secretp@ssword -e VIRTUAL_HOST=someothersite.com -p 80 --name someothersitecom_wordpress_image wordpress"

assert_end deploy_new_wordpress_and_mysql_containers

Script

1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash

if [ $# -ne 1 ]
  then
    echo "$0 domain"
    exit 1
fi
# Remove periods from domain
prefix=${1//.}
docker run -d --restart always --volumes-from "$prefix"_mysql_data -e MYSQL_ROOT_PASSWORD=secretp@ssword -e MYSQL_DATABASE=wordpress --name "$prefix"_mysql_image mysql
docker run -d --restart always --volumes-from "$prefix"_wordpress_data --link "$prefix"_mysql_image:mysql -e WORDPRESS_DB_PASSWORD=secretp@ssword -e VIRTUAL_HOST=$1 -p 80 --name "$prefix"_wordpress_image wordpress

To run these tests, execute the following from the project’s test directory:

1
./deploy_new_wordpress_and_mysql_containers_tests.sh

Put it all together…

There are four steps described here to backing up and restoring a WordPress site and its MySQL data. You may have occassion to execute each script one at a time, but that would generally be too much typing. Combined, these scripts take a total of three parameters:

  • The old domain
  • The new domain
  • The backup directory

I’m going to make one script that will execute each individual script. The command looks like this:

1
./move_compose_configured_wordpress_and_mysql.sh mysite.com someothersite.com backups

Tests

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
#!/bin/bash

# Source the test utilities
. ../deps/assert.sh
. ../deps/stub.sh

#
# Stub the docker command
#
stub docker

#
# Exit if not provided with exactly three command line arguments
#
assert_raises "bash ../move_compose_configured_wordpress_and_mysql.sh" 1
assert_raises "bash ../move_compose_configured_wordpress_and_mysql.sh mysite.com" 1
assert_raises "bash ../move_compose_configured_wordpress_and_mysql.sh mysite.com someothersite.com" 1
assert_raises "bash ../move_compose_configured_wordpress_and_mysql.sh mysite.com someothersite.com backup_source_dir extra" 1

# Make sure the Docker commands were called with the correct parameters
. ../move_compose_configured_wordpress_and_mysql.sh mysite.com someothersite.com backup_dir

assert "stub_called_times docker" 8

## backup_wordpress_and_mysql_tests
assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from mysitecom_mysql_1 -v backup_dir:/backup mysql tar cvf /backup/mysitecom_mysql.tar /var/lib/mysql"
assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from mysitecom_wordpress_1 -v backup_dir:/backup wordpress tar cvf /backup/mysitecom_wordpress.tar /var/www/html"

# create_wordpress_and_mysql_data_only_containers_tests
assert_raises "stub_called_with_exactly_times docker 1 create --name someothersitecom_mysql_data -v /var/lib/mysql mysql"
assert_raises "stub_called_with_exactly_times docker 1 create --name someothersitecom_wordpress_data -v /var/www/html wordpress"

# write_wordpress_and_mysql_data_only_containers_tests
assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from someothersitecom_mysql_data -v backup_dir:/backup mysql tar xvf /backup/mysitecom_mysql.tar"
assert_raises "stub_called_with_exactly_times docker 1 run --rm --volumes-from someothersitecom_wordpress_data -v backup_dir:/backup wordpress tar xvf /backup/mysitecom_wordpress.tar /var/www/html -C /"

# deploy_new_wordpress_and_mysql_containers_tests
assert_raises "stub_called_with_exactly_times docker 1 run -d --restart always --volumes-from someothersitecom_mysql_data -e MYSQL_ROOT_PASSWORD=secretp@ssword -e MYSQL_DATABASE=wordpress --name someothersitecom_mysql_image mysql"
assert_raises "stub_called_with_exactly_times docker 1 run -d --restart always --volumes-from someothersitecom_wordpress_data --link someothersitecom_mysql_image:mysql -e WORDPRESS_DB_PASSWORD=secretp@ssword -e VIRTUAL_HOST=someothersite.com -p 80 --name someothersitecom_wordpress_image wordpress"

assert_end move_compose_configured_wordpress_and_mysql

Script

1
2
3
4
5
6
7
8
9
10
11
#!/bin/bash

if [ $# -ne 1 ]
  then
    echo "$0 domain"
    exit 1
fi
# Remove periods from domain
prefix=${1//.}
docker run -d --restart always --volumes-from "$prefix"_mysql_data -e MYSQL_ROOT_PASSWORD=secretp@ssword -e MYSQL_DATABASE=wordpress --name "$prefix"_mysql_image mysql
docker run -d --restart always --volumes-from "$prefix"_wordpress_data --link "$prefix"_mysql_image:mysql -e WORDPRESS_DB_PASSWORD=secretp@ssword -e VIRTUAL_HOST=$1 -p 80 --name "$prefix"_wordpress_image wordpress

To run these tests, execute the following from the project’s test directory:

1
./move_compose_configured_wordpress_and_mysql_tests.sh

Testing is still pretty manual. Setting yourself up to do something like ./test.sh [test_file] takes a bit of reorganization…