Projects

How I delivered SanctionedList.com with a solo developer setup (Part 3)

This guide series will be about setting up http://sanctionedlist.com with configuration management and TDD.

In the previous post, we can do unit tests to ensure our code doesn’t break in future. For small projects, unit testing in chef would be overkill. However, if you plan to expand and use Chef a lot, then unit testing is a must for one good reason, regression. When we make a change in the complex project, we want to ensure we don’t break something. When we want to ensure that Chef performs the correct actions, we need to do integration testing. For example, creating a database user can be mocked to be correct in unit testing but we want to ensure an user is created in production. This brings us to the next step, Kitchen Test.

First check your .kitchen.yml. I removed – name: centos-7.2 so that kitchen tests only Ubuntu. If you want more through platform-independent tests, add more platforms.

---
driver:
name: vagrant

provisioner:
name: chef_zero
# You may wish to disable always updating cookbooks in CI or other testing environments.
# For example:
# always_update_cookbooks:
always_update_cookbooks: true

verifier:
name: inspec

platforms:
- name: ubuntu-16.04

suites:
- name: default
run_list:
- recipe[sanctionedlist::default]
- recipe[sanctionedlist::database]
verifier: Run tests again
inspec_tests:
- test/smoke/default
attributes:

In your database test, lets add a simple check that PostgreSQL is installed. Note that this is under test/smoke, not spec.

describe package 'postgresql-9.5' do
it { should be_installed }
end

Now run it

$ kitchen test
[…]
Target: ssh://vagrant@127.0.0.1:2222

System Package
✔ postgresql-9.5 should be installed
User root
✔ should exist
↺ This is an example test, replace with your own test.
Port 80
✔ should not be listening
↺ This is an example test, replace with your own test.

Test Summary: 3 successful, 0 failures, 2 skipped
[…]
-----> Kitchen is finished. (1m37.55s)

These are basic tests but at least kitchen is successfully running! You can already see that it takes longer than the unit tests, this is because we’re creating the virtual environment each time we converge. However, we can just run our tests quickly with verify. Lets see. Note: You can do kitchen test -d never to prevent a new environment from being set up each time.

$ kitchen verify
----> Creating ...
[…]
-----> Kitchen is finished. (1m30.73s)
$ kitchen verify
[…]
-----> Kitchen is finished. (0m1.48s)

Lets test for the PostgreSQL user creation

describe command('sudo -u postgres -s psql postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname=\'deploy\'"') do
its(:stdout) { should match /1/ }
end
$ kitchen verify

[…]
Command sudo
∅ -u postgres -s psql postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='deploy'" stdout should match /1/
expected "" to match /1/
Diff:
@@ -1,2 +1,2 @@
-/1/
+""

User root
✔ should exist
↺ This is an example test, replace with your own test.
Port 80
✔ should not be listening
↺ This is an example test, replace with your own test.

Test Summary: 3 successful, 1 failures, 2 skipped

Update the database recipe

include_recipe 'postgresql::server'
# create a PostgreSQL user
include_recipe 'database::postgresql' # This is required for PostgreSQL user creation

# create connection info as an external ruby hash
postgresql_connection_info = {
:host => '127.0.0.1',
:port => 5432,
:username => 'postgres',
:password => ‘password1'
}

# Create a PostgreSQL user but grant no privileges
postgresql_database_user 'deploy' do
connection postgresql_connection_info
password 'super_secret'
action :create
end

Lets run it

$ kitchen converge

[…]
Chef Client finished, 10/45 resources updated in 01 minutes 12 second
[…]

$ kitchen verify
[…]
Command sudo
✔ -u postgres -s psql postgres -tAc "SELECT 1 FROM pg_roles WHERE rolname='deploy'" stdout should match /1/
[…]
Test Summary: 4 successful, 0 failures, 2 skipped
[…]

Refactoring as we don’t want to hard code passwords into recipes

include_recipe 'postgresql::server'

# create a PostgreSQL user
include_recipe 'database::postgresql' # This is required for PostgreSQL user creation

# create connection info as an external ruby hash
postgresql_connection_info = {
:host => '127.0.0.1',
:port => node['postgresql']['config']['port'],
:username => 'postgres',
:password => node['postgresql']['password']['postgres']
}

# Create a PostgreSQL user but grant no privileges
postgresql_database_user node['postgresql']['user']['name'] do
connection postgresql_connection_info
password node['postgresql']['user']['password']
action :create
end
$ chef generate attribute default
[...]
default['postgresql']['config']['port'] = "5432"
default['postgresql']['password']['postgres'] = "password1"
default['postgresql']['user']['name'] = "deploy"
default['postgresql']['user']['password'] = "super_secret"

We will use better passwords later!

$ kitchen converge; kitchen verify

All fine. Lets set up the database. Add the following to end of the smoke test

[…]
describe command('sudo -u postgres -s psql postgres -tAc "SELECT 1 FROM pg_database WHERE datname=\'sanctionedlist\'"') do
its(:stdout) { should match /1/ }
end
$ kitchen verify
[…]
Test Summary: 4 successful, 1 failures, 2 skipped

As expected.

# create a PostgreSQL database
postgresql_database 'sanctionedlist' do
connection postgresql_connection_info
owner 'deploy'
action :create
end
$ kitchen converge; kitchen verify

[…]
Command sudo
✔ -u postgres -s psql postgres -tAc "SELECT 1 FROM pg_database WHERE datname='sanctionedlist'" stdout should match /1/

Lets stop testing the database for now and move to testing roles.

{
"name" : "database",
"description": "A role to configure our database",
"json_class": "Chef::Role",
"chef_type": "role",
"run_list": [
"recipe[sanctionedlist:database]"
]
}

A simple role. We will add attributes later.
Update kitchen yml to include database role and run verify.

[…]
suites:
- name: default
run_list:
- role[database]
[…]
$ kitchen converge; kitchen verify
[…]
INFO: Run List expands to [sanctionedlist::default, sanctionedlist::database]
[…]
Test Summary: 5 successful, 0 failures, 2 skipped

Not much happened with roles but later, it will be useful for management for when the project gets big. We want to show off our project on Github but our (weak) passwords would be exposed! I will show you how to do this with databags and secrets in next post.

Some good sources on integration testing:

2 thoughts on “How I delivered SanctionedList.com with a solo developer setup (Part 3)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s