A big part of software development is quality assurance. We want to make sure that we implement the correct behaviour according to specifications and that we don’t introduce bugs when deploying a new version. For this, every project depends on a rigorous testing process, either manual or automated. We like to let our development process be guided by automated tests so most of the time they become the first code we write.

In the post When 10x becomes 1/10th I described how a project can get derailed by accumulating too much technical debt.

One core-reason for technical debt was the lack of a proper test suite that lets developers recognize regressions — bugs introduced while implementing new features or performing other changes to the system — early on and fix them before they reach the production environments.

There are other reasons for technical debt, which may be mitigated through proper testing processes.

For instance you can write requirement specifications that are backed by automated tests. Thus ensuring that the specifications are precise and verifiable. This also helps with refactoring, because we can ensure to keep the behaviour specified by the acceptance tests.

If done correctly testing can help to keep your team’s velocity at a constant pace and help your team to stay confident when performing code changes.

Levels of Testing

There are many levels of testing and on each level there are different types of tests. When developing there are also many ways on how to go through these levels when developing software products:

  • Top-to-bottom approach, where we develop larger integrated specifications on the system level (how the user is supposed to interact with the system), and work our way down
  • Bottom-up approach, where we develop the most granular levels first and combine them in the upper levels
  • Sandwich approach, where we combine the latter two approaches

At velaluqa we like to work mainly with the top-to-bottom approach.

System Testing

A system test is a kind of black-box test, describing the behaviour of the system as the user sees and interacts with it. The user can be either a real person or another system interacting with an API.

Mostly these tests define scenarios for each feature in which the user interacts with the system through her browser. Thus testing the graphical user interface, functionality and components beneath.

Optimally these tests are written with the customer so that they can be used as validation against the user requirement specifications.

Integration Testing

Integration tests are a level below system tests. They are written by the developers to test the behaviour of single components when they are combined with other components.

This helps testing the software design and the interaction of singular components.

Unit Testing

Unit Tests check single components. Usually they test the smallest part of a system that can be tested independently — a unit. This can be a module but most of the time we test single functions or procedures.

Since we want to test single components we try to make sure that they are tested independently from others. Usually fakes (mock objects, method stubs and similar) are injected to separate the components from the depending parts of the system.

Most of the time this helps to keep the components decoupled which results in cleaner code.

How we test

We favor the top-to-bottom approach. When we start a new project we like to capture the user requirements as user stories. Stories that describe how the user is supposed to interact with the software product.

Together with the customer we describe different scenarios for each feature or user story which describes the user interaction step-by-step in the Gerkhin language. This language is a human-readable, domain specific language in this format:

1
2
3
4
5
6
7
8
9
10
11
12
13
Feature: Some terse yet descriptive text of what is desired
  Textual description of the business value of this feature
  Business rules that govern the scope of the feature
  Any additional information that will make the feature easier to understand

  Scenario: Some determinable business situation
    Given some precondition
    And some other precondition
    When some action by the actor
    And some other action
    And yet another action
    Then some testable outcome is achieved
    And something else we can check happens too

The scenario for a feature may look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Feature: List Products
  As product manager,
  I want to see a list of products in the system.

  Background:
    Given a product "Full-HD Projector" exists for 1299.99 EUR

  Scenario: Unauthorized
    Given I sign in as a user with all permissions
    But I cannot read products
    When I browse to the products list
    Then I see the unauthorized page

  Scenario: Authorized
    Given I sign in as a user with role "Product Manager"
    When I browse to the products list
    Then I see "Products"
    And I see "Full-HD Projector 1299.99 EUR"

Then we provide each step with reusable test code that drives a real browser or mobile emulation environment to perform the actions within the software just as a normal user would.

These tests are system tests that not only test the whole system and all the lower components but also serve as documentation and validation against the user requirement specifications.

The system tests guide us to the next layers of tests: Integration Tests. For a web application for instance we would create routing tests that check for the necessary URL to be handled by the web application server. Then we would test the controller which handles the given parameters and passes them on to the models. Then we test the models to make sure that they communicate correctly with the database and that our data is saved and retrieved correctly. Finally we test the views — what the user sees — and check that they are sent to the user with the correct information.

While doing this, each code change triggers a rerun of the specific component tests or integration tests so that the developer is constantly in the loop to recognize bugs early on.

Conclusion

Obviously this process of software development cannot guarantee bug-free software, but it can imply a certain level of confidence, that the set of defined scenarios are working correctly.

The right amount of test coverage is project-specific but you should never go without testing. It helps to keep velocity and confidence, helps to refactor safely and thus allows to fight technical debt.

Tests provide a sort of living documentation of the system. Developers who are interested in learning what kind of functionality is provided by the system or component can look into the respective system or unit tests to gain a basic understanding.

When using a test-driven approach, as I described in “How we test”, the combination of writing tests from the top — specifying the interface first - working down to the bottom and refactoring along the way may take the place of formal design. For instance each unit test can be seen as a design element specifying classes, methods and observable behaviour.

TL;DR

I hope this piece could give you a basic understanding of what we like about testing and how we incorporate it into our process. If you have any questions, please don’t hesitate to leave a message either in the comments or via e-mail.