In the life of any software project there are bugs. Granted, there are different types of bugs, but you will never encounter a software project in which everything goes as planned. This post describes how to report on bugs in a way that helps your developers to quickly fix them.

Let’s assume you are testing your software product. You are looking forward to test out the new feature that you were looking forward to for the last two weeks.

With your browser you sign in to your web application and try to download the PDF that should be presented as a new report in your App’s management console.

It looks fine. You do some changes in the system and request another report PDF. Something seems off. Your changes are not reflected within the newly generated report.

How to report this issue?

Wrong User Requirement

If your project is backed by user requirement specifications and a proper test suite, then you should first consult the acceptance tests.

At velaluqa we like to write user story scenarios as acceptance tests together with the customer to carve out the behavior of the software system (see What I talk about, when I talk about testing).

Requirement Not Satisfied Anymore

One case could be that the acceptance test fails. This should not happen frequently. Though it could happen if a developer implemented a new feature or fixed a complex bug without properly checking the continuously run test suite and its results. If that is the case, you should report the failing scenario.

1
2
3
4
Failure: Scenario Fails
Feature: Download PDF Report
Scenario: Adding User is reflected in Report PDF
Steps tested manually: Yes/No

If you tested the scenarios manually and observed unexpected behavior it is helpful to report what you were able to observe. See the last section about Unexpected Failing Behavior.

Requirements Changed

Another case could be that the scenarios for a certain feature are not comprehensive enough. Although all requirements as stated by the user story — its scenarios and acceptance tests — are satisfied, you expected some additional or deviating behaviour.

If that is the case you should adjust the requirements. For this, either create a new scenario or create a feature request.

Unexpected Failing Behavior

These are bugs that are not reflected in or caught by any test scenarios. Some examples might be:

  • providing some custom inputs for a form field and when submitting the form you see a blank page
  • requesting an archived download of some data but you are redirected to an error page
  • starting long running export task fails after 2 hours with error message

These are some examples of unexpected behavior. You could report these lines as I presented them here. But they do not give a proper answer to the most important questions:

  • What?
    • What happened with the application?
    • What did the application report as error message?
  • When?
    • When did the error occur?
  • How?
    • Is it reproducible?
    • How did you interact with the system to produce the error?
    • What did you expect and what was the actual behavior?
  • Where?
    • Where did the error occur?
    • Which server?
    • Which environment/IP?
    • Which operating system/browser and version?

The outline for a good bug report should look like this:

1
2
3
4
5
6
7
8
9
- Title
- Affected software version
- Affected operating system incl. version number
- Affected browser incl. version number
- Date & Time (when the error occurred)
- Steps to Reproduce
- Expected behaviour
- Actual behaviour
- Attachments (Screenshots or log files)

Title

The title should provide a concise explanation of the issue.

Cannot add admin role to user in edit user form

Affected software version

This is necessary information. Provide at least one version number for which the bug can be reproduced.

Version 2.1.2

Affected operating system

Especially in the sphere of the web the client-side software can be the cause for unexpected behavior. This includes the operating system.

MacOS, Windows & Linux have different ways of accomplishing the same thing. Even different versions of Windows introduce major changes under the hood which can cause changes in behavior.

Please provide all operating systems which you have tested and provide information about whether the bug is reproducible or not. This will help identifying operating-system-specific bugs.

Affected Operating System:

  • MacOS 10.13.1 (reproducible, see below),
  • ArchLinux 4.14.15-1-ARCH #1 SMP PREEMPT x86_64 GNU/Linux (not-reproducible)

Affected browser

More common are differences in various browser-engines.

Please provide a list of all browser versions which you have tested and provide information about whether the bug is reproducible or not. This will help identifying browser-specific bugs.

Affected Browser:

  • Chrome 59.0.64.1 (reproducible, see below),
  • Firefox 59.0a1 (2018-01-21) (64-bit) (not-reproducible)

Date & Time

Provide the date and time optimally to the second.

If your issue has to do with errors occuring in other — possibly remote — components, then we can look into the log files these components to identify the problem.

Issue at: 2018-02-18 10:33:12

Steps to Reproduce

These explain the how, describing where in the system the action was performed, which role the testing user had within the system and other similar preconditions.

Don’t be too specific

Steps to Reproduce:

  1. Logged in as an admin user
  2. Open admin back-end
  3. Go to users list
  4. Click Edit User (choose any non-admin user)

Expected Behavior

Expected Behavior:
See a list of roles to choose from, then choosing admin and saving user should add the admin role.

Actual Behavior

Actual Behavior:
I do not see a list of roles to choose from.

Attachments

If possible attach additional files for support. Something like a screenshot before and after something went wrong can help the developer to quickly jump into the problem.

Also try to attach all log files that you might have access to. On pure web applications most logs are on the server, but you could attach logs from the JavaScript console.

Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
Title: Cannot add `admin` role to user in edit user form

Version 2.1.2

Affected Operating System:

- MacOS 10.13.1 (reproducible, see below)
- ArchLinux 4.14.15-1-ARCH #1 SMP PREEMPT x86_64 GNU/Linux (not-reproducible)

Affected Browser:

- Chrome 59.0.64.1 (reproducible, see below)
- Firefox 59.0a1 (2018-01-21) (64-bit) (not-reproducible)

Issue at: 2018-02-18 10:33:12

Steps to Reproduce:

1. Logged in as an admin user
2. Open admin back-end
3. Go to users list
4. Click Edit User (choose any non-admin user)

Expected Behavior:

See a list of roles to choose from, then choosing admin and saving
user should add the admin role.

Actual Behavior:

I do not see a list of roles to choose from.

Attachments:

[Screenshot of the view in question]

TL;DR

To ensure the bug can be fixed quickly, you should provide enough information answering these basic questions:

  • What?
  • When?
  • How?
  • Where?

For this you can use the following outline:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Title:

Affected software version:

Affected operating system incl. version number:

Affected browser incl. version number:

Date & Time (when the error occurred):

Steps to Reproduce:

Expected behaviour:

Actual behaviour:

Attachments (Screenshots or log files):