In the life of any software project there are bugs. Granted, there are different types of bugs, but you will never encounter a software project in which everything goes as planned. This post describes how to report on bugs in a way that helps your developers to quickly fix them.
Let’s assume you are testing your software product. You are looking forward to test out the new feature that you were looking forward to for the last two weeks.
With your browser you sign in to your web application and try to download the PDF that should be presented as a new report in your App’s management console.
It looks fine. You do some changes in the system and request another report PDF. Something seems off. Your changes are not reflected within the newly generated report.
How to report this issue?
Wrong User Requirement
If your project is backed by user requirement specifications and a proper test suite, then you should first consult the acceptance tests.
At velaluqa we like to write user story scenarios as acceptance tests together with the customer to carve out the behavior of the software system (see What I talk about, when I talk about testing).
Requirement Not Satisfied Anymore
One case could be that the acceptance test fails. This should not happen frequently. Though it could happen if a developer implemented a new feature or fixed a complex bug without properly checking the continuously run test suite and its results. If that is the case, you should report the failing scenario.
1 2 3 4
Failure: Scenario Fails Feature: Download PDF Report Scenario: Adding User is reflected in Report PDF Steps tested manually: Yes/No
If you tested the scenarios manually and observed unexpected behavior it is helpful to report what you were able to observe. See the last section about Unexpected Failing Behavior.
Another case could be that the scenarios for a certain feature are not comprehensive enough. Although all requirements as stated by the user story — its scenarios and acceptance tests — are satisfied, you expected some additional or deviating behaviour.
If that is the case you should adjust the requirements. For this, either create a new scenario or create a feature request.
Unexpected Failing Behavior
These are bugs that are not reflected in or caught by any test scenarios. Some examples might be:
- providing some custom inputs for a form field and when submitting the form you see a blank page
- requesting an archived download of some data but you are redirected to an error page
- starting long running export task fails after 2 hours with error message
These are some examples of unexpected behavior. You could report these lines as I presented them here. But they do not give a proper answer to the most important questions:
- What happened with the application?
- What did the application report as error message?
- When did the error occur?
- Is it reproducible?
- How did you interact with the system to produce the error?
- What did you expect and what was the actual behavior?
- Where did the error occur?
- Which server?
- Which environment/IP?
- Which operating system/browser and version?
The outline for a good bug report should look like this:
1 2 3 4 5 6 7 8 9
- Title - Affected software version - Affected operating system incl. version number - Affected browser incl. version number - Date & Time (when the error occurred) - Steps to Reproduce - Expected behaviour - Actual behaviour - Attachments (Screenshots or log files)
The title should provide a concise explanation of the issue.
adminrole to user in edit user form
Affected software version
This is necessary information. Provide at least one version number for which the bug can be reproduced.
Affected operating system
Especially in the sphere of the web the client-side software can be the cause for unexpected behavior. This includes the operating system.
MacOS, Windows & Linux have different ways of accomplishing the same thing. Even different versions of Windows introduce major changes under the hood which can cause changes in behavior.
Please provide all operating systems which you have tested and provide information about whether the bug is reproducible or not. This will help identifying operating-system-specific bugs.
Affected Operating System:
- MacOS 10.13.1 (reproducible, see below),
- ArchLinux 4.14.15-1-ARCH #1 SMP PREEMPT x86_64 GNU/Linux (not-reproducible)
More common are differences in various browser-engines.
Please provide a list of all browser versions which you have tested and provide information about whether the bug is reproducible or not. This will help identifying browser-specific bugs.
- Chrome 18.104.22.168 (reproducible, see below),
- Firefox 59.0a1 (2018-01-21) (64-bit) (not-reproducible)
Date & Time
Provide the date and time optimally to the second.
If your issue has to do with errors occuring in other — possibly remote — components, then we can look into the log files these components to identify the problem.
Issue at: 2018-02-18 10:33:12
Steps to Reproduce
These explain the how, describing where in the system the action was performed, which role the testing user had within the system and other similar preconditions.
Don’t be too specific
Steps to Reproduce:
- Logged in as an admin user
- Open admin back-end
- Go to users list
- Click Edit User (choose any non-admin user)
See a list of roles to choose from, then choosing admin and saving user should add the admin role.
I do not see a list of roles to choose from.
If possible attach additional files for support. Something like a screenshot before and after something went wrong can help the developer to quickly jump into the problem.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
Title: Cannot add `admin` role to user in edit user form Version 2.1.2 Affected Operating System: - MacOS 10.13.1 (reproducible, see below) - ArchLinux 4.14.15-1-ARCH #1 SMP PREEMPT x86_64 GNU/Linux (not-reproducible) Affected Browser: - Chrome 22.214.171.124 (reproducible, see below) - Firefox 59.0a1 (2018-01-21) (64-bit) (not-reproducible) Issue at: 2018-02-18 10:33:12 Steps to Reproduce: 1. Logged in as an admin user 2. Open admin back-end 3. Go to users list 4. Click Edit User (choose any non-admin user) Expected Behavior: See a list of roles to choose from, then choosing admin and saving user should add the admin role. Actual Behavior: I do not see a list of roles to choose from. Attachments: [Screenshot of the view in question]
To ensure the bug can be fixed quickly, you should provide enough information answering these basic questions:
For this you can use the following outline:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Title: Affected software version: Affected operating system incl. version number: Affected browser incl. version number: Date & Time (when the error occurred): Steps to Reproduce: Expected behaviour: Actual behaviour: Attachments (Screenshots or log files):