Writing a good defect report or issue report is one of the most important skills required of a software tester. The art of creating a well-written defect report requires a balanced combination of insight, testing and communication skills. The ultimate goal of each defect report is to prevent specific, significant problems within a software system ever being experienced by the customer. This post based on an email I sent to my test team provides advice and tips geared towards helping you create defect reports that are informative and actionable, thus improving their value to the customer.
What Makes a Good Defect Report?
The purpose of defect report is to provide information to the project stakeholders about a problem encountered with the product. However, a “good” or valuable defect report takes that a step further and provides useful information in an efficient way.
To help us understand what is required in order to write a high quality defect report we can examine four important points:
- The Title
- Actions Performed (Steps)
- Expected and Actual Results
- Attachments
The Title
The title is the headline of your defect report. Similar to a newspaper headline it should grab the reader’s attention and give them a complete summary of the problem in a single line. It should be clear, concise and in no more than ten words, tell other users Where the issue can be found, How the user can repeat the issue and What the issue is. e.g.
Where = Win7/Chrm36:Homepage:
How = Clicking the “Next” button
What = Loads 404 Page Not Found
Win7/Chrm36:Homepage: Clicking the “Next” button Loads 404 Page Not Found
A good title helps reduce duplicate defects and can quickly convey a summary of the problem.
It’s best to avoid generic problems in the title. For example, these should never be used:
- XYX is not working properly
- Issue with XYZ
- XYZ is corrupted/does not look right
The above example titles add little value in describing the problem. By nature, every defect report is describing something that is not working as it should. Be specific about why it is “not working.”Instead of: Sorting is not working properly.Try: Sorting is happening in reverse order.
Instead of: Issues with GUI on navigation bar.
Try: Navigation bar is wrapping to a second line.
Bugs are often migrated into the developer’s database that may contain hundreds, if not thousands, of other defects. Imagine trying to search this database for “navigation bar”. That search will return every issue related to the navigation bar. Searching for “wrapping to second line” is much more specific making it easier to find the bug. Your bug report needs to survive (and be useful) beyond the current test cycle; a strong title will help it through its journey.
Actions Performed – Describing what you did to reveal the defect
This is the body of your report. The goal of this section is to show the reader how to reproduce the issue. Since this area usually contains the most information, it’s important to keep it concise and easy to read. Always number your steps and kept them short and to the point.
Tip: Using a prerequisite can reduce the number of steps.
Instead of listing out every step to login in, start your steps with: “Prerequisite: User is logged in”
Tip: Find the direct path to the issue
Testers will often stop at the point where they found a bug and log their last few actions. However, the most helpful bug reports are those that distil the report down to the core reproduction steps.
It’s a good exercise to reproduce the issue by following the steps you’ve just outlined. This will help ensure you’ve included everything the customer will need to reproduce it as well.
Sometimes digging a little deeper below the surface of the issue can add additional value. Here are some examples of how adding a bit more effort or thought will produce a higher quality report.
Example 1: Provide additional useful information
Scenario: You find that a video does not play.
Good: Mention if it happened on all videos and not just the one mentioned in report.
Better: Specify if the issue is reproducible on more than one browser or device.
Best: Upload a speed test showing that bandwidth was adequate when testing was happening.
Lesson: Try to identify and answer follow-up questions before the customer asks them.
Example 2: Report the issue, not a symptom of the issue
Scenario: We are testing an Address input field. We find that the Address field allows “1234567890” and it also allows “!@#$%^&*()_+”
Lesson: These are two different symptoms of the same issue. Closer inspection would reveal that the real issue is the Address field isn’t being validated at all. The problem may be more serious than the first symptom you find.
EXPECTED AND ACTUAL RESULTS
Now that you have described how to reproduce the issue, you need to explain the problem and the desired behaviour.
Tip: When describing expected results, explain what should happen, not what shouldn’t happen.
Instead of: The app shouldn’t crash.
Try: The user is incorrectly taken to XYZ screen.
Tip: When describing actual results, describe what did happen, not what didn’t happen.
Instead of: The user wasn’t taken to the XYZ screen.
Try: The user remained on the ABC screen which is incorrect.
ATTACHMENTS – WHAT TO DO AND WHAT NOT TO DO
Attachments add to the issue’s value by offering evidence of the issue’s existence, enabling the customer to reproduce it or helping the developer fix it. Each attachment should add to the value of the issue report in at least one of these three ways.
The following are some tips and guidelines to keep in mind when adding attachments:
IMAGES
- Adding images is a quick way to add context to your issue. Consider adding an image even if you also have a video.
- Highlight the area(s) of interest in your image.
- Attach the image files directly to the report. Don’t put images in a Word document or a zip file.
- Use images to illustrate static issues.
VIDEOS
- Video confirms your steps were accurate at the time the issue report was created. For example, a screen grab of an error message isn’t as useful as seeing what went into creating that error message.
- Actions in the video should match the steps listed in the issue report.
- Videos should be trimmed to only show the issue.
- Provide video if the steps are complex.
- External/live videos can be more impactful than mirrored videos because you can see hand gestures or you touching a button on the screen.
LOG FILES AND OTHER TIPS
- Avoid proprietary file types (like .docx). Use .txt instead.
Use this advice when creating issue reports for any products you are working on today!
Log in using your favourite account and write your comments here