Connecting Practice

In “How to Report Bugs Effectively?” topic, one can identify the twofold strategy below to provide defect information effectively to developers.

1. Show the defects to the developers personally; Or
2. Provide all the information that the developers need; give them careful and detailed instructions on how to make it fail in the same way; and/or find pattern, symptoms, and/or answers to their own defects that concern them to know the cause. More information is almost always better than less - Simon Tatham. The intention is to enable them to reproduce the defects.

Moreover, reporting defects are task of showing or documenting the defects so that the developers responsible for repairing the defects can reproduce the failures and fix the defects.

Further, one can document the defects using a template for reporting defects, tracking defect status, and generating test summary report. The typical contents of the template for documenting defects includes:

Template Header

1. Project number - It should describe the project number accurately.

2. Test Specification name - It should describe the project test specification name clearly.

Template Details

1. Defect No. - It should describe the defect number to ensure every defects in a project has a unique number for traceability.

2. Tracking No. - It should describe the associated defect tracking number reference with the particular defect for traceability.

3. Priority - It should correspond to how urgent it is that the defect be fixed. If the defect needs to be fixed immediately, it should have its priority set to P1 (P1 being the highest priority and P5 being the lowest). Most serious bugs should have their priority set to P2.

P1 - Very High
P2 - High
P3 - Medium
P4 - Low
P5 - Very Low

Moreover, priority should be based on the failed functionality importance and the risk of failure to the business. Where possible the priority should be assigned based on discussions between QA group, Project Manager, BA, and Software Development Lead.

4. Dev Assigned - It should be the assigned developer for fixing a bug.

5. Reported By - It should be the tester’s name for traceability and accountability.

6. Reported Date - It should be the date when the defect was recorded.

7. Found in Build - It should be the date of the build when the defect was found.

8. State - It should be New, Open, or Closed.

9. Status - It should be:

• Disregarded - Defects that were disregarded in the bug report.
• NTReview - Defects that were New in the bug report for review.
• CNNWE - Defects that were Closed-NeverFix as not worth the effort.
• CWAD - Defects that were closed-Withdrawn as designed.
• CWCR - Defects that were closed-Withdrawn as cannot reproduce.
• CWD - Defects that were closed-Withdrawn as duplicate.
• CWNAB - Defects that were closed-Withdrawn as not a bug.
• CWTP - Defects that were closed-Withdrawn as third party.
• CWUE - Defects that were closed-Withdrawn as user error.
• CDLI - Defects that were closed-Deferred as Low Impact.
• Withdrawn - Defects that were closed-Withdrawn.
• OTClosing - Opened Defects that were found for closing.
• Fixed - Defects that were closed as fixed.
• ToTest - Defects that were opened to test.
• ToFix - Defects that were opened to fix.
• ToDefer - Defects that were opened to defer. It will be addressed in a later release.
• OTNMI - Defects that need more information.
• OTCR - Defects that can’t reproduce, functionality no longer available, or will be available in further phases. It is returned when the person assigned cannot replicate the incident.
• OTHNI - Defects that have new information.
• OTHD - “Defects that have dependency or a bug in another piece of functionality is restricting you from running this test.
• OTFNMI - Opened ToFix Defects that need more information.
• OTFHNI - Opened ToFix Defects that have new information.
• NNMI - New Defects that need more information.
• NCR - New Defects that can’t reproduce.

10. Bug Severity - It should correspond to the seriousness of the defect. it should be set to Critical, Major, Minor, or Low severity type. Severity Types are scheme to assist the project team in identifying the severity of the defect and managing the defect log.

Moreover, the number and severity level of open defects may affect timely completion of a project, so it is important to a have a consistent approach to assign severity level to errors; and critical that everyone know how to identify the severity of defects, and how severity levels affect the delivery of the system.

The following are definition of severity levels for defects. Notice that in addition to identifying what the severity levels are, we also present some example of what types of defects qualify for each severity. This helps the project team subjectively in deciding the severity of an error.

Critical: Business process and system functionality are SEVERELY affected - software HALTS. In most cases, they are defects in which will not allow the tester to proceed with the tests - a system inoperable or showstopper issue. Critical defects must be corrected before the code is released to production. Examples are as follows:

• Endless loop - A sequence of instructions in a computer program that are constantly repeated. In most cases, you get an incomplete page displays, or are unable to control the status of the mouse.
• Fatal errors - A condition that halts processing due to program bugs, read errors, or other anomalies. In most cases, generally you cannot recover from it.
• Program Hangs - A condition that software stops responding.

Major: Business process and system functionality are seriously affected; system and/or data is exposed to potential loss or interruption. It is a severe error that causes a program not to perform properly or to produce unreliable results. Normally, the user cannot find an appropriate “workaround” for this type of error. Also it needs immediate attention. It must be corrected before the code is released to the production. Near-term system workaround is required. Examples are as follows:

• Computation errors - Incorrect/Inaccurate calculation - a great deal. Sign errors are surely the most common errors of all. Causes: Loss of invisible parentheses, belief that a minus sign means a negative number, or out of formula/carelessness.
• Incorrect functionality - Fails to satisfy the target specifications or requirements.
• Erroneously displayed/unexpected display of error messages
• Database/table errors

Minor: Business process and system functionality are moderately affected - software is not at risk. Critical system(s) and/or data are not at risk. It needs to be resolved as soon as possible but not a showstopper. Also it is an error in which a “workaround” solution can be found or available for the user. These defects do not significantly impact the execution or performance of the software and need not necessarily be corrected before the code is released to production. Examples are as follows:

• If you click the button, the code does not work, but if you choose the command from the menu, it works fine.
• Incorrect display - In most cases, it is a condition wherein the display contains inconsistencies with the specification or requirement documents.
• Incorrect location - In most cases, it is a condition wherein the display contains inconsistencies with the specification or requirement documents.
• Data was excluded from display - In most cases, it is a condition wherein the specified or required data is not displayed.
• Unexpected behavior of link - In most cases, link is broken and goes not to the correct/intended page.
• Unexpected display of alert message - In some cases, the system responds with an alert message to the user.
• Major grammatical errors in warning/error messages
• Incorrect value for essential fields in the logs
• Obvious cosmetic errors in screens
• Incorrect default values

Low: Business process and system functionality are marginally affected or unaffected. System(s) and/or data are not at risk. In most cases, they are defects in which minimal error, a cosmetic change, or an enhancement is needed. Low level defects do not need to be corrected before release to production. Examples are as follows:

• Incorrect format
• Undeleted line space
• No line space
• No space between character
• Undeleted character
• Undeleted text
• Wrong alignment
• Wrong caption
• Wrong spelling
• Minor grammatical errors
• Wrong text position on screen
• Wrong data position on screen
• Wrong window position on screen

11. DC No. - It should be the defect category Number for analysis.

12. Subject, Object, or Function Name - It should be the subject, object, or function Name where the defect was found.

13. Defect Description - It should be clear, concise, and accurately describe the defect with actual facts so that developer doesn’t have to come looking for tester to get them. and it should have some mandatory fields like:

• Title - General description of the defect that clearly and accurately describe the defect.
• Steps to Reproduce The Bug - It should be the clear step by step instructions on how to reproduce the bug. It should provide a verbatim transcript of the session and show what commands you typed and what output in response. From Development point of view, it is very important. It should be minimized and easy-to-follow steps.
• Actual Result - It should be the clear and detailed description of what happens after the step by step instruction are followed. If you saw test failures then describe the errors, carefully and precisely, what they were. The developers need to know what has gone wrong. If available, include the error message number to get information or search for clues.
• Expected Behavior - It should be the description of the correct or expected results where the bug is not present.

14. Remarks - It should be any additional comments - keep them as consise as possible.

15. Screen Dump - It should be the captured image that could help in motivating developers to fix bugs that are hard to recreate.

16. Test Time - It should be the estimated (Time estimated to re-test the bug) and actual (Actual time spent re-testing the bug) in hours.

Join QAST Practitioner Group, download the material in MS Excel version, and/or give comment(s) at:
http://daniloalsonado.com/qastforum/index.php?topic=58.0