Approach to Implementing Risk Based Testing

Kishore C.S.

Introduction

Motivated by articles on Risk based testing, I have attempted to implement this in my work and the following is my experience.

Also read my views on use of ‘Risk Based Testing’ in Software. Lack of risk strategies results in lost focus during a test project. Establishing this approach not only at project level but test case level enhances value of this approach.

 Approach

 The following is the pilot approach I have followed in implementation of Risk Based Testing. Initial stages of the project did not contain risk assessment and the test status was not clear. Realizing this the test cases were updated to include Risk and subsequent metrics generated out of the approach were much more meaningful for the project team.

 Step 1: Preparation of Test Cases

 After I was assigned the testing responsibility, I prepared the test plan identify the features to be covered. For each feature I detailed the test cases in the following format (Table 1)

 Table 1: Normal Test Cases

 These test cases were detailed out and reviewed by developers, peers and manager.

 Step 2: Normal Test Execution Results

 Once the test cases are executed the results are updated in the table. The status of the case is given as Pending, In Progress, Failed, Pass. Remarks column is used to add more information about the test execution. Status Date is also entered as and when status is updated.

 Using Excel Pivot Chart I created a summary table showing status of testing. Table 2 shows the test results summary.

Table 2: Test Results Summary

This gave me a metric to communicate to my supervisor about how much testing is completed and how much is pending.

 Step 3: Introduction of Risk Factor

 But the project lead wanted to know how many critical cases have failed. It is at this point of time I put Risk Based Testing into practice

 I updated the Test Case Document to include Risk Column.

 I categorized Risks on 1 – 5 Scale (1 Low, 5 High). Failure of a risk 5 test case would mean the system is not acceptable. Risk 1 test case is a problem that needs to be resolved and is likely to inconvenience the user.

I updated the test case document by giving a risk for each test case.

Table 3 shows the test cases table after introduction of Risk

Table 3: Test Cases with Risk attached to each case.

 I created a new pivot table that gives me test case distribution by risk category. This helps me analyze that the testing is focused on high-risk areas. If the number of test cases in risks 3-5 forms at least 70-80% of my test cases, I can make an assumption that my test cases are good. If that is not the case, I may be testing unimportant aspects the software wasting time and energy.

 Table 4: Summary of Test Cases

Step 4: Analyzing the new status

 And taking the status report based on test risk and status gave a matrix as shown below

The above table reflects status of testing that there are no failures in High Risk Category and there are still some cases to be executed in high-risk category.

The above report acted as a very good metric to measure software quality and will be a key factor in making decisions related to software release.

Advantages of this approach

 -          Quantitative report of Software Risk that enables better decision making

-          Prioritization of Software Defects based on Risk

-          Focused approach on Critical User Requirements at testing stage resulting in easier acceptance of software by customer

Summary

 Implementation of Risk Based Testing starts with an understanding of the importance of the same. A pilot approach can be developed and implemented by any project. Templates and data developed can be applied to subsequent projects. Having a quality metric on a software projects gives a very good advantage for the project manager to make decisions. By reducing risks of the project the project can be a big success. Risk based testing approach eliminates ambiguity in status of test project and injects confidence into the process.