Testing a test case – necessary or not?

How often do you test a test case?

One easily ends up with the idea that the test cases are the answer and therefore correct. However, test cases might need to be adjusted whenever a requirement, product, use case etc. is updated, changed or deleted. Even if there is no change it is always a good practice to test a test case to make sure it still has a value.

Mikael Bolton wrote an interesting article that caught my interest:

“The tester finds a bug, and the test case may play a role in finding the bug”

How do you test a test case? Start by running through the test cases and when you find a bug, you score the test case as follows:

• 3 points – The test case played a crucial part of finding the bug, it would not been found without the test case.
• 2 points – The test case had a considerable part of finding the bug, but it would have been possible to find it without the test case.
• 1 point – The test case was helpful in finding the bug.
• 0 points – The test case neither helped nor interfered with finding the bug.
• -1 point – The test case is time consuming and/or distracts the tester from finding bugs
• -2 points – The test case is extremely time consuming and/or distracts the tester from finding bugs.
• -3 points – The test case interferes and prevents the tester from finding bugs.

What you can see above from this method is that you do not need to find a bug in order to evaluate a test case. But how do you know if a test case prevents you from finding bugs? Once you have finished testing a test case, you should test the same area/function with the exploratory testing method to see if there bugs there.

When you have run your suite with test cases, the results should go into a list with the highest score first according to Bolton’s list mentioned above. All test cases ranging from 3 to 1 can be kept since they contribute to the test process. Test cases ranging from 0 to –3 should either be removed or rewritten in order to help the tester instead of hinder.

Is that it? Unfortunately, not quite. There is another dimension that seems to put a twist on the method above; test cases often exist to verify requirements hence it cannot be removed only because it does not contribute to the test process. In case of this event, re-evaluate the test case and investigate whether it can be rewritten to better help the test process.

How often do you need to test your test cases? It’s about as easy as answering how long is a string? As with so many other questions in the industry, the answer is: It depends.
You will have to look at what type of project you are in, the risk assessment, management etc.

Do you want to talk more about this or need help with your project, do not hesitate to contact us!