Can you trust a tester?
I have worked in the software QA business for more than 20 years, helping clients with software testing and test management primarily. Something that I find interesting is when I can see connections between the IT business and other industries. Today I would like to discuss one of these topics. It’s one that came to mind in the light of a notable medical scandal.
Theranos was founded in the early 2000s by Elizabeth Holmes, a young biotech entrepreneur who had studied chemical engineering at Stanford University. She wanted to revolutionize the healthcare industry by providing innovative tools for blood testing. The idea was to make it possible to detect diseases and medical risk conditions at an early stage, while making it accessible and stress-free for patients. Theranos invested years of research and development to invent a device that would be able to perform a large number of tests based on just a tiny sample. A few drops of blood from a fingerstick would be sufficient. The company was backed by a group of powerful investors, who also had strong connections in politics. Partnerships were established with clinics and insurance companies, as well as a major pharmacy store chain, which offered blood testing by Theranos as a service to its customers. The business was a huge success. By the end of 2014, the corporation was valued at $9 billion. Elizabeth Holmes was seen as a role model and influencer in Silicon Valley. Some even praised her as the next Steve Jobs.
The turn came in the fall of 2015. A Wall Street Journal reporter had investigated Theranos after receiving tips from a previous employee. The investigation showed that the blood testing device invented by the company did not work reliably. In fact, it would often produce incorrect test results. Theranos, aware of the flaws of its own technology, would actually use equipment from other companies for most of their testing, which was against FDA regulations. The claims were initially denied by Theranos, but further investigations carried out by federal agencies later revealed a number of deficiencies at the clinic. This resulted in a ban for Holmes to own, operate or direct a clinical lab. Theranos was sued by several of its business partners. Eventually, the business was shut down completely. Holmes and her business associate were also indicted on federal charges but pleaded not guilty. In early 2022, Holmes was found guilty on four counts of conspiracy and wire fraud. At the time of writing, her sentencing is set for October 2022. [1] [2]
The story about Theranos got me thinking. Without a doubt, there are many ethical, legal, and medical aspects to discuss (and I encourage all readers to go ahead and have vivid discussions in small groups after the read). But in the context of testing, I would like to raise the question about the tester’s credibility. It’s a question that you rarely hear about. For some reason, we take it for granted that testers will always report results objectively and truthfully. But what happens if this isn’t the case?
As a tester, you assess what is right and what is wrong. Whenever you notice deviations from the expected, it’s your job to report them as correctly and objectively as possible. In addition, you need to convey the information so that the recipients can understand and take the appropriate actions. To be successful, you need to have domain knowledge, technical knowledge, and good communicative skills. But that’s not all. You also need credibility. The recipient must trust what you say, or your message will not be recognized. As a tester, you are definitely in the business of trust.
Organizations may choose to promote an ethical work approach through a code of ethics statement. For an example, see the ISTQB® Code of Ethics, which highlights the importance of acting in the best interest of clients, employers, and the public. It also states the importance of delivering results that meet the highest professional standards possible.[3]
In practice, I’m convinced that most test professionals do report test results conscientiously. Whenever a result is not reported accurately, I assume it’s more likely to be a result of misunderstanding or ignorance rather than deliberate forgery. Nonetheless, as we learned from the Theranos story, the risk cannot be ruled out completely. There might be economic or ideological reasons that could lead someone to fabricate test results.
Now that you are aware of the risk, how can you mitigate it? Here are my ideas:
1. Work according to a clearly defined process. First of all, ensure that your organization has a sensible work process in place, that is agreed on and followed by all staff members. For the testing, transparency and traceability are important, so that you can always answer the questions of what, when, where, how and why tests have been run. You may also want to use reviews and audits to help improve the quality of your work products and processes.
2. Try to find alternative test oracles. If possible, check your test results against a different source. Assuming it’s equivalent, you should still get the same results, and you can tell that your test results are valid.
3. Consider introducing some independence. You may choose to distribute the responsibility for testing, rotating the task between several testers. You could also involve someone outside your team or organization to do part of the testing.
How would you address this risk in your test strategy?
