Automation with Purpose: Avoiding the Space Shuttle Trap

What is the first thing that crosses your mind when you see the report that a test has failed in the automated run?

• “O-o, we have a bug here,” or
• “What is wrong with this test case now?”

Sometimes, we put so much effort into automating complex scenarios with lots of dependencies and we still end up with unstable tests that require even more time for maintenance. We get so buried in resolving issues with the test itself that we forget to take a step back, look at the big picture, and remind ourselves of the real purpose of having a test automation project in the first place.

Let’s face it: a test automation solution cannot be a purpose in itself. It has a means to an end. The true purpose of a test automation project is to serve the larger project it is testing. We need to stay focused on that larger project and not lose ourselves in the technical weeds of automation for automation’s sake.

There is certainly value in automating test cases - when we choose the right ones to automate. To get the best bang for our buck, here are a few scenarios to avoid automating:

• Highly interdependent systems: If the test case relies on several interconnected systems, small changes in one system can ripple through and break the test, making it a maintenance headache.
• Heavy reliance on third-party integrations: Third-party systems with high variability can make automated tests fragile and unreliable.
• Complex scenarios requiring validation on multiple systems: These might sound like prime candidates for automation, but they often require extensive setup and debugging, which can end up being more trouble than it's worth.
• Dependencies on frequently updated extensions: If your test case depends on specific extensions that need constant updates, you are signing up for more work than it’s worth.

In many of these cases, automation can become more of a burden than a benefit. I would rather focus on automating test cases that deliver fast, clear results. These tests, where failures signal real issues in the system under test, provide actionable insights into what has changed.

When setting up an automation project, decisions about programming languages, frameworks, and tools play a crucial role. These choices often dictate the extensions we need and the constraints we face. And sometimes, those constraints limit QA efficiency. To work around this, we build “space shuttle solutions”—over-engineered setups that are hard to understand and maintain. My advice? Use the simplest, most efficient tools that provide the best value for the project under test, rather than striving to create a perfect automation solution.

They say, “A jack of all trades is a master of none.” That’s not great advice for a developer but for a QA? It is spot on. QA should prioritize efficiency and flexibility over strictly following specific tools or methods. The goal is to deliver long-term value, not to perfect the automation project itself.

And let’s not forget: some scenarios are better left to manual testing. Manual testing keeps our QA instincts sharp and creates opportunities to identify potential improvements that automation might miss.

Keep it clear. Keep it simple. Serve the purpose.