System Verification Regression and Load Test Tool (ReLoadTT)

System Verification Regression and Load Test Tool (ReLoadTT) was gradually and iteratively developed based on technical requirements and experiences from many test automation projects over the years.

The main purpose of the framework and tool is to fulfill the following generic qualities:

• Robust/reliable test implementation – tests should fail when there is a problem in the tested application and not because of an unreliable test implementation.
• Maintainable test scenarios – efficient update strategy is an intrinsic part of the test development process.
• Reusable functionality – specific and relevant automation concepts are implemented in one place and can be reused in many tests. Code copying and similar logic variants are avoided.
• Scalable architecture – a large number of tests can be handled with efficiency and without overview degradation.
• Build-in logging and test report generation.

At the same time, ReLoadTT provides a tool that addresses the intrinsic requirement of a generic test automation tool with specialised features for UI test automation and load testing.

The generic aspect is achieved by supporting additions/extensions via programming language support. This means that additional functionality or low-level updates can be done in the programming language layer of the framework if necessary.

The specialised features are provided via a dedicated set of framework functions. These functions support the so-called DSL (Domain Specific Language) for the UI test domain.

On the technical level, ReLoadTT supports:

• Test development and execution on different platforms: Windows, MacOS, and Linux.
• Test development for different application types: Web, Mobile, and native.
• Debugging of test logic.
• CI / CD integration.

Finally, ReLoadTT provides a unique feature to perform load testing on the user interaction aspects of the application. This type of load testing provides direct insights into the user interface responsiveness of the application under load. The load can be generated by another tool/framework like Locust and use ReLoadTT to measure user-level application response times.

ReLoadTT DSL

ReLoadTT introduces the concept of a Domain Specific Language for a high-level, simplified, and non-programming approach for writing tests. The DSL idea introduces the following advantages:

• Readability – the ReLoadTT DSL is designed for user interface testing. It provides high-level steps that specifically address the test needs of the UI domain of applications and systems.
• Abstraction – the DSL hides the technical complexity of the underlying implementation. The test definition focuses on the business logic definition rather than the low-level details of the programming language or UI management utilities.
• Productivity – the ReLoadTT DSL provides reusable functions that allow expressing test steps more directly without the need to handle programming constructs. These DSL functions handle all common tasks for manipulating user interface elements like input, buttons, menus, etc. but also table-based data presentation.
• Maintenance – updates, and changes are made more efficiently in DSL-defined tests.

Its framework handles the trade-off of delivering the DSL in a prepared package that is ready for action on day 1 of the test automation project.

ReLoadTT DSL uses the Gauge Test notation language (gauge.org). This notation allows test definitions that are very close to a spoken language. Other popular alternatives in this area are Gherkin and the Robot Framework notation.

Challenges of UI testing

List of ReLoadTT functionality that is relevant and highly important to the UI test domain. The tooling in the framework provides robust implementations for all the listed challenges.

1. UI elements locators’ strategy. ReLoadTT framework provides page object encapsulation of application UI. The page object basically contains a dictionary for page elements of interest. An element is designated with a logical name, has a locator used to find it, and is set up with a set of methods to be manipulated. The element methods are used by the test step definitions related to the element.
2. UI testing timing issues. UI applications often involve asynchronous operations. ReLoadTT framework methods automatically handle timing issues with built-in support for synchronising the test steps with the application UI.
3. Web, Mobile, and native UI support. A unified framework that can handle different UI platforms and/or different types of applications in the same test sequence or test suite.
4. Dynamic elements. Locator support feature for elements that have dynamic properties.
5. Cross-Browser testing. For Web applications, it’s possible to test in different browsers.

The main ReLoadTT advantage is the possibility to provide specific and dedicated coded solutions in the selected programming language. The described DSL exposes the functionality to the test definition and the programming language implementation handles the technical details.

All UI test automation frameworks, both open-source and commercial solutions, need to address the above challenges. Usually, they do that with varying degrees of success and clarity. The main point is that automation projects usually fail miserably when trying to solve the above from scratch.

Programming languages support

ReLoadTT framework implementations exist for C#, Python, and Java. This allows different development groups to integrate the test automation with the development environment and language that they know and use.

The test implementation process

ReLoadTT is provided with a set of example test suites for Web and Mobile applications. The documentation is provided in the popular AsciiDoc format. Both aspects allow for a relatively quick path to using the tool/framework in the test automation work.

For most applications, UI testing is sufficient to create DSL-based test definitions. This part requires no specific programming skills. Obviously, some programming knowledge is required, even if very simplified, for the page object part definitions.

We have experimented with facilitating the popular OpenAI API for streamlining the page object creation. This is described in the next chapter.

OpenAI API experiments by Jens Johansson

The recent AI achievements made available through OpenAI's GPT model have given the possibilities for autogenerating code.

Within the ReLoadTT framework, we are evaluating the possibilities to automate the generation of page objects. By providing the GTP model with an example of the page object design used with the ReLoadTT framework the API could create new page objects, including descriptive element names and valid locators. The GPT creates the relevant page object class by reading an exposed Web URL address. This is usually the most time-consuming and mundane part of the test implementation process. With the help of the GPT API, a lot of time can be saved.

-

In conclusion, it is feasible to provide an implementation of the AI-based page object creator that is supplied together with the ReLoadTT framework. This can potentially further streamline the process of creating automated test scenarios where creating the coded page object is strongly augmented with a possible GPT PO-creator plugin. Writing the test definition itself is presently still beyond the GPT API capabilities because of a total number of tokes limitations.