Tosca functional testing

Tosca from Tricentis automation is in the first place a tool, and like any other tool, you need to know how to use it and be aware that you can misuse it too. This article assumes you are aware of how to write good quality tests in isolation.

The thought behind Tosca is that you should be able to use the tool without having advanced development skills. You scan a screen and Tosca transforms it into a module. You can then use that model to write your test case against, using a graphical way of coding (writing loops, waits, and conditions). As a business user, you should be able to create your first test case with just a few hours of training. You can then create re-useable test blocks to keep your tests maintainable, write a datasheet with different variables and let Tosca instantiate your tests based on the number of fields in combination with the requirements in your datasheet, pretty powerful in my opinion.

• 

• 

Tosca has a few different engines; one of them is a .NET forms engine, which is unique because all the modern day UI automation tools focusing on the web more than desktop applications. Most of the common controls are recognized but when you have a custom control you need to manage, and Tosca does not perform the way you want, so you can write your own implementation of it.

First time screen interaction

Start by selecting your sandbox, you use that to work in your own isolated space in the repository. That is where you can try things without getting into conflict with your colleagues.

• 

Create a module

Add a new module folder and choose scan application –> desktop. Next, choose your application screen you want to capture. Next, the XScan module will pop up and give you the possibility to select, the detected components in your application, and use it in your module. You can even use the slider to address different layers in your design like having a button inside a table or frame. Click save to create the module.

• 

• 

• 

• 

• 

When you choose a web browser, like the chrome browser. You will be asked to install a chrome extension. Afterward, the same screen will appear and you will be able to point out the controls you want in your module and how to recognize them. This is more important when you have a number of the same controls in your web-page like in the example below. Your primary identifier would be the ID but when it is generated or not present you can divert onto other properties. The HREF example below is not the best option but only for demonstration purposes that the tool will inform you that you have unique identifiable properties selected. Just be aware that when you choose XPath you run the risk of not being able to access your controls due to restructuring your page.

• 

• 

Create a test case

Now we have to define our test cases. Like the modules, we are going to start with creating the structure. Followed by creating a test case, because we eventually want to work with test datasheets you need to convert the test case to a template. Now just drag the created module onto the template instance.

• 

• 

• 

• 

Create a test datasheet

Because I want to use multiple values for my test, it is smart to set up a test data sheet up front instead of altering the test case afterward. Therefore, we are going to start by creating the structure, followed by the test sheet and a new instance. Create the attributes you want to use in your test data

• 

• 

• 

• 

• 

• 

Although you cannot see any direct changes now you have created the possibility to create a template instance under the test case and can fill your attributes with values to test.

Mixing it all together

Now everything is set up you can drag and drop the attributes on the related properties of your test case. Make sure the order is input -> ok -> response. You can change the order inside your module. Eventually, you have the actions, insert text in the input box -> click ok -> verify the response. Next, right click on the template instance -> “instantiate instance”. This will apply the test data into actions to perform per test instance.

• 

• 

• 

• 

Give it a spin

Let us make sure our application is running and give it run in scratch book command. When Tosca gives back the focus, you can then see the results in the scratch book log. Remember that when you change anything in your test case definition or test sheet you need to re-instantiate your test case instance before your changes take effect.

• 

• 

Although this was a very simple test case, it is cool you can create a structured test without any real programming.

Distributed execution

The distributed execution gives you the possibility to execute your execution lists on independent test machines. Those test machines are regular computers\VM that have a Tosca agent installed which connects to a central Distributed EXecution server (DEX). The DEX will distribute and manage the requested execution list over the available and capable (through configurations) machines. This gives you the possibility to run tests in isolation while working on another test case locally. The challenge for teams can be to understand how their tests function when they are executed in parallel; the better the tests are structured the fewer difficulties they will have. To monitor your test cases you can use the MonitorService.

• 

• 

CI/CD

To support in CI/CD they provide a “Tosca CI Client”. That client provides you with the possibility to send your execution lists via an XML to the DEX server. Polling every few seconds resulting in an output.xml in JUnit. As of Tosca 12.? there is no need for the workspace to be installed on the Tosca client machine. The DEX server will send only the instructions over, this is nice in theory but I have yet to get experienced with Tosca 12 on that matter.

Tosca SQL & REST API

Tosca gives you a great variety of possible interactions with your workspace. You can create your own scripts, to create a backup of the used test cases after the execution of a regression test using Tosca SQL; you need to have a complete local repository to make this work. Another option is to interact with objects in the workspace using a REST API; you bypass the local workspace requirement via that method. It depends on the specific task you want to execute what your weapon of choice would be. What I would have expected but just is not available is the possibility to execute a DEX through the REST API.

updateAll
jumptonode """/006_XXX/04_Execution/ExecutionLists/002_regression"""
task """Checkout Tree"""
yes
for "=>Items:ExecutionList[ChangesAllowed==\"""True\"""]" taskonall """Clear Log"""
task """Export Subset"""
"""c:\\temp\\ExportTosca.tcu"""
revertAll
Ok
exit

Tosca analytics

A valid question is off course are we testing the right parts good enough and what is the impact of a failed test on your product. Is feature A more important than feature B. Tosca analytics can help you answer those questions. By using requirements, you can create a structural-functional representation of your application. For each node, you enter the frequency it is used by your customers and a damage value when that part would fail. Tosca will calculate the weight over all the layers. Next, you can address test cases to the requirements, giving the test density per requirement. This will help you focus on the most important part of your application first. Tosca analytics gives a nice web report for stakeholders to view.

• 

• 

Concerns

During my time working with teams who use Tosca as their primary UI testing tool we learned of some caveats to be aware of. For instance, Tosca uses its own repository, which makes it near to impossible to align it with changes to your own code changes in your own repository. The test logs/results are also added to the same repository, therefore it is important to clean and maintain the logs to prevent the repository to grow quickly. My experiences tell me that a repository without any logging can be around 50 MB and after a 2-week sprint it is a little over 5GB, this would be for about 21.000 executed tests. Tosca is a well-maintained product and does bring out patches and versions on a regular basis, the downside is that in the four major updates I experienced the teams never blindly could jump to the new Tosca nor did it support backward compatibility the way you want now a day. When working together on a project you must be aware that the repository is a centralized repository system, meaning that you cannot both checkout and work on the same use case.

Summary

Tosca is certainly a good alternative for function-based testing. Although you need some knowledge about programming and reusing objects, working with classes and data management to keep the maintenance manageable. As with all tools, it is just a tool and the quality is a direct result of the people’s knowledge to write good tests and understanding the limitations and requirements of the tool.