Frequently asked questions

Need technical support?

How to Start Specific TestRun Variations in Test Manager

Question

How to start specific variations from a test series?

Solution

This FAQ article explains different approaches to run only certain test variations from a test series. It also shows a solution to start only the variations with the test result "bad".

1. Without Tcl script:
1.1 Run specific variations by manually selecting them
One way to start only particular variations of a Test Series is to manually select them per left mouse click. By clicking and holding the Start button, you can select the option "Run selected".

1.2 Run specific variations in ScriptControl using the variation number.
Another way to start a particular variation from a test series is to use the internal Test Manager function TestMgr::Start {}. For this, the number of the variation to be executed must be entered in the curly brackets. Since all entries in the Test Manager are counted, the first variation does not have the number 1, but a number depending on the previous entries. In the following example, the variation numbers start with 7.

However, it is not sufficient to enter the number of the desired test variation in the curly brackets, as all settings, criteria etc. from the previous lines will then be skipped. Instead, their numbers must also be added. This leads to the following command to start variation 1 (and only variation 1):
          TestMgr::Start {1 2 3 4 5 6 7}
To start further variations, their numbers can simply be added after a blank.

Important note: The line numbers only match the Test Manager IDs if the test series is unchanged after reading in or was created strictly from "top to bottom" without deleting lines. If lines are subsequently changed or deleted, however, the IDs may be assigned crosswise.

Warning: The TestMgr::Start command is not an officially supported ScriptControl command!

2. Using Start and End procedures inside Test Manager:
2.1 Starting specific variations by entering the variation name in script
For the previous approach, the numbers of the global settings and criteria as well as the desired test run variations must be found and entered. There is also a high risk that by adjusting the variations, the Test Manager IDs no longer match the line numbers.
An alternative solution can be realized in a StartProc of the script file to be read by the Test Manager. In comparison to the first picture, the line StartProc must be added in the Global Settings. If you are new to using a script with the StartProc and EndProc functions, the FAQ article How are StartProc and EndProc used in Test Automation? will be helpful.

The process in MyStartProc can be divided into three steps, as can be seen in the following picture. In the first step, the number of the currently active test variation is assigned to a new variable V_number via "TestMgr get ActiveItem". In the second step, the name of the currently active test variation is read and assigned to a new variable V_name by inserting the variable V_number in "TestMgr itemget $V_number -name". In the third step, the name of the current test variation is compared with the names of the test variations to be executed. If the name does not match, the variation is skipped. In the following example, this means that only the variations "Variation 3" and "Variation 5" are executed.

2.2 Starting variations with the result "failed" from the previous run
The script to be read in the Test Manager can also be used to realize the use case that only the variations from a test series with the result "failed" are executed. In this case, in addition to the name and number of the active variation, the result is read and saved in MyEndProc. The result is subsequently  compared with "good" and "bad". Depending on the match, the name of the variation is then entered in the list ::bad_variations_numbers or ::good_variations_numbers.

When the test series is restarted, MyStartProc searches the list of test variations with the result "good" for the number of the current variation. If the number is included in the list, the variation is skipped.

If the parameters of a variation are changed after it has delivered the result "bad" in the first run, the variation may delivers the result "good" in the second run. Then it is also added to the list of variations with result "good" and is skipped in the third run.

Download: TMgr_RunSpecificVariations.tcl

Download: TMgr_SkipGoodVariations.tcl

Need technical support?
  • Date: 10.10.2021
  • Product: CarMaker
  • Version: 10.1
  • Component: Test Automation
  • Language: English

Tags

TestSeries
TestManager
Test Automation
ScriptControl
tcl/tk