Interpret Results
In the previous document, we learned how to write a simple Touca test and run it to submit data for the current version of our software workflow.
Touca Suite Overview page
This page provides an overview of how our workflow changes over time in behavior and performance. We can click on each version to inspect the test results submitted for each test case in that version.
Touca Version Overview page
We can also review the captured data for any one of the test cases.
Touca Test Case Overview page
But submitting results to a remote Touca server makes more sense when we have large amounts of data that is difficult to compare across versions and overwhelming to interpret manually. Touca server automatically compares submitted test results against previous versions and visualizes all differences to make it easy for us to trace those differences back to their root cause.
In this document, we will learn how Touca server can help us detect differences in future versions of our software, inspect those differences to determine whether they are intended, and take action based on our investigation.

Changing the Code

We are going to keep using the same "Parse Profile" software as in the previous document. Let us start with making a small change to our code under test.
Python
C++
TypeScript
Java
While we can make any changes to our code under test, for now we are going to replicate a change in behavior by changing one of Alice's courses from Course("math", 4.0) to Course("english", 3.6).
Once we rebuild our software and its corresponding test tool, we can run our test again, this time for version 2.0 of our code.
1
python ./students_test.py --revision 2.0
Copied!
If you are using the Docker image provided in the examples repository, you have sudo privileges to install your preferred editor, e.g. sudo apt update && sudo apt install vim.
While we can make any changes to our code under test, for now we are going to replicate a change in behavior by changing one of Alice's courses from Course {"math", 4.0} to Course {"english", 3.6}.
Once we rebuild our software and its corresponding test tool, we can run our test again, this time for version 2.0 of our code.
1
./build.sh
2
sudo cmake --install local/build
3
example_cpp_main_api --revision 2.0
Copied!
While we can make any changes to our code under test, for now we are going to replicate a change in behavior by changing one of Alice's courses from { name: 'math', grade: 4.0 } to { name: 'english', grade: 3.6 }.
Once we rebuild our software and its corresponding test tool, we can run our test again, this time for version 2.0 of our code.
1
yarn build
2
node ./students_test.js --revision 2.0
Copied!
While we can make any changes to our code under test, for now we are going to replicate a change in behavior by changing one of Alice's courses from Course("math", 4.0) to Course("english", 3.6).
Once we rebuild our software and its corresponding test tool, we can run our test again, this time for version 2.0 of our code.
1
./gradlew build
2
gradle runExampleMain --args='--revision 2.0'
Copied!
Notice that we are not specifying the list of test cases anymore. When they are not explicitly provided, the SDK fetches this list from the Touca server.
1
Touca Test Framework
2
Suite: main-api
3
Revision: 2.0
4
5
( 1 of 3 ) alice (pass, 233 ms)
6
( 2 of 3 ) bob (pass, 236 ms)
7
( 3 of 3 ) charlie (pass, 228 ms)
8
9
Processed 3 of 3 testcases
10
Test completed in 1223 ms
Copied!
As test cases are executed, the server compares their captured test results and performance benchmarks against the pervious "baseline" version v1.0 and visualizes the differences.
Comparison results for v2.0
We can inspect the differences reported for test case alice to understand the impact of our recent code change.
Comparison results for test case alice in v2.0

Promoting Baseline

The main goal of the Touca server is to make it easy for us to find and understand the side-effects of our code changes. Once we inspect the reported differences for a given version of our workflow, we can decide whether those differences match our expectations. If we determine that the differences are unexpected, we can share the findings of our investigation with our team by adding comments and collaborate to find the root cause of potential defects.
There are times when we may want to change expected behavior or performance of our workflow. We can use the Touca server to promote any given version of our workflow as the new baseline, using the "Promote Version" button in the version overview page.
You can set any version as Baseline for your Suite
Promoting a version as "Baseline" makes the Touca server automatically compare future versions of our workflow against that version. Since this may be an important event in the course of evolution of our software, we could make a note to explain our reasons for taking this action. The server shares our explanation with all team members subscribed to this Suite using their preferred notification settings(s).
In the next section, we will review some of the Touca server features regarding event subscription and notification management.
Last modified 1mo ago