Your First Touca Test
You've made it this far. Great! πŸ‘πŸΌ
We assume you have followed our Setup Your Account tutorial to create an account on Touca. In this document, we will show you how to write and run a simple Touca test to submit your first test results to the Touca server.
You will need your API Key and API URL to submit test results.
This is a hands-on tutorial. It's only fun if you follow along. πŸ‘¨πŸ»β€πŸ’»

Code Under Test

Let us imagine we are building a profile database software that retrieves personal information of students based on their username.
Python
C++
TypeScript
Java
1
def parse_profile(username: str) -> Student:
Copied!
1
Student parse_profile(const std::string& username);
Copied!
1
async function parse_profile(username: string): Promise<Student>;
Copied!
1
public static Student parseProfile(final String username);
Copied!
Where type Student could be defined as follows:
Python
C++
TypeScript
Java
1
@dataclass
2
class Student:
3
username: str
4
fullname: str
5
dob: datetime.date
6
gpa: float
Copied!
1
struct Student {
2
std::string username;
3
std::string fullname;
4
Date dob;
5
float gpa;
6
};
Copied!
1
interface Student {
2
username: string;
3
fullname: string;
4
dob: Date;
5
gpa: number;
6
}
Copied!
1
import java.time.LocalDate;
2
​
3
public final class Student {
4
public String username;
5
public String fullname;
6
public LocalDate dob;
7
public double gpa;
8
}
Copied!
We have created an examples repository that includes a possible implementation for this software.
trytouca/examples repository on GitHub
Clone this repository to a directory of your choice.
1
git clone [email protected]:trytouca/examples.git
Copied!
Navigate to your local copy of this repository and change your working directory to the directory for your preferred programming language. We have added several examples for each language. Each example serves as a standalone hands-on tutorial that showcases different Touca SDK features.
In this document, we will be using the 02_<lang>_main_api example which includes two modules students and students_test. The students module represents our code under test: the production code for our profile database software. Our code under test can have any complexity. It may call various nested functions, connect to database, and scrape the web to return information about a student given their username.
Check out the students module for a possible "current" implementation:
Python
C++
TypeScript
Java
python/02_python_main_api/students.py
1
def parse_profile(username: str) -> Student:
2
sleep(0.2)
3
data = next((k for k in students if k[0] == username), None)
4
if not data:
5
raise ValueError(f"no student found for username: ${username}")
6
return Student(data[0], data[1], data[2], calculate_gpa(data[3]))
Copied!
cpp/02_cpp_main_api/students.cpp
1
Student parse_profile(const std::string& username)
2
{
3
std::this_thread::sleep_for(std::chrono::milliseconds(200));
4
if (!students.count(username)) {
5
throw std::invalid_argument("no student found for username: " + username);
6
}
7
const auto& student = students.at(username);
8
return {
9
student.username,
10
student.fullname,
11
student.dob,
12
calculate_gpa(student.courses)
13
};
14
}
Copied!
javascript/02_node_main_api/students.ts
1
export async function parse_profile(username: string): Promise<Student> {
2
await new Promise((v) => setTimeout(v, 200));
3
const data = students.find((v) => v.username === username);
4
if (!data) {
5
throw new Error(`no student found for username: ${username}`);
6
}
7
const { courses, ...student } = data;
8
return { ...student, gpa: calculate_gpa(courses) };
9
}
Copied!
java/02_java_main_api/Students.java
1
public static Student parseProfile(final String username) {
2
Thread.sleep(200);
3
for (Student student : Students.students) {
4
if (student.username.equals(username)) {
5
return new Student(data.username, data.fullname, data.dob,
6
calculateGPA(data.courses));
7
}
8
}
9
throw new NoSuchElementException(
10
String.format("No student found for username: %s", username));
11
}
Copied!

Writing a Touca Test

With Touca, we call our workflow under test with various inputs and try to describe the behavior and performance of our implementation by capturing values of variables and runtime of functions as results and metrics. While this is similar to unit testing, there are fundamental differences:
    Instead of hard-coding inputs to our code under test, we pass them via the
    testcase parameter to our Touca test workflow.
    Instead of hard-coding expected outputs for each test case, we use Touca data
    capturing functions to record the actual values of important variables.
    Instead of being bound to checking the output value of our code under test, we
    can track value of any variable and runtime of any function in our code under
    test.
These differences in approach stem from a difference in objective. Unlike unit testing, our goal is not to verify that our code behaves correctly. We want to check that it behaves and performs as well as before. This way, we can start changing our implementation without causing regressions in our overall software.
Here is a possible implementation for our first Touca test code:
Python
C++
TypeScript
Java
python/02_python_main_api/students_test.py
1
import touca
2
from students import parse_profile
3
​
4
@touca.Workflow
5
def students_test(username: str):
6
student = parse_profile(username)
7
touca.add_assertion("username", student.username)
8
touca.add_result("fullname", student.fullname)
9
touca.add_result("birth_date", student.dob)
10
touca.add_result("gpa", student.gpa)
11
​
12
if __name__ == "__main__":
13
touca.run()
Copied!
cpp/02_cpp_main_api/students_test.cpp
1
#include "students.hpp"
2
#include "students_types.hpp"
3
#include "touca/touca_main.hpp"
4
​
5
void touca::main(const std::string& username)
6
{
7
const auto& student = parse_profile(username);
8
touca::add_assertion("username", student.username);
9
touca::add_result("fullname", student.fullname);
10
touca::add_result("birth_date", student.dob);
11
touca::add_result("gpa", student.gpa);
12
}
Copied!
javascript/02_node_main_api/students_test.ts
1
import { touca } from "@touca/node";
2
import { parse_profile } from "./students";
3
​
4
touca.workflow("students_test", async (username: string) => {
5
const student = await parse_profile(username);
6
touca.add_assertion("username", student.username);
7
touca.add_result("fullname", student.fullname);
8
touca.add_result("birth_date", student.dob);
9
touca.add_result("gpa", student.gpa);
10
});
11
​
12
touca.run();
Copied!
java/02_java_main_api/StudentsTest.ts
1
import io.touca.Touca;
2
​
3
public final class StudentsTest {
4
​
5
@Touca.Workflow
6
public void parseProfile(final String username) {
7
Student student = Students.parseProfile(username);
8
Touca.addAssertion("username", student.username);
9
Touca.addResult("fullname", student.fullname);
10
Touca.addResult("birth_date", student.dob);
11
Touca.addResult("gpa", student.gpa);
12
}
13
​
14
public static void main(String[] args) {
15
Touca.run(StudentsTest.class, args);
16
}
17
​
18
}
Copied!
Notice the absence of hard-coded inputs and expected outputs. Each Touca workflow, takes a short, unique, and URL-friendly testcase name, maps that to a corresponding input and passes that input to our code under test. In the above code snippet, once we receive the output of our parse_profile workflow, we use add_result to track various characteristics of that output. Touca notifies us if these characteristics change in a future version of our parse_profile workflow.
We can track any number of variables in each Touca test workflow. More importantly, we can track important variables that might not necessarily be exposed through the interface of our code under test. In our example, our software computes the GPA of a student based on their courses and using an internal function calculate_gpa. With Touca, we can check this function for regression by tracking both the calculated GPA and the list courses, without creating a separate test workflow.
Python
C++
TypeScript
Java
python/02_python_main_api/students.py
1
def calculate_gpa(courses: List[Course]):
2
touca.add_result("courses", courses)
3
return sum(k.grade for k in courses) / len(courses) if courses else 0
Copied!
cpp/02_cpp_main_api/students.cpp
1
float calculate_gpa(const std::vector<Course>& courses)
2
{
3
touca::add_result("courses", courses);
4
const auto& sum = std::accumulate(courses.begin(), courses.end(), 0.0f,
5
[](const float sum, const Course& course) {
6
return sum + course.grade;
7
});
8
return courses.empty() ? 0.0f : sum / courses.size();
9
}
Copied!
javascript/02_node_main_api/students.ts
1
function calculate_gpa(courses: Course[]): number {
2
touca.add_result("courses", courses);
3
return courses.length
4
? courses.reduce((sum, v) => sum + v.grade, 0) / courses.length
5
: 0.0;
6
}
Copied!
java/02_java_main_api/Students.java
1
private static double calculateGPA(final Course[] courses) {
2
Touca.addResult("courses", courses);
3
double sum = Arrays.asList(courses).stream().mapToDouble(item -> item.grade).sum();
4
return courses.length == 0 ? sum / courses.length : 0.0;
5
}
Copied!
Notice that we are using Touca add_result inside our production code. Touca data capturing functions are no-op in the production environment. When they are executed by Touca workflow in a test environment, they start capturing values and associating them with the active test case.
Lastly, Touca helps us track changes in the performance of different parts of our code, for any number of test cases. While there are various patterns and facilities for capturing performance benchmarks, the most basic functions are start_timer and stop_timer for measuring runtime of a given piece of code, as shown below.
Python
C++
TypeScript
Java
python/02_python_main_api/students_test.py
1
import touca
2
from students import parse_profile
3
​
4
@touca.Workflow
5
def students_test(username: str):
6
touca.start_timer("parse_profile")
7
student = parse_profile(username)
8
touca.stop_timer("parse_profile")
9
touca.add_assertion("username", student.username)
10
touca.add_result("fullname", student.fullname)
11
touca.add_result("birth_date", student.dob)
12
touca.add_result("gpa", student.gpa)
13
touca.add_metric("external_source", 1500)
14
​
15
if __name__ == "__main__":
16
touca.run()
Copied!
cpp/02_cpp_main_api/students_test.cpp
1
#include "students.hpp"
2
#include "students_types.hpp"
3
#include "touca/touca_main.hpp"
4
​
5
void touca::main(const std::string& username)
6
{
7
const auto& student = parse_profile(username);
8
touca::add_assertion("username", student.username);
9
touca::add_result("fullname", student.fullname);
10
touca::add_result("birth_date", student.dob);
11
touca::add_result("gpa", student.gpa);
12
touca::add_metric("external_source", 1500);
13
}
Copied!
javascript/02_node_main_api/students_test.ts
1
import { touca } from "@touca/node";
2
import { parse_profile } from "./students";
3
​
4
touca.workflow("students_test", async (username: string) => {
5
touca.start_timer("parse_profile");
6
const student = await parse_profile(username);
7
touca.stop_timer("parse_profile");
8
touca.add_assertion("username", student.username);
9
touca.add_result("fullname", student.fullname);
10
touca.add_result("birth_date", student.dob);
11
touca.add_result("gpa", student.gpa);
12
touca.add_metric("external_source", 1500);
13
});
14
​
15
touca.run();
Copied!
java/02_java_main_api/StudentsTest.ts
1
import io.touca.Touca;
2
​
3
public final class StudentsTest {
4
​
5
@Touca.Workflow
6
public void parseProfile(final String username) {
7
Touca.startTimer("parse_profile");
8
Student student = Students.parseProfile(username);
9
Touca.stopTimer("parse_profile");
10
Touca.addAssertion("username", student.username);
11
Touca.addResult("fullname", student.fullname);
12
Touca.addResult("birth_date", student.dob);
13
Touca.addResult("gpa", student.gpa);
14
Touca.addMetric("external_source", 1500);
15
}
16
​
17
public static void main(String[] args) {
18
Touca.run(StudentsTest.class, args);
19
}
20
​
21
}
Copied!
There is so much more that we can cover, but for now, let us accept the above code snippet as the first version of our Touca test code and proceed with running this test.

Running a Touca Test

Let us now use one of Touca SDKs to write a test that could help us detect future changes in the overall behavior or performance of our profile database software.
Python
C++
TypeScript
Java
Navigate to directory python/02_python_main_api in the examples repository and create a virtual environment using Python v3.6 or newer.
1
python -m venv .env
2
source .env/bin/activate
Copied!
Install Touca SDK as a third-party dependency:
1
pip install touca
Copied!
Navigate to directory cpp/02_cpp_main_api in the examples repository and run build.sh or build.bat depending on your platform using CMake 3.14 or newer. This command produces executables in a ./local/dist/bin directory.
1
./build.sh
Copied!
This example uses CMake's FetchContent to pull Touca SDK as a dependency. See our SDK documentation for instructions to use Conan, instead.
Navigate to directory javascript/02_node_main_api in the examples repository and use either of yarn or npm to build examples using Node v12 or newer.
1
npm install
2
npm build
Copied!
Navigate to directory java/02_java_main_api in the examples repository and use gradle to build examples using Java 8 or newer.
1
./gradlew build
Copied!
We can run Touca test from the command line, passing the following information as command line arguments.
    API Key: to authenticate with the Touca server
    API URL: to specify where test results should be submitted to
    Revision: to specify the version of our code under test
    Testcases: to specify what inputs should be given to our workflow under test
We can find API Key and API URL on the Touca server. We can use any string value for Revision. More importantly, we can pass any number of test cases to the code under test, without ever changing our test logic.
Python
C++
TypeScript
Java
1
python3 02_python_main_api/students_test.py
2
--api-key <TOUCA_API_KEY>
3
--api-url <TOUCA_API_URL>
4
--revision v1.0
5
--testcase alice bob charlie
Copied!
1
./local/dist/bin/example_cpp_main_api
2
--api-key <TOUCA_API_KEY>
3
--api-url <TOUCA_API_URL>
4
--revision v1.0
5
--testcase-file testcases.txt
Copied!
1
node 02_node_main_api/dist/students_test.js
2
--api-key <TOUCA_API_KEY>
3
--api-url <TOUCA_API_URL>
4
--revision v1.0
5
--testcase alice bob charlie
Copied!
1
gradle runExampleMain --args='--api-key <TOUCA_API_KEY> --api-url <TOUCA_API_URL> --revision v1.0 --testcase alice bob charlie'
Copied!
In real-world scenarios, we may have too many test cases to specify as command line arguments. We can write our test cases to a file and pass the path to that file using the --testcase-file option. Alternatively, we can add our test cases directly to the Touca server. When test cases are not provided via --testcase or --testcase-file options, Touca SDKs attempt to retrieve them from the Touca server.
The above command produces the following output.
1
Touca Test Framework
2
Suite: students_test
3
Revision: 1.0
4
​
5
( 1 of 3 ) alice (pass, 127 ms)
6
( 2 of 3 ) bob (pass, 123 ms)
7
( 3 of 3 ) charlie (pass, 159 ms)
8
​
9
Processed 3 of 3 testcases
10
Test completed in 565 ms
Copied!
At this point, we should see the results of our test on the Touca server. This is a big milestone. Congratulations! πŸŽ‰
Screenshot of the Touca server with the first version
Notice that this version is shown with a star icon to indicate that it is the baseline version of our Suite. Touca will compare subsequent versions of our software against the test results submitted for this version.
In the next section, we will see how to use Touca to understand the differences between different versions of our software, investigate their root cause, communicate our findings with our team members, and update the baseline version.
Last modified 27d ago