Test Results

Test Results

Note: By default, the test case results are cleaned up at the end of the test run. All that's left is the text file quicktest.log and video timelapse quicktest.mp4. This behaviour can be overridden by the setting the environment variables QT_KEEP_SCREENSHOTS and QT_KEEP_TESSERACT_TEXT to true

Results

• quicktest.log file contains the same output as echoed during the test run.
• quicktest.mp4 is a timelapse of all the screenshots (good and bad) taken during the test case execution.

Results folder

Test case results are in a unique folder under the ./results folder in $OS/$RELEASE/$TESTCASE/$DATE-TIME-STAMP/. For example:

tree results/ubuntu-mate/daily-live/test_install_entire_disk_with_defaults/20240515-111141
results/ubuntu-mate/daily-live/test_install_entire_disk_with_defaults/20240515-111141
├── quicktest.log
└── quicktest.mp4

Results summary

When a test passes or fails, a single line will be written out to quicktest.log. The same line is sent as a desktop notification if enabled.

 tail -n 5 quicktest.log
🚨 FAIL  20240515-003115 20240515-003348 ubuntu-mate 24.04  test_install_entire_disk_with_defaults
🚨 FAIL  20240515-003348 20240515-005014 ubuntu-mate daily-live  test_install_entire_disk_with_defaults
🎉 PASS  20240515-102934 20240515-104123 ubuntu-mate daily-live  test_install_entire_disk_with_defaults
🎉 PASS  20240515-105306 20240515-110313 ubuntu-mate daily-live  test_install_entire_disk_with_defaults
🎉 PASS  20240515-111141 20240515-112135 ubuntu-mate daily-live  test_install_entire_disk_with_defaults