Skip to content

feat(test): add a GUI test library and GUI tests

Lucas SCHMIDT requested to merge gui-qtest into dev

Description

These changes will add automated GUI tests as well as a GUI test library on top of QTest to help in their creation.

This library can wait and get graphic components, interact with them by emulating mouse and keyboard events and check if assertions are true. In order for the graphic components to be fetched more easily, components got objectNames, so they can be easily found using QObject::findChild.

Additionally, the tests will test the 3D rendering, by creating snapshots and comparing them with reference images using 6 different methods (pixel by pixel, mean square error, cosine of the matrices, histogram, Spearman's correlation and an algorithm I found on StackOverflow). When a test fail, it will take a screenshot of the screen, to help in troubleshooting, especially when it runs on an unmanned environment, such as the CI or XVFB.

Speaking of the CI, the generated images are available as artifacts, to check manually if the images look correct, either because you don't trust the machine, or because the test failed are you want to figure out why. Unfortunately, the GUI tests are pretty unstable on the CI, randomly crashing or timeouting, especially when the CI is overloaded.

Closes #784 (closed)

How to test it?

Run the GUI tests on your machine, both with and without XVFB, and check if it passes. You can run the tests by using either bin/exec_gui_tests.sh or ctest -R guitest. You shouldn't use bin/guitest, unless you want to run one specific test, because it will try to run all the GUI tests in one process, which is unsupported.

Merge request reports