Skip to content

(UTest): fix random failures

What does this MR do?

  • fwServicesTest: add missing wait. Also, one second might not be enough when the system is under heavy load, so the waiting time has been increased to 2500ms instead of 1000ms. Remember this is just the worst case and usually the function returns in a shorter time.
  • ioAtomsTest: this divides the test image size by 2 and reduces the number of reconstructions from 15 to 5 in the test model series. This should help a bit to reduce the execution time, see results below.
  • igtlProtocolTest: actually fix a lot of the mesh conversion code. It crashed randomly, but only because the UTest was very poor. If done properly, it should have shown how bad the code was, since apart from positions and cells copy, all the rest of the array copies were broken.
  • All tests:
    • when destroying a worker, we may ask it to join twice the thread if someone else is calling stop(). On top of that, hen stopping a worker, we decide whether we need to join the thread by testing the state of the io_service. However this does not seem to be a reliable way, so we now test the thread state itself,
    • when launching two unit-tests in parallel, they may stop at the same time. Both processes will try to remove temporary folders simultaneously, so we must handle failures properly with exceptions,
    • use temporary folders for writing instead of folders in the build tree,
    • last, we realized it is a bad practice to rely on stopping a worker with the auto-destruction (i.e. using shared pointers), because this could lead the std::thread to be destroyed from its own execution scope. So we deprecated the call of stop() in the WorkerAsio and now we advise people to call stop() from the callee thread (most often the main thread).

All of this solves a lot of random errors when lauching all unit-tests. We can also launch unit-tests in parallel now, making the CI jobs faster.

How to test it?

Run all unit-tests, watch the CI pipeline.

Some results

iAtomsTest

Before:

~/dev/sight/build/debug(:|✔) % time bin/ioAtomsTest  
....
OK (4)
bin/ioAtomsTest  29,20s user 1,06s system 103% cpu 29,314 total

After:

~/dev/sight/build/debug(:|✔) % time bin/ioAtomsTest
....
OK (4)
bin/ioAtomsTest  6,03s user 0,27s system 104% cpu 6,044 total

Additional tests to run

iAtomsTest

  • Ensure other unit-tests do not suffer from the change of the data set.
  • And most important, take a look at the CI logs to verify if the execution time has really decreased.

Associated Issues/Merge Requests

Edited by Flavien BRIDAULT

Merge request reports