fix(test): coretest is unstable
Description
Coretest sometimes fails ON CI. It is very hard to reproduce, however, we managed to have a backtrace:
Thread 301 "coreTest.bin" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffd1ffb640 (LWP 2630887)]
std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node (this=this@entry=0x7ffff7fbe360 <sight::core::tools::fwID::m_dictionary[abi:cxx11]>, __bkt=12, __k="1b8698b5-f6a2-4f5d-aafe-32628a830dbd", __code=1905100017467635897) at /usr/include/c++/11/bits/hashtable.h:1819
1819 if (this->_M_equals(__k, __code, *__p))
#0 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node (this=this@entry=0x7ffff7fbe360 <sight::core::tools::fwID::m_dictionary[abi:cxx11]>, __bkt=12, __k="1b8698b5-f6a2-4f5d-aafe-32628a830dbd", __code=1905100017467635897) at /usr/include/c++/11/bits/hashtable.h:1819
#1 0x00007ffff7f35e9a in std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_node (__c=<optimized out>, __key="1b8698b5-f6a2-4f5d-aafe-32628a830dbd", __bkt=<optimized out>, this=0x7ffff7fbe360 <sight::core::tools::fwID::m_dictionary[abi:cxx11]>) at /usr/include/c++/11/bits/hashtable.h:793
#2 std::_Hashtable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find (this=this@entry=0x7ffff7fbe360 <sight::core::tools::fwID::m_dictionary[abi:cxx11]>, __k="1b8698b5-f6a2-4f5d-aafe-32628a830dbd") at /usr/include/c++/11/bits/hashtable.h:1574
#3 0x00007ffff7f331c9 in std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::weak_ptr<sight::core::tools::Object>, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::weak_ptr<sight::core::tools::Object> > > >::find (__x="", this=0x7ffff7fbe360 <sight::core::tools::fwID::m_dictionary[abi:cxx11]>) at /usr/include/c++/11/bits/unordered_map.h:869
#4 sight::core::tools::fwID::isIdFound (_id="1b8698b5-f6a2-4f5d-aafe-32628a830dbd") at /home/dweckmann/Work/src/sight/libs/core/core/tools/fwID.cpp:59
#5 0x00007ffff7f341db in sight::core::tools::fwID::addIDInDictionary (this=this@entry=0x7fffb8001618, newID="1b8698b5-f6a2-4f5d-aafe-32628a830dbd") at /home/dweckmann/Work/src/sight/libs/core/core/tools/fwID.cpp:82
#6 0x00007ffff7f34abf in sight::core::tools::fwID::setID (this=this@entry=0x7fffb8001618, newID="1b8698b5-f6a2-4f5d-aafe-32628a830dbd") at /home/dweckmann/Work/src/sight/libs/core/core/tools/fwID.cpp:75
#7 0x00005555556f061a in sight::core::tools::ut::FwIDTest::runFwIDCreation (this=<optimized out>) at /home/dweckmann/Work/src/sight/libs/core/core/test/api/tools/FwIDTest.cpp:119
#8 0x00005555556f2a46 in std::__invoke_impl<void, void (sight::core::tools::ut::FwIDTest::*&)(), sight::core::tools::ut::FwIDTest*&> (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#9 std::__invoke<void (sight::core::tools::ut::FwIDTest::*&)(), sight::core::tools::ut::FwIDTest*&> (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#10 std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) (__args=..., this=<optimized out>) at /usr/include/c++/11/functional:420
#11 std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()>::operator()<, void>() (this=<optimized out>) at /usr/include/c++/11/functional:503
#12 std::__invoke_impl<void, std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()>>(std::__invoke_other, std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()>&&) (__f=...) at /usr/include/c++/11/bits/invoke.h:61
#13 std::__invoke<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()>>(std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()>&&) (__fn=...) at /usr/include/c++/11/bits/invoke.h:96
#14 std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:253
#15 std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >::operator()() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:260
#16 std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::operator()() const (this=0x7fffd1ff9a20) at /usr/include/c++/11/future:1409
#17 std::__invoke_impl<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>&>(std::__invoke_other, std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>&) (__f=...) at /usr/include/c++/11/bits/invoke.h:61
#18 std::__invoke_r<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter>, std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>&>(std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>&) (__fn=...) at /usr/include/c++/11/bits/invoke.h:116
#19 std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void> >::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/11/bits/std_function.h:292
#20 0x00005555555ce213 in std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>::operator()() const (this=<optimized out>) at /usr/include/c++/11/bits/std_function.h:560
#21 std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*) (this=0x55555659d1d0, __f=<optimized out>, __did_set=0x7fffd1ff99f0) at /usr/include/c++/11/future:571
#22 0x00005555555ca18e in std::__invoke_impl<void, void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::__invoke_memfun_deref, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#23 std::__invoke<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#24 std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#1}::operator()() const (__closure=<optimized out>) at /usr/include/c++/11/mutex:776
#25 std::once_flag::_Prepare_execution::_Prepare_execution<std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#1}>(void (std::__future_base::_State_baseV2::*&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*))::{lambda()#1}::operator()() const (__closure=0x0) at /usr/include/c++/11/mutex:712
#26 std::once_flag::_Prepare_execution::_Prepare_execution<std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&)::{lambda()#1}>(void (std::__future_base::_State_baseV2::*&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*))::{lambda()#1}::_FUN() () at /usr/include/c++/11/mutex:712
#27 0x00007ffff7735d60 in __pthread_once_slow (once_control=0x55555659d1e8, init_routine=0x7ffff7a9b4a0 <__once_proxy>) at pthread_once.c:117
#28 0x00005555555d187f in __gthread_once (__func=<optimized out>, __once=<optimized out>) at /usr/include/x86_64-linux-gnu/c++/11/bits/gthr-default.h:700
#29 std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__once=..., __f=<optimized out>) at /usr/include/c++/11/mutex:783
#30 0x00005555556f2c95 in std::__future_base::_State_baseV2::_M_set_result(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) (__ignore_failure=false, __res=..., this=0x55555659d1d0) at /usr/include/c++/11/future:411
#31 std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::_M_run() (this=0x55555659d1d0) at /usr/include/c++/11/future:1737
#32 0x00005555556f20f6 in std::__invoke_impl<void, void (std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::*)(), std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>*>(std::__invoke_memfun_deref, void (std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::*&&)(), std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>*&&) (__t=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:74
#33 std::__invoke<void (std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::*)(), std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>*>(void (std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::*&&)(), std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>*&&) (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#34 std::thread::_Invoker<std::tuple<void (std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::*)(), std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:253
#35 std::thread::_Invoker<std::tuple<void (std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::*)(), std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>*> >::operator()() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:260
#36 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>::*)(), std::__future_base::_Async_state_impl<std::thread::_Invoker<std::tuple<std::_Bind<void (sight::core::tools::ut::FwIDTest::*(sight::core::tools::ut::FwIDTest*))()> > >, void>*> > >::_M_run() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:211
#37 0x00007ffff7a9c694 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#38 0x00007ffff7730947 in start_thread (arg=<optimized out>) at pthread_create.c:435
#39 0x00007ffff77c0a44 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100
Steps to reproduce
launch 12x:
ctest --repeat until-fail:99999 --output-on-failure --stop-on-failure -j 12 -R coretest
Functional specifications
The problem occurs because of a library version mismatch when running sight::core::runtime::ut::RuntimeTest
. More generally, it may occur anytime we load a library through sight::core::runtime::loadLibrary()
.
This is because of the algorithm that browses all library folders to find a library filename. It matches the full name of the library with the extension ".so.X.Z.Y". When two versions of the library are present in a library folder, the result is unpredictable and can lead to load two versions of a shared library.
Technical specifications
The problem can not occur in a production environment, because only one version of each library is present. It can only appear in development when working with a build tree when you switch between two versions of sight.
The problem can be resolved locally by removing the unwanted version of the library files. In addition, we also propose to fix the problem by taking, in priority, the library symlink with the ".so" extension. Indeed, it always corresponds to the latest built version of the library, which is what the developer wants normally.
Test plan
- coreTest should pass again.
- all tests should pass.
- try to launch an app.