(SHybridMarkerTracker): Low tracking accuracy
Summary
The tracking accuracy using the service sHybridMarkerTracker
isn't as good as in the original code given with the research paper on its github repository.
Steps to reproduce
The idea is to compare the tracking on the original github repository with SHybridMarkerTracker
.
here are the steps to try the service in an app (ExHybridMarkerTracker):
- Edit the
settings.xml
file inDev/Src/Bundles/tracker/hybridMarkerTracker/rc/
by editing your camera calibration found todo data Make sure you only edit<image_Width></image_Width>
,<image_Width></image_Width>
, the camera matrix and distortion matrix according to the calibration xml file. - In
Dev/Build/Debug
(or release) launchcmake . -DPROJECTS_TO_BUILD=ExHybridMarkerTracker
ninja
./bin/exhybridmarkertracker
- Select a video file todo add a link to public video and play
Dev environment
- OS: (Linux, Windows,
MacOS) - CMake version:
3.14.5
- Compiler:
clang 6
andVS2017
- Build type: debug and release
- Commit:
-
hybridMarkerTracker
branch onsight
-
What is the current bug behaviour?
With the original code, the tracking is precise but with the SHybridMarkerTracker
it's not, the tracking gets lost sometimes, the 2 predicted poses (blue and red rectangles) interchanges on some poses.
What is the expected correct behaviour?
We are supposed to get the same pose estimation as in the original code.
Relevant logs and/or screenshots
- A video of the tracking made with the original code
- A video of the tracking made with the service
SHybridMarkerTracker
On the following videos, the tracking is similar but there is a difference where on the video with sHybrid the red rectangle comes and the blue on disappears like on the picture below: Instead, since there is no ambiguity in that particular frame, the rectangle should be blue.
Fix: for this particular case,
I realized that the method drawRect
was called without a colour
parameter assigned therefore red came by default.
In fact, within sight we use RGB and when no colour assigned, red is the default and with OpenCv (BGR), blue is by default.
So I edited the code to add the colour blue where there is no ambiguity. The edit was made on this commit
Possible fixes
-
Check that the camera calibration accuracy is good-
The camera calibration isn't the issue here, cause with the same camera calibration we still get 2 different trackings with our service and with the original code.
-
Check possible differences on each step between themain
original code and our implementation inSHybridMarkerTracker
(especially thetracking method
).-
Check if there are some differences between the copy/pastedIPPE
in the original code and the one used in our Conan package-
I've removed the copy-pasted Ippe files in the original code and added our IPPE Conan Package and the tracking isn't changed. This means our Conan-IPPE package isn't the issue here. You can find the used code here
-
Check if there are some mistakes in thehybrid_marker_tracker
Conan package-
⚠ The issue might come from the different framerate used. In fact, in our application, unlike in the original example, the video is not processed frame by frame. The used grabber and synchronizer take time what leads to missing frames.
-
[ ] Change service input to receive::arData::FrameTL
and listen toobjectPushed
signal- No need to test this. It was just to check if treating all frames could resolve the difference but we need to drop some frames when the process is too long.
- optimize
process()
method:
- A high-resolution video slows down the process method. Maybe we should find a way to resize the videos before passing them to the method.
- An optimization was performed directly in the
hybrid_marker_track
lib