(videoRealSense): align pointcloud on RGB frame for AR.

What does this MR do?

Align streams to the desired frame coordinate system (Depth, Infrared, Color).
This allow us to have each stream in the same coordinate system, no need to apply transforms in xml configurations.

New option is available trough ExRealSense: "Align frames to"; this allow user to choose target coordinate frame where all frames should be align to.

Add "visible" option in configuration of ::visuOgreAdaptor::SPointList

Closes #317 (closed)

How to test it?

1. Compile ExRealSense

2. Launch It !

3. Press play (forget about the camera selector, it will be bypassed)

OR If you don't have a realsense device:

3. Download data, select "File" in Camera Selector and open the .bag file.

4. You can display the PointCloud on AR on DEPTH and COLOR stream -> Open the Option menu and choose which one you want to display

NOTE: By default the poincloud should be well aligned on Depth stream, but not on Color one.

5. Choose Align to "Color" and the pointcloud should fit perfectly the Color Stream, but not the Depth one.

NOTE: The alignement process align each Streams in a coordinate frame, so that there is no need to apply extrinsic matrices (between each sensors). BUT If we want to do AR will still need to use intrinsic matrix of sensors to reproject the pointcloud on the stream. This is done by extracting intrinsic matrices from cameraSeries and use it in a SCamera adaptor like this:

<service uid="cameraAdaptorAR" type="::visuOgreAdaptor::SCamera">
    <inout key="transform" uid="identity" />
    <in key="calibration" uid="RGBCamera" /> <!-- Projection matrix of Color Sensor -->
     <config layer="AR" />
</service>

That's the reason why if you enable option "Infra to Color" the pointcloud seems misaligned; Infrared intrinsic should be use for this specific case (not implemented in ExRealSense).

Data

If no realsense device available, use [this one](data.

Some results

Pointcloud on Color frame un-aligned (by default)

Pointcloud on Color frame aligned

Additional tests to run

• windows
• macos
• linux

Associated Issues/Merge Requests

• Issues

  • Closes #317 (closed)

• Depends on

  • !195 (merged)
  • !196 (merged) (if merged remove commit 21894822 ✔ )