Skip to content

(videoRealSense): add filter on depth frame

Emilie WERNERT requested to merge 308-videorealsense-add-filter-on-depth-frame into dev

What does this MR do?

Allow to filter the depth frame with the three filters provided by the RealSense SDK: spacial, temporal and hole filling. The filters can be enabled and configured with a SParameter.

The three filters are:

  • Spatial Edge-Preserving filter: it performs a series of 1D horizontal and vertical passes or iterations, to enhance the smoothness of the reconstructed data.
  • Temporal filter: it is intended to improve the depth data persistency by manipulating per-pixel values based on previous frames. The filter performs a single pass on the data, adjusting the depth values while also updating the tracking history. In cases where the pixel data is missing or invalid the filter uses a user-defined persistency mode to decide whether the missing value should be rectified with stored data. Note that due to its reliance on historic data the filter may introduce visible blurring/smearing artifacts, and therefore is best-suited for static scenes.
  • Holes Filling filter: the filter implements several methods to rectify missing data in the resulting image. The filter obtains the four immediate pixel "neighbors" (up, down ,left, right), and selects one of them according to a user-defined rule.

Update ExRealSense to add widgets to change the filters parameters.

Add a service configuration in ARCalibration to playthe RealSense infrared frame without the emitter.

Fix SPointCloudFromDepthMap to emit less signals. It emitted 'modified' signal on the mesh on each update which was causing the entire refresh of the VTK mesh (very slow). But only the vertex and the point colors are modified, so now, only 'vertexModified' and 'pointColorModied' signal are emitted.

How to test it?

You need a RealSense camera.

Test ARCalibration Lauch ARCalibration/Calibration activit, select a device and click to play the grabber. You should see a new grabber on the list: Infrared grabber without IR emitter. Select it and see that the infrared image is displayed without the pattern.

Grabber: Capture_du_2019-05-14_09-08-14

Frame without IR pattern: Capture_du_2019-05-14_09-08-40

Frame with IR pattern: Capture_du_2019-05-14_09-14-34

Test ExRealSense

Launch ExRealSense, and play with the filters.

Capture_du_2019-05-14_09-33-44

Test SPointCloudFromDepthMap service

Launch PocMergePointCloud or another application that uses this service and see that it still works and is faster.

Associated Issues/Merge Requests

Edited by Emilie WERNERT

Merge request reports