Skip to content

fix(SMatrixWriter): add fixed floating point format and set float precision

Maxime GUININ requested to merge fix-smatrixwriter-precision-to-write-float-values into dev

What does this MR do?

Set floating precision fixed to 7 digits.

How to test it?

  • Build ExStereoARCV
  • Load needed data (calibration + left and right views) with the given data set
  • Record matrices with the red Record button

Data

Some results

  • In order to know which value to give in std::setprecision, some tests was performed and results shows more stable results with a value set to 7.
  • Moreover, from Wikipedia:

The IEEE 754 standard specifies a binary32 as having:

  • Sign bit: 1 bit
  • Exponent width: 8 bits
  • Significand precision: 24 bits (23 explicitly stored)

This gives from 6 to 9 significant decimal digits precision.

Additional tests to run

  • Build on Windows
  • Build on Linux

Associated Issues/Merge Requests

Edited by Marc SCHWEITZER

Merge request reports