enh(viz): rework extrusion tool
Description
sight::module::filter::image::SImageExtruder
documentation states:
This service sets all voxels of an image that are inside meshes to the lowest value of the image type.
Sounds good, doesn't work.
This breaks the gradient (and the normals by extension) which we use for lighting the volume. Also, while this is quite okay for CT because range is signed 16-bit integer, this is very bad for ultrasound data which range is 8-bit unsigned (or 16-bit unsigned depending on format).
This also creates an asymmetry between the ROI tool and this tool. The ROI makes rays start later, but this tool simply modifies the image, i.e. sometimes creates interpolation troubles between two areas in the volume. This is notably visible at the boundaries of ultrasound volumes.
Gradient normals 1 | Gradient normals 2 | Lighting 1 | Lighting 2 |
---|---|---|---|
Note that [2]
(resp. [4]
) is worse than [1]
(resp. [3]
) because I slightly changed transfer function to highlight that phenomenon.
Proposal
Version A
- Explanation: I suggest that instead of computing a new image, we compute a binary mask (of the same size) instead. This should synergise well with #408 (closed), because that would mean everything in Sight Viewer could use the same image and thus share the same associated texture (good for memory usage).
- Advantages: Easy to compute (same algorithm we have, simply write something else to another image and update an associated texture passed to volume rendering shader)
-
Drawbacks: Requires high amount of additional VRAM: a
800 x 800 x 500
image will result in additional300
MB mask (assuming storing as R8).
Edit Flav: In the current setup, this is actually not really a drawback but rather an improvement. Since we keep the original image, we currently have twice the image in memory, potentially in R16. With this approach, we no longer need this copy and instead, we have a mask that is always in R8.
- Questions: what do we actually do in the shader? Do those samples participate in volume rendering or do we just make the ray skip those? (I personally think it makes sense to skip them).
It's perhaps possible to "pack" the relevant areas in a smaller texture, though I did not give any thought about how.
Functional specifications
- This tool does not break image gradient.
- If possible, make it so it is possible to use the same image in
SNegatoND
and in volume rendering.
Technical specifications
- Add an input cropMask to the volume renderer service
- Modify ImageExtruder to output a mask in 8 bits instead of a cropped image
- Modify sampleVolume() in
VolumeSampling.inc.glsl
to take into account the crop mask, with bilinear filtering. - For efficiency, also modify
VolumeBricksGrid_FP.glsl
to take into account the mask. Cropping the volume should speed up the rendering.
Test plan
- SightViewer
- Check performances. A slight decrease is expected since we have an extra texture.