(io): SFrameWriter block connection to write slot while writing
Summary
SFrameWriter blocks the connection to write
while writing.
This leads to drop frames when write
slot takes longer than the current fps.
Currently we have several formats for output frames:
- BMP (no compression)
- TIFF (LSW compression)
- JPEG
- PNG (LSW compression)
- JP2 (JPEG2000)
I've tried to record from a 1920x1080 30fps webcam device
Format | Output FPS |
---|---|
BMP | ~30fps |
TIFF | ~14fps |
JPEG | ~30fps |
PNG | ~10fps |
JP2 | ??? doesn't work |
.BMP and .jpeg seems to support a 30fps writing, but in the first case files are heavy (6.2Mo) and for the second files are compressed.
Steps to reproduce
Use ExVideoRecord
and try to record at >= 30fps with all formats.
Possible fixes
We could hope to be able to record in real-time for a certain combination of formats/resolutions/framerate, but actually we could never ensure notably because it depends on the hardware.
I think we should give the user two options:
- drop frame. This is the actual one, but we should be explicit about it, maybe even asking this in a pop-up window or in preferences.
- buffering. This could of course leads to memory saturation. So one proposal could be to have a limited buffer, that can be modified by the user in preferences. When the limit is reached, we should stop the recording and warn the user. Also if we quit the application before the recording is over, we should inform the user recording is still ongoing and wait before it is over. Last, at best, we could also show a progress bar to watch the buffer filling.
Links/references
Here an old discussion on the subject by @fbridault that may help:
> The blocker was introduced to avoid infinite recursive calls. And usually, we employ them when we have cycles in connections. Here I don't think this is the case, but it does something though.
>
> Indeed it prevents the slot from being called while you record the frame. Actually to be more correct, since we work with events loop, it prevents the slot task from being pushed in the worker event loop. So you will not push tasks like crazy even if you can't process them in time. That explains your result.
>
> I have to admit I never thought using blocker this way, but this may be a good idea. However we definitely have an I/O issue, and for this, I would suggest either to write in a different format or to queue the images internally and write them in a different thread.