Latency on MediaSDK
There are three parts which legally can introduce latency in MediaSDK decoder:
- Bitstream parsing: when app calls DecodeFrameAsync with some amount of bistream data, decoder doesn't know whether it's a full frame or not, so decoder starts real decode process only when app sends the beginning of a next frame. If you app receives data from some splitter and you're sure you have a one full frame, then you should set MFX_BITSTREAM_COMPLETE_FRAME flag in mfxBistream object.
- "AsyncDepth" latency - The AysncDepth parameter will impact the latency. To decrease this part latency, just to set AsyncDepth to 1.
- Reordering: According to AVC spec a decoder doesn't have to return a decoded surface immediately for displaying. It's true even in absence of B frames - decoder doesn't know in advance that later it won't meet B frames, reordering might present. So by default decoder returns the first decoded frame after DPB is full. It's possible to tune an encoder to produce a low latency bistream. Here are the rules:
- sps.pic_order_cnt_type = 2
- or if SPS.pic_order_cnt_type = 0 then
- Set VUI.max_dec_frame_buffering >= Number of reference frames used by encoder (e.g. 1)
or SEI.pic_timing.dpb_output_delay = 0