I am Using VideoToolbox VTCompressionSession For Encoding The Frame in H264 Format Which I will send through web socket to a browser. The Received Frames Will be Decoded and Output Will be rendered in the Website. Now, when using Some encoders the video is rendered always with four frame latency.
How Frame is sent to server :
start>------------ f1 ------------ f2 ------------ f3 ------------ f4 ------------- f5 ...
How rendering is happening :
start>-------------------------------------------------------------------------- f1 ------------ f2 ------------ f3 ------------ f4 ----------- ...
This Sometime becomes two frame latency and Sometime it becomes sixteen frame latency so the usability is getting affected.
Im using this configuration in videotoolbox's VTCompressionSession:
kVTCompressionPropertyKey_AverageBitRate=3MB kVTCompressionPropertyKey_ExpectedFrameRate=24 kVTCompressionPropertyKey_RealTime=true kVTCompressionPropertyKey_ProfileLevel=kVTProfileLevel_H264_High_AutoLevel kVTCompressionPropertyKey_AllowFrameReordering = false kVTCompressionPropertyKey_MaxKeyFrameInterval=1000 With Same Configuration i am able to achieve 1 in - 1 out with com.apple.videotoolbox.videoencoder.h264.gva. This Issue Is replication with Encoder com.apple.videotoolbox.videoencoder.ave.avc
Not Sure if its Encoder Specific. I have also seen that there are difference in VUI Parameters between encoded output of both encoders.
I want to know if there is something i could do to solve this issue from the Encoder Configuration or another API which is provided by the VideoToolBox to ensure that frames are decoded and rendered at the same time by Decoder.
Thanks in Advance....
