[mythtv] AVSync2 Improvements

Peter Bennett pb.mythtv at gmail.com
Tue Feb 19 18:53:26 UTC 2019


The problems with AVSync that I tried to address with AVSync2 -
1. If the frame rate from ffmpeg is incorrect or the actual fra,e rate 
rate changes, AVSync takes no heed of that, just uses the specified 
frame rate. The only reason it actually gets it right in these cases is 
because it doubles or drops frames to keep in sync with the audio. So if 
a video is given as 30fps by ffmpeg but the time codes are actually 25 
fps it will display as 30fps and then double some frames to keep up with 
the audio. If there was a video without audio it would effectively 
ignore video time stamps and play at the wrong rate.
2. When getting the video in sync with audio the only strategies it uses 
are doubling or halving frame interval for selected frames.
3. If the audio device is unreliable at telling us how much audio is in 
the buffer (e.g. OpenMax digital audio), avsync produces unacceptably 
stuttery audio and video. This is addressed in avsync with a complex 
arrangement of averaging the error over many frames if OpenMax audio is 
in use.

Avsync2 addresses the problems in this way -
1. Uses individual frame timestamps to determine when to display frames, 
rather than using the frame rate.
2. If audio and video are out of sync adjusts each frame by a small 
amount (from settings - defaults to 10ms) until it gets in sync. This 
solves problem 3 above without the complex averaging scheme.
3. Recent change - if audio and video are out by more than 200ms - 
speeds up the correction so that it corrects over 15 frames at most.

On 2/17/19 2:03 PM, David Engel wrote:
> I compared some live TV cases (entering and changing channels) with
> and without avsync2.  Without avsync2, there were cases where the
> audio and video would sync up fairly quickly.  Most of the time,
> though, it would take a little while to sync up.  In those cases it
> was comparable to avsync2.  Avsync2 was pretty constant in that it
> always took dozens of secongs to sync up.
When starting Live TV there is a different problem from the normal 
playback, that is the system being starved of audio and video data, and 
stuttering while catching up. I don't know why that would be different 
between avsync and avsync2, there is not much change in that area.
> I noticed a couple of other issues caused by avsync2 that I hadn't
> previously tied to it.  I have one, 1080i, h.264 channel that I watch
> fairly regularly at >= 1.5x.
>
> 1. The avsync randomly goes out of phase for short periods (like Mark
> Spieth mentioned).  I figured the CPU load on my Shield was on the
> edge and just couldn't quite keep at those times.  The problem goes
> away completely after I turn off avsync2.  I suspect the logs would
> show the same inabiltiy to correct that other logs have shown.
Comcast has no 1080i h264 channels. If you can give me more details, a 
verbose playback log or sample video I can look into this. Does it 
happen with Linux? Does it happen with mediacodec or software decode or 
both?
> 2. I have my fast forward speeds set to 10x, 20x, 60x and 180x.  10x
> and 20x work normally.  60x and 180x have problems.  A video frame is
> displayed and then there is a very long pause before another video
> frame is displayed.  Basically those speeds are useless.  Like above,
> the problem goes away completely after I turn off avsync2.  I can
> provide logs for this if you want.
Is this only on shield? Where do you set these amounts? I seldom use 
fast forward so I am not familiar with this.
> I've been using/testing avsync for long enough that I guess I'd gotten
> used to it's quirks.  After seeing the old avsync behavior again, I'm
> inclined to go back.  Skips and jump (which I do a lot of) feel like
> they are slower than with avsync2 but the lack of the other issues
> more than makes up for it.

General comment - If you are using mediacodec decoder with interlaced 
content it automatically uses a frame doubling deinterlacer. In this 
case it is not possible for MythTV to switch to a non-doubling 
deinterlacer when using speedup, so it may work better to use software 
decoding in that case, which will switch to a non-doubling deinterlacer 
when speeding up.

Since I do not use speedup or fast forward, my testing in those areas 
has been minimal. AVSync2 changes to improve smoothness are likely moot 
when using speedup.

Peter


More information about the mythtv-dev mailing list