[mythtv] AVSync2 Improvements
Peter Bennett
pb.mythtv at gmail.com
Tue Feb 19 18:53:26 UTC 2019
The problems with AVSync that I tried to address with AVSync2 -
1. If the frame rate from ffmpeg is incorrect or the actual fra,e rate
rate changes, AVSync takes no heed of that, just uses the specified
frame rate. The only reason it actually gets it right in these cases is
because it doubles or drops frames to keep in sync with the audio. So if
a video is given as 30fps by ffmpeg but the time codes are actually 25
fps it will display as 30fps and then double some frames to keep up with
the audio. If there was a video without audio it would effectively
ignore video time stamps and play at the wrong rate.
2. When getting the video in sync with audio the only strategies it uses
are doubling or halving frame interval for selected frames.
3. If the audio device is unreliable at telling us how much audio is in
the buffer (e.g. OpenMax digital audio), avsync produces unacceptably
stuttery audio and video. This is addressed in avsync with a complex
arrangement of averaging the error over many frames if OpenMax audio is
in use.
Avsync2 addresses the problems in this way -
1. Uses individual frame timestamps to determine when to display frames,
rather than using the frame rate.
2. If audio and video are out of sync adjusts each frame by a small
amount (from settings - defaults to 10ms) until it gets in sync. This
solves problem 3 above without the complex averaging scheme.
3. Recent change - if audio and video are out by more than 200ms -
speeds up the correction so that it corrects over 15 frames at most.
On 2/17/19 2:03 PM, David Engel wrote:
> I compared some live TV cases (entering and changing channels) with
> and without avsync2. Without avsync2, there were cases where the
> audio and video would sync up fairly quickly. Most of the time,
> though, it would take a little while to sync up. In those cases it
> was comparable to avsync2. Avsync2 was pretty constant in that it
> always took dozens of secongs to sync up.
When starting Live TV there is a different problem from the normal
playback, that is the system being starved of audio and video data, and
stuttering while catching up. I don't know why that would be different
between avsync and avsync2, there is not much change in that area.
> I noticed a couple of other issues caused by avsync2 that I hadn't
> previously tied to it. I have one, 1080i, h.264 channel that I watch
> fairly regularly at >= 1.5x.
>
> 1. The avsync randomly goes out of phase for short periods (like Mark
> Spieth mentioned). I figured the CPU load on my Shield was on the
> edge and just couldn't quite keep at those times. The problem goes
> away completely after I turn off avsync2. I suspect the logs would
> show the same inabiltiy to correct that other logs have shown.
Comcast has no 1080i h264 channels. If you can give me more details, a
verbose playback log or sample video I can look into this. Does it
happen with Linux? Does it happen with mediacodec or software decode or
both?
> 2. I have my fast forward speeds set to 10x, 20x, 60x and 180x. 10x
> and 20x work normally. 60x and 180x have problems. A video frame is
> displayed and then there is a very long pause before another video
> frame is displayed. Basically those speeds are useless. Like above,
> the problem goes away completely after I turn off avsync2. I can
> provide logs for this if you want.
Is this only on shield? Where do you set these amounts? I seldom use
fast forward so I am not familiar with this.
> I've been using/testing avsync for long enough that I guess I'd gotten
> used to it's quirks. After seeing the old avsync behavior again, I'm
> inclined to go back. Skips and jump (which I do a lot of) feel like
> they are slower than with avsync2 but the lack of the other issues
> more than makes up for it.
General comment - If you are using mediacodec decoder with interlaced
content it automatically uses a frame doubling deinterlacer. In this
case it is not possible for MythTV to switch to a non-doubling
deinterlacer when using speedup, so it may work better to use software
decoding in that case, which will switch to a non-doubling deinterlacer
when speeding up.
Since I do not use speedup or fast forward, my testing in those areas
has been minimal. AVSync2 changes to improve smoothness are likely moot
when using speedup.
Peter
More information about the mythtv-dev
mailing list