[mythtv] Jitter with time stretch

David Engel david at istwok.net
Sun Jul 15 01:52:30 UTC 2018

On Sun, Jul 15, 2018 at 10:14:35AM +1000, Mark Spieth wrote:
> On 7/15/2018 7:27 AM, Peter Bennett wrote:
> > Hi David
> > 
> > I do not use time stretch, so I do not know what is good and what is
> > bad. I tested time stretch with the current master version on Linux (no
> > patches), VAAPI and "-v playback" . With a 30fps recording it can be
> > speeded up to 2x with no log messages about aout of sync audio. With a
> > 60fps recording (720p) from Comcast, it shows hundreds of errors, a
> > bunch of messages saying "Video is 20.0819 frames behind audio (too
> > slow), dropping frame to catch up." followed by a bunch of messages
> > saying "Video is 17.7729 frames ahead of audio, doubling video frame
> > interval to slow down.". This repeats.

I don't go by the playback, log messages except to try to narrow down
the problem when there is one.  Rather, I use my eyeballs to see how
smooth the news ticker is on news channels.  As long as the decoder
can keep up, it should be generally, pretty smooth with just a little
bit of jitter as frames need to be dropped or extended.  At 2x, it
should be completely smooth as every other (60fps) frame gets dropped.

The generally, pretty smooth is exactly what I see on my main frontend
(Linux with VDPAU).  That's also what I see on my development system
(Linux with VAAPI), albeit with a little more jitter.

> > On the shield, this type of thing also happens, including with 30 fps
> > interlaced content, perhaps because it is actually shown at 60 fps.

On my Shields with current master and current Android patch. 720p
starts to get a little more jittery than expected or wanted at just
below 2x.  I believe it was good all the way to 2x before when there
were more video buffers.  I need to retest that.  1080i has always
been more jittery than expected or wanted at any speed greater than
1x.  This is *the* issue keeping from making the Shield my primary
frontend on my main TV.

> > On the shield if you set your audio device to NULL it seems to be
> > perfectly smooth when speeding up to 2x.

Did you try this with 1080i?  720p worked fine for me.  With 1080i,
however, playback doesn't actually speed up for speeds greater than
1x.  It stays at 1x even though MythTV reports it being faster.  It
dows slow down for speeds less than 1x, though.  Strange.

See below for more comments on the NULL audio test.(*)

> > I suspect what you are seeing is a pre-existing bug, not related to the
> > shield specifically, but showing up because of the frame doubling that
> > is happening on the shield.
> > 
> > I think the audio speedup may not be perfect, or maybe showing video at
> > higher rates than 60fps is problematic.

I disagree about the pre-existing bug part.  Once the processing gets
to the rendering stage, there is no real difference between rendering
video and audio that was originally 60fps from that which was
originally 30fps and then doubled to 60fps.

(*)I think the NULL audio test points the blame at the Shield's audio,
or more likely, our handling of audio on it.  Does the audio timing
need to be adjusted in any way to compensate for the video frame rate
being doubled.  Also, are we sure we are getting accureate timeing
information back from the audio subsystem on the Shield?

FWIW, I've often felt that something was a little off on the audio
timing.  Sometimes it seems like my short skips ahead and back are a
few seconds longer or shorter than they should be.  I realized that's
purely anecdotal, so take it with a huge grain of salt.

> I notice with timestretch it is sensitive to the number of audio buffers and
> video buffers to absorb any delta in A/V presentation in the transport
> stream.
> Increasing the audio buffering time helps in one direction (A precedes V)
> and video buffers where V precedes A.

Peter suggested increasing the audio buffering to 300ms or more a few
days ago.  I did see some improvement going to 300.  I didn't see much
improvement, if any, going as high as 2000ms.  I have it set to 1000ms

> As a test, adjust audio sync to the point where you no longer get the Video
> ahead of audio logs and that will give you an idea of how many buffers are
> required.
> Also with mediacodecs pipeline, this will exacerbate this V lagging audio
> issue.
> 20 frames is about 750 ms at 30fps. so up the audio extra buffer to 1 sec
> should fix this.

I hope to do some more testing on this tomorrow and Monday.

David Engel
david at istwok.net

More information about the mythtv-dev mailing list