[mythtv] Jitter with time stretch

[Peter Bennett]

On 07/15/2018 04:50 PM, David Engel wrote:
> On Sat, Jul 14, 2018 at 08:52:30PM -0500, David Engel wrote:
>> On Sun, Jul 15, 2018 at 10:14:35AM +1000, Mark Spieth wrote:
>>> On 7/15/2018 7:27 AM, Peter Bennett wrote:
>>>> Hi David
>>>>
>>>> I do not use time stretch, so I do not know what is good and what is
>>>> bad. I tested time stretch with the current master version on Linux (no
>>>> patches), VAAPI and "-v playback" . With a 30fps recording it can be
>>>> speeded up to 2x with no log messages about aout of sync audio. With a
>>>> 60fps recording (720p) from Comcast, it shows hundreds of errors, a
>>>> bunch of messages saying "Video is 20.0819 frames behind audio (too
>>>> slow), dropping frame to catch up." followed by a bunch of messages
>>>> saying "Video is 17.7729 frames ahead of audio, doubling video frame
>>>> interval to slow down.". This repeats.
>> I don't go by the playback, log messages except to try to narrow down
>> the problem when there is one.  Rather, I use my eyeballs to see how
>> smooth the news ticker is on news channels.  As long as the decoder
>> can keep up, it should be generally, pretty smooth with just a little
>> bit of jitter as frames need to be dropped or extended.  At 2x, it
>> should be completely smooth as every other (60fps) frame gets dropped.
>>
>> The generally, pretty smooth is exactly what I see on my main frontend
>> (Linux with VDPAU).  That's also what I see on my development system
>> (Linux with VAAPI), albeit with a little more jitter.
>>
>>>> On the shield, this type of thing also happens, including with 30 fps
>>>> interlaced content, perhaps because it is actually shown at 60 fps.
>> On my Shields with current master and current Android patch. 720p
>> starts to get a little more jittery than expected or wanted at just
>> below 2x.  I believe it was good all the way to 2x before when there
>> were more video buffers.  I need to retest that.  1080i has always
>> been more jittery than expected or wanted at any speed greater than
>> 1x.  This is *the* issue keeping from making the Shield my primary
>> frontend on my main TV.
>>
>>>> On the shield if you set your audio device to NULL it seems to be
>>>> perfectly smooth when speeding up to 2x.
>> Did you try this with 1080i?  720p worked fine for me.  With 1080i,
>> however, playback doesn't actually speed up for speeds greater than
>> 1x.  It stays at 1x even though MythTV reports it being faster.  It
>> dows slow down for speeds less than 1x, though.  Strange.
>>
>> See below for more comments on the NULL audio test.(*)
>>
>>>> I suspect what you are seeing is a pre-existing bug, not related to the
>>>> shield specifically, but showing up because of the frame doubling that
>>>> is happening on the shield.
>>>>
>>>> I think the audio speedup may not be perfect, or maybe showing video at
>>>> higher rates than 60fps is problematic.
>> I disagree about the pre-existing bug part.  Once the processing gets
>> to the rendering stage, there is no real difference between rendering
>> video and audio that was originally 60fps from that which was
>> originally 30fps and then doubled to 60fps.
> I found it!  In MythPlayer::SetFrameInterval(), the frame_interval and
> avsync_predictor_enabled are getting set wrong because frame_period
> passed in is wrong.  It is still set the 30fps equivalent instead of
> the 60fps equivalent after the frame doubling in decoding.
>
> After forcing frame_interval and avsync_predictor_enabled to the
> correct values, I get smooth playback of my sample, 1080i content on
> my 1080p TV at speed upto 1.90x.  At 1.95x, a little bit of
> non-predicted adjustment is needed, but it's barely noticeable.  At
> 2.00x, it a little more noticeable, but really isn't too bad.
>
> On my 4k TV, the results aren't quite as good.  The video gets
> noticeably jittry at 1.55x.  It seems the scaling from 1080 to 4k
> impacts how fast the decoding and deinterlacing can go.  I wonder what
> effect using Surface rendering would have on that.
>
> Note that the above was all done with using the old numbers of buffers
> (vbuffers.Init(31, true, 1, 12, 4, 2)).  I'm going to see how far I
> can reduce them without degrading the timestretching performance.
>
> Finally, I don't have a proposed fix for the correct setting of
> frame_interval and avsync_predictor_enabled yet.  I'll defer to those
> of you more familiar with the MythPlayer setup for the time being.  It
> probably needs to be tied into the code that Peter added to detect the
> frame doubling during decoding.
>
> David
>
>
I agree - frame_interval needs fixing. We just need to be careful of not 
using the adjusted value when seeking, because for file access it uses 
the original frame number. For example in MythPlayer::UpdateFFRewSkip, I 
amnot quite sure what that is doing with it. I will go through and check 
all the places where frame_interval is used and see if I can get it right.

Peter
Peter