[mythtv] AVSync2 ideas

[Peter Bennett]

Hi Mark

Thank you for looking at this.
Some comments below

The 40ms was based partly on the wikipedia article 
https://en.wikipedia.org/wiki/Audio-to-video_synchronization
"The EBU Recommendation R37 “The relative timing of the sound and vision 
components of a television signal” states that end-to-end audio/video 
sync should be within +40ms and -60ms (audio before / after video, 
respectively) and that each stage should be within +5ms and -15ms."


On 2/8/19 7:59 AM, Mark Spieth wrote:
> Hi Peter
>
> Ive been analysing AVSYNC2 a bit more and have some ideas for some 
> symptoms Ive seen.
>
> Observations:
>
> * Playback data shows audio_adjustment creep slowly up to -40ms and 
> then drops back.
>
> * periodically (when the output rate is near the display rate, I 
> think) it does a catch up where every 2nd frame is dropped for a short 
> period.
>
Possibly some sort of resonance develops between frame rate and refresh 
rate. Let's try the other suggestions and see if this still happens.
> Ideas:
>
> * The point in time that GetAudioTime is called varies and is not 
> synchornous to the display update time. Avsync delta (and 
> GetAudioTime() should be measured just after wait_for_time() or Show() 
> (not for PBP displays) and saved for next pass, where it is acted upon 
> (FB control latency of 1 frame). This should minimize sync/control 
> signal jitter. Measure this jitter before doing anything else that may 
> affect the control signal.
>
Sounds like a good idea. I thought of that but reasoned that maybe the 
difference was not significant. I will give it a try. **
> * 40 ms adjust margin should be removed and use whatever the previous 
> frame's avsyncadjustment/delta was last frame. The adjustment should 
> have no abs minimum but an abs maximum so it doesnt go crazy on large 
> timestamp jumps. The delta should be maintained between 
> +-frameinterval by the control system. If it doesn't something is 
> wrong with the design and or feedback value (see above).
>
"audio_adjustment" is the actual amount the audio is out. If absolute 
value of that is less than 200 ms the fix_amount used is the amount 
specified in AVSyncIncrementMS settings, a value from 1-40 that defaults 
to 10. If abs("audio_adjustment") is more than 200 the fix_amount is the 
adjustment / 15.

By using a fixed small amount (AVSyncIncrementMS) for fix_amount I make 
it so that adjustments are made gradually. It also caters for situations 
where the audio time stamps from the audio card jump around (e.g. 
openmax audio).

With openmax digital audio, the timestamp received jumps around because 
inaccurate values are received from the driver for amount of audio 
cached. The result with original avsync and openmax digital audio was 
stop-start video and stop-start audio. I got around this with an 
"AVSyncAveraging" setting in original avsync.

I know of at least one user who uses non-openmax where the 
AVSyncAveraging solved a stop-start playback problem.

AVSync2 with AVSyncIncrementMS set to 10 does work well with openmax 
digital audio.

Following on your suggestion about no abs minimum, I will give it a try. 
if abs("audio_adjustment") is above 200 use the existing logic of 
dividing it by 15, otherwise remove the test for > 40ms and use the 
amount specified in AVSyncIncrementMS as the maximum but no minimum. 
Also I no longer need to restrict AVSyncIncrementMS to being less than 
40. [That restriction and the > 40 test were necessary if it was always 
using AVSyncIncrementMS as the fix amount]. **

> * Not sure whether pause audio/audo reset is needed unless things are 
> really far out. Seems to work niced with it disabled for my testing.
>
Audio reset helps for me. Without it, there is jerky speeded up video 
after some jumps. Pause will help where audio is a lot ahead, otherwise 
it could take long for the video to catch up.

Original avsync pauses audio every time audio is ahead, avsync2 is only 
pausing if it is more than 200 ms ahead.
> * interlaced fields could be shown at >1.0 if the due time (i.e. 
> frameinterval) is adjusted appropriately. (i.e. 2nd wait_for_time)
>
Are you referring to m_double_framerate? I don't know what you mean by 
"shown at >1.0". The calculation of the time for the second frame is a 
kludge - it is just using the frame_interval value which could be wrong. 
I suppose I could calculate a current frame interval, but I don't know 
if it would have any actual effect, since frames are synchronized to the 
display rate anyway.
> * is the clock used the correct one? The standard clock is tweeked by 
> NTP. Perhaps the monotonic system clock is better to use. This will 
> not jitter/go backwards.
>
I originally had code in that uses the monotonic system clock and got 
compile failures from some of the build bots, in particular MACOS. I 
changed to use a QT clock function that should work everywhere. If the 
clock goes off by more than 10 seconds it triggers the reset in AVSync2, 
so on a time change it should be OK.
> Hopefully these dont sound too dumb (after my last gaff).
>
Not at all
> If it works right, frame prediction is unnecessary. Will look into 
> this a bit more tomorrow (sat).
>
I will create an updated patch with the two suggestions I have marked ** 
above.

Regards
Peter