[mythtv] Closed captions that become "stuck"

[Jim Stichnoth]

On Tue, Oct 20, 2009 at 10:50 AM, Jim Stichnoth <stichnot at gmail.com> wrote:
> I posted about this on the mythtv-users list
> (http://www.gossamer-threads.com/lists/mythtv/users/402858) and got no
> response.  In short, I sometimes see closed caption text that becomes
> "stuck" -- it doesn't go away and no new captions are displayed.
> Jumping backward or forward may unstick the caption until the next
> problematic caption.
>
> I looked into the problem further and I think I found its source.  The
> problematic captions have ridiculously large timecodes, indicating
> that they shouldn't be displayed until far in the future.  The
> previous caption is then never erased.  As the decoder adds new
> captions to the 60-element queue, the queue eventually fills up
> because no captions are being consumed, and messages like this appear
> in the log:
>    2009-10-18 06:33:03.915 NVP::AddTextData(): Text buffer overflow
> Jumping backward or forward clears the queue so that captions with
> reasonable timestamps can once again be displayed.
>
> These recordings are OTA, ATSC, HDHR.  My latest problematic file was
> also losslessly transcoded to remove commercials.  I don't know where
> the bad timecode comes from, but there's a good chance it's from
> errors in the ATSC stream.  For what it's worth, logs with "-v vbi"
> show plenty of messages like this:
>    2009-10-20 06:11:01.654 XDS: failed CRC 76/227
>
> I have noticed that questions and problem reports about closed
> captioning often fall on deaf ears, so I was wondering if any myth
> developers are interested in this problem.  If not, I will just patch
> my local version with code that simply discards captions with
> timestamps more than 10 hours or so in the future.  If there is any
> interest, I can work on a somewhat better solution, as follows: If a
> caption is added to the queue whose timestamp is less than the
> timestamp of the next caption to be consumed, and higher than the
> timestamp of the most recently consumed caption, then assume the
> currently displayed caption is stuck and adjust the timestamp of the
> next caption to be consumed.  Or, I'm open to better ideas about how
> to detect and fix bad timestamps.
>
> $ mythfrontend --version
> Please include all output in bug reports.
> MythTV Version   : Unknown
> MythTV Branch    : trunk
> Network Protocol : 50
> Library API      : 0.22.20091013-4
> QT Version       : 4.5.2
> Options compiled in:
>  linux debug using_oss using_alsa using_backend using_dvb
> using_frontend using_hdhomerun using_hdpvr using_iptv using_ivtv
> using_joystick_menu using_lirc using_mheg using_opengl_video
> using_opengl_vsync using_qtwebkit using_v4l using_x11 using_xrandr
> using_xv using_bindings_perl using_bindings_python using_opengl
> using_vdpau using_ffmpeg_threads using_live using_mheg
>
> I don't know why MythTV Version shows as Unknown, but it's r22463.
>
> Here is the stack trace where the first huge timecode is detected.
> Most of the line numbers in NuppelVideoPlayer.cpp are off by about 10
> lines because of debugging/breakpoint code I added (see diff below).
>
> (gdb) where
> #0  JMS () at NuppelVideoPlayer.cpp:1495
> #1  0x00b73300 in NuppelVideoPlayer::AddTextData (this=0x88ef768,
>    buffer=0x88ef368 "\016", len=47, timecode=1511828489, type=67 'C')
>    at NuppelVideoPlayer.cpp:1519
> #2  0x008cbc75 in CC608Decoder::BufferCC (this=0x892ee58, mode=0, len=47,
>    clr=1) at cc608decoder.cpp:668
> #3  0x008cd254 in CC608Decoder::FormatCCField (this=0x892ee58, tc=1511828489,
>    field=0, data=12180) at cc608decoder.cpp:450
> #4  0x00bd2e1b in AvFormatDecoder::DecodeDTVCC (this=0x8930ce0,
>    buf=0xadd604d8 "J") at avformatdecoder.cpp:2445
> #5  0x00bebcf0 in AvFormatDecoder::GetFrame (this=0x8930ce0, onlyvideo=0)
>    at avformatdecoder.cpp:3960
> #6  0x00b6814a in NuppelVideoPlayer::GetFrameNormal (this=0x88ef768,
>    onlyvideo=0) at NuppelVideoPlayer.cpp:1555
> #7  0x00b68f36 in NuppelVideoPlayer::GetFrame (this=0x88ef768, onlyvideo=0,
>    unsafe=false) at NuppelVideoPlayer.cpp:1643
> #8  0x00b83061 in NuppelVideoPlayer::StartPlaying (this=0x88ef768,
>    openfile=false) at NuppelVideoPlayer.cpp:3863
> #9  0x00ba5395 in SpawnDecode (param=0x88ef768) at playercontext.cpp:26
> #10 0x021ff51f in start_thread () from /lib/libpthread.so.0
> #11 0x015ef04e in clone () from /lib/libc.so.6
>
> $ svn diff
> Index: libmythtv/NuppelVideoPlayer.cpp
> ===================================================================
> --- libmythtv/NuppelVideoPlayer.cpp     (revision 22463)
> +++ libmythtv/NuppelVideoPlayer.cpp     (working copy)
> @@ -1492,6 +1492,11 @@
>     videoOutput->DrawSlice(frame, x, y, w, h);
>  }
>
> +void JMS(void)
> +{
> +  printf("here\n");
> +}
> +
>  void NuppelVideoPlayer::AddTextData(unsigned char *buffer, int len,
>                                     long long timecode, char type)
>  {
> @@ -1509,6 +1514,11 @@
>     if (len > text_size)
>         len = text_size;
>
> +    if (timecode > 10000000)
> +      {
> +       JMS();
> +       VERBOSE(VB_IMPORTANT, "writing huge timecode");
> +      }
>     txtbuffers[wtxt].timecode = timecode;
>     txtbuffers[wtxt].type = type;
>     txtbuffers[wtxt].len = len;
>
>
> avformatdecoder.cpp, line 3892, AvFormatDecoder::GetFrame(int
> onlyvideo) is apparently setting lastccptsu to a bogus value.  This
> appears to be the source of the bad timecode.
>

Some more information on this problem.  It appears mythtranscode is
the culprit.  I have an HDHR ATSC recording whose caption timecodes
are just fine, but if you do a lossless mythtranscode, without a
cutlist (mythtranscode --mpeg2 --video), caption timecode errors are
introduced.  Interestingly, the bogus timecode always seems to be the
same value -- 1511828489 (or 0x5A1CAC09).  I guess this has something
to do with MPEG2 TS to PS conversion.

In the example I have, the first error shows up about 150MB into the
program, so it may be tricky to properly open a ticket.  I'll continue
looking into this myself, but in the meantime, any hints on likely
places in the code to look would be appreciated.

Jim