[mythtv-commits] [MythTV/mythtv] a6e112: Endless loop in HEVCParser::parseVPS

[kmdewaal]

  Branch: refs/heads/master
  Home:   https://github.com/MythTV/mythtv
  Commit: a6e1128e0f2e0ebf8040479e3b6393d940b14a7c
      https://github.com/MythTV/mythtv/commit/a6e1128e0f2e0ebf8040479e3b6393d940b14a7c
  Author: Klaas de Waal <klaas at kldo.nl>
  Date:   2022-08-17 (Wed, 17 Aug 2022)

  Changed paths:
    M mythtv/libs/libmythtv/mpeg/HEVCParser.cpp

  Log Message:
  -----------
  Endless loop in HEVCParser::parseVPS

In parseVPS the value of vps_num_layer_sets_minus1 should always be zero or positive.
The value is returned by get_ue_golomb() but this is an int where a negative value
is used to signal an error status. Copying a negative value into an unsigned int and
using that as a loop counter limit results in an almost endless loop.
The negative value is observed incidentally with 4K HEVC recordings after the
change from FFmpeg to BitReader in commit 7b2ac1eeb593c0da760cf637a07536f874dce19b.
However, also the FFmpeg version of get_ue_golomb() can return negative values
although this has never been observed.

This issue is now fixed by comparing int instead of unsigned int values.
Additionally, a warning message is given when the value returned by get_ue_golomb()
is negative when reading the vps_num_layer_sets_minus1 value.

This problem does happen only incidentally; it is possible that it is caused
by either invalid or unexpected HEVC stream content.
However, that is what is being broadcast and MythTV should be able to handle it.
It might also be possible that this issue is related to how close the reading
is to the head of the recording but this has not been further investigated.
Note that the get_ue_golomb() function is used in more places in HEVCParser.cpp
and in some of these places the code will fail horribly when a negative value
is returned. More fixes might be needed to make HEVCParser more resilient.

Refs #620