[mythtv-commits] Ticket #13074: Incorrect EIT CRID IMI processing on split programme segments

Fri Jul 21 00:52:09 UTC 2017

#13074: Incorrect EIT CRID IMI processing on split programme segments
--------------------------------------+---------------------
     Reporter:  lucylangthorne55@…    |      Owner:  dekarl
         Type:  Bug Report - General  |     Status:  new
     Priority:  minor                 |  Milestone:  unknown
    Component:  MythTV - EIT          |    Version:  0.28.1
     Severity:  medium                |   Keywords:
Ticket locked:  0                     |
--------------------------------------+---------------------
 When a programme is split for news, mythtv incorrectly only records a
 random half of the programme since the other half is incorrectly
 marked as a duplicate due to incorrect IMI handling.

 When using EIT, programmes that have a crid that includes an IMI
 (Instance Metadata ID) can be used to identify programmes where a
 single piece of content has been split in to more than one part.

 An example document stating that the split should be considered one
 programme is in
 http://www.freeviewnz.tv/media/1055/freeview_dtt_transmission_rules_2_1.pdf
 which says in §7.1.7:

    "A CRID in the CID shall be a programme CRID (crid type 0x31) with an
 IMI extension. Where two events have the same CRID and IMI value and the
 gap between each event is less than 3 hours (measured from the end of the
 preceding event to the start of the next event) then they shall be
 considered to be segments of a single item of content.  An item of content
 may be split across more than two events as long as the gap between each
 event remains less than 3 hours."

 So using dvbsnoop I can see that my film has:
 {{{
 (Part 1 of film):
     Event_ID: 2364 (0x093c)
     Start_time: 0xe263190000 [= 2017-07-21 19:00:00 (UTC)]
     Duration: 0x0011000 [=  01:10:00 (UTC)]
     ...
             DVB-DescriptorTag: 118 (0x76)  [=
 content_identifier_descriptor]
             descriptor_length: 14 (0x0e)
             crid_type: 49 (0x31)  [= user defined]
             crid_location: 0 (0x00)  [= Carried explicitly within
 descriptor]
             crid_len: 12 (0x0c)
             crid_bytes:
                  0000:  2f 32 35 36 37 32 34 37  38 38 23 31
 /256724788#1
 ...
 (News)
     Event_ID: 2365 (0x093d)
     Start_time: 0xe263201000 [= 2017-07-21 20:10:00 (UTC)]
     Duration: 0x0000500 [=  00:05:00 (UTC)]
 ...
 (Part 2 of film):
     Event_ID: 2366 (0x093e)
     Start_time: 0xe263201500 [= 2017-07-21 20:15:00 (UTC)]
     Duration: 0x0010000 [=  01:00:00 (UTC)]
 ...
             DVB-DescriptorTag: 118 (0x76)  [=
 content_identifier_descriptor]
             descriptor_length: 14 (0x0e)
             crid_type: 49 (0x31)  [= user defined]
             crid_location: 0 (0x00)  [= Carried explicitly within
 descriptor]
             crid_len: 12 (0x0c)
             crid_bytes:
                  0000:  2f 32 35 36 37 32 34 37  38 38 23 31
 /256724788#1
 }}}

 Note that crid and IMI for both parts are identical.

 Sinced the crid is identical, mythtv thinks the two halves of the
 programme are in fact the same identical programme rather than two
 parts of the programme.

 Ideally the broadcaster and spec would have used {{{#}}}1 and {{{#}}}2,
 but that would be far too sensible.

 I have not yet seen a programme split in to three segments, even
 though that is valid according to the above spec, nor a case where
 there isn't exactly one programme in between the parts such as "part1,
 <news>, part2".

 Looking in my database, I noticed that mythtv currently strips the
 IMI.  That means a simple query can not locate such records.

 For full backwards compatibility, we probably can't store the IMI in
 the database programid since it would mean many existing programmes
 would no longer match the same crid. Besides, a programme may be split
 today but not split on a subsequent showing.

 Anyway, a query to show duplicates (since the split is always short so
 we can work around lack of IMI):

 {{{
 select distinct P1.title, P1.partnumber, P1.programid, P1.starttime,
 P1.endtime, P2.starttime, P2.endtime from program P1, program P2 where
 P1.programid <> '' and P1.programid=P2.programid and
 P1.chanid=P2.chanid and P1.starttime < P2.starttime and
 timestampdiff(MINUTE, P1.endtime, P2.starttime) < 15 and
 P1.title not in ('Teleshopping', 'Home Shopping', 'News') and
 P1.starttime > now();
 }}}

 Even though I use EIT, I thought perhaps manually setting partnumber
 to 1 and 2 and parttotal to 2 in the database would work, but it
 didn't work and I assume the scheduler ignores those fields since
 programid is identical and it is mythfrontend that is reconstructing
 the description to contain "(Part 1 of 2)". Plus I haven't looked at
 exactly when ProgramData would overwrite the data anyway but assume
 it only overwrites the whole data if broadcast EIT was marked as
 changed.

 So, I'm not sure exactly how the bug could easily be fixed since it
 wasn't immediately obvious to me how eitcache interacts with the
 program table.

 Since this splitting seems only to be used for movies, perhaps the
 full crid could be put in to seriesid and a fake (but consistent) crid
 (such as crid+start_hour_min) generated for programid so that at least
 "record all" rules would work, on the premise that it's better to
 record too much than to only have half and that certain types of movie
 are always on at the same times in the day. Unfortunately with all the
 +1 hour channels it would mean every film recorded twice, but still
 arguably better than only half a movie.

 I created this bug purely to get it documented since it has annoyed me
 for a few years and I finally got time to look in to it.

--
Ticket URL: <https://code.mythtv.org/trac/ticket/13074>
MythTV <http://www.mythtv.org>
MythTV Media Center