[mythtv-users] Duplicate detection

Michael T. Dean mtdean at thirdcontact.com
Tue Sep 20 16:39:38 UTC 2016


On 09/20/2016 10:23 AM, Ian Campbell wrote:
> On Tue, 2016-09-20 at 10:19 -0400, Michael T. Dean wrote:
>> On 09/20/2016 10:13 AM, Ian Campbell wrote:
>>> On Tue, 2016-09-20 at 09:14 -0400, Michael T. Dean wrote:
>>>> On 09/19/2016 04:24 AM, Jan Ceuleers wrote:
>>>>> I thought I had solved this problem by including the following in my
>>>>> daily listings cleanup script:
>>>>>
>>>>> # Force duplicate matching to rely on title, subtitle, description (or
>>>>> whatever
>>>>> # is set in the recording rule) by getting rid of program IDs which are not
>>>>> # consistent between listings sources.
>>>>> update program set programid="";
>>>> BTW, if your program IDs are done correctly, this is not required (it's
>>>> actually a bad thing).  If the program IDs specify an authority (i.e.
>>>> they include a '/', making it an ID in the form "authority/ID"), the ID
>>>> is only used for comparison against other programs with the same
>>>> authority.  And, every listings source should use a different
>>>> authority.  Please check to see if your IDs (when inserted into MythTV's
>>>> database) contain an authority.  If not, I think they will if you have a
>>>> default authority in dtv_multiplex (for EIT-using channels) and/or
>>>> channel (for XMLTV-using channels). Just make sure the authority is
>>>> different for each different source of program IDs (and same for each
>>>> individual source of program IDs).
>>> Does this mean that my programids of the form "SH[0-9]*" and "EP[0-9]*"
>>> (which come from xmltv and are most of them) are wrong and should have
>>> a prefix of some sort?
>>>
>> You don't need a program ID authority if all of your program IDs come
>> from (and "will always come from") the same source (and that source will
>> always use the same IDs for the same shows).  Since Jan has different
>> sources for program IDs, and those different sources use different IDs
>> for a given episode, authorities would be very good for the setup.
> Ack, thanks.
>
> The reason I thought of it was because I recent changed which xmltv
> grabber I was using, having "$grabber.xmltv.org" as the authority might
> have been useful during the transition, but that's hopefully a once a
> decade thing (I'd like to say never again, but don't want to tempt
> fate!).

Yeah, this is the unfortunate part--the "will always come from" 
parenthetical I mentioned, above.  Things change over time, even when 
you don't expect they ever will.

We probably should always use program ID authorities for long-term 
maintainability of user systems, but as currently implemented (embedded 
within the program ID), they slow performance, so we don't.  To make 
MythTV better for users over long periods of time, including changes to 
listings source/program ID providers, we would need to rework the 
program ID authority support to break out the authority into a separate 
field to save the database's have to perform LEFT() function on each 
program ID (meaning at least 2x per row).  However, even then, it may be 
sufficient impact on performance that it's determined to be overkill 
(and workarounds--such as scrubbing program IDs from recording history 
(ideally with a nice bindings-based script) when changing providers--are 
considered good enough).

Until then, there is no good answer to dealing with multiple or changing 
program ID authorities/providers--only workarounds.  :(

Mike


More information about the mythtv-users mailing list