[mythtv-users] Duplicate detection

[Ian Campbell]

On Tue, 2016-09-20 at 12:39 -0400, Michael T. Dean wrote:
> On 09/20/2016 10:23 AM, Ian Campbell wrote:
> > On Tue, 2016-09-20 at 10:19 -0400, Michael T. Dean wrote:
> >> On 09/20/2016 10:13 AM, Ian Campbell wrote:
> >>> On Tue, 2016-09-20 at 09:14 -0400, Michael T. Dean wrote:
> >>>> On 09/19/2016 04:24 AM, Jan Ceuleers wrote:
> >>>>> I thought I had solved this problem by including the following in my
> >>>>> daily listings cleanup script:
> >>>>>
> >>>>> # Force duplicate matching to rely on title, subtitle, description (or
> >>>>> whatever
> >>>>> # is set in the recording rule) by getting rid of program IDs which are not
> >>>>> # consistent between listings sources.
> >>>>> update program set programid="";
> >>>> BTW, if your program IDs are done correctly, this is not required (it's
> >>>> actually a bad thing).  If the program IDs specify an authority (i.e.
> >>>> they include a '/', making it an ID in the form "authority/ID"), the ID
> >>>> is only used for comparison against other programs with the same
> >>>> authority.  And, every listings source should use a different
> >>>> authority.  Please check to see if your IDs (when inserted into MythTV's
> >>>> database) contain an authority.  If not, I think they will if you have a
> >>>> default authority in dtv_multiplex (for EIT-using channels) and/or
> >>>> channel (for XMLTV-using channels). Just make sure the authority is
> >>>> different for each different source of program IDs (and same for each
> >>>> individual source of program IDs).
> >>> Does this mean that my programids of the form "SH[0-9]*" and "EP[0-9]*"
> >>> (which come from xmltv and are most of them) are wrong and should have
> >>> a prefix of some sort?
> >>>
> >> You don't need a program ID authority if all of your program IDs come
> >> from (and "will always come from") the same source (and that source will
> >> always use the same IDs for the same shows).  Since Jan has different
> >> sources for program IDs, and those different sources use different IDs
> >> for a given episode, authorities would be very good for the setup.
> > Ack, thanks.
> >
> > The reason I thought of it was because I recent changed which xmltv
> > grabber I was using, having "$grabber.xmltv.org" as the authority might
> > have been useful during the transition, but that's hopefully a once a
> > decade thing (I'd like to say never again, but don't want to tempt
> > fate!).
> 
> Yeah, this is the unfortunate part--the "will always come from" 
> parenthetical I mentioned, above.  Things change over time, even when 
> you don't expect they ever will.
> 
> We probably should always use program ID authorities for long-term 
> maintainability of user systems, but as currently implemented (embedded 
> within the program ID), they slow performance, so we don't.  To make 
> MythTV better for users over long periods of time, including changes to 
> listings source/program ID providers, we would need to rework the 
> program ID authority support to break out the authority into a separate 
> field to save the database's have to perform LEFT() function on each 
> program ID (meaning at least 2x per row).  However, even then, it may be 
> sufficient impact on performance that it's determined to be overkill 
> (and workarounds--such as scrubbing program IDs from recording history 
> (ideally with a nice bindings-based script) when changing providers--are 
> considered good enough).

Thanks for the explanation!

> Until then, there is no good answer to dealing with multiple or changing 
> program ID authorities/providers--only workarounds.  :(

I suppose pragmatically that switching incoming programids from
tv_grab_uk_atlas/* to tv_grab_sd_json/* on some flag day makes little
practical difference to the outcome compared with zapping the
programids on the same flag day, you'd mostly the same observable
behaviour WRT rerecording etc.

Ian.