[mythtv-users] Duplicate matching methods - make program ID an explicit choice

[Michael T. Dean]

On 05/19/2016 09:50 AM, Michael T. Dean wrote:
> On 05/19/2016 06:35 AM, John Veness wrote:
>> On 19/05/2016 10:39, Ian Campbell wrote:
>>> On Wed, 2016-05-18 at 21:54 +0100, Neil Bird wrote:
>>>>     And, purely going by the wiki, it looks like if we change data 
>>>> source
>>>> (and get different programme IDs) we might have to delete the 
>>>> programme
>>>> IDs for all previously recorded episodes.  If both previous and 
>>>> upcoming
>>>> eps. have IDs, it *only* uses them to detect duplicates, and thus will
>>>> re-record everything it comes across.  If one of the other is missing,
>>>> it falls back to the usual mechanism (title/subtitle/etc.).
>>> Since I switched over to tv_grab_sd_json yesterday I have noticed a
>>> tidal wave of old repeats getting recorded ;-)
>>>
>>> Do you know what "delete the programme IDs for all previously recorded
>>> episodes" would involve? A (carefully crafted) SQL rune on the DB I
>>> suppose?
>> The above discussion was taking place on the xmltv-users mailing 
>> list, with subject line "[xmltv-users] UK Atlas load/usage warning 
>> email" although the quoted paragraphs actually refer to MythTV 
>> behaviour specifically. I'm posting it on mythtv-users with a changed 
>> subject line to get a wider audience.
>>
>> This is something that has annoyed me on a mild level for a few 
>> years, but which is going to become a bigger problem with an upcoming 
>> "forced" listings provider change for UK users, which is going to 
>> change program IDs:
>>
>> When making a recording rule in Myth, one can say that duplicates 
>> should be matched on subtitle, description, or subtitle+description. 
>> However, since several versions ago (or possibly, due to changes in 
>> the quality of upstream listings information), I realised that all of 
>> those settings are ignored in preference of the "program ID".
>>
>> I must admit that in most cases, this works OK, but there are some 
>> occasions when a duplicate is marked to be recorded, even though is 
>> as the same subtitle as one that has been previously recorded, 
>> because for some reason the program ID is different. Or worse, 
>> sometimes a new episode is for some reason thought to be a duplicate 
>> and so won't be recorded, presumably because the same program ID was 
>> erroneously supplied. This can especially happen when some listings 
>> information comes from tv_grab_uk_rt and some from EIT, for channels 
>> that aren't in the uk_rt stream.
>>
>> This is annoying, as there is nothing in the user interface to show 
>> that this is happening, and one can fiddle with the duplicate 
>> matching in the recording rules as much as one likes with no success. 
>> Instead you have to somehow spot this is going to happen and add 
>> override rules.
>>
>> I would very much like to see the program ID listed as an explicit 
>> option to pick in a recording rule, alongside subtitle, description, 
>> or subtitle+description. Only if program ID is selected (and that 
>> could be the default), would it match on those, otherwise it would 
>> match on the other choices. That way, we as users can decide whether 
>> we wish to trust the program IDs or to trust the episode subtitles 
>> more, which especially helps in times of transition like this.
>
> It should not be an option as the program ID is the definitive 
> identifier of a program.  If it's available, it should be used.
>
> What you really want is to expand the definition of program ID to 
> recognize its source dependence.  So, the most straightforward 
> approach would be to include some source identifier in the program ID, 
> something like <sourceid>-<programid>.  However, the source identifier 
> could not be anything like a MythTV Video Source sourceid because that 
> can change even when the actual source is the same (i.e. after a 
> Delete all Video Sources).  It could be the Video Source name, but 
> that would require that the user never change that name (and know its 
> additional importance). It almost makes sense for it to be the Video 
> Source xmltvgrabber (for XMLTV or Schedules Direct), except in some 
> areas the grabber name is changed frequently, though it ultimately 
> gets its data from the same upstream source.  Ideally, it would be 
> some value taken from the data provider itself, though I don't know my 
> XMLTV/SD/EIT data well enough to suggest any particular value to use 
> for a listings source ID that would be specific to a given program ID 
> provider.
>
> Any patch making this change would need to appropriately update 
> program, record,  recorded, oldrecorded, and recordedprogram data.

Oh, and then the matching should be done using program ID where both 
episode have a program ID and the program ID source is the same for 
both, else it falls back to the user-specified duplicate matching algorithm.

And, really, the duplicate matching itself would be easier with the 
listings provider ID (program ID source's ID) in a separate column, 
though the patch would be much more difficult (as it would require 
modifying program data in the protocol and such).

Mike