[mythtv-users] Patches that improve mythcommflag accuracy

[faginbagin]

On 11/21/2014 9:40 AM, Karl Newman wrote:
> On Thu, Nov 20, 2014 at 10:37 PM, faginbagin <mythtv at hbuus.com
> <mailto:mythtv at hbuus.com>> wrote:
> 
> On 11/18/2014 1:24 PM, Carl Hunter wrote:
>> On 10/29/2014 6:48 AM, Carl Hunter wrote:
>>> Yes, I still get the numerous decoding errors not just the one
>>> at the end.  My HD recordings show one error at the end but
>>> still detect commercials fine.  I'll look into changing the
>>> recording settings some more.  Your point about different kernels
>>> has me questioning things too.  I have a test box running
>>> Mythbuntu 11.10 with Myth 0.24 on it.  I'll see if I can update
>>> the kernel on that one and compare results.
>>> 
>>> Yes, I'm using HVR-1600's.  Sorry for the typo.  When using the 
>>> analog side of the card mythcommflag fails the same way on those 
>>> recordings.  I also record using the S-Video and RCA inputs 
>>> depending on the different set-ups I've got, not the "cable"
>>> input. Not sure if that makes a difference with my recordings
>>> either.  I wonder if the part that "combines" the audio and video
>>> on these cards has changed with new kernel drivers.
>>> 
>>> Thanks
>>> 
>>> Carl
>>> 
>>> 
>> I've been using these patches ever since your PVR-150 typecast fix
>> was added to 0.27-fixes and have only seen positive results.  My
>> testing is only "watching" recordings.  HD recordings still seem to
>> be fine but SD recordings are much better.  Where I wasn't getting
>> any commercial flags even after running the rebuild on the
>> recordings I'm now getting the right number of flagged commercials.
>> Not all "marks" are perfect but they're definitely on par with Myth
>> 0.24 if not better.  So what's the next step to get this added to
>> the 0.27-fixes branch?  Have you seen any negative results?
>> 
>> Thanks again for all your work.
>> 
>> Carl
> 
> Hi Carl,
> 
> I'm glad my patches are working well for you. They are being reviewed
> by one of the devs. I believe his initial testing has been mostly
> positive, although he's got one recording where accuracy dropped by
> about half. I've shared a patch that helps that one recording, but
> doesn't help others. So ...
> 
> Anyone else have feedback on the patches?
> 
> Regards, Helen
> 
> 
> I think they are a definite improvement for my PVR-500 recordings. I
> don't think it's quite back to the accuracy it had in 0.24, but
> that's a high bar to clear.

0.24 was when lowres decoding was introduced, but it wasn't the default until 0.25. There's probably a tradeoff between the performance boost that lowres decoding provides and some loss of commflag accuracy. I haven't tried to measure accuracy if lowres decoding were disabled. It used to be controllable with a setting set in mythtv-setup (Enable experimental speedup), but that setting is no longer examined, and I believe that's been true since 0.25. FYI, lowres decoding only applies to MPEG-2 recordings (e.g. ATSC, QAM, and PVR-xxx), but not to others (e.g. H.264 made by HD-PVR tuners).

> Comm flagging struggles a bit with The
> Daily Show (even in 0.24 it had both false positives and negatives),
> but that may be a challenge anyway due to the format of the show. By
> the way, on a lot of channels the logo extends into the letterbox
> bars; I know one of your patches adjusts the border for the logo
> detection; I didn't quite understand the formulas you had for the
> different resolutions though. For my 480i recordings, I think it ends
> up being like 2 pixels (but maybe that applies to the low-resolution
> decode used for comm flagging, if I understand correctly).

Without the patches, there was a fixed border of 20 pixels (can be changed with a database or command-line setting) that limited the area examined both for logo detection and for other characteristics, like blank frame, scene change, letterbox and pillarbox. Consider what that means for a 528x480i SD recording. After lowres decoding, its size is 132x120 (132 = 528/4 and 120 = 480/4). If you exclude 20 pixels from the top, bottom, left and right edges, you are only examining the center area of 92x80 (92=132-20*2 and 80=120-20*2). (92x80) / (132x120) = 46.5%. In other words, you're examining less than half of a frame that's already 1/16th the original frame size. With the patches, I used that setting to compute a border based on the decoded frame's height. And for 480i content that ends up being 2 pixels.

I found that border ratio didn't work very well for letterboxed SD recordings. Instead I found that limiting logo detection to a fixed border of 16 pixels was a reasonable compromise for both HD and SD. That can also be changed with a database or command-line setting.

Commercial flagging is a tricky business. It's relatively easy to tune it for a specific recording, but quite often what you do for one recording, seriously screws up the accuracy for another recording. The trick is to tune the algorithms so they work reasonably well on a majority of recordings.

Regards,
Helen