[mythtv] Test (crude) patch for commercial skipping

Chris Pinkham cpinkham at bc2va.org
Sun Feb 16 23:44:03 EST 2003


Attached is a crude patch to help skip commercials.  It uses a little of
Robert Kulagowski's code in NuppelVideoPlayer::FindCommercial() in
NuppelVideoPlayer.cpp and adds functionality.

When the user hits 'Z' while watching (delayed) LiveTV or a recorded
program, the commercial skip code is run. 

Basically it picks X (currently 200) random pixels from the current frame
and determines what is the variance for those X pixels.  If the variance
is less than 20, it assumes the frame is blank (no color detection
though).  Currently I have it hardcoded to assume the blank screen
transition between commericials and the show is at least 10 frames long.
As a precaution, the code will only skip forward up to 4 minutes at a time
looking for the break.  It also should come out of skip mode if it
encounters the end of the recorded video or it catches up to live TV.

Currently the code still has to decode each frame since it is based upon
detecting blank frames.  This means it's not an instantaneous jump to the
end of a commercial break after you hit 'Z', but instead it may take up to
10-15 seconds to scan through the frames till it finds the break.

I'm uploading this as something to stir discussion and for people to try
out.  I don't expect it to be integrated into MythTV just yet.  I wanted
to have a setting option to specify the minimum number of blank frames
that should mark the transition but didn't code that yet since this is
basically a proof of concept.

I've tried this code with a couple of prerecorded (U.S.) TV shows and it
seems to work pretty good.  I have seen a show though where the
commercial <-> show transitions were not separated by but 1-2 frames of
black so this code did not work for those.  The max frames skipped code
kicked in and returned to normal play after skipping 4 minutes meaning I
had to rewind about a minute to get back to the beginning of the show.

I haven't thought of a way to look for breaks in reverse since there isn't
a reverse version of GetFrame().

Can people take a look at this and let me know what they think or if it
works for them?  The patch is based upon CVS as of 11pm EST on 2/16/2003.

Possible enhancements include:
  - option in setup to set minimum number of blank frames to look for
  - option in setup to set maximum number of seconds to skip while looking
  - only look for black blank frames (ie, not totally white screen which
    could cause false triggers)
  - ability to scan backwards
  - auto-detection of beginning of commercials and auto-skip
  - show every X frames as video is skipped (currently video is paused)
  - better commercial detection (how do the VCRs do it???)

Chris

*****************************************************************************
** Chris Pinkham                  Linux v2.2.18, Sane v1.0.4, Cajun v3.0-8 **
** cpinkham at bc2va.org                          http://www.bc2va.org/chris/ **
*****************************************************************************
-------------- next part --------------
Index: libs/libmythtv/NuppelVideoPlayer.cpp
===================================================================
RCS file: /var/lib/cvs/MC/libs/libmythtv/NuppelVideoPlayer.cpp,v
retrieving revision 1.126
diff -u -r1.126 NuppelVideoPlayer.cpp
--- libs/libmythtv/NuppelVideoPlayer.cpp	16 Feb 2003 20:12:04 -0000	1.126
+++ libs/libmythtv/NuppelVideoPlayer.cpp	17 Feb 2003 04:21:09 -0000
@@ -1833,8 +1833,64 @@
         rewindtime = (int)(seconds * video_frame_rate);
 }
 
+void NuppelVideoPlayer::JumpToBlank( int mbf )
+{
+    if ((min_blank_frames == 0) && (mbf))
+    {
+        min_blank_frames = mbf;
+        blank_found = 0;
+        advancevideo = false;
+        disablevideo = true;
+    }
+}
+
+#define VARIANCE_PIXELS 200
+unsigned int NuppelVideoPlayer::GetFrameVariance( int vpos )
+{
+    unsigned int average_Y;
+    unsigned int pixel_Y[VARIANCE_PIXELS];
+    unsigned int variance, temp;
+    int i,x,y,top,bottom;
+
+    average_Y = variance = temp = 0;
+    i = x = y = top = bottom = 0;
+
+    top = (int)(video_height*.10);
+    bottom = (int)(video_height*.90);
+
+    // get the pixel values
+    for (i=0;i<VARIANCE_PIXELS;i++)
+    {
+        x = 10+(rand()%(video_width-10));  // Get away from the edges.
+        y = top+(rand()%(bottom-top));
+        pixel_Y[i] = vbuffer[vpos][y * video_width + x];
+    }
+
+    // get the average
+    for (i=0;i<VARIANCE_PIXELS;i++)
+    {
+        average_Y += pixel_Y[i];
+    }
+    average_Y /= VARIANCE_PIXELS;
+
+    // get the sum of the squared differences
+    for (i=0;i<VARIANCE_PIXELS;i++)
+    {
+        temp += (pixel_Y[i]-average_Y) * (pixel_Y[i]-average_Y);
+    }
+
+    // get the variance
+    variance = (unsigned int)(temp / VARIANCE_PIXELS);
+
+    return( variance );
+}
+
 void NuppelVideoPlayer::StartPlaying(void)
 {
+    int vpos;
+    int blank_found;
+    int frames_skipped = 0;
+
     killplayer = false;
     usepre = 3;
 
@@ -1866,6 +1922,7 @@
 
     weseeked = 0;
     rewindtime = fftime = 0;
+    min_blank_frames = blank_found = frames_skipped = 0;
 
     resetplaying = false;
     
@@ -1931,7 +1988,7 @@
 	    resetplaying = false;
 	    actuallyreset = true;
         }
-	    
+
         if (paused)
 	{ 
             actuallypaused = true;
@@ -2028,10 +2085,65 @@
             fftime = 0;
 	}
 
-        GetFrame(audiofd <= 0); // reads next compressed video frame from the
-                                // ringbuffer, decompresses it, and stores it
-                                // in our local buffer. Also reads and
-                                // decompresses any audio frames it encounters
+        if (!min_blank_frames)
+        {
+            // reads next compressed video frame from the
+            // ringbuffer, decompresses it, and stores it
+            // in our local buffer. Also reads and
+            // decompresses any audio frames it encounters
+            GetFrame(audiofd <= 0);
+        }
+        else
+        {
+            unsigned int variance = 0;
+
+            vpos = wpos;
+            GetFrame(1);
+
+            variance = GetFrameVariance( vpos );
+            if ( variance < 20 )
+            {
+                // blank frame
+                blank_found++;
+                if ( blank_found >= min_blank_frames )
+                    min_blank_frames = 0;
+            }
+            else
+            {
+                // non-blank frame
+                pthread_mutex_lock(&video_buflock);
+                wpos = vpos;
+                pthread_mutex_unlock(&video_buflock);
+                blank_found = 0;
+            }
+
+            if ((livetv) ||
+                (watchingrecording && nvr_enc && nvr_enc->IsValidRecorder()))
+            {
+                if ( nvr_enc->GetFramesWritten()
+                                  < (framesPlayed + min_blank_frames ))
+                    min_blank_frames = 0;
+            }
+            else if (((totalFrames) &&
+                     (totalFrames < (framesPlayed + min_blank_frames ))) ||
+                    ( frames_skipped > ( video_frame_rate * 60 * 4 )))
+            {
+                min_blank_frames = 0;
+            }
+
+            if (min_blank_frames)
+            {
+                frames_skipped++;
+            }
+            else
+            {
+                advancevideo = true;
+                disablevideo = false;
+                frames_skipped = 0;
+            }
+
+            continue;
+        }
 
         if (hasdeletetable && deleteIter.data() == 1 && 
             framesPlayed >= deleteIter.key())
Index: libs/libmythtv/NuppelVideoPlayer.h
===================================================================
RCS file: /var/lib/cvs/MC/libs/libmythtv/NuppelVideoPlayer.h,v
retrieving revision 1.62
diff -u -r1.62 NuppelVideoPlayer.h
--- libs/libmythtv/NuppelVideoPlayer.h	16 Feb 2003 20:12:04 -0000	1.62
+++ libs/libmythtv/NuppelVideoPlayer.h	17 Feb 2003 04:21:10 -0000
@@ -51,6 +51,7 @@
 
     void SetExactSeeks(bool exact) { exactseeks = exact; }
 
+    unsigned int GetFrameVariance( int vpos );
     void StartPlaying(void);
     void StopPlaying(void) { killplayer = true; }
     
@@ -71,6 +72,7 @@
  
     void FastForward(float seconds);
     void Rewind(float seconds);
+    void JumpToBlank(int mbf);
 
     void ResetPlaying(void) { resetplaying = true; actuallyreset = false; }
     bool ResetYet(void) { return actuallyreset; }
@@ -302,6 +304,7 @@
     long long lastKey;
     
     long long rewindtime, fftime;
+    int min_blank_frames, blank_found;
     RemoteEncoder *nvr_enc;
 
     bool resetplaying;
Index: libs/libmythtv/tv_play.cpp
===================================================================
RCS file: /var/lib/cvs/MC/libs/libmythtv/tv_play.cpp,v
retrieving revision 1.29
diff -u -r1.29 tv_play.cpp
--- libs/libmythtv/tv_play.cpp	5 Feb 2003 19:46:51 -0000	1.29
+++ libs/libmythtv/tv_play.cpp	17 Feb 2003 04:21:10 -0000
@@ -748,6 +748,13 @@
             DoFF(); 
             break;
         }
+        case 'z': case 'Z': 
+        {
+            doing_ff = false;
+            doing_rew = false;
+            DoJumpToBlank( 10 );
+            break;
+        }
         case wsLeft: case 'a': case 'A': 
         {
             doing_rew = true;
@@ -1152,6 +1159,22 @@
     }
 
     activenvp->Rewind(jumptime * 60);
+}
+
+void TV::DoJumpToBlank( int min_blank_frames )
+{
+    bool slidertype = false;
+    if (internalState == kState_WatchingLiveTV)
+        slidertype = true;
+
+    if (activenvp == nvp)
+    {
+        QString desc = "";
+        int pos = calcSliderPos((int)(fftime * ff_rew_scaling), desc);
+        osd->StartPause(pos, slidertype, "Commercial", desc, 2);
+    }
+
+    activenvp->JumpToBlank( min_blank_frames );
 }
 
 void TV::ToggleInputs(void)
Index: libs/libmythtv/tv_play.h
===================================================================
RCS file: /var/lib/cvs/MC/libs/libmythtv/tv_play.h,v
retrieving revision 1.16
diff -u -r1.16 tv_play.h
--- libs/libmythtv/tv_play.h	5 Feb 2003 19:46:51 -0000	1.16
+++ libs/libmythtv/tv_play.h	17 Feb 2003 04:21:10 -0000
@@ -80,6 +80,7 @@
     void DoRew(void);
     void DoJumpAhead(void);
     void DoJumpBack(void);
+    void DoJumpToBlank(int min_blank_frames);
     int  calcSliderPos(int offset, QString &desc);
     
     void UpdateOSD(void);


More information about the mythtv-dev mailing list