[mythtv] PATCH: Commercial detection/skip using blank frames
Chris Pinkham
cpinkham at bc2va.org
Mon Feb 24 20:38:06 EST 2003
> > I'd rather do it realtime if it doesn't take too much cpu.
>
> But as we're seeing, I/O is the time culprit for realtime
> during playback. Optimizing the code only chips away at the
> final 1%. I did some skips in the Lakers game tonight and
> couldn't help notice the HDD LED on solid until the skip
> finished ;-).
I've got a patch ready that will speedup the skips during playback. It
does a shortcut on detecting a blank frame and also checks for
commercial lengths in a specific order. Since most (U.S.) commercials
are 30 seconds in length, it checks that first. Then it checks 15
then 60. It only scans 2 seconds worth of video for each of these.
This seemed to speed it up quite a bit on mine. The patch will also
turn on some status messages to show what it's doing. It turns on
the normal sliderbar that shows when you fast forward and updates it
as it finds commercials also telling how long the commercial was.
It also should skip the blank frames at the end of the commercial
break so it starts right at the show.
I'll attach the new patch to this message and hope that Isaac sees it.
> But if you meant realtime during recording, ya, that's clearly
> a win for software encoding. A tougher call for hardware MJPEG.
Yeah, realtime during encoding. Realtime flagging then post-processing
of the flags to determine which are commercials and which are parts of
the show.
> Right now, I think a practical solution might be to write
> out a file with a .blanks extension (or whatever *) while
> recording. A list of blank frame numbers would be under 100K
The blank frame list (in ascii format) for a 1/2 hour TV show that I
tested the other day was only 1881 bytes in size. So figure a 2 hour show
would only be 10K at the most probably.
> compared to 1GB or 2GB per hour video files. During playback,
> check for a .blanks file. If it doesn't exist, fine, just
> do the I/O intensive seeks that you already have working
This looks like it would be good. Then a post-processing function could
use that file to generate a cutlist later as well.
> * There is a convention of having other files with with
> the same *.nuv name and an extra extension. These should
> be deleted along with the video file in SpawnDelete .
I'll keep that in mind.
The patch should be attached to this message. Applies to cvs as of
8pm EST, 2/24/2003.
Chris
*****************************************************************************
** Chris Pinkham Linux v2.2.18, Sane v1.0.4, Cajun v3.0-8 **
** cpinkham at bc2va.org http://www.bc2va.org/chris/ **
*****************************************************************************
-------------- next part --------------
Index: libs/libmythtv/NuppelVideoPlayer.cpp
===================================================================
RCS file: /var/lib/cvs/MC/libs/libmythtv/NuppelVideoPlayer.cpp,v
retrieving revision 1.139
diff -u -r1.139 NuppelVideoPlayer.cpp
--- libs/libmythtv/NuppelVideoPlayer.cpp 24 Feb 2003 16:10:36 -0000 1.139
+++ libs/libmythtv/NuppelVideoPlayer.cpp 25 Feb 2003 01:08:20 -0000
@@ -3407,13 +3407,15 @@
*/
}
-#define VARIANCE_PIXELS 200
-unsigned int NuppelVideoPlayer::GetFrameVariance(int vposition)
+bool NuppelVideoPlayer::FrameIsBlank(int vposition)
{
+ int pixels_to_check = 200;
+ unsigned int max_brightness = 80;
+ int pixels_over_max = 0;
unsigned int average_Y;
- unsigned int pixel_Y[VARIANCE_PIXELS];
unsigned int variance, temp;
- int i,x,y,top,bottom;
+ unsigned int pixel_Y[pixels_to_check];
+ int i,x,y,top,bottom;
average_Y = variance = temp = 0;
i = x = y = top = bottom = 0;
@@ -3422,37 +3424,47 @@
bottom = (int)(video_height * .90);
// get the pixel values
- for (i = 0; i < VARIANCE_PIXELS; i++)
+ for (i = 0; i < pixels_to_check; i++)
{
x = 10 + (rand() % (video_width - 10)); // Get away from the edges.
y = top + (rand() % (bottom - top));
pixel_Y[i] = vbuffer[vposition][y * video_width + x];
- }
- // get the average
- for (i = 0; i < VARIANCE_PIXELS; i++)
+ if ( pixel_Y[i] > max_brightness )
+ {
+ pixels_over_max++;
+ if ( pixels_over_max > 3 )
+ return false;
+ }
+
average_Y += pixel_Y[i];
- average_Y /= VARIANCE_PIXELS;
+ }
+ average_Y /= pixels_to_check;
// get the sum of the squared differences
- for (i = 0; i < VARIANCE_PIXELS; i++)
+ for (i = 0; i < pixels_to_check; i++)
temp += (pixel_Y[i] - average_Y) * (pixel_Y[i] - average_Y);
// get the variance
- variance = (unsigned int)(temp / VARIANCE_PIXELS);
+ variance = (unsigned int)(temp / pixels_to_check);
- return variance;
+ if (variance <= 20)
+ return true;
+ else
+ return false;
}
void NuppelVideoPlayer::AutoCommercialSkip(void)
{
- PauseVideo();
-
if (autocommercialskip == COMMERCIAL_SKIP_BLANKS)
{
- int variance = GetFrameVariance(vpos);
- if (variance < 20)
+ if (LastFrameIsBlank())
{
+ PauseVideo();
+
+ while (!GetVideoPause())
+ usleep(50);
+
consecutive_blanks++;
if (consecutive_blanks >= 3)
{
@@ -3463,7 +3475,7 @@
int tries = 10;
GetFrame(1, true);
- while ((tries > 0) && (GetFrameVariance(vpos) >= 20))
+ while ((tries > 0) && ( ! LastFrameIsBlank()))
{
GetFrame(1, true);
tries--;
@@ -3490,9 +3502,8 @@
{
consecutive_blanks = 0;
}
+ UnpauseVideo();
}
-
- UnpauseVideo();
}
void NuppelVideoPlayer::SkipCommercialsByBlanks(void)
@@ -3503,6 +3514,8 @@
int first_blank_frame;
int saved_position;
int min_blank_frame_seq = 1;
+ int comm_lengths[] = { 30, 15, 60, 0 }; // commercial lengths
+ int comm_window = 2; // seconds to scan for break
// rewind 2 seconds in case user hit Skip right after a break
JumpToNetFrame((long long int)(-2 * video_frame_rate));
@@ -3515,7 +3528,7 @@
while (scanned_frames < (64 * video_frame_rate))
{
GetFrame(1, true);
- if (GetFrameVariance(vpos) < 20)
+ if (LastFrameIsBlank())
{
blanks_found++;
if (!first_blank_frame)
@@ -3531,6 +3544,7 @@
if (SkipTooCloseToEnd(min_blank_frame_seq - blanks_found))
{
JumpToFrame(saved_position);
+ osd->EndPause();
return;
}
@@ -3540,28 +3554,32 @@
if (!first_blank_frame)
{
JumpToFrame(saved_position);
+ osd->EndPause();
return;
}
// if we make it here, then a blank was found
int blank_seq_found = 0;
+ int commercials_found = 0;
do
{
- int jump_seconds = 14;
+ int *comm_length = comm_lengths;
blank_seq_found = 0;
saved_position = framesPlayed;
- while ((framesPlayed - saved_position) < (61 * video_frame_rate))
+
+ while( *comm_length )
{
- JumpToNetFrame((long long int)(jump_seconds * video_frame_rate));
- jump_seconds = 12;
+ long long int new_frame = saved_position
+ + (long long int)(( *comm_length - 1 ) * video_frame_rate);
+ JumpToFrame(new_frame);
scanned_frames = blanks_found = found_blank_seq = 0;
first_blank_frame = 0;
- while (scanned_frames < (3 * video_frame_rate))
+ while (scanned_frames < (comm_window * video_frame_rate))
{
GetFrame(1, true);
- if (GetFrameVariance(vpos) < 20)
+ if (LastFrameIsBlank())
{
blanks_found++;
if (!first_blank_frame)
@@ -3586,13 +3604,44 @@
if (blanks_found >= min_blank_frame_seq)
{
+ char comm_msg[128];
+ double spos = 0.0;
+
blank_seq_found = 1;
+
+ commercials_found++;
+ sprintf(comm_msg, "Found %d sec. commercial",
+ *comm_length );
+
+ if ((livetv) ||
+ (watchingrecording && nvr_enc &&
+ nvr_enc->IsValidRecorder()))
+ {
+ spos = 1000.0 * framesPlayed / nvr_enc->GetFramesWritten();
+ }
+ else if (totalFrames)
+ {
+ spos = 1000.0 * framesPlayed / totalFrames;
+ }
+
+ osd->StartPause((int)spos, false, "SKIP", comm_msg, 5);
break;
}
+
+ comm_length++;
}
} while (blank_seq_found);
+
+ osd->EndPause();
JumpToFrame(saved_position);
+
+ // skip all blank frames to we get right to the show
+ GetFrame(1,true);
+ while(LastFrameIsBlank())
+ GetFrame(1,true);
+
+ ClearAfterSeek();
}
bool NuppelVideoPlayer::DoSkipCommercials(void)
Index: libs/libmythtv/NuppelVideoPlayer.h
===================================================================
RCS file: /var/lib/cvs/MC/libs/libmythtv/NuppelVideoPlayer.h,v
retrieving revision 1.67
diff -u -r1.67 NuppelVideoPlayer.h
--- libs/libmythtv/NuppelVideoPlayer.h 24 Feb 2003 16:10:36 -0000 1.67
+++ libs/libmythtv/NuppelVideoPlayer.h 25 Feb 2003 01:08:21 -0000
@@ -172,7 +172,8 @@
void ClearAfterSeek(); // caller should not hold any locks
- unsigned int GetFrameVariance(int vposition);
+ bool FrameIsBlank(int vposition);
+ bool LastFrameIsBlank(void) { return FrameIsBlank(vpos); }
int SkipTooCloseToEnd(int frames);
void SkipCommercialsByBlanks(void);
bool DoSkipCommercials(void);
Index: libs/libmythtv/tv_play.cpp
===================================================================
RCS file: /var/lib/cvs/MC/libs/libmythtv/tv_play.cpp,v
retrieving revision 1.32
diff -u -r1.32 tv_play.cpp
--- libs/libmythtv/tv_play.cpp 23 Feb 2003 15:47:35 -0000 1.32
+++ libs/libmythtv/tv_play.cpp 25 Feb 2003 01:08:21 -0000
@@ -1181,9 +1181,10 @@
if (activenvp == nvp)
{
- QString desc = "";
- int pos = calcSliderPos((int)(fftime * ff_rew_scaling), desc);
- osd->StartPause(pos, slidertype, "Skip Commercial", desc, 2);
+ QString dummy = "";
+ QString desc = "Searching...";
+ int pos = calcSliderPos(0, dummy);
+ osd->StartPause(pos, slidertype, "SKIP", desc, 6);
}
activenvp->SkipCommercials();
More information about the mythtv-dev
mailing list