[mythtv] Some bugs (0.15)
Chris Pinkham
cpinkham at bc2va.org
Tue May 25 22:41:48 EDT 2004
> >>After a recording completes, the encoder sends an "INSERT" statement
> >>to the database for each keyframe (even though this recordedmarkup
> >>data is updated every 15 seconds during the recording). One hour is
> >>3600 inserts and on a remote slave, a three hour recording can take
> >>over a minute to complete the network traffic before it will begin the
> >>next back to back recording. Remote one hour back to back recordings
> >>usually have about a 17sec delay.
> >
> >
> > Hmm, I never noticed this before, but I see it in 0.14 as well, now that I
> > look. The process only takes about 2 seconds here for a one hour recording
> > (backend and DB on different hosts), though, so it's not surprising that I
> > didn't notice.
>
> I hadn't either until you and David were talking about speeding
> up the scheduler between back to back recordings. The reason the
> scheduler queries are slow is because the encoder is pounding the
> DB with recordedmarkup.
>
> I see about 2sec also for encoders on the database machine. It's
> the remote slaves that suck. cpinkhan knows about this and said
> he has an idea of how he wants to fix it but I imagine he hasn't
> had much time to spend on myth lately.
>
> -- bjm
I've had a little time lately but hadn't got to this yet. It's been
near the top of my TODO list though. I spent some time working on it tonight
and think I have a solution to this which will speed up the positionmap writing
for all the various 4 recorders. I don't have time to give it a good test so I
was thinking of waiting till after 0.15 to commit it with some other stuff I've
been working on. If someone else wants to give this a good runthrough or test
though, a patch is attached.
Basically, along with the positionMap QMap, I created a positionMapDelta to keep
track of the changes since the last positionmap write to the DB. So instead of
having to keep track of the last entry written and then later writing the whole map
out when the recording finishes, the code now just writes out the positionMapDelta
entries and clears the positionMapDelta QMap each time its written. When the
recording finishes, whatever is left in positionMapDelta is written out to make
sure we get the last few items.
Anyway, attached is a patch if anyone wants to give it a try. I'll try to give it
some more testing once I have more time later this week if no one else finds any
bugs in it. :)
--
Chris
-------------- next part --------------
Index: libs/libmythtv/dvbrecorder.cpp
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/dvbrecorder.cpp,v
retrieving revision 1.30
diff -u -r1.30 dvbrecorder.cpp
--- a/libs/libmythtv/dvbrecorder.cpp 30 Mar 2004 06:06:53 -0000 1.30
+++ b/libs/libmythtv/dvbrecorder.cpp 26 May 2004 02:33:11 -0000
@@ -170,7 +170,6 @@
channel_changed = true;
framesWritten = 0;
- prev_gop_save_pos = -1;
memset(prvpkt, 0, 3);
}
@@ -597,7 +596,8 @@
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(positionMap, MARK_GOP_BYFRAME, db_conn);
+ curRecording->SetPositionMapDelta(positionMapDelta, MARK_GOP_BYFRAME,
+ db_conn);
pthread_mutex_unlock(db_lock);
}
}
@@ -665,18 +665,18 @@
long long startpos = ringBuffer->GetFileWritePosition();
positionMap[framesWritten] = startpos;
+ positionMapDelta[framesWritten] = startpos;
if (curRecording && db_lock && db_conn &&
- ((positionMap.size() % 30) == 0))
+ ((positionMapDelta.size() % 30) == 0))
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(
- positionMap, MARK_GOP_BYFRAME,
- db_conn, prev_gop_save_pos,
- framesWritten);
+ curRecording->SetPositionMapDelta(
+ positionMapDelta, MARK_GOP_BYFRAME,
+ db_conn);
pthread_mutex_unlock(db_lock);
- prev_gop_save_pos = framesWritten + 1;
+ positionMapDelta.clear();
}
}
break;
@@ -707,6 +707,7 @@
framesWritten = 0;
positionMap.clear();
+ positionMapDelta.clear();
}
void DVBRecorder::Pause(bool clear)
Index: libs/libmythtv/hdtvrecorder.cpp
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/hdtvrecorder.cpp,v
retrieving revision 1.21
diff -u -r1.21 hdtvrecorder.cpp
--- a/libs/libmythtv/hdtvrecorder.cpp 10 Feb 2004 22:08:29 -0000 1.21
+++ b/libs/libmythtv/hdtvrecorder.cpp 26 May 2004 02:33:12 -0000
@@ -268,8 +268,6 @@
bool HDTVRecorder::SetupRecording(void)
{
- prev_gop_save_pos = -1;
-
return true;
}
@@ -279,7 +277,8 @@
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(positionMap, MARK_GOP_BYFRAME, db_conn);
+ curRecording->SetPositionMapDelta(positionMapDelta, MARK_GOP_BYFRAME,
+ db_conn);
pthread_mutex_unlock(db_lock);
}
}
@@ -399,18 +398,17 @@
ringBuffer->GetFileWritePosition();
positionMap[frameNum] = startpos;
+ positionMapDelta[frameNum] = startpos;
if (curRecording && db_lock && db_conn &&
- ((positionMap.size() % 30) == 0))
+ ((positionMapDelta.size() % 30) == 0))
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(
- positionMap, MARK_GOP_BYFRAME,
- db_conn, prev_gop_save_pos,
- frameNum);
+ curRecording->SetPositionMapDelta(
+ positionMapDelta, MARK_GOP_BYFRAME,
+ db_conn);
pthread_mutex_unlock(db_lock);
- prev_gop_save_pos = frameNum + 1;
}
}
@@ -825,9 +823,8 @@
lowest_video_pid = 0x1fff;
video_pid_packets = 0;
- prev_gop_save_pos = -1;
-
positionMap.clear();
+ positionMapDelta.clear();
if (chanfd > 0)
{
Index: libs/libmythtv/mpegrecorder.cpp
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/mpegrecorder.cpp,v
retrieving revision 1.33
diff -u -r1.33 mpegrecorder.cpp
--- a/libs/libmythtv/mpegrecorder.cpp 21 May 2004 05:30:58 -0000 1.33
+++ b/libs/libmythtv/mpegrecorder.cpp 26 May 2004 02:33:12 -0000
@@ -437,8 +437,6 @@
ic->build_index = 0;
- prev_gop_save_pos = -1;
-
return true;
}
@@ -448,7 +446,8 @@
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(positionMap, MARK_GOP_START, db_conn);
+ curRecording->SetPositionMapDelta(positionMapDelta, MARK_GOP_START,
+ db_conn);
pthread_mutex_unlock(db_lock);
}
}
@@ -536,16 +535,16 @@
long long keyCount = frameNum / keyframedist;
positionMap[keyCount] = startpos;
+ positionMapDelta[keyCount] = startpos;
if (curRecording && db_lock && db_conn &&
- ((positionMap.size() % 30) == 0))
+ ((positionMapDelta.size() % 30) == 0))
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(positionMap, MARK_GOP_START,
- db_conn, prev_gop_save_pos, keyCount);
+ curRecording->SetPositionMapDelta(positionMapDelta,
+ MARK_GOP_START, db_conn);
pthread_mutex_unlock(db_lock);
- prev_gop_save_pos = keyCount + 1;
}
}
}
@@ -578,6 +577,7 @@
framesWritten = 0;
positionMap.clear();
+ positionMapDelta.clear();
}
void MpegRecorder::Pause(bool clear)
Index: libs/libmythtv/NuppelVideoRecorder.cpp
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/NuppelVideoRecorder.cpp,v
retrieving revision 1.168
diff -u -r1.168 NuppelVideoRecorder.cpp
--- a/libs/libmythtv/NuppelVideoRecorder.cpp 10 Apr 2004 19:31:57 -0000 1.168
+++ b/libs/libmythtv/NuppelVideoRecorder.cpp 26 May 2004 02:33:13 -0000
@@ -148,7 +148,6 @@
usingv4l2 = false;
prev_bframe_save_pos = -1;
- prev_keyframe_save_pos = -1;
volume = 100;
}
@@ -834,6 +833,12 @@
StreamAllocate();
positionMap.clear();
+ positionMapDelta.clear();
+
+ pthread_mutex_lock(db_lock);
+ MythContext::KickDatabase(db_conn);
+ curRecording->ClearPositionMap(MARK_KEYFRAME, db_conn);
+ pthread_mutex_unlock(db_lock);
if (codec.lower() == "rtjpeg")
useavcodec = false;
@@ -1752,12 +1757,15 @@
ringBuffer->WriterSeek(0, SEEK_END);
- if (curRecording && positionMap.size() && db_lock && db_conn)
+ if (curRecording && positionMapDelta.size() && db_lock && db_conn)
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(positionMap, MARK_KEYFRAME, db_conn);
+ curRecording->SetPositionMapDelta(positionMapDelta, MARK_KEYFRAME,
+ db_conn);
pthread_mutex_unlock(db_lock);
+
+ positionMapDelta.clear();
}
delete [] seekbuf;
}
@@ -1808,19 +1816,19 @@
ste.file_offset = position;
ste.keyframe_number = frame_num;
positionMap[ste.keyframe_number] = position;
+ positionMapDelta[ste.keyframe_number] = position;
seektable->push_back(ste);
if (use_db && curRecording && db_lock && db_conn &&
- (positionMap.size() % 15) == 0)
+ (positionMapDelta.size() % 15) == 0)
{
pthread_mutex_lock(db_lock);
MythContext::KickDatabase(db_conn);
- curRecording->SetPositionMap(positionMap, MARK_KEYFRAME, db_conn,
- prev_keyframe_save_pos,
- (long long)ste.keyframe_number);
+ curRecording->SetPositionMapDelta(positionMapDelta, MARK_KEYFRAME,
+ db_conn);
pthread_mutex_unlock(db_lock);
- prev_keyframe_save_pos = ste.keyframe_number + 1;
+ positionMapDelta.clear();
}
}
@@ -1895,6 +1903,12 @@
seektable->clear();
positionMap.clear();
+ positionMapDelta.clear();
+
+ pthread_mutex_lock(db_lock);
+ MythContext::KickDatabase(db_conn);
+ curRecording->ClearPositionMap(MARK_KEYFRAME, db_conn);
+ pthread_mutex_unlock(db_lock);
}
void *NuppelVideoRecorder::WriteThread(void *param)
Index: libs/libmythtv/dvbrecorder.h
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/dvbrecorder.h,v
retrieving revision 1.20
diff -u -r1.20 dvbrecorder.h
--- a/libs/libmythtv/dvbrecorder.h 30 Mar 2004 06:06:53 -0000 1.20
+++ b/libs/libmythtv/dvbrecorder.h 26 May 2004 02:33:13 -0000
@@ -104,7 +104,7 @@
bool signal_monitor_quit;
QMap<long long, long long> positionMap;
- long long prev_gop_save_pos;
+ QMap<long long, long long> positionMapDelta;
bool dvb_on_demand;
bool isopen;
Index: libs/libmythtv/hdtvrecorder.h
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/hdtvrecorder.h,v
retrieving revision 1.16
diff -u -r1.16 hdtvrecorder.h
--- a/libs/libmythtv/hdtvrecorder.h 18 Jan 2004 18:39:25 -0000 1.16
+++ b/libs/libmythtv/hdtvrecorder.h 26 May 2004 02:33:13 -0000
@@ -80,8 +80,8 @@
int m_header_sync;
QMap<long long, long long> positionMap;
+ QMap<long long, long long> positionMapDelta;
- long long prev_gop_save_pos;
int firstgoppos;
int desired_program;
Index: libs/libmythtv/mpegrecorder.h
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/mpegrecorder.h,v
retrieving revision 1.11
diff -u -r1.11 mpegrecorder.h
--- a/libs/libmythtv/mpegrecorder.h 18 Jan 2004 18:39:28 -0000 1.11
+++ b/libs/libmythtv/mpegrecorder.h 26 May 2004 02:33:13 -0000
@@ -70,8 +70,8 @@
bool gopset;
QMap<long long, long long> positionMap;
+ QMap<long long, long long> positionMapDelta;
- long long prev_gop_save_pos;
static const int audRateL1[];
static const int audRateL2[];
static const char* streamType[];
Index: libs/libmythtv/NuppelVideoRecorder.h
===================================================================
RCS file: /var/lib/mythcvs/mythtv/libs/libmythtv/NuppelVideoRecorder.h,v
retrieving revision 1.73
diff -u -r1.73 NuppelVideoRecorder.h
--- a/libs/libmythtv/NuppelVideoRecorder.h 10 Apr 2004 19:31:57 -0000 1.73
+++ b/libs/libmythtv/NuppelVideoRecorder.h 26 May 2004 02:33:13 -0000
@@ -203,6 +203,7 @@
int keyframedist;
vector<struct seektable_entry> *seektable;
QMap<long long, long long> positionMap;
+ QMap<long long, long long> positionMapDelta;
long long extendeddataOffset;
@@ -261,7 +262,6 @@
int channelfd;
long long prev_bframe_save_pos;
- long long prev_keyframe_save_pos;
ChannelBase *channelObj;
More information about the mythtv-dev
mailing list