[mythtv] [ivtv-devel] trunk driver

[f-myth-users at media.mit.edu]

    > Date: Tue, 23 Jan 2007 23:46:50 -0500
    > From: Chris Pinkham <cpinkham at bc2va.org>

    > * On Tue Jan 23, 2007 at 11:15:06PM -0500, f-myth-users at media.mit.edu wrote:
    > >> Date: Tue, 23 Jan 2007 22:45:03 -0500
    > >> From: Chris Pinkham <cpinkham at bc2va.org>
    > >
    > >> Is it the recordedmarkup/seek that is hanging or the update of the
    > >> filesize in the recorded table.  The scheduler query does a left join
    > >> on the recorded table and the filesize in the recorded table gets
    > >> updated whenever a recording ends.
    > > 
    > > I have no idea and am not sure how to investigate that.  I was going

    > If anyone who is having this problem wants to test out this theory,
    > they can test by searching for "curRecording->SetFilesize" in
    > libs/libmythtv/mpegrecorder.cpp and comment out the two lines where
    > that text occurs.

I'm currently running 0.18.1 from a package install, but I have 0.18.1
sources and might be willing to compile it & try this.  I'd like to
understand some of the implications first.

In particular, if I simply comment out those lines, will -something-
eventually set the file size?  Does anything really -care- about that
file size?  I'm worried that commflagging or playback might choke if
it never gets set, or might choke if they happen before recording
finishes (in case it gets set at the end even if we're not setting it
continuously during recording).

This has been my #1 problem with Myth for at least the last six
months, so I'm highly motivated to get this fixed.  I've probably
spent a whole work week trying to kluge around it over that interval
(trying all the cargo-cult solutions of increased ivtv buffering, DB
tweaks, multispindling [which was a huge PITA of repartitioning and
shuffling disks around], trying different kernel disk-scheduling
algorithms, alternate DB hosts, and I forget what else].

(If the fix is impossible in anything older than SVN, I'll jump
straight there from 0.18.1, but I'm hoping not to if I can avoid it,
since I have automation written for 0.18.1 that will also have to be
revised and I'm trying to push that out until I have more time to deal
with it.)

    > The filesize is updated ever time we update the seektable, which in
    > the ivtv/mpeg recorder is every 30 keyframes.  It would makse sense

How often do seekframes come along, anyway?  My memory was something
like 0.5 Hz or so on average, but I haven't been able to quickly
confirm that.

    > that we can't update the recorded.filesize field until the scheduler
    > query is finished since it is accessing the recorded table.  A quick
    > preliminary test on my dev box is confirming that as well.  On my
    > dev box, my scheduler takes about 19 seconds to run.  While the
    > scheduler was running, I ran the following SQL update in a mysql
    > session:

    > UPDATE recorded
    >    SET filesize = 0
    >    WHERE chanid = 1010
	  A> ND starttime = 20061213200000;

    > The update blocked until the scheduler query was finished.

    > Inside MpegRecorder::HandleKeyframe(), we call
    > MpegRecorder::SavePositionMap() which calls curRecording->SetFilesize
    > which does an "UPDATE recorded" exactly like I show above.  This
    > means that the main loop that is processing data in the recorder
    > get blocked trying to update the recorded table while the scheduler
    > is doing a select on that table.

    > This same logic would apply to us updating the recordedmarkup/seek
    > table while another process (ie, mythfrontend, mythcommflag, etc.)
    > is reading from that table although these reads take much less time
    > than the time it takes the scheduler to do that big query against
    > the recorded table.

    > I can see the recordedmarkup changes being a part of the problem,
    > but I think the bigger issue is the locking of the recorded table
    > by the scheduler which blocks the recorder's main processing loop
    > because it is trying to update the filesize in the recorded table.

I'm betting that you're right here; I made some other comments a few
minutes ago in other threads talking about "recordedmarkup" when I
meant to say "recorded", but the distinction was probably irrelevant
in those particular threads anyway.

    > > I'd be very interested to know which releases #1660 is in and whether
    > > it's really helped, especially given the OP's contention in 0.20 that
    > > he's got the same problem I've got.

    > The patch in #1660 is not in any releases, it isn't in SVN either.

Aha.

    > I think it is best to move the DB updates out to another thread, but
    > haven't looked the patch in #1660 over.

I'm a little concerned that -that- thread might exhaust some resource
if it blocks for a long time, but I guess we'll see.  If it would be
possible not to block on "recorded" while recording, that might avoid
having to use a separate thread in the first place, but I haven't
looked at that code in any detail, so this is just a guess, and worth
every penny.

In any event, thanks for chasing this down!  This looks like more
progress than the last six months of everything else combined.