[mythtv-users] Mythfilldatabase scheduling

f-myth-users at media.mit.edu f-myth-users at media.mit.edu
Thu Nov 8 22:38:57 UTC 2007

    > Date: Thu, 8 Nov 2007 14:56:52 -0700
    > From: "Brian Phillips" <brian.phillips at gmx.net>

    > I've noticed that mine is trying to run during primetime as well.  This used
    > to not be the case as of maybe a week ago.  Mfdb would reschedule to run 24
    > hours after I ran it and always at 6:30-ish in the morning.  Now it's trying
    > to run at 5:30 PM tomorrow.

    > Is schedulesdirect responsible?  Maybe they're implementing some load
    > distribution and not considering that we don't want to update our db's
    > during primetime...  If they are, I suggest the ticket requesting they don't
    > "suggest" we run during primetime.

    > I'm running on a P3 550 MHz Slot 1.  Needless to say, every database access
    > is noticable, and a mfdb is insane.

Y'know, there -is- a potential way out of this whole problem, at the
cost of (a) some amount of additional code (alas) and (b) some amount
of additional potential for user confusion (alas^2).

Doing the actual -fetch- (e.g., -just- the wget-based network movement
of bits) can take place at any time without unduly loading even the
puniest of backends, since the data just isn't coming in very fast and
all the backend is doing is copying bits to disk.

It's doing the MFDB run (and subsequent rescheduling, too) that
imposes I/O load on the backend, and might lock tables at various
times, and both of those can corrupt recordings or freeze the UI
for obnoxiously long periods.

But there's no reason why MFDB must be run immediately after the
fetch; similarly, SD has no reason to care if the data is fetched but
not used immediately.  The only big concern is that the fetched data
not be allowed to age too far without being used; otherwise the user
might miss late updates and/or the fetch might have to be redone anyway.

So as long as we're honoring SD suggested download times (*), there's
no reason other than (a) & (b) above not to have the actual fetch and
MFDB runs be decoupled.  Do the fetch whenever the servers think the
load will be lowest and let it sit in /tmp.  Do the MFDB run whenever
no recordings are scheduled for the next n minutes (where n can be
slightly tighter now since we don't have to factor in the time to
fetch).  [And respect user settings about when -never- to run MFDB
anyway, e.g., 'cause it disrupts viewing---which could start halfway
through a run, so MFDB -must- just honor a specified user preference
without trying to outsmart whatever the user has said.]  If MFDB
discovers that the cached data is "too old" (whatever that means---
more than 6 hours?  24?) it can arrange to do an immediate fetch, and
then recheck whether it's safe to start running (and, if not, wait).
Yes, there are some screw cases here if you're recording 24x7, but you
get the idea.

I think the biggest issue is (b), because it makes it hard to debug;
you need a way to say "fetch and run with no delay" that won't paper
over other problems, etc etc.

(*) One of the privileges of -paying- for this data is that we no
longer need mince around on tiptoes hoping that TMS won't take their
ball and go home.  One of the implications of -that- is thus, if their
suggested download times are causing users problems, screw 'em.  The
money we're paying should be going to adequate server resources, and,
if it's insufficient, either (1) the charges should go up, or (2) the
grabbing software [which is implemented by the same people who are
running SD and hence collecting the money] should be improved so that
it's not causing users to have problems.  But since (I hope) it's only
a small minority of users who are having these problems in the first
place, ignoring suggested download times presumably is only a small
additional unscheduled/non-suggestable load on the servers, and is
thus presumably only a small or nonexistent problem. ---But Myth had
damned well better -ignore- those suggestions if users have configured
it to do so, and it sounds like it's currently not fully ignoring those
suggested download times when users have also tried to specify times
during which MFDB must not run.

More information about the mythtv-users mailing list