[mythtv-users] Notification of conflicts

[f-myth-users at media.mit.edu]

    > Date: Sat, 19 Sep 2009 09:16:40 -0400
    > From: "Michael T. Dean" <mtdean at thirdcontact.com>

    > http://swatch.sourceforge.net/
    > http://sourceforge.net/projects/swatch/

    > http://kodu.neti.ee/~risto/sec/

Useful in general, but they and things like that were too much for
what I was trying to do.  Just reading their documentation carefully
is a more involved endeavor than the writing the script; the hardest
part of the script was making sure I knew exactly what log line I
wanted to notice in the backend logs (something that use of any
logging tool would require anyway), and that wasn't too hard...

    > >  if it's already in the file, I've already been notified.
    > > The file gets cleared before mfdb runs.  Thus, I only get notified
    > > about any given conflict once a day, instead of every time the
    > > scheduler runs---but since it's monitoring the backend log rather
    > > than doing an explicit printsched every so often, I get notified the
    > > instant the scheduler first creates the conflict. so I find out if it
    > > magically -creates- a conflict right in the middle of primetime when
    > > nothing's been edited and mfdb hasn't run [which it did once]

    > There's absolutely no magic in Myth's scheduler and the scheduler does
    > not create conflicts--people do.  Assuming you have no conflicts one
    > minute, the only way that a conflict could ever be created is if a) your
    > recording rules change (includes extending an ongoing recording or
    > saying not to record a show when you're in LiveTV and it asks for the
    > tuner), b) your listings change, c) some capture cards (i.e. in a remote
    > backend) drop offline.

*sigh*

Okay, look.  I didn't want to have to write yet another thousand-word
message to keep the nitpickers at bay, so I said "magically" instead
of going through all the rigor.

But since you apparently won't rest without it, here's the issue:

People have occasionally complained that Myth might sometimes be
suboptimal in how it allocates tuners, sometimes saying, "But why
doesn't the scheduler -know- that it can use this tuner over here
while it's using that tuner over there, instead of going for the
repeat later?"  The response has always been, of course, that Myth's
scheduler is not a TMS, nor does it employ arbitrary numbers of
passes to guarantee optimality, and that the current behavior is
instead a good heuristic that works most of the time but is not
guaranteed to be optimal at all times.  This means that every time
the scheduler actually reschedules after the situation changes (e.g.,
after a recording has finished, thus changing the exact situation),
it may pick a slightly different solution, and that solution may be
suboptimal.

You know this.

Therefore, what makes you think that scheduler -cannot- trip over its
own feet and -instigate- a conflict on its very own when it's running
after a showing ends?

In my particular case:
(a) I don't use EIT.
(b) I run mfdb from a cronjob, so I know -absolutely- when it runs.
(c) I run mfdb in the morning.
(d) The failure happened during primetime, 12 hours after the mfdb run
    and maybe 5-10 scheduler runs after the last mfdb.
(e) I am the only person with access to any frontend or Mythweb or the
    backend, e.g., the only person who could -possibly- make any
    changes was me.
(f) I made no changes.
(g) I -checked- that I made no changes, by reviewing my logs.
    I didn't touch the box for many hours before the mishap.
    I log everything but jobqueue from the backend, and I log
    all http interactions from Mythweb.
(h) I still had up the Scheduled Recordings page in my browser from
    the afternoon sometime (when no recordings were going on) that
    showed the correct behavior.  That page was computed quite a while
    -after- any possible scheduling change I'd made, e.g., I wasn't
    looking at old data while the scheduler was busy churning away on
    a change.  [In fact, while I don't remember right now, I suspect I
    hadn't made any changes since the previous day, if that.]
(i) I didn't make any scheduling changes after I'd gotten that SR
    page.  The logs were conclusive.
(j) LiveTV was not in use.  (It's never in use, unless I'm debugging a
    tuner or something.)  In fact, the frontend had not been in use
    at all that evening.
(k) When I noticed that Myth had failed to record one of the primetime
    shows, I clicked on that entry in Scheduled Recordings, putting it
    in a new window so I wouldn't lose the existing SR, and the
    program details said that the showing was not recorded due to a
    conflict---a conflict which was NOT called out in the SR page
    sitting in my browser from the afternoon.
(l) Which tuners were in use for other recordings was shuffled
    somewhat from the SR page, which was not surprising, since the
    scheduler had obviously changed its mind at some point.
(m) I reviewed my backend logs, which include "schedule".  I nailed
    exactly which scheduler run changed the intended recording from
    being on a tuner to "C".  Essentially, a scheduler run around 8pm
    at the end of one showing of something caused the 10pm showing of
    something else to suddenly be conflicting, and so when 10pm rolled
    around, the 10pm show was not recorded.  (There were several other
    recordings ongoing from maybe 7pm until probably past midnight
    that night; that was the only one that conflicted.)
(n) The backend was not rebooted that day, not did mythbackend restart
    itself.  (Of course, by your logic, doing so could never affect a
    conflict anyway and is hence irrelevant, but I figured I'd mention
    it anyway.)
(o) No tuners went offline or otherwise logged errors.  I don't
    remember at this late date (months later) whether there was
    actually a -free- tuner during the "conflicted" time, or whether
    Myth shuffled things around to use all tuners but continued to
    think it couldn't record the conflicted show, but it doesn't
    matter---a recording which had shown up as nonconflicting in SR
    hours earlier had, with zero user input nor changes to the
    listings, become conflicting.
(p) The configuration at that time was (and still is) a single MBE
    with no slaves.  Therefore, I did not suddenly lose and then
    reacquire a tuner due to a network glitch or something like that.
(q) I have never, in the entire history of my use of Myth, had a tuner
    drop out in a way that caused Myth to assume it couldn't be used.
    I've had tuners produce zero-length files (long ago, and rarely),
    and I've had all kinds of misbehavior about the signal that was
    actually -getting- to the tuner, but I've never seen Myth just
    spontaneously decide a tuner wasn't there---much less add it right
    back for the next show.
(r) I record in only SD, from PVR-250's.  Therefore, there is no funny
    business going on with Firewire, ATSC, or any of the other zillion
    ways in which a signal source can become invalid in a way that
    causes Myth to think it's not there.
(s) This was not even in the same month as a daylight-savings
    transition.  (That's a separate sore spot between me and Myth.)

Since I'm sure you will no doubt find some excuse for why this was
somehow user error and not the scheduler's fault, in what way is the
list above not exhaustive?  I will happily address it if you think
it matters.

I keep all my logs forever.  (Why?  In part because of the whole "Myth
doesn't deign to keep--in the database---information about which tuner
recorded something" issue that somebody brought up [again] a few weeks
ago, and in part because I want to be able to precisely characterize
problems even if they take weeks to become apparent.)  That means I
-could-, theoretically, go back and find the exact log that showed the
failure.  But that would require me to spend the time to figure out
exactly when it happened, which might take me some time, and the only
purpose would be to play the sorts of one-upmanship games which you
seem to enjoy but which I find incredibly distasteful.

So I'll just leave it at that.  Either you believe me or not.  You
know that the scheduler is not guaranteed to be perfect.  In several
years and many thousands of recordings, this is the only case in which
I can -prove- that it was this heuristic behavior that screwed me and
not some issue with me or with the listings.  But the probability is
-not- zero.  It may be low, but it's -not- zero.

And that's why I wrote the conflict notifier.  I was sufficiently
pissed off by the behavior that I wanted to ensure that if it ever
happened again, I'd have a chance of dealing with it.  Since I wrote
it, it's come in handy in saving me from having to manually check for
conflicts, which is a nice bonus and makes the (relatively small)
amount of work required to write it even more worthwhile.

And that's all.  Since other people were discussing the issue, it
occurred to me that I might chime in and offer to help save people
from a range of similar issues, if they wanted the code.