[mythtv-users] WARNING: SchedulesDirect users: server currently returning bogus results

[f-myth-users at media.mit.edu]

    > Date: Sun, 20 Dec 2009 09:18:38 +0000
    > From: Allan Stirling <dibblah.allan.stirling at googlemail.com>

    > Important things to note here:

    > 1. You're not actually using mythfilldatabase, but self-made 
    > wrappers around it - 

I -am- actually using mythfilldatabase:
  mythfilldatabase --no-delete --dd-file $id 1 -1 $listing_id $grabbed

But I'm not using the internal Myth way of calling it.  I call it with
the results of tv_grab_na_dd, after doing some work in the middle.

I am also not the only person who does this; I've seen others on the
list who call mfdb in custom ways for custom reasons, and no doubt
there are others who've not spoken up on the list about it.

In fact, this very style of a separate grabber call is -exactly-
what's been recommended when debugging an issue with the data, and
that's why my script does so, and why I did so by hand yesterday---
having the data on-hand, separate from Myth, has saved my bacon a
dozen times in diagnosing issues with my own rules, with Myth's
interpretation of the data, and with the data that TMS is generating.
(Fetching later is no substitute, because by then the data may have
changed.)

    > 			   So most "errors" mentioned here are 
    > meaningless to most users.

SD returned broken XML, for the first time I've ever seen since they
started up.  It may have been handled adequately by most Myth
installations, but I couldn't know that at the time, and the
consequences of it -not- being handled well were alarming---
potentially costing lots of people recordings.  The data itself
claimed to be in error, and their own sites claimed they were
having problems.

    > 2. mythfilldatabase by default for most users uses native 
    > (non-xmltv) DD code.

    > 3. You're using a very old version of Myth.

True, but my grabber is quite recent.  Furthermore, Robert Eden's
inspection of current Myth code appears to show that there are
circumstances in which Myth might flush old data, then choke on
erroneous new data, leaving you with nothing---so there's a potential
issue there if SD ever returns something broken but does -not- include
the "Fault" code the grabber scans for.  I haven't independently
verified this, so maybe he's wrong and Myth will always hang onto
its existing data until it has completely and correctly parsed the
new data, but I'll go with his opinion until I or someone else gets
around to studying it further.  (Or unless he says that I've
completely misunderstood him---always a possibility.)

    > Since there's been essentially no-one else complaining of 
    > this issue, I would guess that it's only your (very specific 
    > to you) implementation that is affected.

I'd guess that too---NOW.  Ain't hindsight wonderful?

Let me ask you something:  Would you rather that someone immediately
notify people of a potentially bad, time-sensitive problem that turns
out to affect only a few users, or would you rather that they say,
"Screw it, I'm not going to take the risk of someone harshing on me
in public.  They can lose a day or two of recordings because I'm not
going to risk it."  I wasn't going to wait around for an hour or a day
to fully diagnose the issue, given it had clearly been going on all
day, might continue for an unknown amount of time, and was clearly not
-only- my problem because SD had those big warnings up all over its
site.  And given that every hour I waited was an hour that might cause
people who didn't know about the problem to lose recordings, I figured
that an early warning was warranted.

Did you also notice that Robert Eden said, "Good idea to warn the list"?
I'll accept his judgement.

(This is especially apropos when you consider that maybe 99% of the
traffic on this list is "Please help me solve my specific configuration
problem" or "Please help me solve this totally off-topic problem that
has nothing to do with Myth except that I happen to also run Myth on
a computer in the same house."  At least I was trying to warn people
about a potential problem that might actually cost them recordings.)

Robert's responses have been very informative, polite, and we've
identified a potential small bug or two as well.  So even though
this may have been a false alarm, I'm happy to have participated.
I've learned something, my code's now more robust, and maybe others
have learned something as well.  (Thanks again, Robert!  Very much
appreciated!)

-And-, let me remind you that -Myth- is not only consumer of SD data.
It's quite possible that other, non-Myth applications might have
stumbled over this issue (or might stumble over a similar issue
in the future), and even though this traffic has been on a Myth
list, it's possible that someone searching (say, because they saw
similar responses from the SD server) will find it and become
enlightened.  We'll never know, of course, because they're unlikely
to send mail to a Myth list because of it.

I'm dropping this now; I think this topic has been beaten much further
into the ground than I had any intention of doing, I have other things
to do with my time, and I'm sure no one else cares.  Let's move on.