[mythtv-users] another scheduling strangeness/question

[Michael T. Dean]

  On 08/31/2010 02:10 PM, Brian J. Murrell wrote:
> On Tue, 2010-08-31 at 09:52 -0500, David Engel wrote:
>> Shouldn't the implicit "--refresh-tomorrow" eventually take care of
>> that?

Yeah.

> That seems to be the case.  These situations are typically days out,
> rather than occurring within the next 18h or so.
>
> But in the intervening 12 days, scheduling decisions are being made
> around recording this showing that will, in the 24h leading up to it end
> up changing.
>
> While I can't come up with a use-case where such a situation could end
> up in something not being recorded (i.e. a conflict) because the
> scheduler thinks only until the upcoming 24h that it was going to record
> something, it also doesn't seem like it would be a bad thing for missing
> data at the time of fetching on one channel to be filled in by data
> present for another channel with the exact same xmlid number.
>
> But even without multiple sources and multiple channels with the same
> xmlids, if it's typical for the data on day +13 to have (lots of?)
> "generic episode" data just because details have not yet been made
> available from TMS, but then fetching that same data the next day, when
> it's +12 fills in missing bits[1], would it be so bad to do a
> "--refresh-day-before-last" on each daily mythfilldatabase run?

I'm planning to make changes to mythfilldatabase so that it always 
retrieves all of the data (at minimum tomorrow through +13) for 
Schedules Direct users.  Talking with Robert Eden, it sounds like 
MythTV's usage of TMS data (getting tomorrow and +13 in 2 separate 
requests, plus getting any "substantially empty" days at +12 and/or +11 
...) is actually harder on the server than simply getting all the data 
in a single request.

Unfortunately, to make this happen, we need some changes to how 
mythfilldatabase works because mythfilldatabase is extremely 
resource-intensive, and there are a lot of users with underpowered boxes 
that wouldn't be able to handle a --dd-grab-all with the current code.

>> Brian, when was your last mythfilldatabase run and which Oprah is
>> scheduled for the following day?
> Yeah, per the above, typically the data within 24h is better.  Although
> at the moment it's not for tomorrow, but I am also 26h since my last
> mythfilldatabase.  I tend to expect the next fill should be shortly.
>
> b.
>
> [1] Even back when we were all getting data from TMS for free, I oftent
> thought (and think even suggested to TMS) some sort of "delta" protocol
> would be hugely useful for fetching this data.  If the data were added
> to the (schedules direct) database in a "log" format, when one wanted an
> update one could just say (to schedules direct) "give me all of the data
> changes since log entry $n" and that would include both changes/updates
> to programs/timeslots that you have already fetched as well as programs
> which are new (i.e. +13 days' new data).

Won't be necessary with the planned changes.  :)

That said, IMHO, the likelihood of such issues happening with current 
MythTV is small enough--especially when in season (versus the summer 
doldrums)--that it's not worth (ab)using TMS servers to retrieve extra 
days in extra runs.  And, in /very/ many situations, generics aren't 
actually changed to specific episodes until +7 or so.  So, randomly 
choosing a day to re-grab doesn't help (some will be fixed at +12 or +10 
or +7 or ...).  The only fix is either some kind of change notification 
or grabbing everything every time.  Since grabbing everything in a 
single request every time would actually be less strain on the servers 
than our current usage and it doesn't require any changes to TMS 
servers, that's the best solution--though it is a long-term project.  
(And, BTW, if when we start grabbing everything every time, users 
complain of having bad data on which schedules are based for 23 hours, 
then I'll be /very/ vocal about how they shouldn't be worrying.  :)

It all comes down to the question, "are you getting the shows you want 
recorded," and if the answer is, "No," then it likely means you need 
more capture devices--not better listings data***.

Mike

*** OK, so in Australia, they seem to have really bad data and do need 
better listings data, but that's a whole different story. :)  So, I 
suppose I should say, "You need more capture devices--not 
better-than-Schedules-Direct listings data."