[mythtv-users] NO DATA between 9:30PM and midnight?

Bruce Markey bjm at lvcm.com
Mon Aug 15 22:01:22 UTC 2005


Mark J. Small wrote:
> On August 14, 2005 02:05 am, Anatoly Ivasyuk wrote:
> 
>>I have MythTV 0.18.1 from atrpms installed on FC3 according to Jarod's
>>guide, and I'm using Zap2It DataDirect for guide data.  Normally it gets
>>about 13 or 14 days of complete data, but starting a couple weeks ago
>>I've noticed that data on weekdays between 9:30PM and 12:00AM is showing
>>NO DATA except for shows that began before 9:30PM.
>>
>>Fortunately it updates the current recording day and one day ahead when
>>mythfilldatabase runs overnight, so the data fills in and the recording
>>schedule updates, but this problem does mean that I can't see part of
>>the evening schedule for much of the current and next week so I can't
>>schedule ahead.  Is anyone else having this problem, and is there any
>>way to fix it?
>>
>>Thanks,
>>-Anatoly
>>
> 
> 
> I've had this problem on and off for at least eight months.  The problem is in 
> the way mythtv decides whether the data is incomplete.  
> 
> On a normal update, mythtv will grab data for today, tomorrow, and any days 
> for which it does not have data, and any day which it deems to have 
> incomplete data.

s/today, //
 
> There have been several attempts at improving the way that myth tries to look 
> for incomplete data in the last eight months, but none have them have worked 
> perfectly for me.  

Actually. over the last couple years. Generally when someone gives
me tangible evidence of their actual listings behavior rather than
just their impression and anger. The only change I've made in recent
months is ultimately the only sure way to avoid this problem (and what
others hint at in this thread)...

> If I were to build my own myth instead of using prepared packages, here are 
> two things that I would do to work around this problem without causing too 
> much extra load on the zap2it servers.
> 
> (1) Fetch one less day of data.  It seems to me that when there is empty data, 
> it exists after midnight GMT.  It seems that myth is trying to fetch to the 
> very limits of the data that zap2it provides. If we backed off by a day, this 
> problem may not happen.
> 
> (2) On update, fetch data for today, tomorrow, ONE WEEK FROM NOW, and any 
> empty or incomplete days.  This would only add a little bit of extra load to 
> the zap2it servers.  Perhaps the powers that be would call it too much for 
> official packages, but thats what I would do.

Current SVN has a parameter for "--max-days <number>" so that you
can set it to not grab the problematic last day.

Here's the problem. TV stations report there planned schedule to
the listings clearing houses like GemStar (TV Guide) or Tribune
Media Service (zap2it). These services expect the stations to
report their data up to a certain day and time by some deadline.
The cutoff point most likely is not midnight but 6AM or 5PM or
something. I've only seen a cutoff for the last day of start times
before 5PM. This makes sense. Daytime schedules are fixed where they
may still be contemplating the primetime lineup. Someone says that
there data ends around 9PM. This makes no sense to anyone in the
pipeline. Either the station has decided on their primetime lineup
or they haven't.

In any case, be it 6AM, 4PM, 7PM, etc. there will be no data in
the late hours on the last day. It's obvious that this day should
be refilled tomorrow. The problem is that sometimes most stations
end with shows that start before 5PM Thursday, for example, but some
stations end by 5PM Wed(!). The next day, Thur is replaced but Wed
looks like it is okay.

[Go to your EPG and press End to skip forward by days until you
find everything "Unknown". Back up to find the last shows to see
what your cutoff looks like. If some stations have "Unknown" for
a day earlier than the rest, then you have a problem.]

The first thought would be to check that all stations have data
until midnight and refill all the data if any station does not
have data in the evening. This won't work.

- If the provider moved a station to another channel or dropped
a station, you would have a channel entry (until you removed it)
that wouldn't have data and would cause mfdb to refill all days
every time it ran.

- Some providers have two different stations (callsigns) on the
same channel for different times of the day (local independent
station during the day and CNNHN over night for example). The
one that isn't on in the evening would look like a partial day
and cause everything to refill every day.

- A user may have added a special channel for "VCR" on 3 or 4 that
never gets listings.

- XMLTV does not give endtimes for shows that end after midnight.
Mfdb fills these in when it has the next starttime after midnight.
Therefore it is hard to tell if the data goes to midnight until
after grabbing the next day and doing the post-processing.

- "Headline News" or "Government Access" may have four or six hour
chunks for their schedule and may not have a starttime in the evening.

Therefore, the current approach is to check there are enough start
times in the evening to appear as if there is complete data for the
day. However, this fails if some stations end a day before the others
so it looks like there is enough data but some channels are missing
data in the evening.

If anyone has an algorithm to handle all of these situations plus
other problems not mentioned here, diff -u, spaces, no tabs.

The surefire approach is to cut the snake off at the head (or at least
on the eleventh day ;-). As everyone on this thread seems to suggest,
just don't grab these ambiguous days and stick to the good data. The
first reaction may be 'Whaaa, I want the listings from the first 16
hours of Thursday the week after next'. I hope my phrasing does not
show a bias but this is not a life threatening situation. The scheduler
rarely, if ever, needs to find a later showing more than 7 days away
in order to resolve a conflict. The fact that you don't have a partial
day 12 or 13 days from now has no impact. In fact, in the olden days,
myth used to only grab 8 days of data. Having slightly less data will,
if anything, speed up searchs and scheduler runs. Grabbing one or even
two less days still leaves plenty of time to find new shows (see Search
Lists->New Titles or the "First Episodes" example in Custom Record).

I'd suggest using current SVN, "mysql> truncate table program;" then
start using "mythfilldatabase --max-days 11". If there are still some
channels missing data in the evenings of the last days, try "10". If
11 never fails try 12 in case that works.

--  bjm


More information about the mythtv-users mailing list