[mythtv-users] Scheduler seems to think I am recording too many programs at once
Stephen Worthington
stephen_agent at jsw.gen.nz
Sat May 4 12:26:36 UTC 2013
On Sat, 04 May 2013 12:03:19 +0100, you wrote:
>On 04/05/13 11:03, Stephen Worthington wrote:
>> On Fri, 03 May 2013 13:33:48 -0500, you wrote:
>>
>>> On Sat, May 04, 2013 at 03:50:30AM +1200, Stephen Worthington wrote:
>>>> On Fri, 03 May 2013 09:35:46 -0500, you wrote:
>>>>
>>>>> On Fri, May 03, 2013 at 02:19:52PM +1200, Stephen Worthington wrote:
>>>>>> On Thu, 02 May 2013 13:26:44 -0500, you wrote:
>>>>>>
>>>>>>> On Fri, May 03, 2013 at 03:01:42AM +1200, Stephen Worthington wrote:
>>>>>>>> On Thu, 02 May 2013 09:25:17 -0500, you wrote:
>>>>>>>>
>>>>>>>>> On Fri, May 03, 2013 at 12:04:03AM +1200, Stephen Worthington wrote:
>>>>>>>>>> On Thu, 02 May 2013 07:18:50 -0400, you wrote:
>>>>>>>>>>
>>>>>>>>>>> On 05/02/2013 12:56 AM, Stephen Worthington wrote:
>>>>>>>>>>>> I was just checking the scheduling for the new EPG received today, and
>>>>>>>>>>>> I found that one program "Seven Sharp" was not recording at its usual
>>>>>>>>>>>> time of 19:00 on TV ONE, but instead was recording at 20:00 from the
>>>>>>>>>>>> TVONE PLUS1 channel which retransmits TV ONE an hour later, but in SD
>>>>>>>>>>>> instead of HD. I have occasionally seen this sort of thing happen
>>>>>>>>>>>> before, always when I am recording lots of programs at the same time.
>>>>>>>>>>>> It has always seemed to me that the scheduler must have some limit for
>>>>>>>>>>>> the number of programs it will record at one time and when it goes
>>>>>>>>>>>> over that limit, it tries to reschedule one or more of them. If I add
>>>>>>>>>>>> an override to force "Seven Sharp" to record from TV ONE at 19:00, the
>>>>>>>>>>>> scheduler is happy to do that and does not seem to make any other
>>>>>>>>>>>> changes.
>>>>>>>>>>>>
>>>>>>>>>>>> Here is the schedule for that time period, without the override:
>>>>>>>>>>>> [Time] [Callsign]-[Program name]
>>>>>>>>>>>> 17:30:18:30 DISCO - Mythbusters
>>>>>>>>>>>> 18:00-19:03 TV ONE - One News At 6pm
>>>>>>>>>>>> 18:00-19:03 TV3 - 3 News
>>>>>>>>>>>> 18:30-19:03 ChoiceTV - Bath Crashers
>>>>>>>>>>>> 19:00-19:33 TV ONE - Seven Sharp (yellow, "Find Daily -3 Later
>>>>>>>>>>>> Showing")
>>>>>>>>>>>> 19:00-19:33 TV ONE-S - Seven Sharp (yellow, "Find Daily -7 Later
>>>>>>>>>>>> Showing")
>>>>>>>>>>>> 19:00-19:33 TV3 - Campbell Live
>>>>>>>>>>>> 19:30-19:33 TV2 - Police Ten 7
>>>>>>>>>>>> 19:30-20:30 HISCH - Mysteries At The Museum (yellow, "Channel Record
>>>>>>>>>>>> +0 Later Showing")
>>>>>>>>>>>> 19:30-20:30 KNOWLGE - Who Do You Think You Are? Aus
>>>>>>>>>>>> 19:30-20:33 PRIME - Great Rift: Africas Wild
>>>>>>>>>>>> 19:30-20:33 TV3 - Grand Designs Revisited
>>>>>>>>>>>> 20:00-20:30 CRIME& - Traffic Blues (yellow "Channel Record +0 Later
>>>>>>>>>>>> Showing")
>>>>>>>>>>>> 20:00-20:33 TV2 - RBT
>>>>>>>>>>>> 20:00-20:33 TVONE PLUS1 - Seven Sharp
>>>>>>>>>>>> 20:29-21:33 TV3 - Bones
>>>>>>>>>>>>
>>>>>>>>>>>> In the above, TV ONE, TV2 and TVONE PLUS1 are transmitted on the same
>>>>>>>>>>>> DVB-T multiplex, PRIME and ChoiceTV are on the same DVB-T mux, and TV3
>>>>>>>>>>>> is on the third DVB-T mux. TV ONE-S is an SD version of TV ONE on a
>>>>>>>>>>>> DVB-S mux. The other channels are all on Sky TV which is recorded
>>>>>>>>>>>> from a set top box via one S-Video card (so there is only one tuner
>>>>>>>>>>>> for those channels). The three DVB-T muxes are on one source, the
>>>>>>>>>>>> DVB-S muxes on another source, and Sky TV is on the third source. All
>>>>>>>>>>>> the DVB-T and DVB-S tuners are set up with 4 multirec virtual tuners
>>>>>>>>>>>> each, and I have 3 DVB-T tuners and 2 DVB-S ones. Since I have as
>>>>>>>>>>>> many DVB-T tuners as there are DVB-T muxes being transmitted, I should
>>>>>>>>>>>> never have to use the TVONE PLUS1 channel at all as I should always
>>>>>>>>>>>> have enough tuners to record everything I need on the first showing.
>>>>>>>>>>>>
>>>>>>>>>>>> So, does anyone know if there is some sort of limit in the scheduler
>>>>>>>>>>>> on the number of programs that will record at any one time? And if
>>>>>>>>>>>> so, what it is and how it works? Is there a setting I can adjust for
>>>>>>>>>>>> it?
>>>>>>>>>>>
>>>>>>>>>>> No limit. The only limit is the number of tuners (physical and virtual)
>>>>>>>>>>> you have. MythTV will gladly work your system so hard that the hardware
>>>>>>>>>>> fails to keep up and everything fails miserably if you tell it to.
>>>>>>>>>>>
>>>>>>>>>>>> I have 5 recording drives, so whatever is causing this sort of
>>>>>>>>>>>> rescheduling is not taking that into account, as I can easily record
>>>>>>>>>>>> at least 10 programs simultaneously, and probably more, as long as the
>>>>>>>>>>>> recordings use all the drives (and they normally do).
>>>>>>>>>>>
>>>>>>>>>>> It is almost definitely a priority modifier on the other showing causing
>>>>>>>>>>> it to be preferred. This could be channel or input or HDTV or any of a
>>>>>>>>>>> number of other priority modifiers. You can see what's happening by
>>>>>>>>>>> running:
>>>>>>>>>>>
>>>>>>>>>>> mythbackend -v schedule --loglevel debug --printsched
>>>>>>>>>>>
>>>>>>>>>>> before the shows air and without the override in place.
>>>>>>>>>>>
>>>>>>>>>>>> Also, is there a way to mark the TVONE PLUS1 channel as SD and the TV
>>>>>>>>>>>> ONE channel as HD and tell the scheduler to always record the HD
>>>>>>>>>>>> programming unless there is an actual clash?
>>>>>>>>>>>
>>>>>>>>>>> No, there's no such thing as an "HD" channel--only channels that have
>>>>>>>>>>> programs which may or may not be HDTV. So, it's up to your guide data
>>>>>>>>>>> to properly mark programs as HDTV or not.
>>>>>>>>>>>
>>>>>>>>>>> You can set priority modifiers on channels (and starting with 0.27,
>>>>>>>>>>> they'll work the way you think they would work), but then again,
>>>>>>>>>>> priority modifiers you've already set are probably what's causing MythTV
>>>>>>>>>>> to record shows in such a way that you think the scheduler is wrong... ;)
>>>>>>>>>>>
>>>>>>>>>>> Mike
>>>>>>>>>>
>>>>>>>>>> The --printsched option looks like it is really useful. I have put
>>>>>>>>>> the result on my web server here:
>>>>>>>>>>
>>>>>>>>>> http://www.jsw.gen.nz/mythtv/sched.txt
>>>>>>>>>>
>>>>>>>>>> If I paste it into a post, its gets word wrapped and unreadable. For
>>>>>>>>>> readability I also cut down the output to just the bit between times
>>>>>>>>>> where there is nothing recording, so in theory nothing before or after
>>>>>>>>>> the bit posted would affect the scheduling at the problem time.
>>>>>>>>>>
>>>>>>>>>> If, as I am assuming, the P=Priority column is the fully calculated
>>>>>>>>>> priority value for the scheduled recording, the I do not understand
>>>>>>>>>> what is going on. The "Seven Sharp" scheduled to actually record from
>>>>>>>>>> TVONE PLUS1 has a priority of -4, but the one that is not going to
>>>>>>>>>> record from TV ONE has a priority of -3.
>>>>>>>>>>
>>>>>>>>>> The channel priorities are: TV ONE is 0, TVONE PLUS1 is -1, which
>>>>>>>>>> explains the difference between the two recording priorities. But it
>>>>>>>>>> is still going to record the wrong one.
>>>>>>>>>
>>>>>>>>> I'm pretty sure Mike meant that to be --testsched and not
>>>>>>>>> --printsched. And don't snip any of the output. It's the cryptic
>>>>>>>>> details that explains what is going on and why.
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>
>>>>>>>> OK, that has a lot more detail. I have put it here:
>>>>>>>>
>>>>>>>> http://www.jsw.gen.nz/mythtv/testsched.txt
>>>>>>>
>>>>>>> Here's what's happening in a nutshell. In the initial scheduling
>>>>>>> pass, none of the HD showings can be scheduled due to conflicts. In
>>>>>>> the first retry scheduling pass, none of the conflicting programs can
>>>>>>> be moved. In the second retry pass, lower priority showings are tried
>>>>>>> and the SD showing at 20:00 is chosen.
>>>>>>>
>>>>>>> The reason adding an override sometimes works is it essentially limits
>>>>>>> the second retry pass to the current showing. FWIW, there are some
>>>>>>> minor changes to this area in master (aka pre-0.27). They might or
>>>>>>> might ot help in this particular case.
>>>>>>>
>>>>>>> David
>>>>>>
>>>>>> But what I do not understand is that there are no conflicts at all for
>>>>>> the DVB-T channels. There are enough DVB-T tuners to record all the
>>>>>> programs, so why are there conflicts to resolve? The scheduler should
>>>>>> just be able to assign the available tuners. The conflicts are only
>>>>>> for the S-Video input.
>>>>>
>>>>> There are conflicts with:
>>>>>
>>>>> !Bath Crashers 12 ChoiceT 09 18:30-19:03 1 1 1 C 1 4/1
>>>>> !Grand Designs Revisited 3 TV3 09 19:30-20:33 1 5 5 C 5 4/5
>>>>> !Great Rift: Africa's Wild - 10 PRIME 09 19:30-20:33 1 10 10 C 10 4/9
>>>>>
>>>>> David
>>>>
>>>> As I said, I have 3 DVB-T tuners. There are only 3 DVB-T multiplexes
>>>> being transmitted in New Zealand. So there is no way to get a
>>>> conflict on DVB-T except by running out of virtual tuners. Which
>>>> would require recording from 5 channels at once on one of the DVB-T
>>>> multiplexes. The TVNZ multiplex and the Mediaworks multiplex have
>>>> only 4 channels transmitted on each. The Kordia multiplex has more
>>>> channels, around 12 I think including the radio ones, but I only ever
>>>> record from 3 of them. So I always have enough DVB-T tuners and
>>>> enough virtual tuners on each DVB-T tuner so that there is no conflict
>>>> possible.
>>>>
>>>> So where is the conflict coming from? The source of the non-existent
>>>> conflict would seem to be the source of the problem.
>>>
>>> I told you exactly where the conflicts were, but you aparently refuse
>>> to see them. If you close your eyes, do you think you're invisible,
>>> too?
>>
>> Clearly I do not know what you mean by a conflict here then. Sure,
>> the scheduler is seeing a conflict - it is saying so. But there is no
>> actual conflict for me to relate that back to. Since I have few clues
>> about how the scheduler works, from the outside, it appears to me to
>> be a bug. If it is actually normal and expected behaviour from the
>> scheduler to decide there is a conflict where there is not one (for
>> reasons known only to the scheduler), then it would be helpful to know
>> that is what it does.
>>
>>> You seem to have the mistaken impression the scheduler will
>>> automagically utitlize every tuner and multiplex like it's playing a
>>> perfect game of tetris. It doesn't work that way. The scheduler
>>> tries to balance many different goals using the priority hints given
>>> to it. It's not perfect, doesn't claim to be and never will be.
>>>
>>> David
>>
>> Well, mostly the scheduler does seem to be able to automagically get
>> everything right! It does very well for me with the difficult job of
>> scheduling my one S-Video input from my Sky set top box. So when I am
>> seeing it go wrong, if not badly, in the "easier" job of scheduling my
>> DVB-T tuners where conflicts are impossible, I wonder why. And since
>> I am getting this problem quite repeatably every Thursday at the
>> moment until one of the channels changes their schedule, surely it
>> would be a good idea to use my setup to collect some information on
>> where it is going wrong so that it might be possible to improve the
>> scheduler.
>>
>> As I said in my original post, the problem feels to me, as a person
>> looking on from the outside with no knowledge of the internals of the
>> scheduler, that it is hitting some sort of fixed limit, such as too
>> many channels to schedule at once. For example, maybe it is going
>> around a loop N times and is unable to schedule in those N loops, so
>> it declares a conflict and changes the conditions it is working on and
>> then tries another N loops and succeeds, when if N was a bit bigger
>> (to cope with more channels and/or more tuners), it might have been
>> able to schedule properly on the first try. Or maybe it is filling up
>> a table and stopping when it is full. Just some wild-ass guesses.
>>
>> And if I am right and the problem is triggered by recording more
>> things at once, as the number of channels being broadcast increases
>> and people have more tuners, more people are likely to come up against
>> this problem.
>>
>> So, is there anything more I can do to get data that might illuminate
>> the source of this problem?
>
>You claim that with your setup 'conflicts are impossible', but as I said
>earlier in this thread:
>
>> It's easy to reach the limit on a single mux, especially with pre and post roll. I think the default limit in 0.26-fixes is 5 virtual tuners; you say you are using 4. I haven't examined your schedule in depth, but if all your muxes are in use there's nowhere else to go.
>>
>Two successive sets of two simultaneous recordings on mux A, with pre
>and post roll, will exhaust your 4 virtual tuners during the overlap
>period; if your other hardware is already recording from other muxes
>there will be a conflict if you seek to schedule another recording from
>mux A. That may not be what is happening, but it's _not_ impossible;
>around 8 pm or 9 pm for me.
Hmm... Yes, you are right, with the overlaps, it is not impossible to
run out of virtual tuners. So I will have to increase them to 5, just
in case.
But when I add overrides to fix the problem, there are no conflicts,
so it was not a problem with the overlaps creating a conflict. With
the overrides, it does record everything from the TV One and TV3 HD
channels, as desired. It did that this Thursday, and with the
overrides in place, it is going to again next Thursday.
Here is a table I did, for a manual schedule for next Thursday. I
have ignored the non-DVB-T recordings.
T1 = DVB-T tuner, 4 virtual tuners
T2 = DVB-T tuner, 4 virtual tuners
T3 = DVB-T tuner, 4 virtual tuners
There are three DVB-T multiplexes transmitting:
TVNZ
Mediaworks
Kordia
All DVB-T programs are set up with 3 minutes hard post-roll. Bones
also has 1 minute of hard pre-roll.
There is 60 seconds soft pre-roll and 60 seconds soft post-roll.
Time T1 T2 T3 Program (multiplex)
12:04 0 0 0 -Cowboy Builders (TVNZ) = T1 is free
17:59 1 0 0 +One News (TVNZ) = T1 tunes to TVNZ
17:59 1 1 0 +3 News (Mediaworks) = T2 tunes to Mediaworks
18:29 1 1 1 +Bath Crashers (Kordia) = T3 tunes to Kordia
18:59 2 1 1 +Seven Sharp (TVNZ)
18:59 2 2 1 +Campbell Live (Mediaworks)
19:04 1 2 1 -One News (TVNZ)
19:04 1 1 1 -3 News (Mediaworks)
19:04 1 1 0 -Bath Crashers (Kordia) = T3 is free
19:29 2 1 0 +Police Ten 7 (TVNZ)
19:29 2 2 0 +Grand Designs Revisited (Mediaworks)
19:29 2 2 1 +Great Rift (Kordia) = T3 tunes to Kordia
19:34 1 2 1 -Seven Sharp (TVNZ)
19:34 1 1 1 -Campbell Live (Mediaworks)
19:59 2 1 1 +RBT (TVNZ)
20:04 1 1 1 -Police Ten 7 (TVNZ)
20:28 1 2 1 +Bones (Mediaworks)
20:34 1 1 1 -Grand Designs Revisited (Mediaworks)
20:34 1 1 0 -Great Rift (Kordia) = T3 is free
20:34 0 1 0 -RBT (TV2) = T1 is free
21:29 1 1 0 +Embarrassing Bodies (TVNZ) = T1 tunes to TVNZ
21:34 1 0 0 -Bones (Mediaworks) = T2 is free
21:59 1 1 0 +Cops (Mediaworks) = T2 tunes to Mediaworks
22:34 0 1 0 -Embarrassing Bodies (TVNZ) = T1 is free
22:34 0 1 0 -Cops (Mediaworks) = T2 is free
It is not at all impossible that I have made a mistake in doing this
table - that is why we do not do manual scheduling and need the
scheduler. But if I have not, then you can see that even with the
overlaps no tuner is ever using more than 2 virtual tuners - nowhere
near the limit. And there is clearly no conflict.
So why, without the overrides, does the schedule currently say it will
not record Bath Crashers at 18:29, but will take a later showing? To
get it to record like the above, I have to add a "Record anyway"
override to Bath Crashers at 18:29, and then to Seven Sharp at 18:59,
and then to Campbell Live at 18:59. Then the scheduler stops marking
other programs for later showings and records as desired on the first
showing (and from the HD channels).
More information about the mythtv-users
mailing list