[mythtv-users] scheduler doesn't run ?

[Michael T. Dean]

On 01/14/2016 03:04 AM, Per Jessen wrote:
> Michael T. Dean wrote:
>
>> On 01/13/2016 02:30 PM, Jonatan Lindblad wrote:
>>> Den 2016-01-13 kl. 14:24, skrev Per Jessen:
>>>> Michael T. Dean wrote:
>>>>
>>>>> Usually if the scheduler isn't scheduling properly, it means
>>>>> there's a problem with one or more of your (almost definitely
>>>>> custom) recording
>>>>> rules--usually resulting in a SQL issue.  If you have any custom
>>>>> rules, you may want to write down the details (so you could
>>>>> recreate
>>>>> them) and then delete them and see if things work properly.  If so,
>>>>> start adding them back one by one until you find the culprit.
>>>> I have 77 custom rules, most from October 2015 or older - only two
>>>> new ones:
>>>>
>>>> program.title like '%Walking%dead%'
>>>> program.title like '%Endeavour%'
>>>>
>>>> I did have a double for #1, but deleting it didn't do much.
>>>>
>>>> Can I unload/reloead table 'record' or will that screw up things?
>>>> I have some 440 rules, I'd hate to try to recreate those :-(
>>> 440 rules sounds like it could use an awful lot of memory so make
>>> sure you have enough temporary disk space.
>>>
> http://lists.mythtv.org/pipermail/mythtv-users/2011-December/325302.html
>> Great suggestion.  Those 2 new rules look fine, so it may well be
>> something else (such as MySQL memory/connection issues).  (Look also
>> at whether your MySQL is configured to allow sufficient
>> connections--this has been an issue in the past, and can result in
>> symptoms like you describe, where everything appears to work, but the
>> scheduling doesn't and/or doesn't reliably.)
> Max mysql clients is 151, the default I believe. Currently 28 active
> connections from the master plus one slave.
> mysql is using /var/tmp and there is about 10Gb of available space.
>
> I'm slowly getting the feeling it's somehow related to these messages:
>
> Scheduled 629 items in 1764.2 = 1752.96 match + 11.25 place
> Scheduled 643 items in 164.1 = 155.02 match + 9.06 place
> Scheduled 643 items in 916.3 = 904.87 match + 11.45 place
>
> Judging by the code, this is about how much was scheduled and how long
> it took.  I took a look at those messages in the logs (back to 31/10, I
> only keep the logs for so long).
>
> http://files.jessen.ch/mythtv-items-scheduled-since-20151031a.jpeg
>
> It seems to have been fairly steady, with a rise since December. It's
> gone from around 600 items to more than 800.
>
> This is the interesting one though:
> http://files.jessen.ch/mythtv-time-to-schedule-since-20151031a.jpeg
>
> Very stable until last week of November, then mayhem as of December.
>
> It looks like it used to take 60 seconds, since December up to an hour
> or more - if it takes too long to run a reschedule, might this upset
> the overall scheduling?

Yep.  That's your problem.  It's not that scheduling isn't running, it's 
that it's taking so long to run that you're not noticing the changes.

Reschedules should take (significantly) less than a minute on a properly 
tuned system, though things should generally work fine with them taking 
up to a minute or so (some people run with scheduling taking multiple 
minutes, though, it's not really designed/tested for such slow 
response).  Properly tuned means many things--properly tuned MySQL with 
MySQL data and temp/scratch storage on a properly-configured file system 
with MythTV only being asked to do as much as the hardware can handle 
(or, if you want it to do more, with hardware upgraded to provide more 
capability).

As far as the MythTV side of things go, there are exactly 3 things that 
determine scheduling time:  1) number of programs to consider for 
scheduling (which, generally, is related to number of channels), 2) 
number of recording rules, and 3) number of previous recordings. So, did 
any of those things change in November/December?  Did you add new tuners 
(or virtual tuners) or new sources or reconfigure your sources or ...?  
If so, you may need to pare down one of the 3--i.e. delete unnecessary 
channels (in mythtv-setup's Channel Editor), recording rules (in Manage 
Recordings|Recording Rules), or recording history (in Manage 
Recordings|Previously Recorded). Alternatively, if you don't want to 
pare down the things you're asking MythTV to manage, you may need to 
upgrade the hardware on which MythTV (and, likely, MySQL server) is running.

If none of the above changed in November/December, then it's unlikely to 
be something that you can fix in MythTV.  Instead, it's most likely some 
big change to your configuration (i.e. change of file system on which 
the MySQL data/temp storage lives or change of MySQL configuration--such 
as database Storage Engine--or similar). (Did your distro change the 
file system used for /tmp?  Is it an actual file system or a in-memory 
file system--where in-memory file system will defeat the purpose of 
using it for a MySQL scratch file system for things that won't fit in 
memory.)  Alternatively, it could be down to issues with your MySQL data 
storage (i.e. you need to run optimize_mythdb.pl or similar).

Since your scheduling time didn't grow slowly, it's almost definitely 
not just a problem of "normal" growth of recording rules/history.  The 
large, sudden growth in November/December indicates a big change to 
something around that time.  Also, the change from 60s to

Note, too, that to some extent, the type of recording rule can make some 
difference--though for big differences, the most likely issue is just 
bad SQL (or overly-complex-to-run SQL) in custom rules.

Mike