[mythtv-users] Scheduler runs slowly causes backend to lock up

[Paul Netherwood]

>>On Wed, Nov 26, 2014 at 01:01:20PM +0100, Paul Netherwood wrote:
>>> I've noticed that the scheduler is running very slowly which coincides with the backend locking up. The delay is in the "place" part of the scheduler. I've grep'd out the recent scheduler lines from the log:
>>>
>>> Nov 25 22:06:39 mythbackend mythbackend: mythbackend[22634]: I Scheduler scheduler.cpp:2252 (HandleReschedule) Scheduled 1089 items in 215.2 = 0.00 match + 0.00 check + 215.22 place
>>>
>>> 200 seconds is a long time. Its been worse with it sometimes taking 800 seconds. Its been like this since upgrading from v0.27.3-5-gdbb4ef3 to v0.27.3-164-g629f711. Before that it averaged 20 seconds. I'm now on v0.27.4-6-ge0b0027 and its still happening.
>>>
>>> Any ideas what could be causing this?
>> I'm afraid you'll have to run git bisect to try to identify a specific
>>commit that might be causing your problem.  I don't see anything
>>between those versions that might have caused it.  There certainly
>>weren't any scheduler changes.  Alternatviely, are you sure you didn't
>>change anything alse in your system that could have contributed to the
>>problem?
>>
>>David
> 
>I am on the exact same git version here and no problems.  2 sources, but SSD, 6 cores, and lots of ram.
> 
>Dec  3 03:01:02 mythmasterbackend mythbackend: mythbackend[30359]: I Scheduler scheduler.cpp:2252 (HandleReschedule) Scheduled 986 items in 1.2 = 0.01 match + 0.01 check + 1.18 place

After turning on extra scheduling logging with -v scheduler I was able to identify that it was the place query that was taking the time. Looking at the query I noticed it was creating a temporary table. created_tmp_tables_on_disk variable in mysql was really high so I'm guessing every time it did the place query it was creating a new table on disk. I'm beginning to wonder if its just coincidental that the time increase happened near a version upgrade but what was really happening is that a temporary table that could fit into memory was now being moved onto disk.

It occurred to me that it has been ages since I had a look at my mysql config and did a bit of tuning. After a bit of tweaking I was able to get my place times from well over 200 seconds to just a over a second!

It was quite useful to run mysqltuner.pl to help pick up some optimisations on my config.

Thanks for the comments.

Paul