[mythtv] [mythtv-commits] Ticket #2782: automatic simultaneous jobs scaling

Michael T. Dean mtdean at thirdcontact.com
Sat Dec 9 03:17:55 UTC 2006


On 12/08/2006 02:29 AM, osma ahvenlampi wrote:
> On 12/8/06, Michael T. Dean <mtdean at thirdcontact.com> wrote:
>   
>> The idle value /never/ changes on any of the hosts, thanks to a little
>> program I call AthWarm (most call SETI/BOINC).  :)
>>     
>
> Ah, I didn't consider someone wanting to run their HTPC hot - I've
> been more concerned about running it quiet. How would you propose
> detecting between time spent in SETI vs time spent by jobs? The kernel
> won't distinguish between the niced processes..
>   

Yeah, my 2 backends and 1 frontend are not in the media room.  
Therefore, noise from running hot has no effect on my viewing 
experience.  I like my media room clean--one 67" HDTV, speakers, and 
places to sit--no ugly A/V receivers, STB's, PC's, tangle o' cables, ...

>> No problem, though, I wouldn't mind disabling job scaling.  (Now,
>> disabling BOINC, OTOH, will not happen :).
>>     
>
> I didn't make this configurable, because I would have liked to make it
> "work right" without configuring. That can be done though, but I'd
> rather figure out a better solution.
>   

Not a problem for me--I keep my own patch tree.  ;)

> At least one change would need to happen for this to become
> commit-worthy, I guess: right now jobqueue will not start *any*
> processes if there's no idle time, but it will start them if it can't
> read idle time at all. I'll change it so that it will start *one*
> process in either of those situations, but scale up from there only if
> there's still time available. Would that be satisfactory?
>   

Yep.  With that change, I wouldn't even bother disabling the scaling.  I 
run with a max of 1 job on all my hosts because of the I/O issues that 
Daniel mentioned.  So, if it were allowed to start one job even at 0% 
idle (i.e. with 99% nice), it would be the same as I have currently.  
Basically, I prefer running on the safe side.  After all, I have > 480 
hours of TV (including commercials) to catch up on, so I /never/ have to 
watch something "as soon as possible" after it airs--even waiting until 
the next day to give commflagging plenty of time to finish isn't a problem.*

Perhaps a more "general-purpose" approach that would work for others, 
too, would be detecting the percentage of CPU time spent on idle plus 
the CPU time spent on processes running at a higher nice level than the 
job.  IIRC, jobs are run at nice 17, 10 or 0 depending on the value of 
JobQueueCPU, so any job running "nicer" than the nice setting for the 
job queue should be treated as idle time (i.e. time that could be used 
for the job queue).  (This assumes that we'll still have a JobQueueCPU 
setting after the patch--I haven't looked at the patch, yet.)  I run 
BOINC at nice 19--and expect most people running "hot" do so with 
similarly-low-priority tasks.  Any process running at higher priority 
than the job queue jobs should probably be treated as more important, 
anyway, so this seems like a good way to decide whether a new job is 
warranted.

I don't know if doing this would require parsing every /proc/*/stat to 
retrieve the priority/nice value(s) (fields 18, 19, and/or 38) and the 
per-process CPU usage.  If so, it's probably not worth the effort, but 
it's the thought that counts, right?

Mike

*In fact, I have my Myth box record all new series every season.  Then, 
as networks cancel some of the new series, I delete them without 
watching them.  If "the word on the street" says the show isn't worth my 
time, I delete the series without watching it.  If everyone's talking 
about the show--and saying good things about it--I'll actually watch 
it.  So, generally, I don't watch start to watch shows until they're at 
least a couple months old.  Then, I'm far enough behind "real-time" that 
I don't catch up until after the season finishes.


More information about the mythtv-dev mailing list