[mythtv-users] 0.25 lookup metadata as part of jobqueue?

[Raymond Wagner]

On 3/11/2012 12:04, Steven Adeff wrote:
> On Sun, Mar 11, 2012 at 11:43 AM, Raymond Wagner<raymond at wagnerrp.com>  wrote:
>> On 3/11/2012 08:03, Scott&  Nicole Harris wrote:
>>> It would also be nice to set a different max jobs per job type.  i.e. Allow
>>> 4 commercial flag jobs but only 1 transcode job at a time.
>> That's still far from optimal.  Commercial detection can run in realtime
>> behind a recording, or at full speed behind a recording.  At full speed,
>> it will fairly efficiency balance over two cores, meaning those four
>> jobs would saturate a dual socket, quad core server.  In realtime, all
>> of those commercial detection runs will sit idle for the first ten
>> minutes, waiting for enough data to do icon detection, and then catch up
>> as fast as they can to the in-progress recording, again saturating that
>> high end server.  Depending on how many simultaneous recordings you
>> have, you may not have a choice of which mode the jobs are going to run in.
> So if I have a dual core processor, I run my commercial detection jobs
> "real time", and I have say, 7 tuners, would it make sense to allow
> more than two commercial detection jobs to run at a time so that all
> current recordings are being processed or is it better to only allow
> one job per core?

If you have a dual core processor, are allowing two simultaneous 
commercial detection jobs, and are recording seven shows, the first two 
jobs that get added to the database will run in real time, assuming your 
CPU has enough performance to achieve that.  The remaining five will 
queue up and run full speed once the first two recordings are finished, 
two at a time, and every so slightly less than half their maximum speed, 
due to the two potentially competing over shared resources.

Ideally, a heuristic job scheduler would know you could run up to four 
in real time on your CPU during the recordings, and then drop to 
sequential operation after the recordings have finished.  The job queue 
rewrite I have planned calls for a centralized plugable scheduler that 
would allow for such to be written.  However actually writing one is 
beyond the scope of those initial plans, and I have doubts about how 
successful I would be at trying to write one.