[mythtv] [mythtv-commits] Ticket #2782: automatic simultaneous jobs scaling

[f-myth-users at media.mit.edu]

    > Date: Fri, 8 Dec 2006 16:42:05 +0200
    > From: "osma ahvenlampi" <oa at iki.fi>

    > On 12/8/06, Daniel Kristjansson <danielk at cuymedia.net> wrote:
    > > Simple math, lets say you have 4 ATSC recorders which are
    > > recording, and 3 frontends are watching 3 other streams off
    > > your disk, and your disk read/write speed is 21 MB/s. Since
    > > each stream is about 3 MB/s, adding another frontend process
    > > or starting a commercial flagging or transcode process would
    > > bring us over the edge.

    > This is why I referred to CFQ scheduling. With CFQ enabled and the
    > jobs being niced, they won't be getting a whole lot of I/O bandwidth
    > in this situation, so I really doubt that will be the thing taking you
    > over the edge -- it'll be the too-many-recordings-and-playbacks alone
    > that would be the real contributors, as you said (in the part I
    > clipped).

No I/O scheduling setting did a damn at preventing trashed recordings
when I was debugging issues of MySQL being too disk-intensive a few
months ago.  I tried 'em all.  This was in Ubunto Breezy; later
releases' kernels might implement this better (though I -was- seeing
changes, just not good-enough changes).  But since you can't specify
"thou shalt run a kernel where I/O scheduling must work in
such-and-such a way", you can't depend on it.

What -did- work (sort of) was changing some MySQL parameters (I'll
send followup mail at some point for the archives), and even so, I
have to be cautious not to have too many processes flailing on the
MBE's disk (e.g., via NFS mounts from the SBE/FE, via other programs
moving things around as part of my automation, via commflagging, etc.)
I can force a failure (e.g., dropped video from encoders if the
scheduler decides to run) by loading up the I/O system enough, and
while I've got it so my normal usage won't do it, one part of that is
strictly limiting commflagging to one local-to-the-MBE job and one
local-to-the-SBE job (whose 100baseT network connection acts as a
natural throttle on the demands it can make of the MBE's disk).

I've had enough issues with glitched recordings due to other things
using the disk (even though I'm recording on fairly beefy machines
with modern disks, etc) that, if your patch didn't have an OFF switch,
I wouldn't upgrade.  There are too many variables in how people set up
their Myths (e.g., hardware, system architecture, versions of every
single piece of software under the sun) for you to be assured that
your patch will always do the right thing.  It might, and for those
for whom it does, fine.  But you can't claim "it works so well it
won't need an OFF switch" because you haven't seen everyone's
configurations, and there will always be new ones in the future.