[mythtv-users] Software RAID and ionice

[David Rees]

On 5/24/07, Yeechang Lee <ylee at pobox.com> wrote:
> David Rees <drees76 at gmail.com> says:
> > I would suggest trying the following things:
> > 1. Try removing the sync calls.
> > 2. Try increasing the buffer size to 5-10MB.
> > 3. Combinations of the above.
>
> That's my cue. I've posted several times (such as
> <URL:http://www.gossamer-threads.com/lists/mythtv/users/183808#183808>)
> regarding how I patch ThreadedFileWriter to tweak the
> TFW_DEF_BUF_SIZE, TFW_MAX_WRITE_SIZE, and TFW_MIN_WRITE_SIZE settings
> to minimize IOBOUND errors

Ah, good info. Just for reference you found that a 32MB buffer, ~2.7MB
max write and ~1.3MB min write gave you the most reliable recording
experience in that thread, compared to the stock 2MB buffer, 0.5MB max
write and 0.25MB min write sizes.

> (As you'd imagine, I'd love to have these settings be user-tweakable
> without a recompile.) I'm currently using 128/24/512; any insights
> into what the "right" values should be?

Those values are 128MB buffer, 5MB max write and 0.25MB min write
sizes. Those settings seem pretty sane to me, although I imagine that
the amount of buffer you're willing to allocate is going to depend a
good deal on the available memory in the machine and how many parallel
writers you have.

> What you wrote earlier in the thread to Richard Freeman intrigues me:
> > In theory removing the fsync/fdatasync should allow the OS to group
> > writes and should improve performance. It would be interesting to know
> > if your system using using fdatasync, or fsync, since fsync also
> > flushes the file's metadata which means 2 writes go out instead of 1.
> >
> > The biggest issue with those commands is that they are blocking
> > commands - which is why myth runs into the IOBOUND issues.  Of
> > course, if your machine really is IOBOUND for some reason,
> > eventually the write will block as well.
> >
> > You could just comment out the Sync call in the SyncLoop function,
> > too.
>
> Before I go poking around myself, if you or Richard Freeman want to
> whip a patch up (and it sounds like a pretty simple one), I'm game.

It's pretty straight forward. Just put a // in front of line 336 of
ThreadedFileWriter.cpp which should correspond to a call to Sync().

Ideally we'd be able to turn on/off the SyncLoop thread as well as the
amount of time between SyncLoop calls Sync (which defaults to 1000 ms
if it has written out more than the min write size and 100 ms if it
has written out less than the min write size).

If people are interested I could whip up a patch to log how long
writes/syncs are taking, too along with how much of the buffer is
used.

-Dave