[mythtv-users] Howto prevent MythTV flushing/Syncing disk writes (LiveTV/Recordings) [ SOLVED ]

[David Rees]

On Thu, Mar 20, 2008 at 11:31 AM, Albert Graham <agraham at g-b.net> wrote:
> David Rees wrote:
>  > I don't see any point in syncing when closing the file. This will
>  > cause a large writeout which could cause other activity to lag. It
>  > will be better to just let the OS flush the data out to disk as usual.
>  > The large writeout will be even worse when you have extreme VM
>  > settings like you do. Luckily, you have the hardware to support this
>  > type of activity which is probably why you haven't seen any ill
>  > effects.
>
>  OK, I too am in two minds about this, the idea is that most shows are
>  1/2 - 1 hour long, so this happens at the end of show
>  rather than when the OS wants it to happen - also if the machine did
>  crash, you'll have the last finished recording, I'm using XFS (with
>  extreme settings too :))

You are assuming that a complete recording is some how more valuable
than a partial recording. While I can see your argument, personally I
would prefer to configure the system so that every 5 minutes or so the
data gets flushed to disk by the OS instead of forcing it to flush
everything out once it's done. If you're constantly writing, you're
going to have to schedule writes at some point, anyway. Anyway,
probably not that big a deal.

>  > Not really. The behavior you see once memory has filled up is about as
>  > good as you'd expect. You could reduce the frequency of these writes
>  > in a steady state writing case by re-enabling the
>  > dirty_writeback_centisecs setting and setting it to 5-10 seconds or
>  > so.
>
>  OK, my box is recording 24/7 so after an hour or so Linux starts to slow
>  flush/sync out to disk (in about 8MB chunks) the cache, and the goes on
>  forever!
>
>  >>  echo "1" > /proc/sys/vm/laptop_mode
>  >>  sync
>  >>  echo "3" > /proc/sys/vm/drop_caches
>  >>  sync
>  >>  tw_cli /c0 flush
>  >>  echo "0" > /proc/sys/vm/laptop_mode
>  >
>  > Eek! Why write out all that data to disk that you just worked so hard
>  > to buffer up? And then after syncing it all to disk, you throw it all
>  > away?
>
>  Well, yes, but my goal is "quietness", and having the entire show flush
>  in 1 large write is what I personally was looking for.

I think that your hack of syncing and dropping caches is just that - a
hack. You should be able to return similar results by properly
tweaking the VM dirty writeout settings.

>  > The only reason a sync is required before dropping the caches is
>  > because dropping the caches will only drop data that has been synced
>  > to disk. So when you have aggressive VM settings which don't ever
>  > flush things to disk, trying to drop the caches will not actually drop
>  > anything. You won't lose any data by dropping caches without syncing.
>
>  Well, I was really just going by some docs on this one, e.g.
>
>  "As this is a non-destructive operation, and dirty objects are not
>  freeable, the user should run "sync" first in order to make sure all
>  cached objects are freed."

Isn't that what I said? ;-)

Anyway, I would suggest these VM settings to try to achieve fairly
infrequent large writes:

# Let dirty data consume this percentage of memory (default 10)
dirty_ratio = 80
# Let dirty active data consume this percentage of memory (default 5)
dirty_background_ratio = 80
# Wake up pdflush every minute (default 1499)
dirty_writeback_centisecs = 6000
# How old data can be before it should be considered dirty and written
to disk (5min) (default 2999)
dirty_expire_centisecs = 30000

I have a feeling that by going too extreme in your settings may
actually be detrimental to your goal. In theory, the above settings
should end up flushing a bit of data to disk every minute or so. Tweak
slightly as you see fit and let us know how it works out. :-)

-Dave