[mythtv-users] pdflush is killing me

[R. G. Newbury]

Tony Lill wrote:
> "Steven Adeff" <adeffs.mythtv at gmail.com> writes:
> 
>> On 2/5/07, Tony Lill <ajlill at ajlc.waterloo.on.ca> wrote:
>>> THis worked for me (basicly by making pdflush run more often, but
>>> with less work to do).
>>>
>>> # % of physical memory after which a process doing a write will be paused
>>> # untill dirty pages have been written
>>> # default 40
>>> echo 50 > /proc/sys/vm/dirty_ratio
>>>
>>> # % of physical memory at which pdflush will be woken. Hopefull this will limit
>>> # system pauses for flushes
>>> # default 10
>>> echo 5 > /proc/sys/vm/dirty_background_ratio
>> having seen pdflush run while observing top (not knowing if it was
>> causing any actual issues...), I've taken an interest in what you've
>> written here. Would you mind elaborating on what this does for people
>> like me that don't know what that means?
> 
> When you write data to a file, it doesn't go directly to the file, the
> data is buffered up in memory, and the kernel then tried to find the
> "best" time to write it out. Usually this means when the disk head
> happens to be near where the data should be written. This reduces seek
> time and since a lot of files are small and the data gets read again
> soon after it is written, keeping them in memory is a big performance
> boost.
> 
> This is great for a compiler, because when it creates small temporary
> files, turns around and reads them, then deletes them. By caching
> this data in memory, there's a good chance that they will never
> actually be written to disk.
> 
> Then there's myth, which writes a huge (larger than physical memory)
> file and maybe doesn't read it for hours.
> 
> Modulo other parameters, pdflush will get woken up when
> /proc/sys/vm/dirty_background_ratio percent of memory is full of data
> to be flushed to disk, and then it will clean it out. If you've got 1G
> of memory, that's 100M. 
> 
> So if nothing much else is happening and myth is writing at 1M/s to a
> new file, and when 100M has accumulated, pdflush wakes up and writes
> that out at one shot. That can hog down your system enough to cause
> playback to pause or recording to drop frames. By turning it down to
> 5%, that's only 50M at one shot, so no hiccup. If you have 2G, turn it
> down even more.
> 
> The fun thing is, you can also cure this behaviour by doing something
> like compiling a kernel!
> 
> The other value will let your program to use up more memory for write
> buffers before a write system call will cause the program to be
> suspended. 

So is the number you poke into /proc/sys/vm/dirty_ratio stored? Or 
should these calls be added to rc.local on the mythbox?


Geoff