[mythtv-users] 64 Bit and memory
r-mythtv at thefreemanclan.net
Thu Jan 10 14:23:43 UTC 2008
> I'm running a SVN(15354) frontend/backend on Gentoo with 32 bits and I'm not
> seeing this problem.
> asgard ~ # free -m
I just noticed these posts. If you're concerned about memory leaks /
etc looking at the output of free is just about useless. Look at how
much memory individual processes are taking up instead.
It is completely normal for a linux system with 500GB of memory and at
least as much hard drive space to swap to disk, and report most of the
memory as in use just about all the time.
The reason memory seems to be always in-use is that "free" memory in
linux is memory that is not being used AT ALL for any purpose. Linux
tries to minimize the amount of completely-unused memory - preferring to
use it for buffering or cache. When the memory is needed the cache is
purged and the buffers are flushed. This only improves system
performance by reducing disk access. If memory isn't needed for
something else you might as well use it as cache - it doesn't cost
anything as cache can be instantly purged to be made available.
The reason swapping happens all the time on linux is because of the
"swapiness" setting. If a page of memory seems to be idle for a very
long time the system will swap it out when it is otherwise not busy to
make more room for cache/buffers. The assumption is that something else
will need that memory before the idle app does, or that caching will
generally improve performance. So even if there is free memory the
system will swap out empty pages. This behavior can be tuned with a
setting in /proc if necessary.
A memory leak is when an application allocates memory and then doesn't
use it, or stops using memory but doesn't free it. It causes an
application's virtual memory use to grow, but its real memory use does
not necessarily grow as much (because of the aforementioned swappiness -
linux just swaps the dead pages out to disk and never reads them back).
The ability to swap dead pages would probably depend on whether the
wasted memory fills entire pages - it still isn't ideal.
I haven't observed memory leaks on my 64bit gentoo system, or on my
32-bit frontend (with about 200MB available RAM and no swap). I have
increased my buffer sizes on my backend, so that can consume a fair
amount of virtual memory - but nothing really more than expected.
When my myth system is slow I've found that the usual cause is IO
contention. Swapping contributes to this in part, but only in part.
The iostat tool can help troubleshoot this. The ionice tool can help
mitigate this. I run the backend with a medium realtime ioniceness
setting - this allows the backend access to the disk just about anytime
it needs it without waiting in line. That and increasing my buffers
and getting rid of the buffer fsync have completely eliminated by
In any case, the way to spot a memory leak is looking at the output of
top or ps -o args,vsz,rss -u mythtv (or whatever). Sorting top by
memory use is a good way to find out where all your RAM is going. You
might be surprised to find out that it is something else entirely.
More information about the mythtv-users