[mythtv-users] 64 Bit and memory

[Mitch Gore]

On Jan 10, 2008 8:23 AM, Richard Freeman <r-mythtv at thefreemanclan.net>
wrote:

> Marc wrote:
> > I'm running a SVN(15354) frontend/backend on Gentoo with 32 bits and I'm
> not
> > seeing this problem.
> > asgard ~ # free -m
>  > snip...
>
> I just noticed these posts.  If you're concerned about memory leaks /
> etc looking at the output of free is just about useless.  Look at how
> much memory individual processes are taking up instead.
>
> It is completely normal for a linux system with 500GB of memory and at
> least as much hard drive space to swap to disk, and report most of the
> memory as in use just about all the time.
>
> The reason memory seems to be always in-use is that "free" memory in
> linux is memory that is not being used AT ALL for any purpose.  Linux
> tries to minimize the amount of completely-unused memory - preferring to
> use it for buffering or cache.  When the memory is needed the cache is
> purged and the buffers are flushed.  This only improves system
> performance by reducing disk access.  If memory isn't needed for
> something else you might as well use it as cache - it doesn't cost
> anything as cache can be instantly purged to be made available.
>
> The reason swapping happens all the time on linux is because of the
> "swapiness" setting.  If a page of memory seems to be idle for a very
> long time the system will swap it out when it is otherwise not busy to
> make more room for cache/buffers.  The assumption is that something else
> will need that memory before the idle app does, or that caching will
> generally improve performance.  So even if there is free memory the
> system will swap out empty pages.  This behavior can be tuned with a
> setting in /proc if necessary.
>
> A memory leak is when an application allocates memory and then doesn't
> use it, or stops using memory but doesn't free it.  It causes an
> application's virtual memory use to grow, but its real memory use does
> not necessarily grow as much (because of the aforementioned swappiness -
> linux just swaps the dead pages out to disk and never reads them back).
>  The ability to swap dead pages would probably depend on whether the
> wasted memory fills entire pages - it still isn't ideal.
>
> I haven't observed memory leaks on my 64bit gentoo system, or on my
> 32-bit frontend (with about 200MB available RAM and no swap).  I have
> increased my buffer sizes on my backend, so that can consume a fair
> amount of virtual memory - but nothing really more than expected.
>
> When my myth system is slow I've found that the usual cause is IO
> contention.  Swapping contributes to this in part, but only in part.
> The iostat tool can help troubleshoot this.  The ionice tool can help
> mitigate this.  I run the backend with a medium realtime ioniceness
> setting - this allows the backend access to the disk just about anytime
> it needs it without waiting  in line.  That and increasing my buffers
> and getting rid of the buffer fsync have completely eliminated by
> IOBOUND issues.
>
> In any case, the way to spot a memory leak is looking at the output of
> top or ps -o args,vsz,rss -u mythtv (or whatever).  Sorting top by
> memory use is a good way to find out where all your RAM is going.  You
> might be surprised to find out that it is something else entirely.
>
> _______________________________________________
> mythtv-users mailing list
> mythtv-users at mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>



I understand how linux does its memory usage and what not.  But i am not
talking about using alot of my memory and it being 'slower'.  When I say
slow the mouse wont even move fluidly.  It jumps 1/2 way across the screen
as i move it.  I look at htop and see mem usage is 100% and swap is
1000/2500m used.  The usage of swap increases the longer I leave the
frontend open.  Looking at memory to processes correlation I see the
mythfrontend processes each taking 9.4% of memory.  And when there are 15
processes going for mythfrontend it eats everything up.  When I killall
mythfrontend the problem gets better.  I can atleast run the frontend with
some experience but the only way to really get a fast userspace is to
reboot.


Again I am using atrpms bleeding packages which are SVN r15333.  I believe
this my be a x86-64 issue.  As my 32bit works fine with the same packages.
both are F8 fully updated boxes.

How do i got about figuring out where the issue is?  I am not the only one
seeing this problem and its a SERIOUS one!

If a Dev would like to take a look I would have no problem giving user shell
access to the box.

Thanks,
Mitchell
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mythtv.org/pipermail/mythtv-users/attachments/20080110/c0c60e9f/attachment.htm