[mythtv] [mythtv-commits] Ticket #3317: mythbackend memory leak

[Kevin Kuphal]

Michael T. Dean wrote:
> On 05/07/2007 12:50 PM, Rob Smith wrote:
>   
>> On 5/7/07, Carl Fongheiser <carlfongheiser at gmail.com> wrote:
>>   
>>     
>>> On 5/7/07, Kevin Kuphal <kuphal at dls.net> wrote:
>>>     
>>>       
>>>> Mem:    775856k total,   767484k used,     8372k free,      108k buffers
>>>> Swap:   524280k total,   240864k used,   283416k free,   175920k cached
>>>>
>>>> with the backend process
>>>>
>>>> 31439 root      18   0  987m 499m  12m S  3.3 65.9  45:08.63 mythbackend
>>>>       
>>>>         
>>> Your machine is seriously short on swap space.  A good rule of thumb is to
>>> allocate about 2x the amount of physical RAM for swap space.  As to why
>>> you're getting a fork() error, fork() will attempt to clone the entire
>>> address space.  In this case, that's 987MB worth.  Not all of that will need
>>> to be backed by swap, but most of it will.  Since you've only got 280 MB
>>> free, the fork() fails.
>>>       
>> That's not quite true. Sure, the process will 'take' 987mb worth, but
>> a large chunk of that is in shared libs and the like, so it won't take
>> all that in swap *or* main memory.
>>
>> pmap -d `pidof mythbackend` will tell you how much is truely used and
>> required, under the writable/private line. on mine it had mapped
>> 160640k, and writable/private of 114768k, which means it's 'total'
>> footprint is a tad over 270m, but it is only using 115m by itself and
>> that's all it needs to page into swap.
>>
>> If you really want to see where the memory is being used, just look
>> above the last line and it will tell you what it is using where.
>>   
>>     
>
> Not to mention, from man 2 fork:
> Under  Linux,  fork()  is implemented using copy-on-write pages, so the
> only penalty that it incurs is the time and memory required  to  dupli-
> cate  the  parent's  page tables, and to create a unique task structure
> for the child.
>
> But, if fork were to clone the actual memory contents, it would be a 
> complete waste to clone more than the writable/private data.
>
>   
>> That being said, 2x ram for swap is a huge waste.
>>     
>
> I agree.  Swap size = 2x RAM /was/ the rule of thumb when RAM was 
> expensive (and memory sizes were small).  Anymore (especially on systems 
> with >256MB RAM), the rule doesn't really apply.
>
> Note, though, that I am not recommending any specific swap sizes--I'm 
> only saying the rule doesn't really apply for most modern systems.  
> Google for "linux swap size" (no quotes) to see all the competing 
> theories on the best way to size your swap, then roll a die (or cast 
> some bones, or read some tea leaves, or ...) and figure your required 
> swap size.
>
>   
What's interesting is that after increasing the swap size to address 
this issue, the backend seems to have stabalized at around 1.2G of 
virtual ram.  I'm going to watch it more but it seems that once it 
reaches this level it tops out which would be fine for me with the extra 
swap space now not having the fork problem...

Kevin