Hi again,<br><br>I tried the term trick.<br>I kept the backend running for about 5 mins and the hit it <br><br>After this the log file doubles in size to over 2Mb<br>I now have a four parts in the log file that say 'definitely lost'. (not counting the summary)<br>
<br>Are the 'definitely lost' blocks the parts of the logs that we're looking for ?<br><br>log is below<br><br>Rob Verduijn<br><br>First the leak summary<br>==21026== LEAK SUMMARY:<br>==21026== definitely lost: 4,536 bytes in 5 blocks<br>
==21026== indirectly lost: 0 bytes in 0 blocks<br>==21026== possibly lost: 209,488 bytes in 3,100 blocks<br>==21026== still reachable: 475,959 bytes in 3,156 blocks<br>==21026== suppressed: 0 bytes in 0 blocks<br>
==21026== Reachable blocks (those to which a pointer was found) are not shown.<br>==21026== To see them, rerun with: --leak-check=full --show-reachable=yes<br>==21026==<br>==21026== For counts of detected and suppressed errors, rerun with: -v<br>
==21026== ERROR SUMMARY: 1358 errors from 1358 contexts (suppressed: 3 from 3)<br><br>first block<br><br>==21026== 152 bytes in 1 blocks are definitely lost in loss record 3,742 of 3,963<br>==21026== at 0x40260BB: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)<br>
==21026== by 0x4010E17: allocate_dtv (in /lib/<a href="http://ld-2.11.2.so">ld-2.11.2.so</a>)<br>==21026== by 0x40115B8: _dl_allocate_tls (in /lib/<a href="http://ld-2.11.2.so">ld-2.11.2.so</a>)<br>==21026== by 0x604D610: pthread_create@@GLIBC_2.1 (in /lib/<a href="http://libpthread-2.11.2.so">libpthread-2.11.2.so</a>)<br>
==21026== by 0x9F74525: my_thread_global_init (in /usr/lib/libmysqlclient_r.so.16.0.0)<br>==21026== by 0x9F6EA6B: my_init (in /usr/lib/libmysqlclient_r.so.16.0.0)<br>==21026== by 0x9F68ED9: mysql_server_init (in /usr/lib/libmysqlclient_r.so.16.0.0)<br>
==21026== by 0x9F23923: qLibraryInit() (in /usr/lib/qt4/plugins/sqldrivers/libqsqlmysql.so)<br>==21026== by 0x9F25231: QMYSQLDriver::QMYSQLDriver(QObject*) (in /usr/lib/qt4/plugins/sqldrivers/libqsqlmysql.so)<br>==21026== by 0x9F23145: QMYSQLDriverPlugin::create(QString const&) (in /usr/lib/qt4/plugins/sqldrivers/libqsqlmysql.so)<br>
==21026== by 0x51B52AE: ??? (in /usr/lib/libQtSql.so.4.6.3)<br>==21026== by 0x51B5809: QSqlDatabase::addDatabase(QString const&, QString const&) (in /usr/lib/libQtSql.so.4.6.3)<br><br>second block<br><br>==21026== 152 bytes in 1 blocks are definitely lost in loss record 3,743 of 3,963<br>
==21026== at 0x40260BB: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)<br>==21026== by 0x4010E17: allocate_dtv (in /lib/<a href="http://ld-2.11.2.so">ld-2.11.2.so</a>)<br>==21026== by 0x40115B8: _dl_allocate_tls (in /lib/<a href="http://ld-2.11.2.so">ld-2.11.2.so</a>)<br>
==21026== by 0x604D610: pthread_create@@GLIBC_2.1 (in /lib/<a href="http://libpthread-2.11.2.so">libpthread-2.11.2.so</a>)<br>==21026== by 0x5E2EAD7: QThread::start(QThread::Priority) (in /usr/lib/libQtCore.so.4.6.3)<br>
==21026== by 0x4A7F895: UPnp::Start() (in /usr/lib/libmythupnp-0.23.1.so.0.23.1)<br>==21026== by 0x813C1B3: ??? (in /usr/bin/mythbackend)<br>==21026== by 0x80962D1: ??? (in /usr/bin/mythbackend)<br>==21026== by 0x61B1C0D: (below main) (in /lib/<a href="http://libc-2.11.2.so">libc-2.11.2.so</a>)<br>
<br>third block<br><br>==21026== 152 bytes in 1 blocks are definitely lost in loss record 3,744 of 3,963<br>==21026== at 0x40260BB: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)<br>==21026== by 0x4010E17: allocate_dtv (in /lib/<a href="http://ld-2.11.2.so">ld-2.11.2.so</a>)<br>
==21026== by 0x40115B8: _dl_allocate_tls (in /lib/<a href="http://ld-2.11.2.so">ld-2.11.2.so</a>)<br>==21026== by 0x604D610: pthread_create@@GLIBC_2.1 (in /lib/<a href="http://libpthread-2.11.2.so">libpthread-2.11.2.so</a>)<br>
==21026== by 0x5E2EAD7: QThread::start(QThread::Priority) (in /usr/lib/libQtCore.so.4.6.3)<br>==21026== by 0x80C7B63: ??? (in /usr/bin/mythbackend)<br>==21026== by 0x8096616: ??? (in /usr/bin/mythbackend)<br>==21026== by 0x61B1C0D: (below main) (in /lib/<a href="http://libc-2.11.2.so">libc-2.11.2.so</a>)<br>
<br>fourth block<br><br>==21026== 4,080 bytes in 2 blocks are definitely lost in loss record 3,932 of 3,963<br>==21026== at 0x40260BB: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)<br>==21026== by 0x74D5729: g_malloc0 (in /usr/lib/libglib-2.0.so.0.2400.1)<br>
==21026== by 0x74ED0D2: g_slice_alloc (in /usr/lib/libglib-2.0.so.0.2400.1)<br>==21026== by 0x74A4BEA: g_ptr_array_sized_new (in /usr/lib/libglib-2.0.so.0.2400.1)<br>==21026== by 0x74A4C3E: g_ptr_array_new (in /usr/lib/libglib-2.0.so.0.2400.1)<br>
==21026== by 0x74CBDF0: g_main_context_new (in /usr/lib/libglib-2.0.so.0.2400.1)<br>==21026== by 0x5F54B7B: QEventDispatcherGlibPrivate::QEventDispatcherGlibPrivate(_GMainContext*) (in /usr/lib/libQtCore.so.4.6.3)<br>
==21026== by 0x5F54C1B: QEventDispatcherGlib::QEventDispatcherGlib(QObject*) (in /usr/lib/libQtCore.so.4.6.3)<br>==21026== by 0x5E2E2C5: ??? (in /usr/lib/libQtCore.so.4.6.3)<br>==21026== by 0x5E2F025: ??? (in /usr/lib/libQtCore.so.4.6.3)<br>
==21026== by 0x604CB24: start_thread (in /lib/<a href="http://libpthread-2.11.2.so">libpthread-2.11.2.so</a>)<br>==21026== by 0x626E46D: clone (in /lib/<a href="http://libc-2.11.2.so">libc-2.11.2.so</a>)<br><br><br>
<br><br><div class="gmail_quote">2010/11/1 Brian J. Murrell <span dir="ltr"><<a href="mailto:brian@interlinx.bc.ca">brian@interlinx.bc.ca</a>></span><br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div class="im">On Sun, 2010-10-31 at 15:54 -0700, Gavin Hurlbut wrote:<br>
><br>
> No memory leak. Proof positive. At MOST, you have about 250kByte<br>
> maybe lost and/or still reachable.<br>
<br>
</div>Well, you can't really say that for sure given what happened below...<br>
<div><div></div><div class="h5"><br>
> > untill it crashes at the end with this message<br>
> > ==12720==<br>
> > ==12720== Valgrind's memory management: out of memory:<br>
> > ==12720== newSuperblock's request for 4194304 bytes failed.<br>
> > ==12720== 3034054656 bytes have already been allocated.<br>
> > ==12720== Valgrind cannot continue. Sorry.<br>
> > ==12720==<br>
> > ==12720== There are several possible reasons for this.<br>
> > ==12720== - You have some kind of memory limit in place. Look at the<br>
> > ==12720== output of 'ulimit -a'. Is there a limit on the size of<br>
> > ==12720== virtual memory or address space?<br>
> > ==12720== - You have run out of swap space.<br>
> > ==12720== - Valgrind has a bug. If you think this is the case or you<br>
> > are<br>
> > ==12720== not sure, please let us know and we'll try to fix it.<br>
> > ==12720== Please note that programs can take substantially more memory<br>
> > than<br>
> > ==12720== normal when running under Valgrind tools, eg. up to twice or<br>
> > ==12720== more, depending on the tool. On a 64-bit machine, Valgrind<br>
> > ==12720== should be able to make use of up 32GB memory. On a 32-bit<br>
> > ==12720== machine, Valgrind should be able to use all the memory<br>
> > available<br>
> > ==12720== to a single process, up to 4GB if that's how you have your<br>
> > ==12720== kernel configured. Most 32-bit Linux setups allow a maximum<br>
> > of<br>
> > ==12720== 3GB per process.<br>
> > ==12720==<br>
> > ==12720== Whatever the reason, Valgrind cannot continue. Sorry.<br>
<br>
</div></div>So valgrind was unable to continue due to ENOMEM, so likely it did not<br>
get to finish it's accounting.<br>
<br>
Perhaps if you "exited" (i.e. kill, gracefully, with an INTR or TERM)<br>
the backend process before it or valgrind ENOMEMs so that it exits<br>
properly, valgrind could properly account for it's memory use rather<br>
than having to throw it's hands up in defeat by getting it's own ENOMEM.<br>
<div class="im"><br>
> Secondly: you hit a 3GB process memory limit (whether imposed by<br>
> ulimit or by the kernel). The ulimit -m unlimited may fix that.<br>
<br>
</div>I tend to think that removing the ulimit -m will only delay the ENOMEM<br>
until physical VM is exhausted rather than the ulimit being bumped.<br>
Both will have the same problem, so might as well leave the ulimit in<br>
place to mimic VM exhaustion and not start getting into invoking the OOM<br>
killer.<br>
<div class="im"><br>
> You ran out of memory for that process. There's no way to tell from<br>
> here what you are using your memory for or what you are making it do.<br>
<br>
</div>Right. Which is why he needs to make mythbackend exit before it or<br>
valgrind ENOMEMs.<br>
<div class="im"><br>
> I haven't ever seen the backend approach that kind of memory use, so<br>
> it's likely something in how you are using it, etc.<br>
<br>
</div>Well, of course. But what? That's the question. Valgrind should help<br>
point out the answer.<br>
<font color="#888888"><br>
b.<br>
<br>
</font><br>_______________________________________________<br>
mythtv-users mailing list<br>
<a href="mailto:mythtv-users@mythtv.org">mythtv-users@mythtv.org</a><br>
<a href="http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users" target="_blank">http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users</a><br>
<br></blockquote></div><br>