[mythtv-users] MythFrontend out of memory problem
Roland Ernst
rcrernst at gmail.com
Fri May 20 15:21:48 UTC 2022
On Fri, May 20, 2022 at 5:05 PM Roland Ernst <rcrernst at gmail.com> wrote:
>
>
> On Fri, May 20, 2022 at 10:36 AM James Linder <jam at tigger.ws> wrote:
>
>>
>>
>> > On 20 May 2022, at 4:30 am, Kenneth Emerson <kenneth.emerson at gmail.com>
>> wrote:
>> >
>> >
>> >
>> > On Thu, May 19, 2022 at 12:19 PM Jay Foster <jayf0ster at roadrunner.com>
>> wrote:
>> > On 5/19/2022 10:03 AM, Kenneth Emerson wrote:
>> >>
>> >>
>> >> On Mon, Apr 11, 2022, 2:53 PM Kenneth Emerson <
>> kenneth.emerson at gmail.com> wrote:
>> >>
>> >>
>> >> On Mon, Apr 11, 2022, 1:04 PM Kenneth Emerson <
>> kenneth.emerson at gmail.com> wrote:
>> >>
>> >>
>> >> On Mon, Apr 11, 2022, 12:53 AM Stephen Worthington <
>> stephen_agent at jsw.gen.nz> wrote:
>> >> On Sun, 10 Apr 2022 23:00:17 -0500, you wrote:
>> >>
>> >>
>> >> >> Stephen:
>> >> >
>> >> >
>> >> >
>> >> >The system RAM is 8Gb. There is a 2Gb swap space. There are about
>> 3,000
>> >> >recordings. Here is the output from swap.sh:
>> >> >
>> >> >
>> >> >ken at mythtv:~$ ./swap.sh
>> >> >
>> >> >
>> >> >..............................
>> >> >
>> >> >Overall swap used: 798096 kB
>> >> >
>> >> >
>> >> >========================================
>> >> >
>> >> >kB pid name
>> >> >
>> >> >
>> >> >========================================
>> >> >
>> >> >404072 1408 mythfrontend.re
>> >> >
>> >> >77376 1365 xfdesktop
>> >> >
>> >> >42080 1277 xfwm4
>> >> >
>> >> >38504 1377 applet.py
>> >> >
>> >> >37120 1509 blueman-tray
>> >> >
>> >> >34504 1426 blueman-applet
>> >> >
>> >> >15624 1131 xfce4-session
>> >> >
>> >> >14224 1310 panel-1-whisker
>> >> >
>> >> >12480 1321 panel-10-pulsea
>> >> >
>> >> >12224 1317 panel-8-statusn
>> >> >
>> >> >11536 1314 panel-6-notific
>> >> >
>> >> >10552 1401 update-notifier
>> >> >
>> >> >10216 1411 nm-applet
>> >> >
>> >> >9408 1360 Thunar
>> >> >
>> >> >9336 1282 xfsettingsd
>> >> >
>> >> >8608 1313 panel-5-systray
>> >> >
>> >> >8576 1318 panel-9-power-m
>> >> >
>> >> >8528 1385 xfce4-power-man
>> >> >
>> >> >8240 1315 panel-7-indicat
>> >> >
>> >> >5840 1283 xfce4-panel
>> >> >
>> >> >3280 12076 bash
>> >> >
>> >> >3272 1382 xiccd
>> >> >
>> >> >3032 3791 bash
>> >> >
>> >> >2688 1123 systemd
>> >> >
>> >> >2640 1442 polkit-gnome-au
>> >> >
>> >> >1920 1130 pulseaudio
>> >> >
>> >> >1224 1437 agent
>> >> >
>> >> >504 1145 dbus-daemon
>> >> >
>> >> >248 3784 tmux: server
>> >> >
>> >> >240 1368 mythfrontend
>> >> >
>> >> >
>> >> >ken at mythtv:~$ free -m
>> >> >
>> >> >
>> >> > total used free shared
>> buff/cache
>> >> >available
>> >> >Mem: 7934 2964 202 7 4768
>> >> > 4668
>> >> >
>> >> >If it's meaningful the CPU is an Intel i5-2500K 3.3GHz
>> >> >
>> >> >Perhaps I am hitting some limit on number of recordings, but I don't
>> ever
>> >> >remember there being a (soft?) limit.
>> >> >The system is a bit old, but I would hope it's powerful enough. I
>> only use
>> >> >it as a media center with only MythTv as the main application.
>> >> >
>> >> >Regards,
>> >> >
>> >> >Ken Emerson
>> >>
>> >> With 8 Gibytes of RAM, it should be fine. Only 3000 recordings is not
>> >> a problem (I have 52880 on an 8 Gibyte frontend/backend system). There
>> >> is no fixed limit on recordings - the list can grow as big as your
>> >> system will allow, although things do slow down a bit when you have as
>> >> many as I do.
>> >>
>> >> It does look like the swap space is filling up though - I would
>> >> suggest increasing that to 8 Gibytes. But it is currently using
>> >> 798096 kiB, which is under half full so it is not obvious that it will
>> >> run out of swap.
>> >>
>> >> I have my system and swap partitions on a fast M.2 NVME SSD, which
>> >> helps with swapping.
>> >>
>> >> The free command gives you the total RAM and swap usage. This is my
>> >> system with mythfrontend showing my recordings list:
>> >>
>> >> root at mypvr:~# free -m
>> >> total used free shared buff/cache
>> >> available
>> >> Mem: 7902 2829 120 23 4952
>> >> 4752
>> >> Swap: 10239 1556 8683
>> >>
>> >> (Sorry about the line wrapping - my email client does that and it is
>> >> not able to be turned off).
>> >>
>> >> So your frontend only system with only 3000 recordings should be just
>> >> fine. So my best guess is that something is triggering a bug that is
>> >> causing a memory leak. But the problem might be in another program,
>> >> rather than mythfrontend. The OOM system is known for killing the
>> >> largest RAM user, when the problem program that is suddenly using up
>> >> RAM is a different program.
>> >>
>> >> There was one old bug that caused the cache directory not to be
>> >> cleaned up properly, and mythfrontend would then load the cache and
>> >> run out of RAM. That was fixed a while ago, but it is possible that
>> >> you still somehow have a huge cache. So take a look at
>> >> $(HOME)/.mythtv/cache and see how big it is:
>> >>
>> >> root at mypvr:~# du -hcs /home/stephen/.mythtv/cache/
>> >> 376M /home/stephen/.mythtv/cache/
>> >> 376M total
>> >>
>> >> This mine now, but when the bug was happening, it was many gibytes. If
>> >> yours is too large, shut down mythfrontend and do:
>> >>
>> >> rm -r $(HOME)/.mythtv/cache
>> >>
>> >> Mythfrontend should then rebuild the cache as required.
>> >>
>> >> I would suggest upgrading to v32. That is the currently supported
>> >> version, so if you have a problem that needs the devs to fix it, you
>> >> will need to be on v32 for them to be interested. And it is vaguely
>> >> possible your problem may be fixed already in v32-fixes.
>> >>
>> >>
>> >> Stephen:
>> >> Unfortunately, I decided to take the plunge and upgrade to 32. Looked
>> simple; however, in stuck years ago right at the beginning running
>> mythtv-setup with "Failed to get schema upgrade lock". Can't go back or
>> forward at this point. I have a good DB backup just don't know what to do
>> with it.
>> >>
>> >> Ken Emerson
>> >>
>> >> Update:
>> >>
>> >> I changed the hostname in the setup dialog from localhost to mythtv
>> (the actual hostname) and then rebooted. The master backend ran after the
>> reboot and asked me if I wanted to upgrade the schema; answered yes and all
>> I good now; upgraded from 1361 to 1376. Don't fully understand the problem.
>> There were no clues in the mythtv-setup log. Perhaps the backend takes care
>> of all that now?
>> >>
>> >> Anyway, sorry for all the cruft.
>> >>
>> >> BTW: 681Mb in .mythtv/cache
>> >> Will have to wait several weeks to see if the OOM kill problem is gone.
>> >>
>> >> Thanks for your time and help.
>> >>
>> >> Regards,
>> >> Ken Emerson
>> >>
>> >> It's been a month now, and I haven't seen a re-occurance of the OOM
>> killer taking out the frontend; however, by keeping an eye out on resources
>> I notice that memory is still tight. I've updated myth to version 32 and
>> did maintenance on the cache disk space as suggested. This morning when a
>> OTA program was recording with no one watching any recordings, the
>> frontend.real task (using top) was using the majority of resources, both
>> cpu and memory. CPU was 47-80% range with memory 80-90% range. Along with
>> the frontend, mythcommflag was also running using minimal resources as was
>> the backend. frontend.real was always first in listing using top looking at
>> both cpu and memory.
>> >>
>> >> Using free, there was only about 60mb memory available (system has 8
>> gb installed). I don't understand what the frontend is doing when there is
>> nothing being played back: just sitting at the recordings menu. I also have
>> a 2gb swap partition that is also more than 90% consumed.
>> >>
>> >> FYI: This is a dedicated frontend/backend server with no other
>> services running other than what myth requires.
>> >>
>> >> I've ordered and am now waiting for an additional 8gb of memory but
>> would appreciate any enlightenment on what the frontend is busy doing.
>> >>
>> >> Regards,
>> >>
>> >> Ken Emerson
>> >>
>> >>
>> >> _______________________________________________
>> >> mythtv-users mailing list
>> >>
>> >> mythtv-users at mythtv.org
>> >> http://lists.mythtv.org/mailman/listinfo/mythtv-users
>> >> http://wiki.mythtv.org/Mailing_List_etiquette
>> >>
>> >> MythTV Forums:
>> >> https://forum.mythtv.org
>> > My combined FE/BE also has 8GB or RAM and has no issues (running
>> 0.28). Something to keep in mind when looking at how much RAM is 'free'
>> (/proc/meminfo) is that the linux kernel will use any available RAM for
>> caching, so the reported free RAM is usually low. This is not indicative
>> of a problem (i.e., memory leak).
>> > Jay
>> >
>> > Thanks for the input. The problem seems to be intermittent. I'm
>> looking at the memory now with two programs recording (just for a test).
>> The results from the 'free' command are:
>> >
>> > total used free shared buff/cache
>> available
>> > Mem: 7.7Gi 6.8Gi 130Mi 5.0Mi 884Mi
>> 752Mi
>> > Swap: 2.0Gi 2.0Gi 0B
>> >
>> > With the exception of the swap numbers, this seems reasonable. Maybe
>> everything is fine. Watching the numbers it appears the kernel is doing
>> what it's supposed to do. I still don't understand why swap is getting
>> used. I'll see what happens when I add the addtional 8Gb of RAM next week.
>>
>> The trouble is when us mere mortals try to understand ,,,
>> If you have oodles of ram and some of it is untouched (as opposed to
>> unused) then the kernel will swap that out so that you’ve got more cache
>> that is *touched* and there are *signicant* arguments on ram vs swap with
>> tuning available.
>> So you might go from
>>
>> 4G used 4G free swap 0
>> to
>> 2G used 6G free swap 2G
>>
>> eg consider these
>>
>> 'echo 'vm.swappiness=1' > /etc/sysctl.d/99-sysctl.conf'
>> 'echo 'vm.vfs_cache_pressure=50' >> /etc/sysctl.d/99-sysctl.conf’
>>
>> James
>>
>>
> Kenneth,
> please state the theme you are using (including version).
> Do you use plugins from MythTV?
> Is mythwelcome part of the game?
> How often does the frontend restart because of an error?
> What is the output of
> dpkg -l | grep -i mythtv
>
> Roland
>
>
Sorry, I had a thinko:
What is the output of
dpkg -l | grep -i myth
This is the better way to check.
Roland
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mythtv.org/pipermail/mythtv-users/attachments/20220520/02c809d9/attachment.htm>
More information about the mythtv-users
mailing list