[mythtv-users] Frontend locking up and unable to kill it.

[Allen Edwards]

On Thu, Jul 4, 2019 at 2:58 AM Stephen Worthington <stephen_agent at jsw.gen.nz>
wrote:

> On Wed, 3 Jul 2019 22:33:43 -0700, you wrote:
>
> >On Wed, Jul 3, 2019 at 9:27 PM Stephen Worthington <
> stephen_agent at jsw.gen.nz>
> >wrote:
> >
> >> On Wed, 3 Jul 2019 17:44:59 -0700, you wrote:
> >>
> >> >Twice now the frontend has frozen. If that were not bad enough, I am
> >> unable
> >> >to kill it. I have remotely tried to kill it and what I find is a
> status
> >> >message on the main screen saying the program has become unresponsive
> and
> >> >do I want to shut it down.  But with no visible cursor I cannot click
> on
> >> >"Yes" and it does not respond to "Y" from the keyboard. I did a "sudo
> >> >shutdown now -r" from the remote and the link was immediately cut but
> the
> >> >myth computer did not shut down. It is as if mythfrontend was blocking
> all
> >> >attempts to close it. I finally held the power button down long enough
> to
> >> >turn the computer off and when I restarted it, all was fine. There is
> just
> >> >a short section in the frontend log between when my wife started
> watching
> >> >her show and the freeze. No log entries after the freeze until the
> >> restart.
> >> >
> >> >Jul  3 07:33:33 NewMyth mythfrontend.real: mythfrontend[1555]: I
> >> >CoreContext tv_play.cpp:2532 (HandleStateChange)
> TV::HandleStateChange():
> >> >Changing from None to WatchingRecording
> >> >Jul  3 07:33:34 NewMyth mythfrontend.real: mythfrontend[1555]: I
> >> >CoreContext tv_play.cpp:2638 (HandleStateChange)
> TV::HandleStateChange():
> >> >Main UI disabled.
> >> >Jul  3 07:33:34 NewMyth mythfrontend.real: mythfrontend[1555]: I
> >> >CoreContext tv_play.cpp:424 (StartTV) TV::StartTV(): Entering main
> >> playback
> >> >loop.
> >> >Jul  3 07:33:34 NewMyth mythfrontend.real: mythfrontend[1555]: N
> >> >CoreContext mythplayer.cpp:2231 (PrebufferEnoughFrames) Player(f):
> Waited
> >> >101ms for video buffers AAAAAAAAAAAAAA
> >> >Jul  3 07:33:34 NewMyth mythfrontend.real: mythfrontend[1555]: N
> >> >CoreContext mythplayer.cpp:2231 (PrebufferEnoughFrames) Player(f):
> Waited
> >> >203ms for video buffers AAAAAAAAAAAAAA
> >> >Jul  3 07:33:34 NewMyth mythfrontend.real: mythfrontend[1555]: N
> >> >CoreContext mythplayer.cpp:2231 (PrebufferEnoughFrames) Player(f):
> Waited
> >> >305ms for video buffers LAAAAAAAAAAAAA
> >> >Jul  3 07:33:34 NewMyth mythfrontend.real: mythfrontend[1555]: I
> >> >CoreContext mythrender_vdpau.cpp:615 (CheckOutputSurfaces) VDPAU:
> Added 2
> >> >output surfaces (total 4, max 4)
> >> >Jul  3 07:33:34 NewMyth mythfrontend.real: mythfrontend[1555]: N
> >> >CoreContext mythplayer.cpp:2231 (PrebufferEnoughFrames) Player(f):
> Waited
> >> >102ms for video buffers dLAAAAAAAAAAAA
> >> >Jul  3 07:35:29 NewMyth mythfrontend.real: mythfrontend[1555]: E
> Decoder
> >> >mythplayer.cpp:3504 (DecoderGetFrame) Player(f): Decoder timed out
> waiting
> >> >for free video buffers.
> >> >
> >> >Nothing here that means anything to me that tells me what happened or
> what
> >> >to do.
> >> >
> >> >Ideally someone can help figure out what is causing the freezes but
> short
> >> >of that, how can I shutdown the system?
> >> >
> >> > What I tried:
> >> >sudo kill 1555
> >> >sudo kill -9 1555
> >> > sudo killall  mythfrontend.re
> >> > sudo shutdown now -r
> >> >
> >> >Nothing  above would kill the frontend.
> >> >
> >> >Allen
> >>
> >> That sounds like something else other than mythfrontend was locked up,
> >> like the kernel.  Check syslog.   When that happens, you can get a
> >> complete lockup, or it can just make the system very slow, where it
> >> takes an hour or two to shutdown.  The latter usually happens when one
> >> or more cores of your CPU are locked up or "stalled" by the software
> >> running on them.
> >>
> >> What distro is it?  If it is Ubuntu, some of the "magic" keys are
> >> enabled, so that in such a situation, you may be able to get the
> >> filesystem shut down safely and force a reboot.  Try holding down the
> >> Alt key, and the SysReq key (PrintScrn if there is no marked SysReq
> >> key), and then with both those keys down, hit and release all these
> >> keys in sequence, slowly, with a second or two between keys: REISUB.
> >>
> >> See this page for what all those keys do:
> >>
> >>
> >>
> http://www.ubuntugeek.com/how-to-use-magic-system-request-keys-in-ubuntu-linux.html
> >>
> >> Note that the information on that page is for a very old version of
> >> Ubuntu.  The O key (shutdown) does not work for some reason in modern
> >> Ubuntus, so you need to use B and have it reboot, and then you can
> >> shut it down when the BIOS display comes up if that is what you want.
> >>
> >> You did manage a remote login - did that take longer than usual to
> >> work?  Was it slow to respond to commands?
> >>
> >> Another option is to enable the systemd debug console:
> >>
> >> systemctl enable debug-shell.service
> >>
> >> That creates a logged-in root shell on Ctrl-Alt-F9 that comes up early
> >> in the boot process and does not go away until as late as possible on
> >> shutdown.  It allows you to use Ctrl-Alt-F9 to see what is preventing
> >> a shutdown by using "systemctl status" or "systemctl list-units", or
> >> any other commands.  However, it is a big security hole as it allows
> >> anyone who knows about it to use Ctrl-Alt-F9 to get root access while
> >> it is enabled.
> >>
> >> And remember that if you did not get the filesystem shut down properly
> >> before you powered off or rebooted the PC, you must do a full
> >> filesystem check (fsck -C -f) on all partitions that were mounted at
> >> the time.  The easiest way to do that is to have a separate small boot
> >> partition on the PC to do it from, so that you do not have the root
> >> partition locked.  Or you can use a compatible live DVD or USB stick
> >> to boot from - one that has the same or better level of filesystem
> >> drivers.  Keep repeating the fsck -C -f command for each partition
> >> until there are no more errors reported.
> >> _______________________________________________
> >>
> >
> >Thanks for the reply. I looked at the syslog and other than I am no expert
> >on what is normal and what is not the best I can tell the system was
> >running just fine except mythfrontend was locked. When I logged in
> remotely
> >it seemed normal. I ran "top" and was able to see normal sort of readouts
> >which is how I found the PID of mythfrontend. I ran the sudo kill 1555 and
> >it did nothing.  PID 1555 was still running.  The UI on the local display
> >(I have two displays, a TV and a LCD display) looked to be working
> normally
> >except there was no cursor. But I could more the invisible cursor over to
> >the menu area and click and get drop downs and such. From syslog it looks
> >like the system ran a daily cron job after mythfrontend crashed. It seemed
> >to terminate normally.I can see my remote login in syslog and I can see
> >when that ended, which was when I did the shutdown.
>
> So you were able to do all the normal things you want on the PC with
> no sign of slowdown.  Only mythfrontend seemed to be locked up.
> Mythfrontend, like a number of full screen programs, suppresses the
> mouse cursor when over its window, so that is normal.
>
> When killing mythfrontend with a normal kill (no -9), you have to do
> it twice.  That seems to be a result of the multithreading of
> mythfrontend and is probably a bug, but it has been like that for a
> long time.  The normal kill command (kill -15 or kill -SIGTERM) is
> only a request to a program for it to terminate itself.  If it has
> installed a handler for that signal (which mythfrontend does), the
> program can ignore or change what a SIGTERM does.  So next time
> mythfrontend locks up, try running the same normal kill <pid> command
> twice.
>
> However, kill -9 should always work.  A kill -9 or kill -SIGKILL is
> done by the operating system from outside the program.  The program
> has no control of it.  The only time it does not work is if the
> program is waiting on some external condition (say in a driver) and
> that is not able to be undone by a SIGKILL for some reason.  Then you
> can wind up with a dead program that does not exit.  I consider this
> to be a bug in the kernel that this can happen, but it is possible. It
> can result in the system taking a very long time to shut down if you
> try that.  But normally, after one or more long timeouts, the shutdown
> should succeed.  Using a Ctrl-Alt-F9 debug console you should be able
> to tell it this is what is happening.
>
> It is possible that your having done only one kill -SIGTERM and that
> having left mythfrontend only partially shut down made kill -9 fail
> and caused the shutdown problems.  I have never tried that to find
> out.
>
> >I find this set of entries odd.  It starts with the stopping of my remote
> >session (dad) and there is nothing of the same sime sequence above that
> one
> >(well there is one).  What is odd is that the time stamp on the second to
> >last one before the start is out of time sequence.
> >
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopped Stop ureadahead data
> collection
> >45s after completed startup.
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopping Session c1 of user dad.
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopping ACPI event daemon...
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopping RealtimeKit Scheduling Policy
> >Service...
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopped target Timers.
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopped Daily Cleanup of Temporary
> >Directories.
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopping Session 596 of user dad.
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopping Disk Manager...
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopped target Sound Card.
> >Jul  3 07:51:36 NewMyth systemd[1]: Stopping User Manager for UID 1000...
> >Jul  3 07:35:06 NewMyth rsyslogd: message repeated 8 times: [ [origin
> >software="rsyslogd" swVersion="8.16.0" x-pid="777" x-info="
> >http://www.rsyslog.com"] rsyslogd was HU
> >Ped]
> >Jul  3 07:51:38 NewMyth rsyslogd: [origin software="rsyslogd"
> >swVersion="8.16.0" x-pid="777" x-info="http://www.rsyslog.com"] exiting
> on
> >signal 15.
> >Jul  3 07:55:21 NewMyth rsyslogd: [origin software="rsyslogd"
> >swVersion="8.16.0" x-pid="821" x-info="http://www.rsyslog.com"] start
>
> I agree that this looks odd, but I think it is an artifact of how
> rsyslogd handles repeats of the same message.  I would have to dig
> into the rsyslog documentation to be sure and it is not worth that as
> I am pretty sure it is irrelevant.
>
> >I am running the latest Mythbuntu
> >Ubuntu 16.04.1 LTS
>
> It is worth upgrading to Ubuntu 18.04 as the updates to systemd and
> how startup and shutdown work have made it quite a bit faster and have
> also got rid of a number of problems with shutdown that made 16.04
> much more likely to lockup while shutting down.  And it is much faster
> shutting down if it has a locked up dead program.
> _______________________________________________
>
> Here is what I did:
top (to find the PID)
sudo kill 1555
top (still running)
sudo kill 1555
top (still running)
sudo kill -9 1555
ps aux | grep mythfrontend (still running)
sudo killall  mythfrontend.re
ps aux | grep mythfrontend (still running)
I believe it was at this point I went downstairs and found the popup asking
if I really wanted to kill the program but with no cursor I was unable to
randomly land on the "Yes" area and it did not respond to a "Y" from the
keyboard. The UI was still interactive and I did a right click on the
mythfrontend tack on the taskbar and said close. The computer was still
acting normally at this time except no cursor. I then went back upstairs to
my remote session.
sudo shutdown now -r
Then I went downstairs to the computer and pressed the power button, which
normally does a graceful shutdown. Nothing happened so then I held the
power button down and the computer turned off.

So as you can see, I did kill PID twice.

I was under the impression that Ubuntu 18.04 would break my LIRC code and
become another project to get the system working again. I ran Mythbuntu
8.04 until just this year and never had the problems I have had with the
latest install. I thought I had it all fixed but then these two lock ups
happened. Very frustrating.

Allen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mythtv.org/pipermail/mythtv-users/attachments/20190704/803fc73a/attachment.html>