[mythtv-users] fixes branch: can't kill mythbackend

[George Mari]

Steven Adeff wrote:
> On Mon, Dec 1, 2008 at 9:09 PM, Steven Adeff <adeffs.mythtv at gmail.com> wrote:
>> On Wed, Nov 19, 2008 at 9:07 AM, Steven Adeff <adeffs.mythtv at gmail.com> wrote:
>>> On Wed, Nov 19, 2008 at 8:34 AM, George Mari
>>> <george_mythusers at mari1938.org> wrote:
>>>> Steven Adeff wrote:
>>>>> lately mythbackend has gone rogue on me. it sits the computer at 100%
>>>>> on the wa meter in top and doesn't respond to any of the frontends, as
>>>>> well I can't kill -9 it, I *have* to reboot. I have no idea why it
>>>>> gets like this or why it's only been lately that this happens, but
>>>>> being that it forces me to reboot the computer manually I then have to
>>>>> wait for my RAID array to resync which is not fun.
>>>>>
>>>>> anyone have any ideas?
>>>>> thanks!
>>>>>
>>>> If it's 100% on wa, that means its waiting for some IO operation to
>>>> complete.  Use lsof to see what files that PID has open.
>>>>
>>>> Any messages in /var/log/messages or the mythbackend log?
>>>>
>>>> Does it only happen while it's recording, or playing back, or doing
>>>> nothing, all of the above?
>>> Ah, I tried fuser but did not think of using lsof... damnit.
>>>
>>> I can't tell what it's doing other than it stops writing to the log
>>> and puts a lock on my RAID array such that I can't stop it. I have not
>>> noticed it happen during playback, so far it has only occured when no
>>> frontend is streaming anything (they may be on, but just at a menu),
>>> and at least the last few times mythcommflag had two processes running
>>> (which is what I have it capped to).
>>>
>>> I'll try lsof next time it happens. If it is a file operation on the
>>> RAID array, is there a way to see why it cannot complete the IO
>>> operation?
>>>
>>> thanks!
>>
>> ok, finally occurred again, I can't open the log since it's on the
>> same RAID array as I seem to be having problems with.
>> I did run lsof and have a huge list, but don't really know what I'm looking for?
>>
>> Thanks for the help!
> 
> on a whim I "looked" at all the files being accessed on the RAID array
> and it turns out I can't even do an 'ls' of the log directory which
> makes me thing perhaps it could be myth having an issue writing to the
> log file? perhaps its having an issue with the log rolling over?
> 
> Thanks!

I ran into the exact same problem over the last couple of weeks on a 
different machine I have with a RAID5 array, and I think it was the 
kernel.  I was in the process of yum upgrading from FC6 to FC9, and left 
   the machine on FC7 for a while.  It was during these few days that it 
was on FC7 that I noticed the RAID5 array just hanging every once in a 
while, when it was being accessed.

I ended up going to FC8, then FC9, and the problem went away.

But I had the exact same symptoms - any process accessing the array just 
hung - couldn't even do a directory listing, couldn't even shutdown the 
machine cleanly, which ended up in needing to re-sync the array upon 
re-boot.

You may want to see what version of the kernel you are on, and see if 
there is a newer one available for your distro.