[mythtv-users] /find_orphans.py crashes mythbackend

[Michael T. Dean]

On 09/02/2016 10:17 AM, Udo van den Heuvel wrote:
> On 02-09-16 16:03, Stuart Auchterlonie wrote:
>> How do you prove that without a backtrace, or at very least an
>> strace showing that the return code from a system call indicated
>> the out of file descriptor condition?
> Huh?
> If the backend (!) logs in /var/log/messages:
>
> /var/log/messages-20160821:Aug 30 08:57:27 recorder mythbackend:
> (process:1905): GLib-ERROR **: Creating pipes for GWakeup: Too many open
> files
> /var/log/messages-20160821:Aug 30 08:57:27 recorder mythbackend:
> (process:1905): GLib-ERROR **: Creating pipes for GWakeup: Too many open
> files
> /var/log/messages-20160821:Aug 30 08:57:27 recorder mythbackend:
> (process:1905): GLib-ERROR **: Creating pipes for GWakeup: Too many open
> files
> /var/log/messages-20160821:Sep  1 17:15:24 recorder mythbackend:
> (process:12476): GLib-ERROR **: Creating pipes for GWakeup: Too many
> open files
> /var/log/messages-20160821:Sep  1 17:20:51 recorder mythbackend:
> (process:28205): GLib-ERROR **: Creating pipes for GWakeup: Too many
> open files
> /var/log/messages-20160821:Sep  1 17:31:15 recorder mythbackend:
> (process:29226): GLib-ERROR **: Creating pipes for GWakeup: Too many
> open files
>
> Who am I to doubt it's utterings?
> You do doubt the backend.
>
> If you doubt the code itself, while more or less defending it by putting
> the work on my side, then what am I to think in this simple situation?
>
> That the code is not very stable? That the error messages cannot be
> trusted? That logic is not enough to debug this thing?

You know, antagonizing the one developer who's actually expressed an 
interest in helping/making MythTV code more robust against your (corner) 
case isn't likely to motivate him further.  (To say nothing of the 
incorrect idea you expressed about how setting up your system to allow 
generating a proper backtrace "pollute[s] the system with debug-info.")

> Well then.
> I did a few of the very same thing:
> I tried to delete 4000+ orphaned recordings via find_orphans.

That's a lot of files to be deleting at once--likely far more files than 
your system's limits allow a single user to open at once.  It's also not 
normal for a MythTV user to have 4000+ orphans, and it's also not normal 
for a user to try to delete 4000+ recordings (orphaned or not-orphaned) 
at a time.

> Every time the backend would stop being a process. Exit. Vanish. Go away.

No, it would "crash"--it was likely was killed by your OS because you or 
your distro maintainers have configured your system to limit file handle 
usage.

> Every time it logged the irrelevant messages that I pasted a few lines
> above this one.
> I do not see any relation between these things but I want them to go away.
> Maybe you want them to go away as well because another user might run
> into the same stuff and be not so handy with sql and a tiny bit of shell
> magic.


FWIW, your "handiness" with SQL probably left stuff in the DB related to 
the orphans.  There's a reason that find_orphans.py doesn't actually do 
any deletion at all--it just asks the backend to do it--because deletion 
from MythTV requires cleanup of data from multiple connected tables.  
But, whatever...

Anyway, what do you get from:

ulimit -Hn
ulimit -Sn
grep nofile /etc/security/limits.conf
  and
cat /proc/sys/fs/file-max
  or
sysctl fs.file-max

Note, too, that the ulimit -n will show the limit for the current shell 
and any processes started by that shell, so, really, to get the most 
useful information, you should find out what ulimit -n gives for the 
shell that starts mythbackend process.  But, in the absence of better 
information, the ulimit -n information run from a login shell (at 
minimum su - mythtv )

If you change the limits to allow the user running mythbackend to have 
sufficient files open for the task you're hoping to achieve (in other 
words, if you configure your system to allow it to do what you 
ask/expect of it), do you still get a crash?

Mike