[mythtv-users] Automatically restart backend on failure? (Gentoo)

[Richard Freeman]

Dan Wilga wrote:
> The problem with cron-based solutions to this problem is the lag between
> the time of the crash and when cron next runs. If MBA had been recording
> when it died, you can miss a big part of the program.
> 
> Instead, here's a daemon I wrote to start MBA and monitor how it ends.
> If it ends gracefully (i.e.: it got a TERM signal), then this script
> exits. If not, MBA is immediately relaunched.
> 

Gentoo has an ebuild for monit - I would recommend checking it out.  The
following monitrc snippet works for me:

check process mysqld with pidfile /var/run/mysqld/mysqld.pid
  start program = "/usr/local/sbin/kickservice start mysql"
  stop program = "/usr/local/sbin/kickservice stop mysql kill mysqld"

  if cpu is greater than 60% for 2 cycles then alert
  if cpu > 80% for 15 cycles then restart
  if mem > 1000.0 MB for 5 cycles then restart
  if children > 250 then restart

  if 3 restarts within 5 cycles then timeout
  group server

check process mythbackend with pidfile /var/run/mythbackend.pid
  start program = "/usr/local/sbin/kickservice start mythbackend"
  stop program =  "/usr/local/sbin/kickservice stop mythbackend kill
mythbackend"
  if failed host localhost port 6544 proto http with timeout 60 seconds
then restart
  if mem > 1000.0 MB for 5 cycles then restart
  if children > 250 then restart
  if 5 restarts within 5 cycles then timeout
  depends on mysqld
  group server


kickservice script:

#!/bin/sh
# kickservice by Eric Radman
# Script used by monit to kill off and restart daemons

function stop {
  /etc/init.d/$1 pause
  /etc/init.d/$1 zap
  if [ "$2" == "kill" ]
  then
    /bin/killall $3
  fi
}

function start {
  /etc/init.d/$1 zap
  /etc/init.d/$1 start
}

case "$1" in
start)  start $2 $3 $4 ;;
stop )  stop $2 $3 $4 ;;
*    )  echo "Usage: kickservice [start|stop] [service] kill [daemon]" ;;
esac