[mythtv-users] delaying backend startup until the network is up

[Stephen Worthington]

On Tue, 18 Dec 2018 16:43:35 +0100, you wrote:

>On 17/12/2018 22:31, James Abernathy wrote:
>> "full-internet.service"
>> 
>> Description=Wait for full internet service to be available including DNS
>> After=NetworkManager-wait-online.service
>> 
>> [Service]
>> Type=simple
>> ExecStartPre=/home/jim/delay-backend-till-network-up.py
>> ExecStart=/bin/true
>> 
>> [Install]
>> WantedBy=multi-user.target
>
>If there are ever any problems with your network configuration you won't
>be able to log in to fix them. Not even on the console - there will
>simply not be a login prompt.
>
>Same problem is caused by the mythtv-backend.service file: if there is a
>problem of any kind (e.g. with a tuner that's been added to After= and
>Wants= lines) then the system doesn't complete booting and you can't log
>in to troubleshoot. Truly horrendous design.
>
>I would like multi-user.target to be reached ASAP and with as few
>dependencies as possible. It can take 3 minutes for all of my pvrusb2
>tuners to be initialised, and I don't want to wait for them to be able
>to log in.
>
>Thx, Jan

There are timeouts on most things in systemd, so it will eventually
start up anyway.  Eventually can be a very long time though - many
minutes.  But in 16.04, Ubuntu seem to have their systemd settings
screwed up a bit, so even when it eventually starts up, there can be
things that do not work (sometimes including networking), and usually
it will not reach a workable GUI screen.  In particular, the startup
does not work well when several of my EXT4 partitions need an
automatic fsck at startup - fsck takes quite a while and you get no
access to see what the boot is waiting for until it they are all
complete.  And after they complete, some other timeouts seem to have
expired and critical things do not get started at all.

So if the boot gets screwed up by having to wait too long for things,
just wait (for many minutes) until it does finish booting as much as
it can.  Then you just go to the physical PC, do Ctrl-Alt-F1 to get a
console and work from there to fix things.  The usual thing in 16.04
that caused a bad problem like that (other than fsck), is having a
partition that was required on boot, but was not mounting (eg required
its filestructure to be fixed, or is currently unplugged).  You can
edit fstab to not require the partition, then reboot.

In 18.04, systemd works much better.  Ubuntu have put a fair bit of
effort into making systemd work better, and they seem to have fixed
the major problems.  Booting is significantly faster and makes more
use of my NVMe SSD's ability to support parallel activity.  So if you
have not already upgraded to 18.04 and you hate how systemd works in
16.04, I do recommend upgrading.

In 16.04, I did this:

sudo systemctl enable debug-shell.service

That enables a root console on Ctrl-Alt-F9 that is up very early in
the booting process, and allowed me to see what systemd was doing when
things went wrong.  It is also a very big security hole, as it allows
anyone who knows about it to get root access.  If you have not enabled
it, you can before booting by temporarily adding:

systemd.debug-shell=1

to the kernel boot line from the grub menu before you let it boot.

See here for how to debug systemd problems:

https://freedesktop.org/wiki/Software/systemd/Debugging/

Systemd is very different and still being developed to fix some of the
problems it has introduced.  But I really like how easy it makes it to
create a new daemon program.  Instead of having to mess around writing
a massive, complicated (and time consuming) /etc/init.d script to get
a daemon to work, and adding special daemonising code to the daemon
itself and getting that to work, you can just write your daemon as
though it was a normal program and let the systemd unit file handle
all the daemonising stuff.  No more having to fork and mess around
with PID files and the like.