[mythtv-users] Total system upgrade: Am I screwed?

[Michael T. Dean]

On 11/29/2009 08:14 PM, f-myth-users wrote:
>     > Date: Sun, 29 Nov 2009 13:22:56 -0500
>     > From: "Michael T. Dean"
>
>     > On 11/29/2009 09:50 AM, Richard Shaw wrote:
>     > > Here's the error:
>     > >
>     > > ERROR: Invalid database information file, stopped at
>     > > bin/mythconverg_restore.pl line 686.
>     > >
>     > > I used --partial_restore just to get the recordings and schedule info
>     > > but got the same error trying without that option as well. I'm
>     > > specifiying the full name of the backup file using tab completion so I
>     > > know I'm not mistyping it...
>
>     > But you're not specifying the command-line option that tells the script 
>     > you're specifying a filename.
>
>     >  - What you're doing:
>
>     > ./mythconverg_restore.pl file.sql.gz
>
>     > (which will never work) or
>
> I'd argue that's a usability bug right there.
>
> Having never actually looked at mythconverg_restore.pl until right
> this second, and having just made a pass through the very long output
> of the --help option [I'm looking at the source to the version in
> trunk], I can completely see why (a) he missed it in reading all of
> that, and (b) why "[options|database_information_file]" completely
> confused him.
>
> The misleading error message obviously didn't help, either.
>
> I understand why you have --directory and --filename options, but the
> vast, vast majority of Unix command-line tools typically expect a pile
> of (optional :) options, optionally or mandatorily followed by a filename.
> Some of them are pretty screwed up vis a vis this informal standard (dd
> and find spring immediately to mind, and _The Unix-Haters Handbook_ has
> some other great examples), but that's the typical behavior.
>
> I would opine that a nice way to fix this would be to recognize the
> situation where no options are present but it looks like a filename
> is (either because it has slashes in it, or because a slashless string
> seems to match a filename in the current working directory), and then
> just run with that, perhaps printing out a message saying something
> like "I'm guessing you meant --directory blah --filename biff").
>   

No.  Read the rest of the script.  Figure out what a database 
information file is--the thing he claimed to be passing.  Then you'll 
realize why the script--with only one non-option argument--assumes a 
database information file.  You'll also realize why having it accept a 
backup filename is ambiguous.

If the user uses the wrong format, they can get an error and fix it.  I 
will *not* screw up users' databases by attempting to read their minds.

You, after all, are the one who wants it to work like *nix tools.  And, 
*nix tools do what they're told to do, not, "Well, I think you might 
have wanted to ...".

I purposefully made the decision early on that if there's /any/ 
ambiguity, the scripts will fail rather than do bad things--especially 
the restore script.  Fail fast is a design principle of the restore script.

> If that's too dangerous, perhaps it could at least print out a guess,
> like "Did you mean to say $0 --directory blah --filename biff?",
> though that's not as friendly because -my- immediate reaction would
> be, "Yes, you stupid program, and why are you making me type all this
> in twice when you obviously figured it out the first time?"
>   

Thanks for a long response that suggests exactly what I told people I 
was going to do.

> (It's -especially- weird that the program -requires- you to break up a
> pathname into separate --directory and --filename args; virtually all
> other Unix programs expect a single pathname,

What should I do when the user has a backup resource file that specifies 
a directory.  The user then invokes the program:

./mythconverg_restore.pl --filename this_is_my_backup.sql.gz

which they used tab-autocomplete to find in the current directory.  
However, the directory specified in the backup resource file /also/ 
contains a same-named file?  Which do I restore?

Well, the "closer" one makes sense.  That would be the one in the 
current directory.  Ooops...  Now, just convincing someone to run the 
program when their current working directory is the wrong directory does 
the wrong thing.  That is bad security.

If I do the opposite--and /always/ use the directory in the backup 
resource file if the , then the user will have to know to verify that 
there are no files in the backup-resource-file-specified directory with 
the same name as the file in the current directory they're trying to 
restore.

I will *not* guess what the user wants.  They can resubmit the command.

Are the examples at 
http://www.mythtv.org/wiki/Database_Backup_and_Restore really so hard?

>  and requiring this both
> surprises the user and makes it harder to call from a script, which
> must separate the pieces first.  I know why you did it, but it's
> surprising.)
>   

I doubt that you did know why I did it.  You probably think I did it to 
make it easy to program, but I didn't.  I did it for the reason 
above--to /never/ have an ambiguous invocation that does something that 
may not be what the user wants.

Unlike you, I spent a great deal of time thinking about all the possible 
ways the script may be invoked and made a plan that prevents breaking 
someone's database by doing bad things.

(I know that sounds inflammatory, but honest to god, that's not my 
intention here.  I'm just trying to point out that there are usually 
other options -besides- "the developer is stupid and didn't think to 
make the script work in the most obvious way.")

Show of hands.  How many people here have run the backup and/or restore 
scripts more than 1000 times?  (I'm just guessing, but I seriously 
believe I'm beyond 2000.  With some of the tasks I was working on, I'd 
run them more than 50 times in an evening.)

> Remember, this is a program that should be very rarely used.

Exactly.  So when they do it wrong, they can re-invoke it correctly.

>   Forcing
> the user to have a steep learning curve, read a whole pile of options
> that are probably irrelevant to him, and then barfing on the obvious
> way to call it seems unfriendly---it's not as if the typical user
> should be running this often enough to remember how.
>
>     > For future reference, if you tell people what command you ran, they can 
>     > help you figure out what you did wrong
>
> Very true.
>
>     > 					     (even if they don't know the 
>     > script inside-out)...  The possibility exists--though I'm sure it's a 
>     > tiny possibility--that the problem lies not with the backup or with the 
>     > restore script...
>
> Yes.  Like maybe it's a usability bug and not the user's fault.
>
> (I know that sounds inflammatory, but honest to god, that's not my
> intention here.  I'm just trying to point out that there are usually
> other options -besides- "the user is stupid and it's all his fault.")
>
>     > Also, there /is/ a mythconverg_restore.pl --help so you can help 
>     > yourself.  Though it has a lot of info, the quick start is generally 
>     > enough.
>
> ...but it's buried -waaaay- far down a giant block of text,


The Quick Start section is purposefully at the bottom of the --help 
output so it's what's showing in the terminal and what's easily 
available with the scroll wheel.

As for "giant block of text", the backup and restore script --help 
output are each about the same length as the gzip manual page--and the 
vast majority of that is redundant information (where each database 
information file property has a corresponding argument whose description 
is a shortened version of the DB info file description).  And, if you 
actually take time to read the --help output, you'll find that they have 
a /lot/ of options because they do a /lot/ of things.

>  and it
> -still- doesn't actually have as examples the most common way you
> might want to be calling this script. 

Actually, it does.  The way the OP was using it was not a common way.

>  [Certainly, my quick skim
> of that section still didn't give me any info about calling it.]
>
>     > 	       I'll change it to do the quick start info for --help and have 
>     > an extended help with everything.
>
> One suggestion for the very top of the help might be something like
>
>   HERE ARE THE MOST LIKELY THINGS YOU'LL WANT TO DO:
>
>     $0 
>     $0 --directory /biff/boom
>     $0 --directory /biff/boom --filename bar
>
> and (especially if you don't intend to make the script DTRT if only a
> pathname is specified and nothing else)
>
>     Please note that $0 /some/path/to/file will -not- work.  You must
>     specify --directory and/or --filename options if you intend to choose
>     which backup to restore from.
>
>     > I'll also put an error message in there that tells people to fix their 
>     > command line when they specify an invalid database information file.
>
> I'm sure that's a good idea.
>
> [I'd be willing to make patches for all of the suggestions above,
> if that matters, though I won't be able to get to it in the next
> few days, and I'm sure it'd be about a 10-minute job for you.]

The proper way to restore the most recent backup is:

./mythconverg_restore.pl

I /don't/ think it's possible to make it any easier than that.  Again, 
read about how the script works ( 
http://www.mythtv.org/wiki/Database_Backup_and_Restore ) and you'll see 
that the approach the OP used was the hard approach.

And, even though you're running 0.18.1, you can still use the scripts.  
So please at least use them a while before complaining that they're not 
user-friendly.

Mike