[mythtv] Schema updates

[f-myth-users at media.mit.edu]

    > Date: Sun, 4 Mar 2007 13:43:21 -0500
    > From: Isaac Richards <ijr at case.edu>

[Some design questions appear below, where relevant.]

    > On Sunday 04 March 2007 12:02:24 am f-myth-users at media.mit.edu wrote:
    > >     > Date: Sat, 3 Mar 2007 22:38:50 -0500
    > >     > From: Isaac Richards <ijr at case.edu>
    > >     >
    > >     > > That's part of why I don't understand the reluctance to making DB
    > >     > > schema upgrades NOT happen AUTOMATICALLY from a FRONTEND.
    > >     >
    > >     > There are users that don't use the tv recording functionality, and
    > >     > thus don't ever have the backend running, or ever run mythtv-setup. 
    > >     > That's reason enough for me to keep things as they are for now.
    > >
    > > Isn't this a tiny percentage of Myth users?  Couldn't they set the
    > > "dangerous" flag?  (Assuming we therefore -do- let frontends update;
    > > another idea would be "this is a frontend-ONLY installation", which
    > > is semantically cleaner and allows updates 'cause we know there's no
    > > MBE.  Of course, multiple frontends get to fight it out among themselves.)

    > So, your solution would be to only fix the problem for you, and leave people 
    > with other types of configurations out?

No, my solution was to attempt to fix this for the people who post
better than once a month saying, "Damn!  I accidentally ugraded my
DB!"  -My- problem is solved (though not very well) by firewalling
my MBE so it can't talk to any FE/SBE I don't control, and by being
very careful about what versions I run where.  It may also be solved
by Chris's "9999 hack", which was something new to come out of the
discussion (this time around) and which I appreciate him mentioning.

If "one trivial question you answer once and never revisit in the
entire history of running Myth, even after changing versions" is
"leaving people out", then explain the rest of the configuration
process...

I must admit, I was -really- surprised when you started talking about
FE-only systems that never see a backend; it took me a few moments to
even figure out what you were talking about.  It wasn't my impression
-at all- that Myth was designed as a standalone video player using
content imported from elsewhere---I mean, sure, you can use a FE to do
so, but why not just use mplayer?  It seems to me, from reading the
documentation, the architecture, and all the discussion I see on the
lists, that it's vastly more common to use it to -record- stuff you'd
like to see later, and that requires an MBE somewhere.

    >   				     Adding additional manual actions for 
    > people who obviously want the simplest setup doesn't really make sense to me.

Okay, then what -would- you suggest?  You seem to have set up an
impossibly contradictory set of requirements:
(a) No additional manual actions, even to say, "Never ask me again."
(b) No queries about whether to proceed with an update.

The two of those together imply "we can't ever ask people anything,
and we can't ever let them set a switch to avoid being asked."  How,
then, do we prevent an accidental upgrade?  You tell me.  I'm baffled.

For starters, is there some way a "standalone FE" (or whatever you
call "running a FE when you've never ever run an MBE or mythtv-setup")
can -tell- it's in that situation?  If so, it can, in essence, never
ask about upgrading the DB and can act as if my "dangerous" switch is
set all the time.  The only way -I- see that it can know this is "I
don't happen to be connected to an MBE right now", which seems fragile.
[For example, the first time somebody runs a non-standalone FE when
the MBE happens to be crashed or down, the FE will update the DB and
the MBE will be screwed when it comes back up.  Yucko.]

    > > Otherwise, it seems we're allowing the vast majority of users to screw
    > > themselves with simple mistakes to avoid asking a tiny minority to set
    > > a flag just once for their unusual usage pattern and forget it forever
    > > after.

    > The vast majority of users use packages and have single machine setups, and so 
    > don't tend to have this problem. =)

But there are a ton of users with multiple-machine setups; that is,
after all, one of Myth's greatest strengths & features.  EVEN IF they
use packages, what prevents someone from installing a too-new package
on a FE and thereby screwing his MBE?  -That- is what I'm trying to
solve here.  (Or, alternatively, someone simply plugs in a new FE
[visiting laptop, new machine, whatever] and forgets/doesn't know that
the FE is running a newer version---wham, the MBE gets irreversibly
updated, forcing updates to all -other- FE's/SBE's.)

Do we actually -have- any statistics at all on:
(a) FE -only- installs?
(b) Single-machine installs?
(c) Multiple-machine installs?

[Seems like a ton of traffic to the -users list is people w/FE's on
laptops or something; those couldn't possibly also be MBE's in most
cases.  But it's all anecdotal and horribly sample-biased.]

    > >     > It all becomes a non-issue if we move to an embedded db, too - the
    > >     > database (server/backend/whatever) itself will be the only thing
    > >     > upgrading the schema, then.
    > >
    > > What about the frontend-only users above?  Are they talking to the
    > > embedded DB and hence running a backend -anyway- in the new regime?
    > > [I thought the embedded DB was architecturally identical to sucking
    > > MySQL into the backend.  Apparently that's not the case if frontend-
    > > only users currently don't talk to a backend at all, but will have to
    > > with the EDB.  And I still don't understand how it prevents, e.g., a
    > > -slave- backend from updating an MBE, or a frontend from doing so,
    > > etc etc.  It seems that EDB is completely orthogonal to the whole
    > > discussion, but I obviously don't know what you have in mind here.
    > > Unless you're saying, "Once we go to EDB, FE's and SBE's will no
    > > longer -need- to update the MBE because we'll always make sure the MBE
    > > can speak in a backwards-compatible manner to them", which is a pretty
    > > big change in paradigm and, again, doesn't seem necessarily related to
    > > whether the database is MySQL or EDB.]

    > This has been discussed before, but a rough outline is:

    > - separate server app to handle the db access.  Owns the schema, not the 
    > fe/be.  
    > - autodiscoverable, with as minimal configuration as possible for initial 
    > setup/etc.
    > - add various canned queries to lessen impact of a direct mysql dependency on 
    > slim clients.  Slowly move at least some queries directly from the 
    > frontend/backend to use canned queries as well (ie, takes care of at least 
    > some backwards compatibility issues)

    > Basically, among other benefits, yes, it will completely make this issue go 
    > away.  Unless you 'accidently' stop your mythdbserver or whatever it's called 
    > and run a new version, I guess. =)

Hmm, interesting.  This sounds great, but it leaves me a little
confused, because so many changes to the way myth apps work (FE,
SBE, MBE, etc) seem to require DB schema changes.  It seems like
the "server app that owns the schema" [I don't have a good name
for this piece] would either have to get updated anyway, or the
way that FE/SBE/MBE development keeps revving the schema must
change.  I'm assuming you have a plan worked out for how this
needs to work; I'd be very interested in reading it.  Is there
a pointer somewhere to a design document or a set of threads on
the -dev list or something?

    > > And it seems like it'd be a tiny amount of work now to protect
    > > people, even if at some indeterminate time in the future (a year?
    > > more?) we go to an embedded DB.  No doubt there'll be plenty of other
    > > code thrown away at that point, anyway.

    > Why would code be thrown away?  I don't see the need for that, unless we start 
    > removing functionality or something.

I was guessing that some of the hair for auto-updating the DB schema
that's currently in the FE & SBE (at least) would become redundant,
plus that a change to an embedded DB would obsolete at least
-something- that was originally designed for MySQL.  But it was
just a guess.

    > > In any event, it sounds like you're saying, "No patch to implement any
    > > functionality to prevent users from screwing themselves (or being
    > > screwed by a rogue frontend) due to a DB update will be accepted."
    > >
    > > Is this true?

    > Where exactly did I say that?  If something's implemented sanely, sure, it'll 
    > be accepted.  If it adds extra work for most people, though, then it wouldn't 
    > be something I'd consider sane.

I was reading between the lines---see my question above re a
conflicting set of requirements.  I don't see any way to implement
what I'm talking about in a way you'd consider "sane", because you
seem to have specified that it's not allowed to change the behavior
enough to allow the protection to be implemented.  If I've actually
misunderstood the requirements, great!  It means there's some room to
actually make this happen, but at the moment I don't see what that
room can come from.

So I'd be very interested in seeing if we can hash out what the
requirements and constraints -are-, so it's -possible- to actually
make Myth safer in a way that "everyone" finds acceptable.

[For example, you say "most" people above, but I just flatly disbelieve
that "most" people run standalone FE's that never ever talk to an MBE.
Yet the reason you shot down my "ask or configure not to ask" proposal
was precisely because of the standalone-FE case, which can't be "most".
Do you see why I find this confusingly contradictory?]

    > > If so, those other people who said, "Write up a patch and submit it
    > > and it'll get looked at" are basically saying, "Go waste your time
    > > because there's no chance it'll be committed."  That's part of why
    > > I was trying to figure out whether it was worth -anybody's- time to
    > > even -try- to address this issue.  Sounds like you'd step on any such
    > > attempt.  True?

    > Uh, no.  I believe by this point, you've complained about this often and 
    > verbosely enough that you likely could have written a patch in less time than 
    > all the various emails you've written about it.

Certainly not; this discussion has only taken a couple of hours for
me, and it would take longer than that just to acquire and set up the
HW I'd need to make a sandboxed test environment that couldn't screw
up my current production setup.

[Remember also that this -started- as a 10-second toss-off comment
I made at the end of an unrelated piece of mail about fixing the
recorder not to hang on scheduler-locked tables; I hadn't expected
it to blow up into a giant discussion the way it has.]

    >     					     It's certainly wasted more 
    > developer time than reviewing and commenting (and possibly committing) a 
    > patch would have.

Well, sure, because in 10 seconds you would have said, "I disagree
that anything like this patch is needed" (as you seem to have said
above) and it would have taken no time at all to review & flush
without even reading a single line of code. :)

Okay, that's uncharitable.  But it's certainly -possible-.

    > Write a patch.

...but it's pretty obvious that the way -I- would have done it would
have been totally unacceptable to you and the patch would have been
rejected.

I'm a big believer in doing the design -first- and writing the code
-second-.  You seem to be implying that the only time design can
happen is after a patch has -already- been submitted, sits around
until somebody get around to looking at it, it's rejected, and -then-
it's permissable to say, "Hey, how should this have worked?"  I'd like
to short-circuit all but the last step.

    >         It'll work for you, and I can promise that it'll be considered 
    > for inclusion, at the very least.

Well, no, it won't work for me.  Why?  Because you insist (quite
reasonably) that patches be against current SVN.  Yet, if my MBE
ran current SVN, I wouldn't -need- the patch, because it would always
be more recent than any conceivable other component and hence wouldn't
be getting updated.  -And- it means that then I'd have this patch I'd
have to keep merging every time I pulled a new SVN.  So really, I'd
have to make this patch against 0.18.1, but there's no point at all in
my submitting such a patch, because you'd quite sensibly reject it
solely on the basis of not being against current SVN.  Catch-22.

The only time it would actually start working for me is when it got
included in a release, at which point I could run a released version
and start lagging SVN.  (This is also the time it could start working
for others, of course.)

    > >     > Really, though - the unapplied patch in question doesn't really do
    > >     > anything aside from remove some fairly low-overhead extraneous tests.
    > >     >  The valid portions of the guy's other patch (the freebsd stuff) got
    > >     > applied rather quickly.  The cosmetic one will get applied when
    > >     > someone has enough free time.  I believe that's rather reasonable.
    > >
    > > I guess I managed to miss a commit message.  I sure didn't miss him
    > > getting yelled at and threatened, though.  (I agree wrt assigning
    > > cosmetic patches low priority, but I wonder if he'll ever bother to
    > > submit anything else.  Guess we'll see.)

    > I think you're reading a different list than I am.

Or we're reading it in different ways.  That wouldn't surprise me.