[mythtv] Storage Groups functionality

f-myth-users@media.mit.edu f-myth-users at media.mit.edu
Tue Feb 14 09:59:24 UTC 2006

    > Date: Mon, 6 Feb 2006 23:33:46 -0500 (EST)
    > From: "Chris Pinkham" <cpinkham at bc2va.org>

[Prolonging this one more message since it might give somebody an idea,
and because "Paul Harrison" <mythtv at dsl.pipex.com> and---I think---at
least one other person in this thread have also said they're talking
about making some sort of offlining tool.]

    > > One unsolved problem:  It'd be really, really nice to be able to take
    > > an archived program (which has presumably been deleted from "recorded"
    > > because it's been deleted from the UI) and reinsert all the relevant
    > > table data again once the video bits have been restored to disk.  Is
    > > it sufficient to simply save the relevant row from the "recorded"
    > > table and reinsert it?  (I'm assuming that mythcommflag --rebuild will

    > IMHO, the archiving discussion is drifting off-topic, but why even delete
    > the data from the database at all?  All you need to do is flag the
    > recording so that it is not shown in the recordings list if it is
    > archived.  When an 'archive' was restored (by either mounting a disk
    > or external drive, etc., then you just have to mark the archived
    > recordings on that media as available and they show up in the Watch
    > Recordings screen.

If you're moving recordings offline, but will later be hauling them
back online, the one table that it might make sense to move offline
with them seems to be recordedmarkup.  That table currently consumes
75% of all the space my DB consumes, and seems to correspond to about
1 megabyte per hour of recorded video.  (If you export the data and
compress it, it's about 60K or so.)  A meg per hour doesn't seem like
much, but if the whole idea of moving things offline is because you
save a lot of things, it can bloat the database fairly effectively---
a terabyte of online storage means a gigabyte of recordedmarkup in the
DB, and lots of people -have- a terabyte (or sizeable fraction thereof)
online, so it seems safe to assume that offline storers are probably
storing 10x that or more---and now we're talking 10GB of database and
a fairly substantial figure.

(I used to think, "just regenerate recordedmarkup with mythcommflag
--rebuild when the recording is put back online", but recent
discussions indicate that this only works for ivtv cards until 19, and
worse, I think you'd have to rerun commercial-flagging and get all the
cutpoints accurate again, which would use a lot of runtime and human
time.  Easier to just store a 60K compressed file that's associated
with the video when you migrate it offline and stick it back in the
table when it comes back.  Or maybe just store the -compressed-
version in the DB?  Then, 10TB of recordings is 600meg of space in
mysql, which isn't too horrible, though still, even that space is
wasted w/o the original video it refers to.  I'm a little surprised
that mysql isn't already using a much more compact format for this
data, but my naive analysis of this (export whole table, check its
size, and compare to the size of everything under /var/lib/mysql---and
they're about equal!) seems to indicate that this might not be the
case.)  [Hmm---though I suppose, if you write out this table w/a
recording, you'd better also write out DBSchemaVer with it in case the
schema changes in the future in an incompatible way, so you can know
how to fix up what you read in...  ick.]

And until the UI -does- have an "archived" flag (and probably a way of
saying "and don't show them to me, either" so you don't drown in them),
I'm going to have to actually delete offlined recordings, which means
I'm back to my original question of, "And what tables must I dump to
make later reinsertion work?"  Since nobody answered that one, I'll
try to figure out all the DB calls made when a recording is deleted,
which should tell me which tables to look at more closely.  (Or I'll
export mythconverg, delete a recording, export again, and diff---that
could be a lot faster, easier, and less error-prone... :)

P.S.  My question about the "recorded" table above might be moot, since
I reported earlier that only shows with no pre/postroll were winding up
in there (at least in 18; nobody said anything about whether this was
a bug or a feature, but it sure seems strange to me), and yet recordings
not being in there doesn't -seem- to have affected anything, so it's
not clear to me why this table is there until I take a closer look at
the sources.  And clearly I'm not going to touch oldrecorded; that's
keeping track of where any offline files went, via some added columns.

More information about the mythtv-dev mailing list