[mythtv] Storage Groups functionality

[f-myth-users@media.mit.edu]

    > Date: Mon, 6 Feb 2006 20:09:16 -0000
    > From: "Paul Harrison" <mythtv at dsl.pipex.com>

    > Do you plan to make the scripts to do this public?

Sure, I'd be happy to make them available once they're reasonably
stable.  Some of them might be a bit too specialized for anyone else
to use, but you never know...

    >				 		       The reason I ask is
    > I'm thinking of writing a generic import/export plugin for mythtv that eventually
    > will allow you to archive tv recording and video files as well as music files to
    > various formats (raw files, dvd format, ipod, psp, mp3 cd etc.). Alot of this can already
    > be done using various scripts/tools scattered around the web. I'd just like to create
    > some sort of generic frontend that people can extend to suit there needs. 

That sounds like a really nice utility to have.

    > You would be able to choose to archive a recording/file from various points in the myth
    > interface, say the watch recording screen, or from within MythVideo or MythMusic
    > which would probably just add some details into a db table for later use. Later you 
    > could open the archive plugin which would show you the files you have chosen
    > to archive and allow you choose what format you would like to archive the files
    > to and to enter any options required etc. Finally the plugin would just run the
    > tool/script that does all the hard work passing any options chosen etc.

Sounds good to me!

    > One of the first import/exporters I was going to include was a way to save myth 
    > recordings to DVD along with all the metadata to recreate the recording in myth.
    > It sound like you have already done that =) at least the exporting part. 

Well, sort of.  I'm not sure yet that I really -do- have the exporting
part, at least when it comes to which DB information needs to be
saved.  I'm in the process of seeing what's really required there, and
furthermore this might change once 0.19 comes out (my work so far is
based on 0.18.1 because I need stability).

    Out of interest why don't you save all the recordedmarkup info for a recording so 
    you don't have to recreate it later?

My impression was that it's quite a bit of data per hour just to save
the info needed to making skipping work.  Granted, it's probably not
all that much, and -could- just get written to the DVD along with the
actual mpeg and not consume much space.  I haven't actually verified
whether that supposition is correct, though, and my impression was
that just doing the --rebuild was almost as fast as simply reading the
raw data from the disk. Commflagging is another story, though, so it
might be worth just saving everything so as not to have to wait for
all that work to be redone if a recording is re-imported.  (Can I save
commflagging data without saving all the data required to make
skipping work?  Haven't checked yet.)

Btw, in case anyone's wondering why I'm bothering to save (or
regenerate) commflagging data instead of just saving the video
without the ads:
(a)  I can't (yet) do mpeg2->mpeg2 cutting (though in theory this
     should work in .19, yes?), which means I'd have to transcode.
     [Even if I -could-, unless it was "minimally invasive" along
     the lines of dvbcut, it might lose the closed-captioning data.]
(b)  I haven't been able to make transcoding to megp4 work (at least,
     using whatever Myth 0.18.1 does by default if enabled):
     (1) Audio level goes down many dB.
     (2) The transcoded mpeg4 blows up my mplayer.
     (3) There's no agreed-upon way to keep the closed-captioning data
         in the resulting video stream.
     [I could cope with (3) if (1) & (2) looked solvable, but I've
     asked on the -users list and haven't gotten any solutions.  It
     sure would be nice to halve my data requirements, though---given
     that I'm already stripping the CC data into ASCII text, I could
     stand to toss it from the video stream (meaning no on-screen
     captioning) if the transcoding step worked for me.]
(c)  A lot of this data may get written directly to DVD in a fairly
     automated process, having been checked only superficially to make
     sure the video isn't blank or messed up (as I said, this isn't
     the typical "I'm watching TV" sort of application :).  Since
     commflagging isn't 100% accurate, we can't just chuck those parts
     of the video that the commflagger suspected were ads, but since
     nobody's necessarily going to spend the time making sure the
     cutpoints are accurate, the whole thing gets archived and dealt
     with on-demand later if somebody actually needs a comm-free video.
     So given how cheap disk (and especially DVD) storage is vs human
     time, it's easier to just save everything.

    > One other point. I guess you value your recordings more that me but I really don't want
    > to have a machine chugging away for 6 hours just so I can create a backup. For what blank
    > DVD's cost these days if a recording is that important I would rather just burn a second
    > copy. Just a thought.

The problem with a second copy is that it uses twice as many DVDs, and
I don't expect every original DVD to fail.  Properly done, a PAR2
archive with 20% redundancy means that any single DVD in a group of
five can be recovered no matter what happens to it, while only
increasing the number of DVDs required by 20%.  That's probably still
too much redundancy; 10% with groups of 10 DVDs would work fine, too,
but then either generating PAR or recovering a single DVD would take 12
hours, and would require over 50GB of temporary disk space for either
operation.  That's still feasible (and reduces the total number of
DVDs burned by 10%), but anything bigger would start to get pretty
irritating.  And since the machine doing the computation is otherwise
pretty much unloaded, the CPU time is just going to waste anyway.

(How many DVDs -do- I expect to fail?  I dunno.  I've heard wild
lifetimes that are all over the map, from months to centuries.  I'm
using Taiyo Yuden DVDs, which are generally highly regarded, but even
if reliability per-disk is quite high, if we potentially might have
thousands of them, then the chances of a single-disk failure are
actually non-negligible if the failures are independent.  And media
longevity might not be the dominant failure mode, anyway---it doesn't
include the possibility that something will go wrong with the robot
that will shatter a disk! :)