[mythtv-users] Storage Groups, free space, (and auto-expire?)

[Brian Guilfoos]

Michael T. Dean wrote:
> Note, though, that if your recording directories are not accessible to
> all backends, you /must/ modify the hostname when you move a recording
> from a directory on one hostname to a directory on another.

Yeah - I'm still mentally processing all the possible setups that would
have to be accommodated by robust code.

Aside: it seems that it would be possible to have a Storage Group
directory that is only mounted on a slave backend - how does playback of
those recordings happen on a frontend that doesn't mount that share?

>> As I only have one Storage Group, I could easily automate what I'm doing
>> manually every few days now.  If you've got multiple storage groups, I
>> still think you'd need to query the database to make sure you only move
>> recordings around within that SG.
> 
> No.  If a recording is not found in the directories of the specified
> storage group, Myth punts and looks at all (locally accessible) storage
> groups (directories for the current backend).   So, there's no need to
> change the storage group or to be limited to moving only within storage
> groups.

OK, so it seems that you'd take a tiny performance hit due to the lookup
time.  That's probably the least-important issue, when compared to
having all possible drives available for load-balancing and ensuring
that auto-expires "make sense".

>>   It seems you could still do that in a
>> bash script with a "mysql --execute" command.  Again, I'm willing to
>> work on something more robust if there is interest.
> 
> If you write a script that you wish to be included in the MythTV source
> distribution, please do so using the Perl bindings.  They provide all
> the capability you need for finding information about the recordings and
> the storage groups--you'd basically just have to provide the code to do
> the actual moving of the file and the logic for when to do it.

I had never looked at the source for those contrib scripts, and holy
smokes are they good.  It never occurred to me that somebody might have
made Perl bindings to provide reusable MythTV-specific code.  Oops.

> See http://www.gossamer-threads.com/lists/mythtv/commits/239035#239035
> for the best info on storage groups.  See, especially, "<spoiler, ie.
> not in the current code>" and feel free to implement that, instead, for
> a /much/ more capable solution.  Doing that /and/ adding GUI control to
> the frontend for moving/archiving recordings would be a very nice
> addition (and save someone--likely Chris Pinkham) a lot of time.  See,
> also http://www.gossamer-threads.com/lists/mythtv/users/256679#256679
> and http://www.gossamer-threads.com/lists/mythtv/users/269092#269092 for
> a way to influence the choice of dirs used.

I had read that spoiler before, and it had slipped my mind while
thinking about this particular problem.  (I'm already using those
per-drive adjustments to bias the scheduler against using the
worst-performing of my network mounts.)

So it seems to me that it'd be VeryUseful(tm) if a patch can be provided
against trunk that would provide a "bool MoveThis(file, [destination])"
functionality (and determine the "optimal" destination automatically if
it's not provided).  That could allow someone to "fix" AutoExpire to try
and move files first, and then delete them only if moving fails.
Second, a routine that could be run periodically that would attempt to
balance free space across storage devices (that could reuse MoveThis())
would be handy.  Finally, adding a way for users to manually move files
via the frontend GUI would be useful.

That specification still raises a few questions.  First, what should be
the algorithm for determining the "optimal" destination for a recording?
1) Move the file to whichever storage device has the most free space,
provided the file will fit, AND assuming the move won't make the
destination device have less free space than the source device.  Ignore
storage groups.
2) Move the file to whichever storage device in the Storage Group has
the most space, provided the file will fit, AND assuming the move won't
make the destination device have less free space than the source device.
 If this fails, fall back to mode 1.
3) Some esoteric algorithm I haven't thought of that handles some use
case I also haven't thought of.

Second, where should "MoveThis()" or whatever it ends up being called be
bolted on?  ProgramInfo?  StorageGroup?

I appreciate the feedback.  I think I'm convinced this is useful enough
for me to bother with doing.  I wouldn't mind some additional input from
some of the other "power users" utilizing Storage Groups.