[mythtv-users] MythicalLibrarian, symlinks and Myth: Will this work or will Myth lose its mind?

[Matt Emmott]

On Tue, Jan 31, 2012 at 3:27 PM, Raymond Wagner <raymond at wagnerrp.com>wrote:

>  On 1/31/2012 14:07, Matt Emmott wrote:
>
> I think I've come up with a solution, but I was wondering if it will cause
> issues. My old recording directory is /var/lib/mythtv/recordings. It's a
> local 50GB partition running EXT3, I believe. I have found using
> mythlink.pl that I can create symlinks in that folder that will point to
> the destination CIFS share of /var/lib/mythtv/recordings3 (Don't ask where
> recordings2 went, it's a long story). So, I could repoint my storage group
> to recordings, have ML move the files to recordings3/Episodes, and leave
> symlinks behind. My question is, what kind of weird side effects would I
> run into? I've already told the BE to follow symbolic links when deleting
> files, so that shouldn't be an issue. I guess my biggest concern would be
> with the reported space available to myth. My /recordings3 folder is a 2TB
> mount, while /recordings is only 50GB. I have Myth set to leave 1GB free at
> all times. How will Myth calculate its available storage? Is it smart
> enough to see the true destination's space used? Or will it always think
> there is 50GB free, or some other weird scenario that I haven't thought of
> yet?
>
>
> MythTV will be recording directly to the 50GB partition.  Hopefully this
> isn't a partition on the same disk that holds your database, and is not
> mounted directly on the recording path.  Since your average digital
> recording will be 6-8GB/hr, you've got less than eight hours of recording
> time before MythTV starts expiring content.  That could potentially be
> filled by a single block of primetime TV.  You must run mythicalibrarian at
> least that frequently.
>
> When mythicalibrarian runs, it will copy the content over CIFS to your
> Windows server, leaving behind a symlink and freeing up room.  Honestly, I
> got lost in a mess of bash, writing new bash scripts in huge one-liners,
> that it the runs through to do the moving, but in the event you fill your
> destination filesystem, the best case scenario is that the content stays on
> your recording disk and mythicalibrarian fails out gracefully.  As before,
> after a couple hours of recording, your filesystem will be filled, and
> MythTV will begin expiring content.
>
> So what will happen when you start expiring content?  Deletes follow
> symlinks, but the autoexpirer knows nothing of that.  All it knows is how
> large the file is, and in what order it is for deletion.  As you drop below
> 1GB (or higher if recording), the expirer will kick in.  Default rate is
> every fifteen minutes, but as you close in on empty, that will accelerate
> to every three minutes.  On each pass, it will delete a recording.  Since
> it doesn't know deleting a symlink is of no worth, it will simply start
> working through your archived recordings over CIFS, clearing out the lowest
> ranking (usually the oldest) recordings.  Your recording filesystem will
> run out of space, the recording will fail, and the autoexpirer will just
> continue happily deleting those symlinks and emptying your CIFS share until
> it finally gets to your real recording directory, leaving only the latest
> six or seven hours of recordings.
>
> _______________________________________________
>

Thanks Raymond. I appreciate the detailed description of how Myth works///
It's always helpful with the mad scientist-esque ideas that pop into my
head.


Before you had replied, I did some experimenting and set it up so that my
SG only contained the 50GB partition and came to the same conclusion you
spelled out above. One potential workaround I thought of was to present
the 2TB partition to the storage group after the 50GB partition. In my
experience, Myth will always record to the first disk in the SG until that
one fills up, and then move on to the second. My thought was that since
Myth could see the second disk's free space, it would remove recordings as
needed. But, the more I thought about this, the more problematic scenarios
kept cropping up and I decided that the whole idea just had way too many
points of failure to be used in my "production" environment.

I'm going to revisit my idea of mounting the drives via NFS, and at that
point the symlinks should work. I hope that this won't have any ill effects
later on down the road... Say in six months when I want to upgrade to
.25-fixes or something, and Myth freaks out when all the recordings are
actually symlinks. But I'm the kind of guy that learns by breaking things,
so it's a win-win in my eyes :-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.mythtv.org/pipermail/mythtv-users/attachments/20120201/285f88ca/attachment.html