[mythtv] 'Special' storage groups and free space calculations
raymond at wagnerrp.com
Sat Dec 8 20:15:10 UTC 2012
On 12/8/2012 13:15, Alexander Fisher wrote:
> On 8 December 2012 17:30, Raymond Wagner <raymond at wagnerrp.com> wrote:
>> On 12/8/2012 08:45, Alexander Fisher wrote:
>>> That would still leave the livetv filesystem.
>>> Why is the livetv filesystem free space being included in the free
>>> space statistics? It is already listed as a special group (and I've
>>> verified you can't schedule a recording to go to this group). Does
>>> this make it a special special group?:)
>> The only groups that get included in freespace calculations are ones that
>> get recorded to. That means 'Default', 'LiveTV', and any other non-special
>> groups you define on your own. If no livetv paths are defined, livetv gets
>> stored in the default paths. If no default paths are defined, it drops back
>> to a compiled in '/mnt/store/'.
> I didn't define the 'Streaming' storage group myself. It was part of
> the mythbuntu 12.04 install.
> From http://code.mythtv.org/trac/changeset/7e1a77063e3f002c118149c91d2e5e86b3bb2c5a/mythtv
> 'Live Stream files are written to a 'Streaming' Storage Group if
> one is defined, otherwise they will go in ~/.mythtv/tmp/hls.'
> So I think that 'Streaming' should be added to the list of special
> groups as it's not a user-defined group that is created to hold
While HLS temporary files are related to recordings, they are currently
outside the scope of the AutoExpirer, so how much room they take is
irrelevant to storage available for recordings.
There are plans to internally restructure the database schema in a
manner that will support multiple files tied to a single recording,
potentially in time for 0.27. This would allow multiple transcoded
copies of a recording to remain known to MythTV, and would also allow
HLS transcodes to be tracked as well. At such time, what you suggest
would need to be revisited.
> As for the Live TV group free space being included in the total, I'm
> still not sure I understand the rational. I'm probably just
> misunderstanding something. :)
> What if default group filesystems are almost almost full, but the Live
> Tv group happens to have terabytes available on a separate filesystem?
> It will appear as if there is plenty of room for new recordings. But
> this won't be true as scheduled recordings can't be to the Live Tv
Autoexpiration is a secondary task. The storage available to each
configured storage path is calculated, and that is used by the Scheduler
to determine where a specific recording is going. Once that is decided,
the AutoExpirer determines whether content needs to be cleared off that
path, or if there is sufficient free space to handle the recording.
Looking through the code, it seems any custom paths defined for the
LiveTV group to _not_ get added to the free space calculations, so they
are not matched up with other paths on the same partition. I'd like to
get confirmation from someone who knows more about the process, but it
appears the AutoExpirer assumes anyone using a specific LiveTV path will
have put it on it's own partition.
> But when autoexpire kicks in, I better not also have any snapshots in
> existence! I can envisage a situation where the auto expire code
> deletes all recordings :(
To be honest, I don't see any reason to snapshot recordings for that to
be a problem. In any case, it's just going to be something that cannot
be recommended, for many of the same reasons that mythlink.pl does not
support hardlinks, and requires softlinks.
A bigger issue I could see would be a zpool with a number of zvols, and
someone defining "copies" to be something greater than on, to make a
pseudo-mirror. The actual recording throughput would be multiplied
potentially several times what MythTV otherwise expects it to be. Since
the AutoExpirer makes decisions on how frequently it needs to run, based
off how fast the current recording load should consume disk space, it
could potentially result in you running out of free disk space before
the AutoExpirer runs again.
> I'd be interested to know how many other people are using zfs and
> whether they've run into this.
I run ZFS, but my recordings are still handled as if I weren't, with
each drive on it's own zpool, and one path defined to Default for each
drive. Now that I think about it, I could have defined them all to a
shared zpool, and let ZFS load balance instead of MythTV, but the end
result isn't much different.
More information about the mythtv-dev