[mythtv-users] Immediate autoexpire recent rec. despite plenty of old rec. to expire

Michael T. Dean mtdean at thirdcontact.com
Wed Oct 30 13:27:32 UTC 2013


On 10/30/2013 05:48 AM, warpme wrote:
>
> Hi *
>
> I need small consultancy regarding unpleasant issue witch started 
> recently in my production system: random recent recordings are quickly 
> 'disappearing' causing users complain like 'argh my favorite show 
> again wasn't recorded'.
>
> Checking on logs it looks like those recording were recorded OK - but 
> they expired immediately (within minutes after rec. finishes).
> My hypothesis is that myth expiring them for making room for new 
> recordings. But in the same time few weeks old recordings are not 
> expired...
> It was surprise for me as on 6T Default SG I have 2T space which is 
> "Used by Auto-expirable Recordings" - so theoretically old. recordings 
> from this pool should be used first for making room for new recordings.
>
> My 6T Default SG is sum of 2 volumes: 2T partition used since initial 
> system build (/mythtv/tv) and 4T drive (/myth/tv1) added half Year ago 
> as storage expansion.
> StorageScheduler is "Combination".
> Looking on few 'too quickly' expired recordings - they always were on 
> '/mythtv/tv' (old drive).
>
> Now I see potential root cause: if 'old drive' is almost full with 
> non-expirable recordings - then if scheduler schedules multiple new 
> rec (my wife can schedule 4-7 concurrent on prime time) - some of them 
> will be assigned to this drive and will cause expiration of other, 
> recent recordings despite other drive (/myth/tv1) had candidates 
> 1-month old rec. to expire. This is just theory.
>

What Storage Group Disk Scheduler have you set?  Most likely it's 
Balanced Free Space (which was made the default because users said 
that's what they wanted--even though they didn't know that they don't 
want it).  If all your file systems used by your Storage Group (the one 
being used for a given recording) are full (meaning the directories in 
the SG directory list exist on full file systems) and you have chosen 
Balanced Free Space or Balanced Percent Free Space, you could easily get 
this behavior.

Because recordings are only expired when new recordings are in progress 
and *only* on the file system(s) to which the new recordings are being 
written, if you've chosen Balanced *Free Space (meaning either plain or 
Percent), when MythTV is about to start a new recording, it will check 
to see which file system has the most space free.  Once it finds that 
file system, it starts a new recording (or multiple) on /that/ file 
system.  Then, it has to start expiring recordings from that file system 
to make room for the new recordings (meaning that the file system will 
likely end up as the one with the most space free, again, for when it 
chooses the file system for the next set of recordings).  After a few 
iterations of this, all of the old/high-auto-expire-priority recordings 
on that file system have been deleted, leaving only new recordings to 
delete, but because you've said, "Only consider free space when 
selecting a file system," MythTV does exactly what you've 
asked--regardless of what you want.

Note that this is not a bug, but an inevitable consequence of choosing 
file system solely based upon "which file system has the most space 
available at the time when a recording starts."

Therefore, if you have full file systems (and rely on auto-expiration to 
make room for new recordings) and you use the Balanced *Percent Free 
Space Storage Group Disk Scheduler, you will have problems unless.  I 
recommend all users who rely on auto-expiration and have one or more 
(even mostly-)full file systems use only Combination or Balanced I/O as 
their Storage Group Disk Scheduler.


> To verify this hypothesis will be good to count space occupied by 
> non-expirable recordings per SG member ( '/mythtv/tv' here).
> Is there easy way to do this?
>
> Generally, looking on this issue from overall perspective: potentially 
> I see where I do mistake: I assume SG is nice way to expand storage 
> when user has lack of space.
> I was happy not going with things like LVM (as member failure brings 
> whole volume down).
>
> Hypothesis from beginning of this post (if verified as correct one) 
> shows that SG is perfect for IO scaling - but not for space scaling.
> I suspect only proper solution of my issue is moving from application 
> level striping (mythtv SG) to fs level striping (LVM) or mass storage 
> level striping (RAID)...
> Unfortunately this is quite difficult to do me as I don't have 6T temp 
> storage needed for such conversion.
>
> Do anybody see other solution here?
>

SGs are perfect for space scaling, too.  The only requirement is that 
you must (currently) balance recordings manually--making sure you have 
auto-expirable recordings on all file systems, and that recordings on 
all file systems tend to span the range of auto-expire priority.  To 
balance recordings, simply use file system commands to move recording 
files from one directory to any other directory in any Storage Group 
(whether it's in the same SG or not).  There are no changes needed in 
MythTV data--just move the file to where you want it.

Alternatively, when your file systems begin to fill up, you can manually 
go through and expire recordings yourself to make room.  This is the 
best approach because it allows you to decide which recordings are least 
important to you (rather than hoping the auto-expire priority is 
"right"--which is almost definitely not the case because of the amount 
of time MythTV has been recording things, the addition of new recording 
rules (that should, but doesn't, change expire priority of even old, 
existing recordings), and a lot of other factors).  You can do this 
easily by going into the System Status section of mythfrontend and 
selecting Auto-Expire List and, as you find recordings you don't care 
about, use the MENU (in that screen) to delete the recordings *now*, 
instead of waiting for new recordings to start.

Also, another way to balance your recordings is to move any 
non-auto-expirable (important/planning to keep) recordings from Watch 
Recordings to Video Library--where you (and not MythTV) manage disk/file 
system placement.  It's also a much better location for (TBs of) 
recordings you plan to keep long term, anyway, because it has a better 
UI for handling large numbers of videos--after all it was designed for 
exactly that purpose.  See 
http://www.gossamer-threads.com/lists/mythtv/users/510542#510542 and 
http://www.gossamer-threads.com/lists/mythtv/users/519758#519758 and 
http://www.gossamer-threads.com/lists/mythtv/users/519996#519996 .

Mike


More information about the mythtv-users mailing list