[mythtv-users] AutoExpire: Oldest Show First, yet newly recorded shows still expire...

[Michael T. Dean]

On 07/18/2011 11:33 AM, Steven Adeff wrote:
> On Mon, Jul 18, 2011 at 12:05 AM, Michael T. Dean wrote:
>> On 07/16/2011 04:05 PM, Steven Adeff wrote:
>>> I have had Myth make some odd decisions in what shows to expire to
>>> clear up space. I have a lot of space so I decided to set it to Oldest
>>> Show First since that basically means stuff recorded 2+ weeks ago get
>>> deleted which is more than enough time for us to actually watch them.
>>>
>>> anyway, since then I've noticed that Myth still expires new-ish
>>> recordings (from as soon as earlier that day). can someone help me
>>> troubleshoot to find out why this is happening?
>>>
>>> thanks!!
>> For a start:
>>
>> http://code.mythtv.org/trac/ticket/7936#comment:16
>> and, more importantly,
>> http://code.mythtv.org/trac/ticket/7936#comment:18
>>
>> And note that MythTV will only ever expire recordings when recordings
>> are in progress--and then only on the file systems to which it's
>> recording.  So if you have a full file system with only new recordings...
>>
>> Also, have you recently added a new backend host?
> I added a slave backend a while back. Can different backends have
> different expire settings?

No, auto-expire settings are global.  I'm just thinking that if you have 
a new backend, depending on your file system layout, you may have a case 
where there aren't any old recordings to expire to make room for new 
recordings on it.

> one thing I just noticed is that in my Deleted recordings list via
> Mythweb there's about 100GB worth of "expired" recordings that don't
> need to be there, they can just as well reside in the main Default
> listing until they actually need to be deleted. This is very confusing
> as well.

Not sure what exactly you meant by this, but Deleted recordings are 
always expired first when you enable:

Auto-Expire instead of delete recording
If enabled, move deleted recordings to the 'Deleted' recgroup and turn 
on auto-expire instead of deleting immediately.

So, if you have any Deleted recordings on the file system to which 
MythTV is recording, they will be deleted first--regardless of when they 
were recorded and what priority and...  After Deleted recordings is Live 
TV recordings, and only then are recordings expired--according to the 
priority specified--possibly modified by:

Watched before UNwatched
If set, programs that have been marked as watched will be expired before 
programs that have not been watched.

(in which case high-expire-priority shows that are not marked as watched 
will be ignored until all expirable watched recordings--even 
low-expire-priority ones--are gone).  Again, though, expiry happens on a 
per-file-system basis and only on file systems to which MythTV is 
recording something.  That said, the frontend's status screen or 
mythbackend --printexpire should still show the order in which things 
will be expired, taking into account all of the factors above (but 
without file system information--meaning you'd have to ignore all the 
ones on other file systems to determine exactly which one will be 
expired when recording to a certain file system).

To get closer to the expiry behavior most users seem to want--where 
things are expired in auto-expire order (as much as possible, where 
having multiple hosts with not-shared file systems may make this 
impossible)--someone would need to create a new Storage Group Disk 
Scheduler ( 
http://www.mythtv.org/wiki?title=Storage_Groups&redirect=no#Storage_Group_Disk_Scheduler 
) algorithm that chooses the file system using some "default" criteria 
(most likely the same as Balanced Free Space), but when the chosen file 
system is "full" (within some number of MB of the "Extra disk space 
(GB)" limit), it instead goes through the auto-expire list and chooses a 
file system based on what should expire next.

The biggest challenge, though, will be handling multiple recordings 
starting at the same time (since expiry happens after recording starts) 
and the fact that you may need, for example, 6GiB to record a 1hr show, 
but the highest-priority-for-expiry show is a 3GiB file from a 30-min 
recording.  Then, there may be 100 other recording files in the list 
before you find the next to-be-expired recording on the same file 
system.  (So, MythTV would delete the highest-priority-for-expiry show, 
then delete a recording 100 farther down the list to clear enough room 
for the new 1hr recording).  (And, as I alluded to, if 2 recordings 
start at the same time, one of them should choose the file system that 
contains the 2nd-highest-priority-for-expiry recording, even though the 
highest-priority-for-expiry recording won't be deleted until the next 
auto-expire run.)

Some scheduled changes to the database schema will make writing such a 
storage scheduler easier, but the changes won't be in use until at least 
0.26.

Mike