[mythtv-users] AutoExpire: Oldest Show First, yet newly recorded shows still expire...

Tue Jul 19 14:48:16 UTC 2011

On Mon, Jul 18, 2011 at 12:20 PM, Michael T. Dean
<mtdean at thirdcontact.com> wrote:
> On 07/18/2011 11:33 AM, Steven Adeff wrote:
>> On Mon, Jul 18, 2011 at 12:05 AM, Michael T. Dean wrote:
>>> On 07/16/2011 04:05 PM, Steven Adeff wrote:
>>>> I have had Myth make some odd decisions in what shows to expire to
>>>> clear up space. I have a lot of space so I decided to set it to Oldest
>>>> Show First since that basically means stuff recorded 2+ weeks ago get
>>>> deleted which is more than enough time for us to actually watch them.
>>>>
>>>> anyway, since then I've noticed that Myth still expires new-ish
>>>> recordings (from as soon as earlier that day). can someone help me
>>>> troubleshoot to find out why this is happening?
>>>>
>>>> thanks!!
>>> For a start:
>>>
>>> http://code.mythtv.org/trac/ticket/7936#comment:16
>>> and, more importantly,
>>> http://code.mythtv.org/trac/ticket/7936#comment:18
>>>
>>> And note that MythTV will only ever expire recordings when recordings
>>> are in progress--and then only on the file systems to which it's
>>> recording.  So if you have a full file system with only new recordings...
>>>
>>> Also, have you recently added a new backend host?
>> I added a slave backend a while back. Can different backends have
>> different expire settings?
>
> No, auto-expire settings are global.  I'm just thinking that if you have
> a new backend, depending on your file system layout, you may have a case
> where there aren't any old recordings to expire to make room for new
> recordings on it.

ahh, ok, the main backend is "old", I added a slave but the slave
records only to drives on the main backend.

>> one thing I just noticed is that in my Deleted recordings list via
>> Mythweb there's about 100GB worth of "expired" recordings that don't
>> need to be there, they can just as well reside in the main Default
>> listing until they actually need to be deleted. This is very confusing
>> as well.
>
> Not sure what exactly you meant by this, but Deleted recordings are
> always expired first when you enable:
>
> Auto-Expire instead of delete recording
> If enabled, move deleted recordings to the 'Deleted' recgroup and turn
> on auto-expire instead of deleting immediately.
>
> So, if you have any Deleted recordings on the file system to which
> MythTV is recording, they will be deleted first--regardless of when they
> were recorded and what priority and...  After Deleted recordings is Live
> TV recordings, and only then are recordings expired--according to the
> priority specified--possibly modified by:
>
> Watched before UNwatched
> If set, programs that have been marked as watched will be expired before
> programs that have not been watched.
>
> (in which case high-expire-priority shows that are not marked as watched
> will be ignored until all expirable watched recordings--even
> low-expire-priority ones--are gone).  Again, though, expiry happens on a
> per-file-system basis and only on file systems to which MythTV is
> recording something.  That said, the frontend's status screen or
> mythbackend --printexpire should still show the order in which things
> will be expired, taking into account all of the factors above (but
> without file system information--meaning you'd have to ignore all the
> ones on other file systems to determine exactly which one will be
> expired when recording to a certain file system).
>
> To get closer to the expiry behavior most users seem to want--where
> things are expired in auto-expire order (as much as possible, where
> having multiple hosts with not-shared file systems may make this
> impossible)--someone would need to create a new Storage Group Disk
> Scheduler (
> http://www.mythtv.org/wiki?title=Storage_Groups&redirect=no#Storage_Group_Disk_Scheduler
> ) algorithm that chooses the file system using some "default" criteria
> (most likely the same as Balanced Free Space), but when the chosen file
> system is "full" (within some number of MB of the "Extra disk space
> (GB)" limit), it instead goes through the auto-expire list and chooses a
> file system based on what should expire next.
>
> The biggest challenge, though, will be handling multiple recordings
> starting at the same time (since expiry happens after recording starts)
> and the fact that you may need, for example, 6GiB to record a 1hr show,
> but the highest-priority-for-expiry show is a 3GiB file from a 30-min
> recording.  Then, there may be 100 other recording files in the list
> before you find the next to-be-expired recording on the same file
> system.  (So, MythTV would delete the highest-priority-for-expiry show,
> then delete a recording 100 farther down the list to clear enough room
> for the new 1hr recording).  (And, as I alluded to, if 2 recordings
> start at the same time, one of them should choose the file system that
> contains the 2nd-highest-priority-for-expiry recording, even though the
> highest-priority-for-expiry recording won't be deleted until the next
> auto-expire run.)
>
> Some scheduled changes to the database schema will make writing such a
> storage scheduler easier, but the changes won't be in use until at least
> 0.26.

right, I understand, but why do I have so much in the "deleted" group
that doesn't need to be there? ie. some of it won't actually be
deleted for a while yet they still have been moved there. It would
seem more intuitive for recordings that still remain on the drive to
still be visible in the All Programs group and then when they need to
be deleted they are then actually deleted and removed. I understand
the idea of having a list of the order in which recordings will be
deleted, but I guess I don't understand why they are removed from the
All Programs group when they are set to "expire"?

that said, I don't have the expirer using the watched status (we have
4+ people that use the system so this feature doesn't work for us,
I've had it turned off for a while), and I have enough storage space
to hold about 3-4weeks worth of shows (according to my currently
recorded listing) so I have tried to set it up so it will only use the
recording age to determine expiration. the users know how to set
something to not expire if they won't be able to watch it within that
~3 weeks of recording age.

I pulled everything out of the "deleted" group and am hoping
mythbackend "re-figures" the expiration thing, which so far it looks
to be doing. I'll see how it goes for now and report back.

-- 
Steve
http://www.mythtv.org/wiki/User:Steveadeff
Before you ask, read the FAQ!
http://www.mythtv.org/wiki/index.php/Frequently_Asked_Questions
then search the Wiki, and this list,
http://www.gossamer-threads.com/lists/mythtv/
Mailinglist etiquette -
http://www.mythtv.org/wiki/index.php/Mailing_List_etiquette