[mythtv-users] 0.22 autoexpire behavior

[Michael T. Dean]

On 11/15/2009 08:01 PM, Yan Seiner wrote:
> Michael T. Dean wrote:
>> On 11/14/2009 10:48 AM, Yan Seiner wrote:
>>  
>>> I have myth set to always keep 60 GB free.  This worked in 0.21 but
>>> appears to be broken in 0.22.
>>>
>>> Individual programs that have episode limits get expired.  Myth shows
>>> the correct autoexpire list.  But AFAICT it doesn't actually expire
>>> anything to meet the 60GB free space requirement.
>>>
>>> From the logs:
>>>
>>> 2009-11-14 07:31:13.336 AutoExpire: CalcParams(): Max required Free
>>> Space: 60.0 GB w/freq: 15 min
>>> 2009-11-14 07:31:13.352 Expiring 548 MBytes for 1282 @ Wed Aug 19
>>> 07:00:00 2009 => Curious George.  Too many episodes, we only want to
>>> keep 5.
>>>
>>> So Curious George gets dumped, but nothing else gets expired,
>>> eventhough the system only has 11GB left...
>>>
>>> Has the behavior in 0.22 changed?  Or is something broken in my
>>> setup?  With 11 GB left, it should expire a bunch of stuff on its list
>>> below:
>>>
>>> selene:/data20/mythtv/recordings# mythbackend -v file --printexpire 
>> Since you chopped out the end resolution of the autoexpire run, I can't
>> tell if this is relevant, but autoexpire only occurs when a recording is
>> made on that specific file system.
> Here's the whole thing. Myth only uses a single file system, /data20.
> It filled up over the weekend while I was gone,and the backend was
> grinding away complaining it couldn't find any space.
...
> /dev/md20 1922866224 1903331044 0 100% /data20

OK, first you might need to delete something until you're showing some
space available for use (if you're currently within the reserved blocks
specified for your filesystem, only root can write to the filesystem, so
Myth never will, so nothing will get expired).  Or, actually, rather
than delete the file, just truncate it:

# > /path/to/1021_20091116200000.mpg

and so on.  (Yes, "> /path/to/file" truncates any existing file--at
least in sane shells (bash), don't know for sure if that's the case in
dash.  If not, "cat /dev/null > /path/to/1021_20091116200000.mpg".)

If that doesn't work, keep reading.

...
> MythTV AutoExpire List (programs listed in order of expiration)
> Nature: "Chimpanzees: An Unnatural Hist 0 MB út pro 16 00:00:00 2008 [ 0]
>
...
> Motorcycle Racing: "Monster Energy AMA 0 MB ne úno 15 09:00:00 2009 [ 0]
> Teen Kids News 0 MB po úno 16 07:00:00 2009 [ 0]
> Horseland: "The Princess" 0 MB so úno 21 07:30:00 2009 [ 0]
...
> Horseland: "The Secret" 0 MB so bře 14 07:30:00 2009 [ 0]
>
...
> Ask This Old House: "Banging Pipes; Ins 0 MB út bře 31 18:00:00 2009 [ 0]
>
...
> Nightly Business Report 0 MB út kvě 26 18:30:00 2009 [ 0]
> Nightly Business Report 0 MB út kvě 26 18:30:00 2009 [ 0]
> Nightly Business Report 0 MB út kvě 26 18:30:00 2009 [ 0]
>
...
> Nightly Business Report 0 MB st kvě 27 18:30:00 2009 [ 0]
>
...
> Bob the Builder: "The Sunflower Farm Ta 0 MB so kvě 30 07:30:00 2009 [ 0]
>
...
> Unknown 0 MB st črn 10 18:40:29 2009 [ 0]
> Unknown 0 MB st črn 10 18:45:19 2009 [ 0]
> Unknown 0 MB st črn 10 18:52:37 2009 [ 0]
> Unknown 0 MB po črn 29 18:02:39 2009 [ 0]
> KEZI 9 News 0 MB po črc 20 18:30:00 2009 [ 0]
> KEZI 9 News 0 MB po črc 20 18:30:00 2009 [ 0]
> KEZI 9 News 0 MB po črc 20 18:30:00 2009 [ 0]
> Entertainment Tonight 0 MB po črc 20 19:00:00 2009 [ 0]
> Entertainment Tonight 0 MB po črc 20 19:00:00 2009 [ 0]
> Entertainment Tonight 0 MB po črc 20 19:00:00 2009 [ 0]
> The Bachelorette: "The Men Tell All" 0 MB po črc 20 20:00:00 2009 [ 0]
> The Bachelorette: "The Men Tell All" 0 MB po črc 20 20:00:00 2009 [ 0]
>
...
> Horseland: "No News Is Good News" 0 MB so črc 25 07:30:00 2009 [ 0]
> Sushi Pack: "Collect 'Em All; Jigsaw Su 0 MB so črc 25 08:30:00 2009 [ 0]
...
> Wild About Animals 0 MB ne črc 26 17:30:00 2009 [ 0]

and so on...  Now if the suggestion of truncating stuff until there's
enough available space for Myth to start a recording doesn't work, it
may be that you need to delete these 0-byte recordings
.  Autoexpire /should/ work properly even with 0-byte files, but I find
it a rather interesting coincidence that the top-most (next-to-expire)
recording in your expire list is a 0-byte file and your autoexpire isn't
working.

I'd prefer--if you're willing to help out--that you try the truncate
then wait a while (until the system starts recording again--and then
give the system some time to realize it needs to expire) before you just
delete the 0-byte recordings.  Once you're sure Myth isn't going to
delete the 0-byte recordings, then try deleting only the top-most 0-byte
recording and see if it starts deleting other shows.  Ideally, then
you'd let it continue to expire real shows until it gets to the next
0-byte recording at which point you'd see if it "hangs" the
autoexpires.  If so, we'd need a bug report indicating this is
happening.  If not, then, the 0-byte files are likely not an issue and
"other factors" contributed to it's not resuming before you deleted the
0-byte file.

Note, also, that this test approach requires that you skip some of the
non-0-byte recordings when you start truncating files so you have some
>0-byte recordings for it to expire--as each truncated file will become
a 0-byte file.

So, what happened to cause the issue in the first place?  If it is just
the problem that Myth can't delete because there's no room to record,
why didn't Myth delete shows before it ran out of space?  I don't know
for sure, but I wouldn't be surprised if Myth were busy trying to expire
a bunch of tiny recordings while your system was making one or more
high-bitrate (HDTV) recordings and the filesystem was filling faster
than it was emptying.  It won't do this with large files--as Myth will
take care to make sure it's deleting faster than all the recorders could
possibly write, but with small files, it may be spending enough time in
between deletes that it can't keep up.

If that's the case, make sure you increase the "Extra Disk Space"
setting so you have a little more "breathing room" before the file
system fills up and Bad Things Happen (TM).

Mike