[mythtv-users] 0.22 autoexpire behavior

Mon Nov 16 15:54:52 UTC 2009

On Sun, November 15, 2009 9:19 pm, Michael T. Dean wrote:
> On 11/15/2009 08:01 PM, Yan Seiner wrote:
>> Michael T. Dean wrote:
>>> On 11/14/2009 10:48 AM, Yan Seiner wrote:
>>>
>>>> I have myth set to always keep 60 GB free.  This worked in 0.21 but
>>>> appears to be broken in 0.22.
>>>>
>>>> Individual programs that have episode limits get expired.  Myth shows
>>>> the correct autoexpire list.  But AFAICT it doesn't actually expire
>>>> anything to meet the 60GB free space requirement.
>>>>
>>>> From the logs:
>>>>
>>>> 2009-11-14 07:31:13.336 AutoExpire: CalcParams(): Max required Free
>>>> Space: 60.0 GB w/freq: 15 min
>>>> 2009-11-14 07:31:13.352 Expiring 548 MBytes for 1282 @ Wed Aug 19
>>>> 07:00:00 2009 => Curious George.  Too many episodes, we only want to
>>>> keep 5.
>>>>
>>>> So Curious George gets dumped, but nothing else gets expired,
>>>> eventhough the system only has 11GB left...
>>>>
>>>> Has the behavior in 0.22 changed?  Or is something broken in my
>>>> setup?  With 11 GB left, it should expire a bunch of stuff on its list
>>>> below:
>>>>
>>>> selene:/data20/mythtv/recordings# mythbackend -v file --printexpire
>>> Since you chopped out the end resolution of the autoexpire run, I can't
>>> tell if this is relevant, but autoexpire only occurs when a recording
>>> is
>>> made on that specific file system.
>> Here's the whole thing. Myth only uses a single file system, /data20.
>> It filled up over the weekend while I was gone,and the backend was
>> grinding away complaining it couldn't find any space.
> ...
>> /dev/md20 1922866224 1903331044 0 100% /data20
>
> OK, first you might need to delete something until you're showing some
> space available for use (if you're currently within the reserved blocks
> specified for your filesystem, only root can write to the filesystem, so
> Myth never will, so nothing will get expired).  Or, actually, rather
> than delete the file, just truncate it:
>
> # > /path/to/1021_20091116200000.mpg
>
> and so on.  (Yes, "> /path/to/file" truncates any existing file--at
> least in sane shells (bash), don't know for sure if that's the case in
> dash.  If not, "cat /dev/null > /path/to/1021_20091116200000.mpg".)
>
> If that doesn't work, keep reading.
>
> ...
>> MythTV AutoExpire List (programs listed in order of expiration)
>> Nature: "Chimpanzees: An Unnatural Hist 0 MB út pro 16 00:00:00 2008 [
>> 0]
>>
> ...
>> Motorcycle Racing: "Monster Energy AMA 0 MB ne úno 15 09:00:00 2009 [ 0]
>> Teen Kids News 0 MB po úno 16 07:00:00 2009 [ 0]
>> Horseland: "The Princess" 0 MB so úno 21 07:30:00 2009 [ 0]
> ...
>> Horseland: "The Secret" 0 MB so bře 14 07:30:00 2009 [ 0]
>>
> ...
>> Ask This Old House: "Banging Pipes; Ins 0 MB út bře 31 18:00:00 2009 [
>> 0]
>>
> ...
>> Nightly Business Report 0 MB út kvě 26 18:30:00 2009 [ 0]
>> Nightly Business Report 0 MB út kvě 26 18:30:00 2009 [ 0]
>> Nightly Business Report 0 MB út kvě 26 18:30:00 2009 [ 0]
>>
> ...
>> Nightly Business Report 0 MB st kvě 27 18:30:00 2009 [ 0]
>>
> ...
>> Bob the Builder: "The Sunflower Farm Ta 0 MB so kvě 30 07:30:00 2009 [
>> 0]
>>
> ...
>> Unknown 0 MB st črn 10 18:40:29 2009 [ 0]
>> Unknown 0 MB st črn 10 18:45:19 2009 [ 0]
>> Unknown 0 MB st črn 10 18:52:37 2009 [ 0]
>> Unknown 0 MB po črn 29 18:02:39 2009 [ 0]
>> KEZI 9 News 0 MB po črc 20 18:30:00 2009 [ 0]
>> KEZI 9 News 0 MB po črc 20 18:30:00 2009 [ 0]
>> KEZI 9 News 0 MB po črc 20 18:30:00 2009 [ 0]
>> Entertainment Tonight 0 MB po črc 20 19:00:00 2009 [ 0]
>> Entertainment Tonight 0 MB po črc 20 19:00:00 2009 [ 0]
>> Entertainment Tonight 0 MB po črc 20 19:00:00 2009 [ 0]
>> The Bachelorette: "The Men Tell All" 0 MB po črc 20 20:00:00 2009 [ 0]
>> The Bachelorette: "The Men Tell All" 0 MB po črc 20 20:00:00 2009 [ 0]
>>
> ...
>> Horseland: "No News Is Good News" 0 MB so črc 25 07:30:00 2009 [ 0]
>> Sushi Pack: "Collect 'Em All; Jigsaw Su 0 MB so črc 25 08:30:00 2009 [
>> 0]
> ...
>> Wild About Animals 0 MB ne črc 26 17:30:00 2009 [ 0]
>
> and so on...  Now if the suggestion of truncating stuff until there's
> enough available space for Myth to start a recording doesn't work, it
> may be that you need to delete these 0-byte recordings
> .  Autoexpire /should/ work properly even with 0-byte files, but I find
> it a rather interesting coincidence that the top-most (next-to-expire)
> recording in your expire list is a 0-byte file and your autoexpire isn't
> working.
>
> I'd prefer--if you're willing to help out--that you try the truncate
> then wait a while (until the system starts recording again--and then
> give the system some time to realize it needs to expire) before you just
> delete the 0-byte recordings.  Once you're sure Myth isn't going to
> delete the 0-byte recordings, then try deleting only the top-most 0-byte
> recording and see if it starts deleting other shows.  Ideally, then
> you'd let it continue to expire real shows until it gets to the next
> 0-byte recording at which point you'd see if it "hangs" the
> autoexpires.  If so, we'd need a bug report indicating this is
> happening.  If not, then, the 0-byte files are likely not an issue and
> "other factors" contributed to it's not resuming before you deleted the
> 0-byte file.
>
> Note, also, that this test approach requires that you skip some of the
> non-0-byte recordings when you start truncating files so you have some
>>0-byte recordings for it to expire--as each truncated file will become
> a 0-byte file.

I'll play with it a bit.  It may be a few days; things are going to get
nuts at work until Thursday this week, and the kids are not home, so the
SO and I are enjoying the peace and quiet of nothing.... :-)

>
> So, what happened to cause the issue in the first place?  If it is just
> the problem that Myth can't delete because there's no room to record,
> why didn't Myth delete shows before it ran out of space?  I don't know
> for sure, but I wouldn't be surprised if Myth were busy trying to expire
> a bunch of tiny recordings while your system was making one or more
> high-bitrate (HDTV) recordings and the filesystem was filling faster
> than it was emptying.  It won't do this with large files--as Myth will
> take care to make sure it's deleting faster than all the recorders could
> possibly write, but with small files, it may be spending enough time in
> between deletes that it can't keep up.

I think I have a part of the problem.  My database is seriously fubared. 
I did a dump and looked at it; seems that over the years I have collected
tons and tons of cruft.  (mythmusic is looking for music in
/lib/modules/*....)  There's multiple backends, frontends, etc. that no
longer exist.

I've posted the dump at http://www.seiner.com/mythtv.sql .  I'd appreciate
any and all help in fixing it.  I don't mind dumping everything but
recording data.  All the recordings should be on selene at
/data20/mythtv/recordings; everything else can be deleted.

selene is the only backend and frontend ATM.  I need to add eos as a
secondary BE/FE but I need to clean up the database first.

How do I do that?

As for how this started, I reserve 60 GB on /data20, so myth should have
lots of room.  I saw this coming over several days; I messed around with
stuff but never could get it to delete and finally over the weekend it
choked.

>
> If that's the case, make sure you increase the "Extra Disk Space"
> setting so you have a little more "breathing room" before the file
> system fills up and Bad Things Happen (TM).

60GB should be plenty.

>
> Mike
> _______________________________________________
> mythtv-users mailing list
> mythtv-users at mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
> !DSPAM:4b00e0db184251804284693!
>
>

-- 
   o__
   ,>/'_         o__
   (_)\(_)       ,>/'_          o__
A day may come  (_)\(_)         ,>/'_      o__
when the courage of men fails, (_)\(_)     ,>/'_                      o__
when we forsake our friends and break all (_)\(_)                     ,>/'_
bonds of fellowship, but it is not this day. But it is not this day! (_)\(_)

This day we ride!