[mythtv-users] Autoexpire issues

[Aran Cox]

I wish I would have read this thread before I enabled auto expire... I
use RTJPEG at 640x480 so it takes a while to delete a file.  Tonight I
ran low on disk space and autoexpire nuked every single show on the hard
drive except the one it was recording.  

I'm not bitchin' mind you, but you should hear my wife ;)

Anyway, is this fixed in CVS?  I looked at it and I didn't see any
obvious differences in the autoexpire.cpp code, but maybe it was a
dispatchNow() bug?

The nice thing is that I just added a tiny patch to delete the
oldrecorded entry for a program being auto-expired, so at least I'll
catch them the next time around.  Now I think I'll add a nice long sleep
to autoexpire.cpp as well but it will be a while before it's needed ;)

Thanks,
Aran


On Fri, 2003-09-12 at 16:28, Chris Pinkham wrote:
> > I added the printf lines and pasted the output below. The 'freespace'
> > line appeared to be accurate prior to any actual autoexpiring and
> > autoexpire kicked in at the appropriate time (5G free). Unfortunately,
> > it seems to have then deleted more (~9G) than necessary. I now have ~14G
> > free in the partition, even though autoexpire thinks 5G:
> 
> It looks like the auto-expire loop isn't waiting until the file has
> been deleted like it should be.  It should be waiting on the line:
> 
> 	gContext->dispatchNow(me);
> 
> I had the add the dispatchNow() method which uses QApplication::sendEvent()
> instead of QApplication::postEvent() which gContext->dispatch() uses.
> sendEvent() is supposed to wait until the message has been processed while
> postEvent() just sends the message and continues on.
> 
> > Is it supposed to be freeing up that much space or keep the system right
> > at my 5G free limit?
> 
> It should keep the system right at the 5G limit.  Since it's not waiting
> for the first file to be deleted, it's going ahead and deleting a second
> and third, etc. until it sees enough free space (which is because the
> first file was eventually deleted by the mainserver thread).
> 
> Are you running multiple backends over nfs or anything like that?  I'm
> not sure why sendEvent() isn't waiting like it should.
> 
> > This is CVS from 8/28. I saw you did a commit yesterday, not sure if
> > that would be related to this or not...
> 
> My commit was to stop the case where the statfs() fails for whatever
> reason and the auto-expire loop still goes and trys to free space by
> deleting files (which would result in all files being deleted from
> the database).
> 
> 1st loop though auto-expire thread
> > bavail  = 1341439
> > bsize   = 4096
> > bfree   = 2721723
> > freespace = 5
> 
> 2nd time
> > bavail  = 1181702
> > bsize   = 4096
> > bfree   = 2561986
> > freespace = 4
> 
> Now we go inside the "while not enough freespace" loop
> 
> > 2003-09-12 12:35:59 Running AutoExpire: Want 5 Gigs free but only have
> > 4.
> > 2003-09-12 12:35:59 AutoExpiring: Law & Order Tue Aug 26 19:00:00 2003
> > 3773 MBytes
> > bavail  = 1181382
> > bsize   = 4096
> > bfree   = 2561666
> > freespace = 4
> 
> This should have said 8 Gigs free but since sendEvent() in dispatchNow()
> didn't wait, it goes ahead and tries to delete another show.
> 
> > 2003-09-12 12:36:00 AutoExpiring: Dilbert Thu Aug 28 01:00:00 2003 1936
> > MBytes
> > bavail  = 1229802
> > bsize   = 4096
> > bfree   = 2610086
> > freespace = 4
> 
> It's tried to delete 2 shows but still only 4 gigs free so it deletes
> another.
> 
> > 2003-09-12 12:36:01 AutoExpiring: That '70s Show Fri Aug 29 18:00:00
> > 2003 1900 MBytes
> > IOBOUND - blocking in ThreadedFileWriter::Write()
> > bavail  = 1266692
> > bsize   = 4096
> > bfree   = 2646976
> > freespace = 4
> 
> Probably started deleting the first file somewhere in there causing
> the recorder to have problems trying to write.
> 
> All this happened in 2 seconds.  Quick and dirty fix would be to insert
> a sleep() around line 95 like this:
> 
> 
>                 expireList.erase(expireList.end() - 1);
> 
>                 sleep(10);
> 
>                 if (statfs(recordfileprefix.ascii(), &statbuf) == 0)
>                 {
> 
> 
> I don't particularly like that though, I'd like to know why sendEvent()
> isn't waiting.  Worked fine here when I wrote and tested the auto-expire
> code to begin with.
> 
> I just put a note in my TODO list to add another check as well.  I think
> it would be good to test the freespace after the delete and if it doesn't
> look like there is more freespace now then don't delete anymore.  Either
> that or only delete 1 file every time there's not enough space, then
> sleep for the time the user configured in setup before trying to delete
> another file.  That would fix the problem you're seeing as well.