[mythtv-users] deleted files not gone, taking up space

[Michael T. Dean]

On 06/17/2013 07:17 PM, David Scammell wrote:
> Hi all,
>
> While browsing about myth I happened to be looking at
> http://mbe-address:6544/ in the information/backend status area I found
> 'space used by deleted recordings to have a large amount (100GB+). While no
> significant deletes or auto expires were underway.
>
> With a little poking about in the database I found the deletepending column
> of the recordings table had a number of entries with a value of 1, most of
> these stretching back to when I had a mythtv bug causing the backend to
> delete/expire recordings prematurely and then die, last year.
>
> I speculate that if a recording is deleted or expired and then the backend
> fails before the actual recording and metadata are removed, the file and
> recording row will become orphaned but still take up space on disk.
>
> I guess its possible many people would have one of more files like this, if
> your backend has ever failed then its possible.  This bit of sql should
> show all pending deletes, those in the delete recording group and any
> 'extras' you may have:
> select * from recorded where deletepending=1;
>
> I ran into the database:
> update recorded set deletepending=0 where deletepending=1;
> [the where clause may well be unnecessary]
>
> It did kill my mbe, so I recommend doing this with your backend(s)
> shutdown. After restarting I can now see all of the recordings which were
> supposedly removed previously while I had the bug last year, these will
> shortly expire (being old) and I'll get my space back.
>
> Perhaps mythbackend should clear this flag on startup, to make the files
> re-appear or restart the slow delete process with data from the recorded
> table where the flag set. I dunno exactly what the correct behaviour should
> be.
>
> I hope this helps someone else. have fun.

This is a known issue that only happens in unusual circumstances (such 
as upgrading to 0.26 with the release version instead of the -fixes 
version that has a fix for a bug that can cause this in some systems 
with a specific pre-upgrade configuration, or doing very bad things like 
hard-killing the backend right after deleting a ton of files).

And, yes, the SQL you issued is a /very/ bad thing to send when (any 
part of) MythTV is running.

There will be a workaround for the issue, eventually, but it's not as 
easy as just resetting the flag on startup.  It's on my TODO list, but 
since this should only happen if people are doing bad things, and since 
it's likely not that much space wasted on any given affected system, I 
recommend not worrying about it too much.

For now, find_orphans.py should be used to clean up any messes, but I'm 
not sure where the updated version that handles it is.

Mike