[mythtv-users] mass deleting shows?

[Michael T. Dean]

On 10/24/2007 02:04 PM, Jay R. Ashworth wrote:
> On Wed, Oct 24, 2007 at 01:39:07PM -0400, Michael T. Dean wrote:
>   
>> I'm a big believer in this setting.  With it, I can use
>> rock-solid-stable ext3 without feeling the effects of its one
>> downside--the fact that it "locks" up I/O for a long time when deleting
>> large files in one fell swoop.  I no longer have to schedule deletes
>> during periods when no recordings are occurring.
> I presume it uses ftrunc in a loop to shorten the file and return
> blocks to the free list, and *then* does the unlink?

Actually, it opens the file, unlinks the file, then begins truncating
the file until it's smaller than the truncate size (the 4MiB I mentioned
above), at which point it closes the file and the filesystem deletes it
since no processes have it open.  This way, the file is already
"deleted" in case anything happens to crash mythbackend (or the user
kills mythbackend) or the system itself crashes (and the remainder of
the file will be removed on journal recovery/fsck--likely when the
system is booted).  If we waited to unlink the file, we could leave
orphans (possibly taking up many, many GiB's of space).

To see how it works, shut down your mythbackend and notice how long it
takes.  Then, restart mythbackend and go in and delete some extremely
large file(s)--you can even artificially increase the size of one of
your recordings with "dd if=/dev/zero ..." to really see how it works
(if you're using ext3, you'll notice a difference with about 10GB or
more), making sure you have slow deletes enabled.  Then, immediately
shut down the backend.  The shutdown will take "forever" in comparison
because at the point of shutting down the backend, the filesystem begins
deleting the large file(s) (a "normal" fast delete, but limited by the
speed of deletes on your filesystem).  Eventually, the file9s) will be
deleted and the shutdown script will finish.

Note that there are some "rough edges" if you use a CIFS/NFS share
because those filesystems are stateless, so they don't truly support
"delete on last close", so you may end up with small ".cifsXXXXX" files
laying around to clean up externally, or whatever.  Unfortunately,
there's nothing Myth can do about it--only the filesystem/filesystem
driver could.

Mike