[mythtv-users] File System Recommendation?

Steven Adeff adeffs.mythtv at gmail.com
Mon Apr 3 13:31:26 UTC 2006


On 4/3/06, Brian Wood <beww at beww.org> wrote:
> On Apr 2, 2006, at 10:10 PM, f-myth-users at media.mit.edu wrote:
> >     Date: Mon, 3 Apr 2006 13:01:21 +1000
> >     From: "Richard Dale" <richard at interlink.com.au>
> >
> >> I'm using ext3 on a RAID-0 system. Who the heck cares if it takes a
> >> second or two to delete a 1-hour show ??
> >
> >     Here's an interesting solution - the recording could be flagged
> > for deletion
> >     and then deleted in a background thread.  So, even if it takes
> > a while it's
> >     the background thread that has to wait on I/O, not the
> > foreground user
> >     interface.
> >
> > The problem isn't making the -user- wait for the disk, it's making
> > -Myth- wait for the disk.
> >
> > While ext3fs is deleting a file, all other writers block.  If you're
> > currently writing a stream (or several) to the same filesystem, -they-
> > will block.  Eventually your buffers will overflow, and you'll lose
> > data, and your recordings in progress will have glitches in them.
> >
> > I verified this experimentally with ext3fs on an ordinary disk (no
> > LVM or RAID), using all SD NTSC feeds from ivtv.  Deleting a one-hour
> > recording was typically okay, even if multiple streams were writing,
> > but deleting (say) 10G (e.g., a 5hr recording) was not.  I don't
> > recall exactly where the breakpoint was (I could look it up in my
> > notes), but it also varies based on how many other streams you're
> > writing, and whether the database is doing anything, etc.  Of course,
> > the breakpoint might be different on your machine, and if something
> > decides to delete too many 1-hour recordings at the same time (such as
> > expiration), you'll be in the many-GB case and lose in the same way.
> >
> > This is simple to reproduce---just use dd or something to create files
> > of arbitrary sizes, and time their deletion.  That'll give you an idea
> > of the work the filesystem is going through.  Then (if you want) try
> > recording n streams to that filesystem while doing a deletion.  There
> > will come a point in ext3fs where the amount of time it takes for the
> > deletion to happen will exceed your RAM buffers, and you'll lose data.
> > (In JFS, the deletions are in the tens of milliseconds or less no
> > matter
> > how large the file is, so that won't happen there.  Ditto XFS, I
> > believe.)
> >
> > If you queued all deletions until no writers were writing---or due to
> > write in the next 5-10 seconds---this would be fine, but you might
> > wind up waiting hours for a deletion to happen.  You'd have to be
> > pretty sure you wouldn't run out of space in the meantime.
> >
> >
>
> Thank you for a good explanation, now everything I have read makes
> sense, including the "bilgewater". I guess I have never noticed a
> problem because I almost never record anything longer than a 1-hour
> show, and thus never delete anything longer either. I also have a lot
> of RAM.
>
> It seems pretty stupid for a deletion to block all writes, but I'm
> sure there is a reason this was done, I can see possible problems if
> it were otherwise.
>
> I guess every filesystem is a compromise based on performance,
> safety, size of code, speed of code execution and other factors, and
> what is good for storing characters from a newswire is different from
> what's best for storing HD video streams.
>
> Thanks again.

When I was setting up my RAID 5 I read in a couple places that XFS was
preferable over JFS. I wish I had saved the links. From what I read,
for non-redundant file systems JFS has the advantage, and with RAID 5
style setups XFS was preferable.

I've been using XFS on my RAID 5 array for about 2 months now without
a problem. I force Myth to keep 10GB free of the 600GB total, and I've
yet to have any problems with it.

--
Steve


More information about the mythtv-users mailing list