[mythtv-users] What is best filesystem for recordings?

[Stephen Worthington]

On Tue, 30 Jun 2015 09:47:20 +0200, you wrote:

>Hi *
>
>I just got some free space on my server (8T Seagate Archive for videos) 
>which temporarily gives me spare capacity allowing change filesystem for 
>recordings volumes.
>
>Currently I have 2x4T HDD allocated purely for recordings. Both volumes 
>are now on ext3.
>
>After years of relaying on ext3 I think it is not best filesystem for 
>recordings as:
>-delete is very throughput expensive
>-chkdsk is really long
>-starting loosless cut makes hell of:
>TFW(/myth/tv/21101_20150630054000.ts:67): write(57152) cnt 35 total 
>2079656 -- took a long time, 2571 ms
>
>So what FS will be better: XFS or JFS or ...?
>
>I think following criteria should be compared:
>1.concurrency (frequently I have 8-10 concurrent HD recordings);
>2.CPU consumption under high FS load;
>3.reliability (I don't have UPS and sometimes power is lost when many 
>recordings are ongoing);
>4.stability/maturity (i.e. recovery from bad/deep FS corruption);
>5.any other I forgot..

I have always used JFS for recording partitions and been very happy
with the results.  Since I have been happy, I have not tried any other
filesystems.

Deletes of huge files are rapid and low load on JFS.

I do not push my recording drives too hard, as most of them are now
"green" drives that do not have the highest specifications.  So I
usually organise things so that there are no more than two concurrent
recordings on one drive.  I prefer to have more drives to do more
recordings, and with seven recording drives at present the load is
well spread even at the worst peak I have seen recently where I had 14
recordings at once during an overlap period.  But when I had fewer
drives and an older CPU that could not do commercial flagging in
real-time, I used to have four recordings at once on one JFS drive and
also mythcommflag reading from one or two of those files at the same
time.  I never had a problem with that, but if I remember correctly,
five recordings at once was pretty well guaranteed to cause missing
bits in one or more recordings.  I personally like to have a bit more
safety margin, so I prefer to keep the concurrency down to no more
than three recordings at once per drive, and now I have the luxury of
only two at once.

CPU consumption for JFS is consistently low.  The only time I see the
JFS tasks at the top of the CPU use is just a flicker before they
disappear again, even when I am copying video files from one JFS
partition to another.

I do not have many power outages or the like, but when they have
happened, including one CPU overheat shutdown in the middle of one of
my peak recording times (8+ recordings running), I have never had any
JFS file system corruption.  I always run a full fsck on my
filesystems after any event like that, and the JFS partitions normally
only have to run their journal processing and then everything is OK
again.  My ext3 system partitions have occasionally had a slight
problem with one or two files that were being updated at the time of
failure, but nothing that fsck could not easily fix.  I do not think I
have had a power failure since I moved my main system partition to
ext4.

But... you have to realise that writing a big video file sequentially
is not a complex operation for a filesystem.  If you have your
database on JFS and it was being written to heavily, the results of a
power outage might be more problematical.  Since I only use my JFS
drives for large file storage, I have no experience with other
scenarios.

I have also had a fairly new 1 Tbyte recording drive with JFS go bad
with many bad sectors appearing and more every hour.  JFS handled that
well, and I was able to use GNU ddrescue to recover all the files and
only about six files ended up damaged where a sector was completely
unreadable.  I may have just been lucky where the bad sectors were and
they were not in the JFS data areas, just in the data files, but there
were over 300 bad sectors, so I was very pleased with the result.

If you want good speed from any video editing task, you need to make
sure that the file being read from is on a different drive from the
file being written to.  There is quite a lot of video processing now
(especially lossless cutting) where the processing is disk bound, not
CPU bound.