[mythtv-users] Raid Performance Tweaking

[Blammo]

On 7/9/06, Debabrata Banerjee <davatar at comcast.net> wrote:
> Based on this I've been hammering my system, and I still see no problems,
> but I don't use continues seek modes, why would you want to do 60X FF in
> real-time? This is a ridiculous requirement to put on the system. Between
> commercial skip and single FF/RW buttons there just isn't need at all.

Keep in mind that most people use Myth as a VCR-type device, Tivo
replacement, etc. Even with a 98% success rate on commercial flagging,
you still have a break be wrong every few programs. That requires you
to FF/REW to get to the right spot. "Just isn't any need at all" isn't
true. "Seldom a need" might be more accurate.

Since the heaviest users of Myth, at least in my house, are not the
same people who maintain the myth cluster (namely me) they don't care
about how crazy a requirement is. They care about watching TV. It's my
job to make it work.  (I'm sure I'm not the only one in this position
on this list.... )

> FYI now that I think about I recall that when I originally tested by system
> for stability I ran loadgens on every disk (including a 6-disk software raid
> 5 and 2 disk raid 1 also in the box) while simultaneously encoding on the
> WinTV card for more than a day. (It's been like 3 years now) I'm sure this
> was a much more rigorous ordeal than adding 2 more video sources. Since the
> WinTV cards actually end up overwriting important areas of memory if they
> get ignored for too long, this test was quite in danger of hard locks and
> kernel panics in case of bottlenecks. None observed.

I would agree that with a motherboard like yours, you would have those
type of results. Trying that same thing with a non-serverworks chipset
board, consumer ram, etc, hasn't yielded the same stellar results for
me. I've churned (like you did) through motherboards, CPU's, ram,
drives, cards, etc, before finding a recipe that worked for me.

Let me state my original point more precisely.

If you are using consumer grade componants, you will likely find that
a single spindle will not provide adequate or dependable performance
for a serious mythTV setup.

Of equal importance to the system as a whole are:

1. CPU performance
2. RAM amount/performance
3. Disk performance

While you can make a "passable" mythTV system on a P2-333, 256M ram,
single disk, NV PCI card, and a PVR-250, you are aware of it's
limitations when you use it

(Aside from the redundancy missing in a single spindle setup, which
isn't even an arguement compared to even mirroring)

> I haven't looked at what the read and write threads look like in myth, but
> if you do indeed have a problem with a single disk they could probably be
> solved by optimizing the read and write sizes. Almost everything happens in
> a sequential fashion and simply increasing the size will give you less head
> movement. However again I doubt this since you had to even resort to
> hardware raid, that's just excessive.

Just for historical purposes, here was my path:

1. Single Drive
2. OS drive + 160G myth drive
3. OS drive + 4-160G IDE myth drives - SW Raid5 (experimented with 0 0+1)
4. OS Drive  +4-160G 2IDE/2SATA - SW Raid5
5. OS Drive + 4-160G SATA + promise card- SW Raid5
6. OS Drive + 6-160G 2IDE 4SATA (etc) - SW Raid5
7. OS Drive + 6-160G SATA + 3ware 9500S - HW Raid5
8. 2xOS Drive (mirrored) + 6-160G SATA + 3ware - HW Raid5

Along the way I swapped through 5 motherboards (A7N8X, AN35N, NF7SG,
etc) , 2 CPU's,  3 pairs of ram, 5 different NIC cards, 3 switches
(including a cisco cat3500) 2 power supplies, 3 OS Distros (Knopp,
Fedora, RHEL), more kernel versions than I can count, etc. I was using
NFS + XFS/JFS + SWRaid, which doesn't seem to be stable together under
hefty IO Load, on anything other than a server class motherboard with
ECC Ram, which I didn't have.  All stability and performance issues
ended with step 7.

I'll be the first to say that most people have no need to end up at
step #8.However, it gave me what I was after. Lots of space. Lots of
performance, redundancy, etc. All things you're hard-pressed to get
from a single spindle.