<br><br><div class="gmail_quote">On Tue, Mar 9, 2010 at 4:19 PM, Graham Mitchell <span dir="ltr"><<a href="mailto:gmitch@woodlea.com">gmitch@woodlea.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
> Are all drives on the pci-e based sata controllers? What CPU?<br>
<br>
There are 2 PCIe 1x cards, Sil3132 based, with 2 SATA 3GB ports on each<br>
card. Each port is connected to a 5 way port multiplier (so I'm expecting<br>
some speed reduction there too). The CPU is an older E2100, running at 2GHz,<br>
with 4GB RAM - so not exactly overpowered, but it's only running a minimal<br>
FC, and all it's doing is file serving.. Currently there are 17 x 500GB<br>
drives connected in a RAID5, and the chunk size is set to 512K, and the<br>
stride on the ext3 filesystem is set to 256.<br></blockquote></div><br><br>My guess is that those controller cards and/or the port multipliers are overloaded. Unfortunately, better cards get expensive fast. I'm running 2 Supermicro USAS-L8i cards, they are PCIe x4 cards and can control 8 SATA drives directly. I have them sitting in 2 PCIe x16 slots, though I did have to bend the brackets up to get them to fit the case. You're braver than me with a 17 disk RAID5, I just don't trust HDDs enough for single parity with that many drives. <br>
<br>I'm running OpenSolaris w/ZFS, so my setup is a little different, but the last time I did a dd test on the array I got about 400MB/s. I'm running 2 6-disk raidz2 (RAID6) arrays stripped together. They are also 5400 RPM drives, so I'm quite satisfied with the speed. One of these days, I'll get a managed switch and try link aggregation so the rest of the network can get better overall speeds to and from that server. <br>