[mythtv-users] WAY OT: Storage designs

[Kenni Lund]

2010/8/31 belcampo <belcampo at zonnet.nl>:
> Kenni Lund wrote:
>>
>> 2010/8/31 Greg Oliver <oliver.greg at gmail.com>:
>>>
>>> On Mon, Aug 30, 2010 at 6:55 PM, Scott <scott at frak.ms> wrote:
>>>>
>>>> On Aug 30, 2010, at 6:25 PM, David Scammell wrote:
>>>>>
>>>>> iSCSI:
>>>>> no performance figures, i'm sorry, in my tests about a couple of years
>>>>> ago NFSv3 vs. iscsi, iscsi won hands down.
>>>>
>>>> I'm not to shocked to hear this. But for a home server, is it really
>>>> needed? I went the other direction and setup a Linux NAS serving data over
>>>> both AFP (Time Machine backups and general Apple file sharing storage) and
>>>> NFSv3 (MythTV BE storage for ISOs). To keep heat and noise down I decided on
>>>> using only 5400 RPM drives. I configured 8 of them in a RAID6 array.
>>>>
>>>> From the MythTV BE a single threaded write over NFS  to the array from
>>>> /dev/zero gives me an average 88MB/s. A single threaded read over NFS from
>>>> the array gives me 30MB/s oddly enough. I mention it's odd because a single
>>>> threaded file read over AFP of the same file shows 75MB/s from the array.
>>>> Clearly something is not optimal in my Linux MythTV BE NFS read performance.
>>>> Still, 30MB/s read and 88MB/s write is easily good enough for streams.
>>>
>>> Yep - the comms between BE/FE ar eonly dictated by the stream..
>>> Unfortunately, I am attemptimg the "whole home solution"..  Like I
>>> said, teh NFSroot I already have going is acceptable for mythtv..  I
>>> specified OT because I need (or would really like) some rsync
>>> performance....
>>
>> After following this thread for the past few days, I decided to give
>> iSCSI a go, to replace my NFS-diskless system, since my NFS-system has
>> been bugging me for a while. The NFS-based system works fine as a
>> MythTV frontend and as a 1080p VDPAU player, but updating the system
>> with kernelupdates etc. is a pain in the *** and package handling and
>> compiling is quite slow as well.
>>
>> I wanted the new frontend to run Mythbuntu to get the autobuilds, but
>> I noticed that the installer in Ubuntu server had received iSCSI
>> support in 9.10, so I went ahead and downloaded Ubuntu 10.04 _SERVER_
>> and installed it onto the new iSCSI target on my server. I then used
>> gPXE to boot it on my frontend and did a "apt-get install
>> ubuntu-desktop" to convert it into a reguler Ubuntu 10.04 Desktop (but
>> with iSCSI boot support out-of-the-box :) ). It worked perfectly and
>> I'm now in the progress of installing the Mythbuntu packages to
>> complete my installation :)
>>
>> Quite a lot easier to setup/maintain than my old NFS-based setup with
>> kernel files in the TFTP root...and I'm still missing the best part:
>> It's much faster than my NFS system, especially at handling small
>> files. I did a quick test uncompressing the Linux kernel 2.6.36-rc3
>> source files from and to the local NFS/iSCSI mount (with tar xjf
>> linux-2.6.36-rc3.tar.bz2):
>> 1st try:
>> NFSv4: 10 min 2 sec
>> iSCSI: 1 min 5 sec
>>
>> 2nd try (rebooted since 1st try):
>> NFSv4: 10min 15 sec
>> iSCSI: 1 min 10 sec
>>
>> Copying the extracted kernel source directory into a new local
>> directory (cp -a linux-2.6.36-rc3 linux-2.6.36-rc3.2):
>> 1st try:
>> NFSv4: 9 min 6 sec
>> iSCSI: 10,6 sec
>>
>> 2nd try (rebooted since 1st try):
>> NFSv4: 9 min 21 sec
>> iSCSI: 9,9 sec
>>
>> When working with bigger files the performance difference is much
>> smaller...copying the ~650MB Ubuntu 10.04 server ISO from/to local NFS
>> and iSCSI mount:
>> NFSv4: 12,3 sec
>> iSCSI: 9,2 sec
>>
>> NFS info: Was mounted with the options "v3,rsize=16384,wsize=16384"
>> with NFS-share stored on a XFS filesystem.
>
> These rsize/wsize values are way below current kernels defaults.
> From /proc/mounts rsize=1048576,wsize=1048576

I don't know about the details of NFSv4, but I can confirm that I have
the same values for my NFSv4 mount. As I remember it, a blocksize of
between 4k and 32k is recommended for NFSv3, but I might be wrong
about this one. I've had serious issues with my diskless systems not
being able to boot consistently (eg. sometimes they boot, sometimes
they don't) with a blocksize of 32k and above, therefore I've been
stuck at 16k for a long time.