[mythtv-users] Gigabit network (Was: Re: mythvideo and NFS)

[Matt Mossholder]

On Sun, 2006-09-17 at 11:48 -0500, jack snodgrass wrote:
> 
> I use a 9000 byte MTU.
> 
> I've found in my testing that gigabit doesn't really buy you anyting
> UNLESS you can get your hardware to do at least a 9000 byte MTU.
> 
> The switch is the hardest part to find cheap.   Recently, the
> NETGEAR GS608 came out with a V2 version that works with
> 9000 byte MTUs. It says it on the box... the V1 version did not
> do jumbo frames and did not say it on the box. You need the V2
> version... they run $60 from new egg. Besides the Netgear V2 GS608,
> the SMC 850 ( there is a 5 port and 8 port version ) support 9K MTUs.
> they are under $100 too.... most other gigabit switches do NOT support
> jumbo frames.
> 
> On the NICs, my motherboard have onboard gigabit nics and they
> happend to support jumbo frames. I'm using the skge and forcedeath
> drivers. As with the switches, you have to make sure that your
> NICs do 9000 byte jumbo frames. Google has a couple of good
> jumbo frame supported hardware pages. ( the GS608 V2 stuff is
> new so some info still says that the GS608 does not do jumbo
> frames )
> 
> jack

9000 byte MTUs help, but you can do a LOT to help your performance by
playing with the kernel's IP stack. For example, on my systems, here are
my nttcp throughput results, with no special options from fairly modest
systems:

     Bytes  Real s   CPU s Real-MBit/s  CPU-MBit/s   Calls  Real-C/s
CPU-C/s
l  8388608    0.07    0.05    923.6012   1290.4806    2048  28186.07
39382.3
1  8388608    0.13    0.06    506.7880   1198.3084    5473  41330.61
97726.9


Here are the /etc/sysctl.conf settings I use:

net.ipv4.tcp_window_scaling=1
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.rmem_default = 5242880
net.core.wmem_default = 5242880
net.ipv4.tcp_rmem = 262144 5242880 16777216
net.ipv4.tcp_wmem = 262144 5242880 16777216
net.core.netdev_max_backlog=2500

I seem to remember, prior to working out those settings, only managing
to get something on the order of 120Mbs throughput. 

The real benefit of the big MTUs are that they allow for the higher
through puts without as much processor overhead. i.e. less packets means
less packet assembly/disassembly, which in turn means less need for the
CPU to be involved.

I do have a question for those of you who are using the 9000 byte MTU...
how do you get to the internet? Do you have a 1Gb firewall, or do you
setup one of the systems with 2 NICs, so it can do the fragmentation? Or
do you set ALL the systems up with 2 nics?

	...just curious...


		--Matt