[mythtv-users] OT: Troubleshooting write speed problems

[Joel Michael]

nasa01 at comcast.net wrote:
> Hi,
> 
> I know, this is way off topic, but I figured there must be a lot of smarter people on the list than I.
> I was troubleshooting what I thought was a network issue, when I decided to test my harddrive speeds... Looking at the results, my aim how changed.  How do I fix these write speed issues?  Info follows:
> 
Ok, I'll throw in a few comments from what I'm seeing.  This post isn't 
exactly easy to follow, but I'll try and make the most of it.

>  I have three machines (Main, mythtv, MythTV_Server).  All machines are running a version of mandrake (later than 2006).  
> 
I'm not too familiar with Mandrake, but a linux kernel is a linux kernel ;-)

> mythtv:
> Kernel:  2.6.22-rc1
> 
YIKES!  Is there any particular reason you're running a -rc1 kernel, 
especially a -rc1 from the last stable kernel?  You should probably be 
running 2.6.22 (maybe 2.6.22.something), or the latest 2.6.23-rc 
(whatever it might be at this moment in time).  Just to make sure, does 
'uname -r' say 2.6.22-rc1?

> GIGABYTE GV-NX73G256D-RH GeForce 7300GS 512MB(256MB on Board) GDDR2 PCI Express x16 Video Card 
> WINTEC AMPO 1GB (2 x 512MB) 240-Pin DDR2 SDRAM DDR2 533 (PC2 4200) Dual Channel Kit Desktop Memory Model 3AMD2533-1G1K-R 
> AMD Athlon 64 X2 3800+(65W) Windsor 2.0GHz Socket AM2 Processor 
> Seagate ST3300831A ATA Drive
> MSI K9N Platinum Socket AM2 NVIDIA nForce 570 Ultra MCP ATX AMD Motherboard 
> 
> hdparm -v /dev/hda
> 
The other thing you may want to investigate is using libata to access 
your hard drives, but YMMV as PATA support isn't quite as mature as SATA 
support for some chipsets.

> multcount = 16(on)
> IO_Support = 1 (32-bit)
> unmaskirq = 1 (on)
> using_dma = 1 (on)
> keepsettings = 0 (off)
> readahead = 256 (on)
> geometry = 19457/255/63, sectors = 312581808, start = 0
> 
> dd results very similar to dd results for hda on MythTV_Server
> 
> *****************************************************************************************
> MythTV_Server:
> Kernel:  2.6.17-13mdv
> 
> HighPoint Technologies, RocketRaid 1740 
> DFI Infinity RS482 Motherboard 
> CPU AMD|A64 3400+ 2.2G 939 
> 756M generic ram 
> 4x320G Harddrives in Raid 5 
> 
> hdparm -v /dev/hda (same as mythtv, except for geometry)
> ...will update hdparm -v /dev/sda later (would this be usefull?)
> 
> hdparm -tT /dev/sda
> 
> /dev/sda:
>  Timing cached reads:   2432 MB in  2.00 seconds = 1214.37 MB/sec
>  Timing buffered disk reads:  184 MB in  3.01 seconds =  61.05 MB/sec
> 
> time dd if=/dev/zero of=./test.tmp bs=1024k count=4000
> 2066+0 records in
> 2066+0 records out
> 2166358016 bytes (2.2 GB) copied, 129.33 seconds, 16.8 MB/s
> Command terminated by signal 2
> 0.02user 19.98system 2:09.49elapsed 15%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (1major+275minor)pagefaults 0swaps
> 
> sync; time bash -c "dd if=/dev/zero bs=1024k count=20 48 of=/mnt/Raid/testspeed; sync"
> 2048+0 records in
> 2048+0 records out
> 2147483648 bytes (2.1 GB) copied, 63.3346 seconds, 33.9 MB/s
> 0.01user 4.42system 1:13.53elapsed 6%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (4major+943minor)pagefaults 0swaps
> 
So, what I'm seeing here (and correct me if I'm wrong), is that the "dd" 
test isn't quite living up to the speed that hdparm suggests.

Off the top of my head, I can think of a few reasons for this.

First thing that comes to mind is disk contention.  Is there anything 
else doing a reasonable amount of I/O on the disk during your test?  To 
make sure, try booting to single user mode and re-run the dd test.

The next thing I can think of, is disk layout.  How is your disk 
partitioned?  Do you run LVM?  Software RAID?  On your RAID set, how big 
is the stripe size?

Next contender is the file system.  What kind of file system is it (for 
MythTV systems, people tend to use XFS)?  Did you remember to tell the 
file system about the stripe size of your RAID set?  What other 
performance related metrics did you adjust (e.g. block size, inodes per 
block, journal size, etc)?  What mount options are you using (e.g. 
noatime, journal options)?

Lastly, check the hardware (it's unlikely that hardware is a problem, 
given that you have two machines displaying the same symptoms).  Is your 
power supply ok?  Are all the cables plugged in right?  Are any of the 
cables kinked or pinched?  Do your kernel logs say anything about disk 
errors?  Controller errors?

A well optimised system will be within a few MB/sec of the theoretical 
throughput as reported by hdparm.  My Myth box is 2MB/sec under on a 
software mirror, and 10MB/sec under the sum of two disks on a RAID-0 set 
that is actually being used (for commflaging and recording at the 
moment, if you're wondering).

But, the biggest question is, is it really a problem?  Put on your 
normal load, then fire up iostat and take a look at the last 4 columns 
(avgqu-sz, await, svctm, %util).  You want avgqu-sz to usually be less 
than 10, certainly no more than 100.  The next two columns are only 
really useful if you're tuning for latency (which isn't much of a factor 
in a MythTV box, but can be for something like a home directory server). 
  The %util is good to look at as well, but can be a bit misleading. 
Don't worry too much if you're 100% utilised, but your queue size is 
still low.