[mythtv] Solving my performance problems...

[Steve Brown]

Mark Frey wrote:

>I've been trying to track down performance problems with my machine and
>MythTV. After spending some time instrumenting the code , and figuring out
>what RingBuffer, RemoteFile, etc. are doing, I determined the following:
>
>1. I'm not CPU limited (<50% cpu used)
>2. I'm not HD bandwidth limited (<30% bandwidth used)
>
>After timing some things I found that the bandwidth at the raw read(...)
>level is very high (straight from the drive, hitting the cache most of the
>time), the effective bandwidth at the RingBuffer::safe_read(RemoteFile *...)
>call is very low, ~1.5 to 2.0 MB/s (the drive transfer rate is ~30MB/s
>reads). It seems like the Qt event loop processing etc. is the bottleneck. I
>know this is no big suprise, I read a post from Isaac in July where he said
>as much. What I have been trying to do is figure out if there is some way to
>improve this. I could go get a new hard drive, use a kernel with both the
>low latency and preempt patches and possibly get things running acceptably
>for my current uses, but I wonder whether any amount of hardware would
>provide enough performance for HDTV given the current overhead of the whole
>socket-request-response stuff.
>
>The obvious way to minimize the event loop overhead is to ask for more in a
>single request, amortize the overhead cost over more bytes. The problem is,
>that because of the socket buffer size, requests are currently limited to
>64000 byte blocks. I've been hacking away trying to raise this limit to
>determine how much performance I can gain. I've managed to test 128000 byte
>blocks, and it looks like the bandwidth at the safe_read level scales by
>block size (to some limit obviously). With 128000 byte blocks my bandwidth
>increases to ~4MB/s. The problem is things aren't stable with the changes I
>made (I essentially request the larger size on the frontend side, then pull
>from the socket when the data becomes available, while on the backend
>pushing bytes in buffer-sized chunks and waiting for the buffer to clear,
>until I've sent the total requested). I believe I understand some of the
>causes of the instability, but before I spend the time to figure it out I
>wanted to make sure I'm not wasting my time.
>
>So, my questions are:
>
>1. Am I misunderstanding something in a big way here? Is there some
>fundamental reason this won't work (I am not a sockets expert)?
>2. Am I wrong that the socket request stuff is the bottleneck?
>3. Requesting larger blocks might be great for frontend/backend on the same
>machine, but stink for networked machines. The request size might have to be
>configurable in some way. Would this be acceptable?
>
>-Mark
>
>  
>
I've been trying to track down the same thing. My symptoms are hdtv 
playing haltingly whether from disk or ota. Also, channel changing 
really aggrevates the halting/stuttering problem and sometimes results 
in lots of complaints of a badly broken stream.  Escaping to the menu 
and hitting "play" clears this up. It's interesting to note that the 
stream errors are not accompanied by continuity counter errors from the 
ts demuxer. I suspect that these are really three different problems.

I hypothesized that part of the problem was the latency of the usleep 
spinlocks in NuppelVideoPlayer. As an experiment, I instrumented all the 
waits and found that only 2 of them were frequently used, specifically, 
the ones at ::GetFrame:614 (appx) and ::OutputVideoLoop:1284 (appx). I 
replaced them with QWaitConditions. Also, ::setPrebuffering  is called 
by 2 threads, but doesn't protect the "prebuffering" variable. I didn't 
find any evidence of a race in my testing, but added a lock anyway.  
Hdtv now plays without stuttering whether ota or from disk. The channel 
changing problems remain.

If you are interested in playing with the patch, let me know off-list. I 
don't want to clutter the list with a half-baked, experimental patch.

I'm running a 2.8GHz P4, SATA disks and xvmc on a nvidia MX440. CPU 
utilization is about 60%.

Steve