[mythtv-commits] Ticket #6330: [RFC] New ThreadedFileWriter (lockless)

[MythTV]

#6330: [RFC] New ThreadedFileWriter (lockless)
--------------------------------------------------------------+-------------
 Reporter:  Matthias "mortalmatt" Dahl <devel at mortal-soul.de  |       Owner:  ijr    
     Type:  enhancement                                       |      Status:  new    
 Priority:  minor                                             |   Milestone:  unknown
Component:  MythTV - General                                  |     Version:  head   
 Severity:  medium                                            |     Mlocked:  0      
--------------------------------------------------------------+-------------
 The last few weeks I did a lot of profiling on MythTV to find some of its
 hot spots and try to optimize it for better behaviour under high cpu load.
 Even though the TFW did not explicitly come up, mutex operations are very
 expensive and used heavily within MythTV.

 The following is my first attempt at doing some optimization. This new TFW
 (for now called ThreadedFileWriterLockless) is a replacement for the old
 TFW. Those are the differences:

 - no mutexes in the normal path of execution

 - use of atomic operations (requires >= Qt 4.4.x)

 - aims to be thread-safe (in contrast to the old TFW) and reentrant

 - writes to the buffer don't interrupt the disk writer (true
 concurrency)

 I have tested the TFWL extensively on my machine with some test programs I
 wrote and naturally within MythTV. All written data was 100% correct, no
 corruption.

 The patch also adds a few lines to the libmythtv project file which
 basically detects if support for atomic operations is available and
 compiles the lockless TFW instead.

 So I'd really like to ask everyone interested in testing this and
 reporting back. Be warned though, there is no guarantee that this
 implementation won't eat your kitten. You're welcome to test it but don't
 use it on a production system... you've been warned.

 What's not yet done:

 - the realtime setting code is there but commented for the moment until I
 have more time to do more benchmarks if this helps or hurts

 - the RingBuffer class still contains a rwlock for accessing the TFWL
 which, after further tests and some modifications to RingBuffer, can
 hopefully be removed

 - further documentation of the code

 - better synchronisation for some corner cases

 - support for writing data larger than the ring buffer size

 Also since not every architecture supports atomic operations in a way
 useful to the TFW, a compile time check needs to be added which checks if
 the support is native or not and compiles the appropriate TFW version. Or
 if the "emulation" done by Qt is fast enough or at least as fast as the
 old TFW, that would be unnecessary (benchmarks and tests needed).

 Since this is my first bigger patch to MythTV, I tried to get as close to
 the implicit code style as possible. I hope it's not too awful. I based
 the TFWL on the original TFW to keep the public API.

 Like I said, feedback is very welcome. Thanks.

-- 
Ticket URL: <http://svn.mythtv.org/trac/ticket/6330>
MythTV <http://www.mythtv.org/>
MythTV