[mythtv-users] Commercial Detection during recording

[Simon Hobson]

Joseph Fry wrote:
> Do you k now for certain that it will use cached data if it's available?
Absolutely no doubt.
This is a standard part of pretty well any mainstream OS, caching disk I/O. When an application writes data to disk (and omitting a few details here) it is actually only written into cache and the cache page marked as dirty. At some point in the future, the OS will write that data to disk, leave it in the cache, and just mark the cache page as clean. Several things can cause the write to disk to happen - the mains ones being: time (there's usually a cleanup process that simply writes 'old' data to disk), spacve (when the cache is getting full, dirty pages are written to disk to enable them to be flushed), or if the application requests it (as is the case with MythTV when it's recording a program). So basically, the program stream that's been written to disk will hang around in cache until the space is needed - the more space you have, the longer it hangs around. As long as your commflag process reads it before it gets flushed, it will already be in memory and so won't have to be read back in. The data is generally only cleared from the cache when the space is needed, this if you have copious amounts of memory, stuff can hang around for a long time.

> If so, I wonder if it wouldn't be worthwhile for a patch to adjust the commercial detection job priority to always favor realtime comflagging. IIRC, when I record multiple shows on multiple channels back to back, only one of the first shows is comflaged in realtime, Then it does the rest of the shows FIFO, the comflagger may never catch back up. For example:


8 -9 pm      
Show 1 
Show 2
9 - 10 pm
Show 3
Show 4


Currently, I thing the comflag jobs would run in order, shows 1, 2, 3, then 4. To reduce IO loads, it would be optimal to run them 1, 3, 2, then 4.  Obviously this would require additional logic where multiple comflag threads are allowed. In fact, I believe that if I could sustain multiple realtime threads if I knew it would favor them over on disk recordings... my fear is that enabling multiple threads would thrash my drives when I am recording 3 shows, watching one, and doing 2 comflags; but if I knew that it would prioritize realtime, and that it would use cached data rather than going to disk, there would be far less chance of thrashing.

The simplest thing to do, if you hardware can cope, is to allow for more than one commflag job. Then 1&2 can be done in realtime, followed by 3&4 during readtime. The job queue system isn't actually that sophisticated, and it would probably be a fair bit of work to have it run as you suggest - and then some people would complain as they want 2 doing before 3.