[mythtv-users] Dropped frames, simultaneous recordings and process priorities.

[Frank Feuerbacher]

On 13-05-13 04:28 AM, John Pilkington wrote:
> On 13/05/13 08:45, Stephen Worthington wrote:
>> On Sun, 12 May 2013 21:40:46 -0500, you wrote:
>>
>>> Our box is a 4 CPU 3.4GHz i5, 8G ram, Ubuntu 12.10, with a PCIe 
>>> InfiniTV
>>> 4-tuner card (which produces mpeg-ts content). It usually has XBMC
>>> running idle on it and mythtv. We also do background Handbrake 
>>> encodings
>>> and editing with Avidemux (as needed). I notice that under some
>>> circumstances, our recordings have considerable noise, especially 
>>> during
>>> the first few minutes. My theory is that this occurs when consecutive
>>> movies are recorded, or perhaps when more than one is recorded at once.
>>> I do have the default set to record 1 minute before and after a 
>>> show, so
>>> we could be using two tuners and double the I/O for a minute for
>>> consecutive recordings. On some occasions we might record two
>>> consecutive shows on two channels simultaneously, so I guess we could
>>> briefly have 4 recordings at once.
>>>
>>> After a recording is complete, I make a copy of the .mpg to the same
>>> disk to simplify the task of my wife copying the file to her machine 
>>> for
>>> Avidemux (avoids symlink issues). I also create an Avidemux index file.
>>>
>>> I see that mythbackend runs with a normal *nice* value and with *ionice
>>> idle*. I just modified my post-record script to run the copy and
>>> avidemux create index with ionice idle and nice 10. Perhaps this will
>>> solve my problem. But I am curious why mythbackend doesn't run with a
>>> normal *ionice* value. Is the content from the InfiniTV considered I/O
>>> by ionice and impacted by it? Does mythtv buffer frames as needed when
>>> the I/O is backed up a bit?
>>>
>>> If I have to I'll rearrange the disks or acquire more or faster (SSD)
>>> disks. But I'm not sure if *ionice**idle* requires ALL I/O to be idle,
>>> or if it is on a per-device basis. Today I have a disk for the normal
>>> Ubuntu stuff (root, /usr, /home, /tmp). I have one, /pvr, for mthtv and
>>> a staging area for movies to be edited or encoded and one for movies
>>> after they have been encoded for XBMC. They are not fast disks (5400)
>>
>> You did not say how many hard drives you are recording to.  If you
>> only have the one hard drive with the system on it and are doing all
>> your recordings to that drive also, then yes, you are probably
>> overloading the drive.  Modern hard drives are very fast at sequential
>> writing, but when they have to move the heads back and forth between
>> four different recordings, plus the usual system accesses and the
>> occasionally heavy database use by MythTV (especially if the scheduler
>> runs), then they do get overloaded and blocks of recordings go
>> missing.  The head movement speed has not increased nearly as much as
>> the sequential write speeds have as drive technology has improved. The
>> fix is normally to add one or more extra hard drives to record to, and
>> move enough recording files over onto the new drives so that there is
>> reasonably equal spare space on all the recording partitions.
>>
>> Also, it is very important to use JFS or XFS for recording partitions,
>> as deleting huge video files from EXT partitions frequently causes so
>> much activity that it will prevent recordings from working.  If you
>> are set up with an EXT type partition for recordings, then you can
>> help with that problem by setting the option for slow deletes. Sorry,
>> I can not remember exactly where in the setup screens to find it.  If
>> your recording drives are full and having to expire old programs, that
>> can happen during recordings and on an EXT partition that will cause
>> problems without the slow deletes option.  Just one delete at the same
>> time as one recording on an EXT partition can cause problems.
>>
>> Do you also run mythcommflag (commercial skip processing enabled)?  If
>> so, change its options in MythTV to use a lower priority also. And as
>> you seem to have lots of RAM, you might like to set it to start at the
>> same time as the recording starts.  If you do that and have enough
>> RAM, mythcommflag will be getting all its data from the buffers in RAM
>> before the data is written to disk, so it will not be reading disk all
>> the time and moving the heads on the drives.
>
> It seems strange to me that your backend is running with ionice set to 
> idle; during recording there are many operations that are time 
> critical, even if there is lots of buffering.  Where did that setting 
> come from?   ionice idle ought to be only for operations that can be 
> delayed without causing problems, such as commflagging and transcoding 
> completed recordings.
I modified the /etc/init/mythtv-backend.conf to add a call to *ionice* 
and *renice*. I confirmed that this did change the priorities using 
*top* and *ionice -p*.

script
         test -f /etc/default/locale && . /etc/default/locale || true
         ionice -c 2 -n 2 -p $$
         renice -n -5 -p $$
         LANG=$LANG exec /usr/bin/mythbackend --syslog local7 --user mythtv
end script

I am guessing that Upstart itself is running with ionice -c idle and 
mythbackend was inheriting it. I will see how this works out.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mythtv.org/pipermail/mythtv-users/attachments/20130513/a66a9344/attachment.html>