[mythtv-users] Dropped frames, simultaneous recordings and process priorities.

[Frank Feuerbacher]

On 13-05-13 04:28 AM, John Pilkington wrote:
> On 13/05/13 08:45, Stephen Worthington wrote:
>> On Sun, 12 May 2013 21:40:46 -0500, you wrote:
>>
>>> Our box is a 4 CPU 3.4GHz i5, 8G ram, Ubuntu 12.10, with a PCIe 
>>> InfiniTV
>>> 4-tuner card (which produces mpeg-ts content). It usually has XBMC
>>> running idle on it and mythtv. We also do background Handbrake 
>>> encodings
>>> and editing with Avidemux (as needed). I notice that under some
>>> circumstances, our recordings have considerable noise, especially 
>>> during
>>> the first few minutes. My theory is that this occurs when consecutive
>>> movies are recorded, or perhaps when more than one is recorded at once.
>>> I do have the default set to record 1 minute before and after a 
>>> show, so
>>> we could be using two tuners and double the I/O for a minute for
>>> consecutive recordings. On some occasions we might record two
>>> consecutive shows on two channels simultaneously, so I guess we could
>>> briefly have 4 recordings at once.
>>>
>>> After a recording is complete, I make a copy of the .mpg to the same
>>> disk to simplify the task of my wife copying the file to her machine 
>>> for
>>> Avidemux (avoids symlink issues). I also create an Avidemux index file.
>>>
>>> I see that mythbackend runs with a normal *nice* value and with *ionice
>>> idle*. I just modified my post-record script to run the copy and
>>> avidemux create index with ionice idle and nice 10. Perhaps this will
>>> solve my problem. But I am curious why mythbackend doesn't run with a
>>> normal *ionice* value. Is the content from the InfiniTV considered I/O
>>> by ionice and impacted by it? Does mythtv buffer frames as needed when
>>> the I/O is backed up a bit?
>>>
>>> If I have to I'll rearrange the disks or acquire more or faster (SSD)
>>> disks. But I'm not sure if *ionice**idle* requires ALL I/O to be idle,
>>> or if it is on a per-device basis. Today I have a disk for the normal
>>> Ubuntu stuff (root, /usr, /home, /tmp). I have one, /pvr, for mthtv and
>>> a staging area for movies to be edited or encoded and one for movies
>>> after they have been encoded for XBMC. They are not fast disks (5400)
>>
>> You did not say how many hard drives you are recording to.  If you
>> only have the one hard drive with the system on it and are doing all
>> your recordings to that drive also, then yes, you are probably
>> overloading the drive.  Modern hard drives are very fast at sequential
>> writing, but when they have to move the heads back and forth between
>> four different recordings, plus the usual system accesses and the
>> occasionally heavy database use by MythTV (especially if the scheduler
>> runs), then they do get overloaded and blocks of recordings go
>> missing.  The head movement speed has not increased nearly as much as
>> the sequential write speeds have as drive technology has improved. The
>> fix is normally to add one or more extra hard drives to record to, and
>> move enough recording files over onto the new drives so that there is
>> reasonably equal spare space on all the recording partitions.
>>
>> Also, it is very important to use JFS or XFS for recording partitions,
>> as deleting huge video files from EXT partitions frequently causes so
>> much activity that it will prevent recordings from working.  If you
>> are set up with an EXT type partition for recordings, then you can
>> help with that problem by setting the option for slow deletes. Sorry,
>> I can not remember exactly where in the setup screens to find it.  If
>> your recording drives are full and having to expire old programs, that
>> can happen during recordings and on an EXT partition that will cause
>> problems without the slow deletes option.  Just one delete at the same
>> time as one recording on an EXT partition can cause problems.
>>
>> Do you also run mythcommflag (commercial skip processing enabled)?  If
>> so, change its options in MythTV to use a lower priority also. And as
>> you seem to have lots of RAM, you might like to set it to start at the
>> same time as the recording starts.  If you do that and have enough
>> RAM, mythcommflag will be getting all its data from the buffers in RAM
>> before the data is written to disk, so it will not be reading disk all
>> the time and moving the heads on the drives.
>
> It seems strange to me that your backend is running with ionice set to 
> idle; during recording there are many operations that are time 
> critical, even if there is lots of buffering.  Where did that setting 
> come from?   ionice idle ought to be only for operations that can be 
> delayed without causing problems, such as commflagging and transcoding 
> completed recordings.
>
I don't know where the "ionice" came from. I will continue to track that 
down. I plan on changing it to give a higher than default io priority, 
perhaps best-effort, 0 or 1.

Today, my /mythtv drive is a 2T 5400 rpm WD GREEN, ext4, lvm, JFS drive. 
The database, configuration and ALL recordings go there. In addition, it 
is where edited recordings go, and Handbrake reads from. The Handbrake 
output goes to another drive.  Typically, we record ~3 movies from 
Turner Classic movies in a day. We also record 3 MSNBC shows daily.

I will look at reorganizing things, along with optimizing what I have. I 
wanted to see if there was some rational behind the "ionice" values that 
I see as well as try to learn something about buffering and other 
implementation details. I think I achieved what I wanted.


Thanks