[mythtv-users] Increasing ivtv and/or Myth buffering?

Sun Aug 27 20:38:58 UTC 2006

I managed to find what might be a solution, although I still have a
couple of questions---namely, "why so much more of a memory footprint
than predicted?" and "what about YUV?".  See below.

(Apparently searching for ``ivtv buffering'' doesn't yield much
useful, but ``ivtv buffers'' and ``ivtv "number of buffers"'' provides
a toehold, and turns out to trip over a page on the 350 in the mythtv
wiki.)

I added this line to /etc/modprobe.d/ivtv (ivtv 0.4.0/0.4.1, Ubuntu Breezy):

    options ivtv yuv_buffers=32 mpg_buffers=16 vbi_buffers=16 pcm_buffers=16 dec_osd_buffers=2

...and my logfile entries on the machine with a sole 350 went from this:

    Aug 21 14:50:38 myth-frontend kernel: [4294694.988000] ivtv0: Allocate DMA encoder MPEG stream: 128 x 32768 buffers (4096KB total)
    Aug 21 14:50:38 myth-frontend kernel: [4294695.002000] ivtv0: Allocate DMA encoder YUV stream: 194 x 10800 buffers (2048KB total)
    Aug 21 14:50:38 myth-frontend kernel: [4294695.016000] ivtv0: Allocate DMA encoder VBI stream: 120 x 17472 buffers (2048KB total)
    Aug 21 14:50:38 myth-frontend kernel: [4294695.030000] ivtv0: Allocate DMA encoder PCM audio stream: 455 x 4608 buffers (2048KB total)
    Aug 21 14:50:38 myth-frontend kernel: [4294695.045000] ivtv0: Create encoder radio stream
    Aug 21 14:50:38 myth-frontend kernel: [4294695.059000] ivtv0: Allocate DMA decoder MPEG stream: 16 x 65536 buffers (1024KB total)
    Aug 21 14:50:38 myth-frontend kernel: [4294695.074000] ivtv0: Allocate DMA decoder VBI stream: 512 x 2048 buffers (1024KB total)
    Aug 21 14:50:38 myth-frontend kernel: [4294695.088000] ivtv0: Create decoder VOUT stream
    Aug 21 14:50:38 myth-frontend kernel: [4294695.102000] ivtv0: Allocate DMA decoder YUV stream: 24 x 43200 buffers (1024KB total)

to this:

    Aug 27 11:46:19 myth-frontend kernel: [4294693.248000] ivtv0: Allocate DMA encoder MPEG stream: 512 x 32768 buffers (16384KB total)
    Aug 27 11:46:19 myth-frontend kernel: [4294693.275000] ivtv0: Allocate DMA encoder YUV stream: 3106 x 10800 buffers (32768KB total)
    Aug 27 11:46:19 myth-frontend kernel: [4294693.279000] ivtv0: Allocate DMA encoder VBI stream: 960 x 17472 buffers (16384KB total)
    Aug 27 11:46:19 myth-frontend kernel: [4294693.307000] ivtv0: Allocate DMA encoder PCM audio stream: 3640 x 4608 buffers (16384KB total)
    Aug 27 11:46:19 myth-frontend kernel: [4294693.311000] ivtv0: Create encoder radio stream
    Aug 27 11:46:19 myth-frontend kernel: [4294693.337000] ivtv0: Allocate DMA decoder MPEG stream: 16 x 65536 buffers (1024KB total)
    Aug 27 11:46:19 myth-frontend kernel: [4294693.338000] ivtv0: Allocate DMA decoder VBI stream: 512 x 2048 buffers (1024KB total)
    Aug 27 11:46:19 myth-frontend kernel: [4294693.365000] ivtv0: Create decoder VOUT stream
    Aug 27 11:46:19 myth-frontend kernel: [4294693.366000] ivtv0: Allocate DMA decoder YUV stream: 24 x 43200 buffers (1024KB total)

That's apparently an additional 70 meg of buffers per card (but see below).

So I did the same on my master backend, which has 5 PVR-250's, and saw
the following change in what top reports, in both cases on a machine
that had been freshly booted:

Before:

    top - 15:03:00 up 1 min,  1 user,  load average: 0.91, 0.32, 0.11
    Tasks: 111 total,   1 running, 110 sleeping,   0 stopped,   0 zombie
    Cpu(s): 26.9% us, 14.5% sy,  0.0% ni, 34.1% id, 23.9% wa,  0.1% hi,  0.5% si
    Mem:   1036648k total,   335660k used,   700988k free,     7796k buffers
    Swap:  1510068k total,        0k used,  1510068k free,   177820k cached

After:

    top - 15:11:07 up 1 min,  1 user,  load average: 0.68, 0.29, 0.10
    Tasks: 111 total,   1 running, 110 sleeping,   0 stopped,   0 zombie
    Cpu(s): 25.6% us, 13.6% sy,  0.0% ni, 39.5% id, 20.9% wa,  0.1% hi,  0.3% si
    Mem:   1036648k total,   898424k used,   138224k free,     7716k buffers
    Swap:  1510068k total,        0k used,  1510068k free,   177828k cached

Geez, good thing I added that other 512meg memory stick before I started!

What's interesting about this is that I seem to now be using an
additional 898424 - 335660 = 562764k of RAM, or over 110meg/card!
This doesn't square with what's getting printed out in the logs.
Why the extra 40meg/card?

So, two questions:
(a) What's going on here?
(b) Do I even need to increase the YUV buffering at all?  After all,
    I'm using the mpeg encoder and not dumping raw frames, and the
    settings I used (the claimed "maximum" settings from the wiki)
    certainly seem to add 30meg of buffering per card there.  (I'm
    presuming that the audio & VBI stuff -is- necessary, since I
    listen to shows and keep closed-captioning data, and don't want
    that corrupted any more than the actual video stream.)

P.S.  It's a little early to tell if this has helped, because I
haven't been able to -reproduceably- stress the backend disk system
enough.  I tried recording a couple streams and watching one in as
close to real-time as possible (e.g., a few seconds' lag, but -not-
via LiveTV), and then kicked off my optimize-and-dump-the-db job.
That thrashed the disk hard enough that my frontend lost its video
stream (e.g., dropped me back to the menu), presumably because it
suddenly ran out of "almost live" video when it got delayed slightly.
But when I reentered and watched starting before it dropped until a
minute after (long after the script had finished), there were no
dropped frames, so that's encouraging.  Neither did doing -exactly-
the reschedule that had screwed me yesterday (changing Duplicate Check
method from None to Title & Description and/or back again in a case
that caused 30+ shows to be re/un-scheduled).

But reverting to only a few buffers and trying again didn't hit the
disk hard enough that time to even drop the frontend for either test;
I'll try again tomorrow, or if I can come up with something else to
stress the frontend enough.  (Most ideas I've had are running into the
problem that the OS is nicely caching filesystem I/O for me and this
machine just doesn't have very many files on it 'cause it's only a
backend.)

[Again, in case this matters, I'm still on mythtv 0.18.1.]