[mythtv] Conversion in the background, can it reduce systemrequirements?

[Henk Poley]

> Van: Geoffrey Hausheer <ou401cru02 at sneakemail.com>
> 
> Since, in theory, we'll want multipass encoding, the transcoding can't
> start until the recording of a show stops.  that means that if you record
> 3 2-hour movies in a row (as I recently did for 'Children of Dune') I
> need enough disk space for 6 hours of rtjpeg (or whatever) movies at best
> quality, as well as the final compressed form.
> 
> So my setup is that i have about 40GB for 'temporary' data, and 160GB for
> final-storage.
>
> at RTJPEG quality 255 at 640x480, I believe it runs about 10GB/hour, and
> on my 1.6GHz machine, I can encode to ~divx quality in about 1/2 real
> time, so I can record as many 2-hour movies back-to-back from a single
> encoder as I like, but don't have enough space to do the same with 3hour
> movies, nor to use two encoders simultaneously for more than 2hours.

You can calculated the amount of left recording time. Even better,
calculate the amount of space that can be freed at the time of the next
recording. (nettoFreeSpace is the amount of free space minus the amount
needed for pending reencodes etc.)

maxRecordingTime = nettoFreeSpace / (recordedSizePerHour +
reencodeSizePerHour)

So you could safely record, say 200GB / (10GB/hr + 1.5GB/hr) => ~17h20.
Since you normaly record in smaller chunks (not in big multiple hour
blocks), you could record a little bit more. But then you would desperately
need to leave your machine chunking away a couple of hours to free some
space. And probably a smarter space conflict resolution.

A scheduler solution:

Recording rule:
Only record a show if there will be enough space left for reencoding.

Scheduler rules:
- Currently not or barely enough free space:
Find the biggest pending reencode that can be done before the next
scheduled recording. Adding in a new recording that will start earlier in
time than the reencode is finished could bring troubles. But...

You could 'stop' the transcoder and start a new one if there's still enough
time, and this smaller chunk of free space will also suffice.

Else you should bring up a space conflict resolution screen. Problem is
that space shortage can occur on times that the user is not available (eg,
using MythTV). Is there a way to prevent that from happening?

- Next recording(s?) will fit into free space:
Fire up the biggest pending reencode. 'stop' (pause) it when a recording is
running. The question is, how much recordings ahead will/need to be checked
before you decide that you have "enough free space". All? no...

Something left to solve...

> There is also a secondary memory issue.  Since multiple threads are
> runing at the same time, you can suck up a lot of memory pretty quickly,
> so I don't think I'd want to use the transcoder on a machine with less
> than 256Megs of RAM.

I've recently seen a program that would 'stop' a program when certain
criteria are met, for example CPU load. The kernel won't give time to that
process anymore, and can even swap it to disk. The process is resumed
afterwards. This way it won't interupt running tasks.

The most natural way would be to let the MythScheduler handle the
start/stop/resuming of the reencode threads. Possibly only based upon
scheduled recordings, so indepedent upon CPU load. Harddisk activivity
could interfer with normal encoding, I think.

Having this in the central scheduler might be handy for the shutdown/sleep
thingy too.

To make things easier we could add a column to the scheduling which tells
how much space is needed at runtime, and how much will approx. be used by
the scheduled entry? This last one can be negative... :-)

	Henk Poley <><


PS: I've been interrupted at least 6 times when writing this email, so I
hope this all makes sense...