[mythtv] Conversion in the background, can it reduce systemrequirements?

[Geoffrey Hausheer]

On Sat, 12 Apr 2003 13:15:37 +0200, "Henk Poley hpoley-at-dds.nl
|mythtv/1.0-Allow|" <r0hvhd7uft0t at sneakemail.com> said:
> > Van: Geoffrey Hausheer <ou401cru02 at sneakemail.com>
> You can calculated the amount of left recording time. Even better,
> calculate the amount of space that can be freed at the time of the next
> recording. (nettoFreeSpace is the amount of free space minus the amount
> needed for pending reencodes etc.)
> 
> maxRecordingTime = nettoFreeSpace / (recordedSizePerHour +
> reencodeSizePerHour)

This is a different tact than the way I've gone.  My current solution is
to have two directories, RecordFilePrefix and RecordAltFilePrefix.  Myth
works as-is today, and the transcoder writes into the 'Alt' directory. 
mythbackend knows to look in the Alt dir before the normal dir, since
there are times when both files are availiable concurrently.  Your
solution might actually be easier, but I have multiple drives to record
to, so this maximizes my space.  I haven't even looked at checking the
free-space on the 'Alt' drive yet, so I'll need a second max-usage
variable which will make things more complicated.

> A scheduler solution:
> 
> Recording rule:
> Only record a show if there will be enough space left for reencoding.

Yes I still need to do this.

> Scheduler rules:
> - Currently not or barely enough free space:
> Find the biggest pending reencode that can be done before the next
> scheduled recording. Adding in a new recording that will start earlier in
> time than the reencode is finished could bring troubles. But...

This is much more complex.  Today multiple transcoders run at the same
time as seperate processes (not threads) all at lower priority.  There is
no way to know how long they will take to complete, so it is possible
that they will be done before the upcoming-program runs out of space, and
it is possible they won't.  My current implementation will have seperate
rules for the first encoding and second encoding dirs, so this is a
non-issue for me.

> Else you should bring up a space conflict resolution screen. Problem is
> that space shortage can occur on times that the user is not available
> (eg,
> using MythTV). Is there a way to prevent that from happening?

I haven't even begun to deal with mythfrontend yet, but my current
implementation only needs changes to the setup screen (I need to find a
way to make it easy to use too)

> I've recently seen a program that would 'stop' a program when certain
> criteria are met, for example CPU load. The kernel won't give time to
> that
> process anymore, and can even swap it to disk. The process is resumed
> afterwards. This way it won't interupt running tasks.
> 
> The most natural way would be to let the MythScheduler handle the
> start/stop/resuming of the reencode threads. Possibly only based upon
> scheduled recordings, so indepedent upon CPU load. Harddisk activivity
> could interfer with normal encoding, I think.

This would be much easier if the libavcodec were thread-safe.  Today I
need to do a fork() for each transcoder which makes it much more
difficult to control (I could do IPC, but that is more pain than it is
worth currently).  If that issue gets solved, the transcoders can become
threads again, and managing the transcoders will be easier.

However, I was playing around, and on my system, I can have 10
transcoders running at once and still encode fine.  obviously the
transcoders don't run very fast, but I only have 384Megs of RAM, so I'm
not too concerned about this at the moment.

This happens for instance the first time you switch on transcoding, and
myth decides to transcode all the files you've got recorded at the
moment.  This may not be a desired effect though.  Easy enough to
disable.

> 
> Having this in the central scheduler might be handy for the
> shutdown/sleep
> thingy too.
> 
> To make things easier we could add a column to the scheduling which tells
> how much space is needed at runtime, and how much will approx. be used by
> the scheduled entry? This last one can be negative... :-)
> 

Another option, as you imply, is to queue up transcoding jobs, and only
execute one at a time.  This is definetely an option, and perhaps a
better overall solution.

Anyhow, I'll clean up my current code and send it out so people who are
interested can play with it.  

Thanks,
.Geoff
-- 
  Geoffrey Hausheer
  XXXXXXXXXXXXXXXXXXXXX