[mythtv] Multi-pass encoding (was: [PATCH] Add new MPEG4 encoding options)

[Geoffrey Hausheer]

On Sat, 29 Mar 2003 12:35:34 +0100, "J.D. Bakker
bakker-at-thorgal.et.tudelft.nl |mythtv/1.0-Allow|"
<kj88hcidor0t at sneakemail.com> said:
> Separating the encoding options is a possible first step. In most 
> situations an end-user could care less which coding options are used; 
> having general classes ("stereo movie", "bottom shelf storage") would 
> be enough for Joe Public (or me, for that matter). These classes 
> could expand to single-stage or multi-stage encodings.
This should not be hard.  Having multiple encoding classes is already
supported, we just need way to tag the first and 2nd encoders.  As far as
I can tell so far, the recording-profiles are automatic.  Never having
tried to use 2 cards at the same time, I'm not sure, but I can't find
where in the mysql database the recording profile is tied to a card. 
Perhaps adding a field to 'cardinput' to select the recording-profiles
would be the best way.

> Once the classes are defined I do not see major obstacles for the 
> transcoding. The moment the recording ends a (nice 19) background 
> transcoding session can start to a temp file; this should not impact 
> systems with modern hard disks. Once the transcoding is finished and 
> there are no users watching the recording, the new file can 
> atomically replace the old one.
This (to me) looks like more work.  While the encoding/decoding code is
all there already in libmythtv, I'm not familiar enough with it yet to
create a thread that will do the transcoding in the background.  I was
actually considering stopping  the 2nd pass whenever encoding is going
on.

> 
> AFAICT people want advanced codecs like MPEG4 to reduce storage size, 
> right ? In most cases it should not matter if a larger temp file is 
> used in the process.
I am not convinced of this.  In general, I want the 2nd pass to allow me
to get better quality encoding at the same bitrate (so that I can store
nice-looking 640x480 shows, better than I can encode at real time on my
processor).  However, storing lossless data for a 2-3-hour movie (or even
lightly rtjpeged) requires a huge amount of disk-space.  In a case like
this I likely need a very large temp drive (perhaps even a separate
drive).  That adds more cost to the system.  I'm not sure how much temp
space is needed, but I'd guess it  is over 30GB for my setup.

> I see only three corner cases: (a) the remote frontend is too slow 
> for the stage-2 codec, (b) the network is too slow for the stage-1 
> stream (in the case of LiveTV or playback before the second stage has 
> finished) or (c) the user records more shows than the stage-2 codec 
> can keep up with.
> 
as a single piece of data for (a) my 366PII can decode and deinterlace a
640x480 MPEG4 stream, so I doubt this is a real issue.  It  does struggle
at decoding mpeg2 at dvd resolution, which is a format I could see
wanting the transcoder to go to (since I could foresee wanting to burn
episodes/movies to DVDR).  But as you say, we'd likely want to do all
pre-processing (deinterlacing/cropping) in the 2nd pass.

For (b) I don't think this is really a concern.  I wouldn't recommend
this method for live-tv, since there is no 'end' to start  the
transcoding at.  And this 2nd pass transcoding is only an option, not  a
requirement.  If the user needs this ability, and doesn't have the
hardware, they need to make choices. 

(c) is just a disk-space limitation, and I think it is non-trivial (in
that somecodingis needed).  Ideally we need the ability to read
recordings from multiple drives, and I hven'tseen a way to do that yet. 
We'd also need to determine what to delete as disk space fills up (do you
delete archived movies?  You'd need to delete 10 for every 1 new recorded
movie).  And I'm predicting that myP4-1600can do about 12fps with the
encoding parameters I'd ideally use.  When it is niced down, this will
likely drop by a 3rd.  That means it'll take ~6 times longer to encode a
movie than to record it, assuming we are still recording.  So if I want
to encode a marathon on TNT,I better have a LOT of disk space on hand.

> And yes, I am actually willing to write some code for all this, if 
> this were the Way To Go.
Well, I've always been told 'put your code where your mouth is'  I will
probably attempt to get a threaded transcoder working sometime soon as a
proof of concept.  Of course my mythbox lives at a different home than I
do, so actually testing it is a pain in the butt.
> 
> JDB
> [who also has a thesis to finish...]
.Geoff
-- 
  Geoffrey Hausheer
  XXXXXXXXXXXXXXXXXXXXX