[mythtv] anyone with mythtranscode issues, please read

Geoffrey Hausheer mythtv0368 at phracturedblue.com
Mon Mar 8 18:27:21 EST 2004

On Mon, 8 Mar 2004 23:06:09 +0100, "Paul Koster" said:
> From: "Geoffrey Hausheer" 
> congrats :)
> > As I have no interest in reading a whole months worth of mailing list
> > postings, I'll just say that if you have had any mythtranscode issues
> > that have not been resolved in the past month or so, and you'd like me to
> > look into it, I'd recommend reiterating your problems either to this
> > list, or in a personal mail to me.
> I'll give a summary of two messages which I've posted before and can be
> found here:
> http://www.gossamer-threads.com/archive/MythTV_C2/Dev_F10/mythtranscode_MPEG2-%3EMPEG2__issues_P107556/
> I included a patch to fix handling command line arguments. It fixes wrong
> offsets in argc etc. and also adds a delete() for a corresponding new().
I'll look into that.  should be easy enough.

> I also worked on other parts of mythtranscode, mainly the muxing, but
> that
> hasn't lead to good results. So the problems I repeat below are still
> valid.

Yeah the muxing is very hokey.  I will answer the below from memory, but
as I ahven't looked at the code for quite some time, I may be talking out
my ass on some of this stuff.

> - 1 file failed the transcoding because it encountered a block type of
> 0xba
> indicating a pack header

we shouldn't run into pack headers in theory because the demuxer should
remove them.  So the demuxer is likely broken.  Adding a patch to work
around this shouldn't be too hard though.

> - 2 files failed the transcoding because mythtranscode encountered a
> block
> type of 0xf4 (I've not been able to find out what kind of object this is.
> Something must be wrong in the input stream or in the demuxing, I could
> not
> find it in the mpeg spec.)

'0xf*' frames are reserved I beleive, but more likely this isn't a valid
frame header, and either the stream or demuxer is broken.  fixing this
may be harder, as I rely on the demuxer to work right.

> For one of the files that completed the initial transcoding successfullly
> I
> repeated it with a cutlist (cutting of the end of a show) and tested the
> resulting file in a number of players:
> - it seems that the cutlist works okay, except that the last (few)
> frame(s)
> are still in there just at the end of the resulting file (this should be
> an
> easy fix)

The en-of-file stuff isn't handled very rigorously, so I have no idea
what happens there.  It should be easy to fix though.

> - the resulting file suffers from many artifacts at certain points
> (unrelated to the cutpoints ) when playing in Windows Media Player with
> the
> Elegard MPEG2 decoder (compared to the original)

This is likely due to not correctlyu applying padding to the stream (the
padding keeps the correct bitrate given that audio and video packets
occur at substatially different rates).  In theory the muxer would do
this for us, but libavformat doesn't do so currently, and I haven't
written the needed code to do it right.  I haven't even looked into how
to do it right yet.

> - the resulting file suffers from minor artifacts at certain points
> (unrelated to the cutpoints ) when playing in PowerDVD (compared to the
> original)

Likely the same as above.  I know that the streams aren't strictly MPEG2
compliant as is, it is just non-trivial to fix (and i was sort of hoping
the libavformat guys would do it for me)

> For the 3 files that failed processing:
> - the loop in mythtrans.cpp can't handle unknown object types and will
> get
> into an endless loop because it does not increase framelen, etc.
> - I tried to include support for type 0xba, but it seems that just
> skipping
> 12 (or 16?) bytes in the stream does not solve the problem as I've seen
> in
> comparable demuxers.

This is likely in part due to relying n libavformat's demuxer.  I'd need
to see the actual video file in question to decide how to fix it.

> - should program streams work or is it just luck that they worked and is
> the
> current code still aimed at transport streams (as mentioned somewhere in
> a
> very old thread)?

both PS and TS streams shoul work correctly with the current code (the
output will always be a TS stream)....Where 'correct' is defined as being
able to run through the tool, and the output being playable with mythtv.

> All recordings were made with my PVR 250 ivtv 0.19 576x576 MPEG2 Program
> Stream 5000 kbps video, 384 kbps audio (mp2). To do the transcoding I
> used
> mythtranscode -i filename.nuv .
> After the initial testing above I dived into the code itself to see what
> happened, especially with the artifacts introduced with the transcoded
> stream (even without cutlists!):
> - the muxer throws away the first audio frames
> - the muxer drops the last video frames

As I said, there is no rigor in how the end of the stream is handled. 
However, the beginning should work correctly (sort of).  the code should
remove all audio frames before the first video frame.  however, in theory
this will only happen once, so rerunning on a transcoded file shouldnt
remove any more frames from the begining.  I don't think there is anthing
wrong with this policy (why would you need audio frames before the video
started?), but i don't have any proof that the code actualy does what it
was designed to do.

> - mythtranscode does the above again and again with as results that files
> get smaller and smaller (it's not too hard to fix this actually), also
> artifacts increase.

Artifcats increasing is somewhat strange.  In theory a file transocded n
times should be identical to the first time.  (whether they are correct
or not is a different issue)

> Actually I think we should not throw away data too
> easily, but only when there's only audio or video at a certain time).

Well, the idea is to make a 100% compliant DVD stream.  If there is data
present which prevents us from obtaining that goal, it needs to be
removed.  However, the code still needs a lot of work to reach the goal. 
Some data is thrown away by the demuxer (the padding data), and there is
currently no way to get it back (which will result in drastically smaller
(and less compliant) files than the original.

> - the way audio and video are muxed is completely unclear to me. I'd
> expect
> that muxing is done using dts values, but these are not taken into
> account.

libavformat did not support dealing the dts when I originally wrote the
code, and the timing of MPEG2 streams is primarily dependant on PTS. 
However, the DTS defines how packets should be spaced (and I think is
what is causing much of the incompatibility).  This needs quite a bit of
work still.

> - generated streams cannot be processed again, because one of the errors
> above occurs (or running out of buffers for example)

That's not good. I am sure it used to work.

> - some more that I do not remember by heart
> Most of the problems seem to relate to the muxing and related buffering
> in
> mythtranscode. If I've understood the code well, libavformat only writes
> the
> packets and sets some header values but is not responsible for muxing.
> Should we rely on a good library to do the muxing, including taking care
> of
> the mpeg buffer model? (Somebody replied to my email in above link to
> suggest the use of such a multiplexer.)

Yes, that would be great.  I believe there is such a tool in dvb-tools
(dvb-mplex?), and someone mailed me before I left about ripping it out
(and perhaps replacing the libavformat stuff with it).  This is probably
the right solution. (possibly it would be easier to rewrite the
mpeg2trans stuff altogether using those routines).

I haven't looked at your previous posts yet, and will do so soon.


More information about the mythtv-dev mailing list