[mythtv-users] My experience with recording profiles and transcoding HD MPEG-2 to MPEG-4

[Yeechang Lee]

(This is part of an occasional series on how I've successfully [or
unsuccessfully] accomplished something in MythTV on my Fedora Core
4-based frontend/backend system using the ATrpms packages. I will
assume that readers can do basic Linux tasks, such as install RPMs,
edit /etc files, and generate xorg.conf. I hope to cover the mystery
areas where others may most often fall astray in.)

After months of bafflement, I've (mostly) figured out out to transcode
my FireWire- and ATSC card-captured MPEG-2 recordings into MPEG-4
versions with (mostly) satisfactory results.

WHY TRANSCODE?
* MPEG-4 files are much smaller than their MPEG-2 counterparts. In my
  experience, MPEG-4 files are about 35-40% of the size of the MPEG-2
  originals, even without resizing to smaller resolutions. Given that
  one-hour HD recordings are about 7.5GB, the appeal of reducing that
  size to about 3GB is obvious. See
  <URL:http://www.gossamer-threads.com/lists/mythtv/users/175520#175520>
  for more on my experiences with HD files' sizes.

WHY NOT TRANSCODE?
* As Robert Heinlein pointed out to the world, There Ain't No Such
  Thing as a Free Lunch. Creating MPEG-4 files costs a) time and b)
  visual quality.
  a) As I wrote in
     <URL:http://www.gossamer-threads.com/lists/mythtv/users/213820#213820>:

	My 3.0GHz Pentium 4 frontend/backend took 458 minutes to
	transcode, without resizing and at a 5000Mbps bitrate, a
	15.7GB 125-minute HDNet (18000Mbps) Movies recording to a
	5.7GB MPEG-4, a 3.67:1 time ratio and a 64% space
	savings. (Both mythbackend and mytfrontend were active at
	various times during the process.)

  b) Again quoting from the above message:

	The quality [of MPEG-4 transcodes] is OK; sometimes the sky
	appears a bit muddy due to compression blocks, as if the camera
	lens needed a wipe with a clean cloth.

     Some recordings transcode better than others; more on this later.

* Don't transcode if your frontend needs XvMC to play high-definition
  files, as there is no support for MPEG-4 hardware-assisted decoding
  in current drivers, whether for Nvidia, Intel, or Unichrome (see
  <URL:http://www.mythtv.org/wiki/index.php/XvMC>).

* There are some serious bugs in mythtranscode, at least as of
  0.19-fixes. More on these later.
   
GETTING STARTED
I began with the instructions at
<URL:http://www.myhdbox.com/mythtips/2006/05/tip-4-transcode-profiles-for-hdtv.php>.
Here are my comments:

* "[Mythtranscode is] also blazingly fast!" Perhaps, if compared to
  watching paint dry.
* "Because of all the combinations, I wanted the following Transcode
  profiles:" We will indeed go ahead and create all of the profiles
  John lists, although in practice I suspect you'll only end up using
  the 1080i, 720p, and 480p NoResize profiles.
* "From mythfrontend, select: Setup -> TV Settings -> Profile Groups
  -> Transcoders. This will drop you into the recordings profile
  screen where you will see the four default profiles: Default, Live
  TV, High Quality, Low Quality." On my system, I believe I only had
  "Autodetect from MRTjpeg/MPEG4" and "Autodetect from MPEG2." We want
  to *keep* these profiles. Thus, skip down to the mysql command-line
  example (If this is a mystery to you, refresh your memory at
  <URL:http://www.mythtv.org/docs/mythtv-HOWTO-6.html>.) and, instead
  of running one mysql command, run seven:

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values ('1080i
    NoResize','MPEG-4','Uncompressed','6');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values ('1080i
    Resize High','MPEG-4','Uncompressed','6');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values ('1080i
    Resize Med','MPEG-4','Uncompressed','6');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values ('720p
    NoResize','MPEG-4','Uncompressed','6');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values ('720p
    Resize High','MPEG-4','Uncompressed','6');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values ('720p
    Resize Med','MPEG-4','Uncompressed','6');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values ('480i
    NoResize','MPEG-4','Uncompressed','6');

* Finish with

    mysql> flush privileges;

  (I don't actually know if this is necessary, but it can't hurt.)

* Go back into mythfrontend Setup|TV Settings|Recording
  Profiles|Transcoders and enter the values John suggests for each
  entry starting with 1080i NoResize. Don't worry about not being able
  to set the bitrate exactly as he specifies; just pick the nearest
  round number. (For 1080i and 480p NoResize, I personally use 5000 as
  the bitrate, not 4600, but as John notes these values may be too
  high.) I'd use 100 for the volume, not 90.

MANUAL TRANSCODING
* Go to a future recording. Assuming it is a high-definition
  recording, in Post Recording Processing, set it to "Transcode using
  '1080i NoResize'" (if ABC or FOX, use '720p NoResize'; if non
  high-definition, select '480p NoResize').

* After the recording completes, go to Media Library|Watch Recordings,
  find the recording, hit ->, then Job Options|Begin Transcoding. Go
  to Information Center|System Status|Job Queue. You'll see the
  transcoding queued up.

AUTOMATIC TRANSCODING
If you prefer to have MythTV automatically add newly-completed
recordings to the transcode queue:

* Set the Setup|General|General (Jobs)|Default Auto Transcode
  setting. Also here, set "Default Transcoder" to the transcode
  profile you'll most likely use, probably 1080i NoResize. (It's easy
  to change the actual transcode profile for a particular recording,
  as per above, if desired.) These two settings are used to fill in
  the defaults under Post Recording Processing for recordings entries
  you create from now on.

* Return to Setup|TV Settings|Recording Profiles. If you have an ATSC
  capture card (I use a Technisat AirStar HD-5000), go to Hardware DVB
  Encoders|Default. Set 'Enable auto-transcode after recording.'

* If you use FireWire (as opposed to an ATSC capture card) to record
  MPEG-2, you'll find that Setup|TV Settings|Recording
  Profiles|FireWire Input is mysteriously empty. Thus, return to
  mysql:

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values
    ('Default','MPEG-4','Uncompressed','7');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values
    ('Live TV','MPEG-4','Uncompressed','7');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values
    ('High Quality','MPEG-4','Uncompressed','7');

    mysql> insert into recordingprofiles
    (name,videocodec,audiocodec,profilegroup) values
    ('Low Quality','MPEG-4','Uncompressed','7');

    mysql> flush privileges;

  Go to Setup|TV Settings|Recording Profiles|FireWire Input|Default
  and set 'Enable auto-transcode after recording.' (The remaining
  settings here are irrelevant.)

EDITING PROGRAM
Before the transcode starts, 'E'dit the recording to create a cutlist,
i.e. the locations of commercials, bumpers, and other portions you
want the trancoded version to omit. Mythtranscode will obey your
requested cuts.

WHAT ABOUT LOSSLESS RECORDING?
Remember those two profiles, "Autodetect from MRTjpeg/MPEG4" and
"Autodetect from MPEG2," I advised you keep? They are meant to
losslessly transcode; that is, without resizing, compressing, or
anything else. (Actually, that's not *quite* true; there is enough
minor cleaning up of duplicate frames and other things that a 15GB
file might end up shrinking by a couple hundred MB or so.) In other
words, MPEG-2 recordings stay MPEG-2.

Since "Autodetect" transcoding also obeys cutlists, this is a good way
to strip out commercials from recordings you want to preserve with the
highest-possible visual fidelity.

TRANSCODING PROBLEMS
As mentioned, MPEG-4 transcoding works better for some recordings than
for others. Most "NoResize" transcodes will look almost as good as the
MPEG-2 original, barring the occasional "muddy camera lens" issue I
mentioned above. However, the quality of the original source material
is key. For example, the Tom Cruise version of _War of the Worlds_ on
HBO HD looks *horrible* transcoded, with MPEG compression blocks
everywhere; I think it's the combination of unusually-grainy video and
an unusually-high number of special effects.

Many lossless transcode attempts just don't work for me; see
<URL:http://www.gossamer-threads.com/lists/mythtv/users/207204#207204>,
from before I figured out how to do MPEG-4 transcoding.

An issue with lossless recordings that do work is that, when using a
cutlist, the cutlist's lists of locations in the video will, get out
of sync with reality the farther one goes into the recording. For
example, I carefully edited an accurate cutlist for ABC's recent
airing of _Lilo & Stitch_, only to find that by the last third or so
of the recording mythtranscode removed significant chunks of the movie
itself. What *may* work in such cases is to first losslessly transcode
without a cutlist--thus removing those aforementioned duplicate frames
and otherwise cleaning up the recording--then transcoding to MPEG-4,
but of course that'll have to wait for the fix to the mythtranscode
UI.

There is no way, using mythfrontend, to change the transcode profile
setting for completed recordings once the recording entry expires a
day or less after completion. (This isn't an issue with TV shows and
other recurring recordings, as their recording entries don't expire.)
I know there is some mysql command to change the setting manually, but
it's beyond me. Fortunately, this has been fixed in SVN (see
<URL:http://svn.mythtv.org/trac/ticket/1416>).

The most serious issue is what Brad Templeton mentioned at
<URL:http://www.gossamer-threads.com/lists/mythtv/users/213824#213824>:

    Most people who have tried to transocde HDTV mp2 recordings to mp4
    have reported annoying audio sync problems after doing this. This
    is with mencoder which is used by myth's transcoder.

This isn't an issue with the vast majority of the transcodings I've
done over the past five weeks. However, some recordings definitely end
up this way. Two examples I've seen are: a) A 720p NoResize transcode
of _The Princess Diaries_ on ABC in which, after a video glitch about
one third through, the audio pitch started to rise so that by the end
Mia Thermopolis and her friends were in full-blow chipmunk mode. b)
_Crash_, in which the audio began stuttering at some point through the
film (quite possibly after a video glitch here as well, although I
don't know for sure). Given the _Diaries_ example, I suspect that
many, if not all, such cases occur when a glitch in the capture
process--whether over FireWire or the airwaves--produces a flaw in the
recording that, although it only causes a momentary jump in the video,
irreparably throws the audio out of sync with the video. (I've
recently upgraded the mplayer RPM from mplayer-1.0-53_pre8.rhfc4.at to
mplayer-1.0-54_pre8.rhfc4.at, which may or may not make a difference.)
Thus, it's always a good idea to zip to the end of transcoded
recordings to make sure the audio and video match up well and appear
OK.

-- 
Yeechang Lee <ylee at pobox.com> | +1 650 776 7763 | San Francisco CA US