[mythtv] Experimental HDTV Recording patch updated

[Daniel Thor Kristjansson]

I've begun cleaning it up for eventual inclusion in MythTV. It's not
ready yet. But I would love it if some people gave it a test.

  http://www.mrl.nyu.edu/~danielk/mythtv/hdtv-recorder-v10.tbz

By default this patch no longer prints out as many of the ATSC Tables,
but you can change this behavior in the TSPacketProcessor::HandleTables()
function. (You can also disable the ones it does print out, look for the
if (0|1) {} sections.) My goal is to both simplify debugging the code
and add features. I won't be doing any coding on this for the next two
weeks so I thought it might be a good time to solicit comments.

The way it works is that HDTVRecorder::ProcessData() instanciates a
TSPacketProcessor class for each ts packet it finds in the stream (there
is be one every 188 bytes unless the stream is corrupted.) It then calls
TSPacketProcessor's Process function. This function hands the packet off
to HandleVideo(), HandleAudio() or HandleTables() depending on the PID.

PID's 0 (PAT) and 1fff (PSIP) are always handled by HandleTable(), this
lets it see the initial PAT table, and save the PMT PID. Once the PMT is
seen it begins listening to the Video and Audio PID's. It does this
better than the current implementation. It still only looks at the PMT
and not the ATSC tables for this determination though, so you can still
pick the wrong language. This was done because as Doug suggested the
ATSC tables are not always correct. But at some point I'd like to add
multi-language support to MythTV's recording, this will involve some
ffmpeg fixing though.

In HandleVideo() and HandleAudio() it just writes the TS Packet to the
stream (with the PID rewriting hacks), it does not try to assemble PES
packets to check the CRC. But on the HandleTables() path it always
creates a PES packet, even if it consists of just one tspacket. This
lets us check the CRC, and lets discard the packet if it is not a
current PSIP Table (We might want to look at the 'next' packets in the
future because they allow for atomic updates of all the PSIP info, but
for now it's just noise.) Once we've determined the table is valid, we
check if it's redundant, if it is we toss it. If it's good there are
several options.

For good PAT or PMT tables we create a new PAT or PMT table with just
our program, and with fixed video and audio PID's. Via the
REWRITE_BASEPID_HACK and REWRITE_AVPIDS_HACK defines, you can disable
the fixed video and audio PID's hack, which allows HandleVideo() and
HandleAudio() to avoid PID rewriting the ts packets, but this breaks
ffmpeg in LiveTV mode. This bug in ffmpeg needs to be fixed for multiple
language support though.

For the MasterGuideTable we record it's version, and parse it. It
functions as the PAT of the ATSC tables, it tells us on which pids we
can find the various ATSC tables, their type, and what version they are
at. Once we've parsed it we add all the tables to the list of pids we
are listening to, just as we add the PMT pid from the PAT and the audio
and video pids from the PMT table to the list of pids we are listening
to.

The other ATSC tables are parsed only if we have the MasterGuideTable. I
haven't done anything to try to parse the Cable Virtual Channel Table,
the Region Rating Table, System Time Table, or the directed channel
change tables. I'm assuming anyone with MythTV is setting their time
with NTP or something like it, but for non-internet connected boxes
perhaps we should parse the STT at some point. The tables I do parse are
the Terrestrial Virtual Channel Table, the Extended Information Table
and the Extended Text Table. The VCTt contains the names of streams
being multiplexed and some additional information (subchannel, freq,
location code, etc.) The Extended Information Table contains the program
guide for the streams, and usually program data for the broadcaster's
NTSC channel. The Extended Text Table contains program data and
subtitles. I'm only decoding Unicode strings, not compressed strings,
but I haven't run into any compressed strings yet.

There are also a number of ATSC Descriptors I parse, these are found in
various tables, but most importantly the Virtual Channel Table. The ones
I parse are: ISO639 Language Code, Multiple String Structure,
Caption Service Descriptor, Component Name Descriptor, Registration
Descriptor, and Audio Stream Descriptor. I do not parse the
ac-3, stuffing, content advisory, extended channel name, service
location, time-shifted service, component name, DCC departing request,
DCC arriving request, or DRM control tables, mostly because they are
used by tables I'm not parsing yet. I also don't parse MPEG descriptors,
but only because I don't have a copy of the standard. I'm indebted to
the ATSC for putting their standards online for free.

The Caption Service Descriptor will allow us to find the PIDs with the
ETT tables containing the captions. The Audio Stream Descriptor should
let us find the audio stream we want, not just the first one listed in
the PMT. I don't know why would need the Registration Descriptor but
it's trivial to parse. The ISO639 Language code and Multiple String
Structure are of course essential for parsing many of the other
descriptors and tables.

atscdescriptors.{h,cpp}, atscdescriptorsmap.cpp: ATSC Descriptor Classes
atsctables.{h,cpp}: ATSC Table Classes: MGT, VCTt, EIT, ETT
mpegtables.{h,cpp}: MPEG Table Classes: PAT, PMT
tspacket.{h,cpp}: TSPacket and TSPacket_nonconst (inherits TSPacket)
tspacketprocessor.{h,cpp}: TSPacketProcessor (inherits TSPacket_nonconst)
pespacket.{h, cpp}: PESPacket (can assemble itself from TSPackets)
mpegstreamdata.{h,cpp}: MPEGStreamData (handles simplified PAT & PMT creation.)
atscstreamdata.{h,cpp}: ATSCStreamData (inherits MPEGStreamData)

I also should mention that keeping track of information about the
stream is done by MPEGStreamData and ATSCStreamData, not the
HDTVRecorder class as before. This is being done to simplify multiple
stream recording once we get there (you can record multiple "channels"
on each frequency, but this requires work on the scheduler and a little
more work on HDTVRecorder.) I also separated out MPEGStreamData from
ATSCStreamData in the hopes that the DVB folks can use some of this
work.

So cleanup work that still needs doing... I'm thinking perhaps the
TSPacketProcessor stuff should go into ATSCStreamData. Also things like
finding a GOP should be taken out of HDTVRecorder, and placed in
MPEGStreamData, until we can get rid of it (GOPs are not required in
MPEG2, and are sometimes absent.) TSPacket_nonconst is only needed
because of the rewriting hacks for ffmpeg, perhaps I should get rid of
this and simply use const_cast's within the #if ..HACK.. sections.

Other than cleanup, what I want to do before trying to get this into
MythTV is to actually parse out the subtitles and send them to the
screen. I might also want to cache some information so "tuning" to a
channel in LiveTV mode can be quicker. Right now we have to wait for a
PAT before we even wait for a PMT, and then we need to wait for a PMT
before we wait for a GOP in the video stream, all before we begin
sending data to the ringbuffer. It is a lot of waiting we mostly don't
have to do.


A Note on terminology: I've used "ATSC Tables", to describe what are
normally referred to as PSIP tables. I did this because the PAT and PMT
are also referred to as PSIP tables, but are almost completely unrelated
to the other ATSC specific PSIP tables. They are also handled very
differently in this code.

-- Daniel