[mythtv-users] Active EIT data collection gives some defective recordings; passive scanning not working.

[Stephen Worthington]

On Sat, 2 Mar 2019 14:09:19 +0000, you wrote:

>I posted about this last year on the dev list and have just encountered 
>it again.  I'm using DVB-T in the UK with one PCI single-tuner card and 
>a twin-tuner USB bevice, running master.
>
>https://lists.gt.net/mythtv/dev/618967
>
>Last year I found that, during my post-processing of recordings from the 
>USB device, project-x often reported missing sync bytes and/or packets 
>out-of-order.  Although it applied its best corrections, some defects 
>were still noticeable.  They seemed less frequent when I stopped active 
>EIT scanning with the USB tuners, using only the PCI card.
>
>Recently the number of defects rose again.  I may well have made changes 
>during mythtv-setup, trees may be growing further into my line of sight, 
>or perhaps there might be confusion between the numbering sequences of 
>real and virtual tuners;  I haven't really looked for that.
>
>I tried disabling active scanning on *all* tuners.  I had understood 
>that passive scanning would continue to collect EIT data, for all muxes, 
>as long as some recordings were being made - but yesterday only 2 days 
>had data and late-edits that might have been expected hadn't been done. 
>Things recovered after re-enabling active scanning by the PCI card only.
>
>John P

I have never heard of a passive EIT scanning option.  The only way EIT
scanning works that I know of is to take over a tuner temporarily.  In
theory, it would be possible to get a tuner that is recording to also
receive the EIT data at the same time, but I do not think MythTV does
that.  However, I thought that use of a tuner for EIT reception gave
way automatically when the tuner was needed for a recording, and
started again automatically afterwards.

When I was using USB tuners, I found that they were less reliable than
PCI or PCIe tuners, due to the way the USB cables work.  I had USB
tuners stop working when the cables were disturbed, or after a while
when there was force on the cable from the weight of the cable or
tuner.  The rate of such problems occurring seemed to be associated
with the USB socket on the PC - some were much worse than others, so I
always thought that such sockets were a slightly looser fit, and the
cable could then move so that eventually one or more pins stopped
making proper contact.  I got better reliability when I changed to
using USB cables that had a tighter fit.  And on my mother's MythTV
box, I have one USB tuner that never causes problems that way at all.

However, I never had bad recordings caused by USB problems.  What
happened was that the USB tuners always worked, or did not work.  They
did not appear to be working, but give bad recordings.  On occasion, I
would have a USB tuner stop in the middle of a recording, but I never
had bad data or missing data in a recording that did not stop.

My recommendations for using USB tuners:

1) Use close fitting USB cables.
2) Make sure that the tuner is flat on a surface and not putting its
weight on the USB cable.  If necessary, put something underneath the
tuner to support it higher so that the weight is off the cable.  Or
use a longer USB cable.  Make sure the aerial cable into the USB tuner
is not pulling on the tuner.  I have not tried this, but it might be a
good idea to tape down the aerial and USB cables to the surface you
have the tuners on.
3) Make sure the tuners and cables are not going to be bumped (eg when
dusting or vacuuming).  Bumping the aerial cable bumps the tuner also.
4) Make sure the cat is not allowed anywhere near the USB tuners. Kids
are a bad idea also.

My ultimate solution for greatest reliability was to change to using
PCIe tuners.  I bought an 8 tuner DVB-T2 PCIe tuner card and an 8
tuner DVB-S2 tuner card.  They have both been completely reliable,
even though the DVB-T2 card is actually running externally to the PC
on the end of a PCIe x1 extender cable (which is actually a USB 3
cable).  I had to do that due to the only remaining PCIe slot being
under the heatsink of my silent Nvidia card.

There is another way to do EIT EPG grabbing, which is the way I used
to do it.  I ran a script external to MythTV that once a day grabbed
the EIT data and saved it as an XMLTV file.  Then mythfilldatabase was
run to read the XMLTV data.  That also allowed me to do extra
processing on the EIT data before running mythfilldatabase - for
example, to remove the obnoxious "All New" that is often put on the
front of the subtitle data here.  I have since had to change to using
the OpenTV data from my satellite provider, as their EIT data stopped
providing full coverage of all their channels.  But that works in a
similar manner, with the EPG script running a program that grabs a
tuner and collects the OpenTV data and stores it as an XMLTV file.

The downside of using external EPG collection like that is that you
need to be able to use a tuner when MythTV is not using it.  The easy
way to do that is to simply have a spare tuner that is not assigned to
MythTV, which is what I am now able to do due to having 8 of each sort
of tuner.  But it is possible to work out when MythTV is not going to
be recording, and how long the gap in recordings will be, and then use
a tuner when there is a large enough gap for the EIT data to be fully
collected.  It should only take a few minutes at most to collect all
the EIT data.  I wrote a Python program that tells me when there are
gaps in my recording schedule.  It is on my web server here:

http://www.jsw.gen.nz/mythtv/gaps

So you could use it to work out a safe time to run EIT collection.
Note that the gaps program does not see the extra time at the end of
recordings that is added by mythbackend from the "RecordOverTime"
setting.  That setting is normally 60 seconds and no-one usually
changes it, so you need to allow an extra minute beyond what the gaps
program says is when all the recordings will be finished.  Similarly,
it does not see the extra time at the start of the next recording from
the "RecordPreRoll" setting, again normally 60 seconds.

External EIT data collection is done use software such as epgsnoop:

https://launchpad.net/epgsnoop

Where there is MHEG5 EPG data (such as here in New Zealand where it is
the official source of DVB-T free-to-air EPG data), that can be
collected using mhegepgsnoop:

https://sourceforge.net/projects/mhegepgsnooppy

On Windows, there is the EPGCollector program which handles EIT,
MHEG5, OpenTV and several other similar EPG data formats:

https://sourceforge.net/projects/epgcollector