[mythtv] Has anyone tested channel scanning in a us-cable Clear QAM environment?
Klaas de Waal
klaas.de.waal at gmail.com
Sun Mar 1 14:40:25 UTC 2020
On Tue, 11 Feb 2020 at 01:22, faginbagin <mythtv at hbuus.com> wrote:
> On 2/10/2020 2:53 PM, Klaas de Waal wrote:
> > Hi faginbagin,
> >
> > On Mon, 10 Feb 2020 at 03:15, faginbagin <mythtv at hbuus.com <mailto:
> mythtv at hbuus.com>> wrote:
> >
> > On 2/8/2020 3:52 AM, Klaas de Waal wrote:
> > > Hi faginbagin,
> > >
> > > On Sat, 8 Feb 2020 at 02:39, faginbagin <mythtv at hbuus.com <mailto:
> mythtv at hbuus.com> <mailto:mythtv at hbuus.com <mailto:mythtv at hbuus.com>>>
> wrote:
> > >
> > > On 2/7/2020 1:55 PM, Klaas de Waal wrote:
> > > > On Fri, 7 Feb 2020 at 08:00, faginbagin <mythtv at hbuus.com
> <mailto:mythtv at hbuus.com> <mailto:mythtv at hbuus.com <mailto:
> mythtv at hbuus.com>> <mailto:mythtv at hbuus.com <mailto:mythtv at hbuus.com>
> <mailto:mythtv at hbuus.com <mailto:mythtv at hbuus.com>>>> wrote:
> > > >
> > > > I ask because I can't get mythtv-setup to find any
> channels. I first
> > > > tried a scan before running mythfilldatabase as
> mentioned in this email
> > > > thread on mythtv-users:
> > > >
> http://lists.mythtv.org/pipermail/mythtv-users/2020-January/402291.html
> > > >
> > > > I just tried another channel scan after running
> mythfilldatabase.
> > > > mythfilldatabase added lots of channels and listings to
> the database,
> > > > but the channel data looked incomplete. So I did a full
> channel scan,
> > > > which said it was scanning for ATSC channels, not QAM,
> and found no
> > > > channels. I can't find any channel scan settings that
> distinguish
> > > > between ATSC and QAM.
> > > >
> > > > Not surprisingly, the channel data still looks wrong.
> For example, all
> > > > of the channel records have a NULL mplexid and the
> dtv_multiplex table
> > > > is empty. Has tuning a clear qam channel changed so much
> that channel
> > > > records don't need to point to valid dtv_multiplex
> records any more?
> > > >
> > > > FWIW, both "General -> Locale Settings -> Channel
> frequency table" and
> > > > "Video Sources -> MyVideoSource -> Channel frequency
> table" are set to
> > > > us-cable and the listings grabber is "Multinational
> (Schedules Direct
> > > > JSON web services w..." and the listings data looks
> correct.
> > > >
> > > >
> > > > It is interesting to know which version of MythTV you are
> running. There are lots of changes in the latest master / v31 compared with
> previous versions.
> > > > You might have missed the Scan Type/ Full Scan settings, see
> the end of this thread:
> > > > https://forum.mythtv.org/viewtopic.php?f=36&t=3567
> > > > Also described here:
> > > > https://www.mythtv.org/wiki/Channel_Scanning#Scan_Type_2
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > mythtv-dev mailing list
> > > > mythtv-dev at mythtv.org <mailto:mythtv-dev at mythtv.org>
> <mailto:mythtv-dev at mythtv.org <mailto:mythtv-dev at mythtv.org>>
> > > > http://lists.mythtv.org/mailman/listinfo/mythtv-dev
> > > > http://wiki.mythtv.org/Mailing_List_etiquette
> > > > MythTV Forums: https://forum.mythtv.org
> > > >
> > >
> > > Hi Klaas,
> > >
> > > My apologies for not mentioning the version, it's v31 from the
> ubuntu PPA, 2:31.0~master.202002051824.e81d4cc~ubuntu19.10.1 and the distro
> is lubuntu 19.10 with the lxqt desktop.
> > >
> > > Both the forum thread and images got me on the right track.
> Instead of scanning 51 ATSC channels, I'm now scanning 130+ QAM-256
> channels. It's still in progress. I'll report back once I've done further
> testing.
> > >
> > > But I must say the new settings UI is quite confusing and a
> departure from how other parts of mythtv's UI work. It hadn't occurred to
> me that right arrow would do something different than OK. Also, that left
> arrow behaves differently than ESC.
> > >
> > > Another point of confusion: after navigating to the scan type
> page and changing the frequency table and modulation to cable and qam-256,
> followed by pressing ESC and being asked if I want to save my changes, and
> saying yes, why am I kicked out of not only the scan type page, but also
> out of the channel scan page? And when I go back into the scan type page,
> why are the frequency table and modulation settings back to the original
> broadcast and 8-vsb settings? Maybe something can be done to explain these
> points in the help text and the wiki?
> > >
> > > And, if I've specified us-cable as the channel frequency table
> in the channel scan page and also in the Global -> Locale page, why does
> the frequency table in the scan type page always revert to Broadcast, not
> to mention 8-vsb modulation? Seems to me there's room for improvement in
> setting more reasonable defaults based on the us-cable setting as well as
> saving the settings the user specifies?
> > >
> > > Food for thought?
> > >
> > >
> > > I am glad that the scanning does work for you now.
> > > About your painful experiences with the GUI, you have made a good
> summary of what can be improved. It is largely in line with what I have in
> mind. For the time being I have documented the way it works in the WIki in
> https://www.mythtv.org/wiki/Channel_Scanning but I do not expect people
> to actually read all of this, But it is there for when desperation sets
> in....
> > > Interesting is also that most of the changes in v31 are "under the
> hood", the GUI is not really changed since v30 except for a few new options.
> >
> > Hi Klaas,
> >
> > I haven't used mythtv-setup to do channel scans in years. My cable
> company has only made incremental changes over the years, most of which
> haven't been important to me. When they do make a change I care about, I've
> used hdhomerun_config to scan for changes, then crafted sql inserts and
> updates to the channel and dtv_multiplex tables. I tend to upgrade mythtv
> with every other release, e.g. 0.27 to 29 and now to 31. And I didn't need
> to spend much time in mythtv-setup to get 29 configured for my needs.
> >
> > Now I'm getting ready to upgrade to 31 and find it necessary to
> re-learn how to setup a video source because of the elimination of the
> DataDirect video source. So, I figured it was also time to see how channel
> scanning works these days. I had hoped, since the channel table was
> populated by the new Schedules Direct video source, that maybe the channel
> scan would be able to update those channel records. But it didn't happen,
> at least not in my case. Instead, it just added all new channel records for
> those found in the scan.
> >
> > In my case, the channel scan only found 68 channels when there are
> actually just over 100 channels. Maybe increasing timeouts would help. But
> if I've still got to populate the xmltvid columns, then I think I'm better
> off starting with my production system's channel and dtv_multiplex tables
> and using the switch_to_json.py script Bill Meek sent me to automate
> conversion from the old to the new xmltvids.
> >
> > Many Thanks for your help in getting channel scanning working.
> > _______________________________________________
> >
> >
> > Thanks for testing on USA cable. I appreciate getting feedback on this
> since I can test only on DVB sources. I have made changes over time that
> also can influence the ATSC behaviour but there has no serious regression
> testing been done on this.
>
> I'd be happy to help.
>
> > About finding only 68 channels when there are actually just over 100
> channels.
> > Could you try with larger timeouts to see if that makes a difference? In
> v31 the default timeout values have increased from 1000/3000 to 3000/6000
> but maybe even bigger timeouts are needed. Note that this new default is
> only for new capture cards, the timeout values of existing capture card
> entries are not changed.
>
> I have tested with larger timeouts, 6000/9000. No change and I'm pretty
> sure timeouts aren't the issue. However, I made some changes to the tuner
> and network environment to rule out some questionable things. I was using
> an HDHomeRun DUAL Model: HDHR3-US tuner and it was separated from the
> backend by 2 gigabit switches and a gigabit router. Now I'm using an
> HDHomeRun CONNECT Model: HDHR4-2US and it's on the same gigabit switch as
> the backend.
>
> > It could also be that there are frequencies used that are not being
> tested with a channel scan. The way it works is that a predefined list of
> frequencies is scanned but maybe that does not cover all frequencies.
> > To figure this out, could you provide me with a list of channels as
> found by hdhomerun_config and also the output of a mythtv-setup channel
> scan with the "-v chanscan" option? E.g. "mythtv-setup -v chanscan 2>&1 |
> tee ms-20200210-2033.log".
>
> I have some data that may or may not interest you. If not, I'd be happy to
> perform whatever tests you want. I blew away the database and started over
> with a backup that I made before running mythfilldatabase. I changed the
> "capturecard" to the newer and closer HDHR4. I then ran a channel scan with
> the default 3000/6000 timeouts as well as the default scanning options like
> "Complete scan data required" enabled. Same results as before, Only 68
> channels added, although I can see many more were found looking at the
> logs, 399 to be exact. Most are encrypted, but many aren't. They weren't
> added, presumably because they didn't meet the "Complete scan data
> required". I have the output from mythtv-setup -v general,chanscan (but not
> the stderr output which only complained about XDG_RUNTIME_DIR) and I'm
> pretty sure the missing channels were found, but were missing callsigns,
> for example. So I repeated the scan with only "Unencrypted only" enabled.
> When the scan completed, I answered in the affirmative to all prompts about
> adding or updating channels. Now there are 150 channels in the table. Some
> are duplicates my cable provider broadcasts on different frequencies, but
> others are unique channels that I'm recording on my production system. Also
> missing are the channel records that were added by mythfilldatabase and
> that I blew away when I restored an earlier backup. That's the reason why
> you may not want what I have so far.
>
> As for an hdhomerun_config channel scan. I did one on Jan 12 on my
> production system, although I don't recall which HDHR tuner I used,
> probably the old one. The output clearly shows hdhomerun_config found call
> signs that mythtv-setup didn't. You may want a more recent scan, just in
> case my cable provider has changed something.
>
> > About creating new channels vs. updating existing channels.
> > The v31 code does try to update existing channels more than before so
> that non-scanned channel information like xmltvid is preserved across scans.
> > If I understand you correct then you have tested with a channel list
> created by/from Schedules Direct and then do a mythtv-setup channel scan.
> > The expected/wanted behaviour is that the channels in the database are
> updated, but the actual behaviour is that mythtv-setup creates new channels.
> > Correct?
>
> Yes that's correct, but I blew away the database and logs with the
> schedules direct channels and the scan done at the time.
>
> > To figure if the expected behaviour can be realized I need a dump of the
> tables channel and dtv_multiplex as created by/from SchedulesDirect and
> also as created by mythtv-setup.
> >
> > N.B. the default Channel frequency table for ATSC "Full Scan" is now
> obtained from the Video Source table value or, if that is "default", from
> the General / Locale settings table value, in master and in fixes/31. Of
> course it might take a few days before this is in a repository near you.
>
> Sounds good, it will be nice not to have to drill down to the "Scan Type"
> page to make those changes. Are you also setting the default modulation as
> well?
>
> > Thanks,
> > Klaas.
>
> If time permits, I will redo things: first running mythfilldatabase to
> populate the channel table with schedules direct channel data, then an
> hdhomerun_config scan, followed by a mythtv-setup scan, in the hopes of
> giving you something to look at tomorrow. But if time doesn't permit, I'll
> send the data I have now later tonight. In either case, I'll send the data
> offlist.
>
> One other detail. I have since edited the sqlite database from schedules
> direct to unselect channels I can't get because they're encrypted. I assume
> it's a non-issue. If it is, I did save a copy of the original schedules
> direct database.
>
> More offline...
>
> For all reading this thread, the ATSC Clear QAM channel scanning is fixed
now and the fixes have been committed in the mythtv master today.
Thanks a lot to FaginBagin who has done all the testing and who has
provided me with many gigabytes of captured ATSC streams.
Thanks,
Klaas.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mythtv.org/pipermail/mythtv-dev/attachments/20200301/7718559f/attachment.htm>
More information about the mythtv-dev
mailing list