[mythtv] Hack to retrieving digital channel listings

Brad Allen Ulmo-Myth-Dev at Usenet.Q.Net
Sat Jan 24 02:20:20 EST 2004


From: "Kirk Lapray" <kirk at backfence.net>
Subject: [mythtv] Hack to retrieving digital channel listings
Date: Wed, 21 Jan 2004 10:37:32 -0700

> I do not yet have a tuner card, but I am looking at getting the HDTV
> one from pcHDTV.

While the drivers and SW are not complete, they are stable and easy to
fix compared to IVTV.  I heartily people recommend getting as many
pcHDTV HD-2000 cards as they can before the end of 2004 when they
won't be able to sell them in the same way any more.

> It has kind of bothered me that there were not any listings for
> digital channels available. When I found out that XMLTV used
> zap2it.com I visited their website and noticed that digital channel
> listings were available, but to view more than one channel at a time
> you had to sign up and create a personal channel list.

It's difficult to figure out how to sign up.  Here's the FAQ for that:

  How do I customize my listings?

  To customize your listings, click on the "Customize" link from above
  the listings grid. You will be taken to a login page, where you can
  click on a link to register for a user name and password. After you
  sign up, you can adjust the channels you want listed, along with the
  size of the grid, the types of categories and more

Indeed it is there.  Netscape gave me an error trying to register a
variation on my usual name which was taken.  I'll try something random
this time first using w3m; cookies needed enabling.  It wouldn't work
because the screen didn't function.  I am going to mozilla, then try
from IE on MacOS 9, and some other things.

Still getting this with mozilla on fresh user with liberal settings:
  Your request could not be processed at this time.
  This incident has been logged.
  An unknown error has occurred.

>From a fresh IP# with fresh homedir:  didn't help.
>From MacOS 9 IE:  didn't help.

An old Netscape (Mozilla) I used a lot had data in it which let me
know I had tried to sign up before with it, and I went to "remember my
password" and tried a lot of usernames, and I found what I had
registered before.  I was able to log in just fine, then.  I'm happy I
went to create that profile in the past in the hope it would prevent
problems in the future when I needed it: I was right.

Sorry, new zap2it members: you have to talk to them about debugging
adding new users.  I've run into this problem before months ago, too.

This is a really big problem.  Once we get through that though, we
have to collect cookies.

> Once my channel list was correct I was able to perform a crude hack
> on ZapListings.pm to have it load my personal cookie.  After this I
> was able to have XMLTV retrieve all channel information, including
> the digital ones.
> 
> To get it to work you have to patch ZapListings.pm and place the
> cookie.txt file in /usr/local/etc. You will also have to edit the

I put my cookie.txt in my myth backend account holder directories.

> cookie.txt file with the appropriate values for your location and
> your user id. These values can be obtained from the
> tvlistings2.zap2it cookie in your cookie directory after you have
> gone and customized your zap2it listings.

Ok.  My "UserID" is not the same as my login; you really need to
compare cookies to see this, like you said.  This was very easy.


> With all of this done you should now be able to have XMLTV retrieve
> all the channels that are listed on your personalized zap2it
> listings.
> 
> In the future it would be nice if there was a simple option to pass
> the user id into tv_grag_na and have it pull the channels without
> the need for a customized cookie.
> 
> I am hoping with this that MythTV can now have the channel listings
> it needs to better work with the pcHDTV card.

This is great.  I think there will be a problem since there are
multiple listings for one area and zap2it doesn't allow you to create
multiple tables, like TitanTV has ever since I started using it.

What we really still needs is TitanTV, using their OSD protocol.

Anyway:

To test:  run mythfilldatabase:


I forgot to change my setup files.  Here were the hacks that would
pull down the NTSC listings for the HDTV channels:

retry limit: 4
retry delay: 1
postal code: 95003
provider: 172857 Local Broadcast Listings
# Air 10:
channel: 81 KSBW-DT
channel: 8 KSBW
# Air 11:
channel: 111 KNTV-DT
channel: 112 KNTV-DT
channel: 11 KNTV
# Air 32:
#channel: 461 KION-DT
#channel: 46 KION
# Air 13:
channel: 351 KCBA-DT
channel: 35 KCBA

here was the first run result:

Day 2004-1-24 schedule for station 8 KSBW has:29 programs
Day 2004-1-24 schedule for station 11 KNTV has:29 programs
Day 2004-1-24 schedule for station 35 KCBA has:36 programs
skipping unavailable channel 81 KSBW-DT
skipping unavailable channel 111 KNTV-DT
skipping unavailable channel 112 KNTV-DT
skipping unavailable channel 351 KCBA-DT

I think the dashes hurt it.

First test:  remove dashes & comment out NTSC channels:

retry limit: 4
retry delay: 1
postal code: 95003
provider: 172857 Local Broadcast Listings
# Air 10:
channel: 81 KSBWDT
#channel: 8 KSBW
# Air 11:
channel: 111 KNTVDT
channel: 112 KNTVDT
#channel: 11 KNTV
# Air 32:
#channel: 461 KIONDT
#channel: 46 KION
# Air 13:
channel: 351 KCBADT
#channel: 35 KCBA

Not helping:

noticed station unavailable (81 KSBWDT), re-run --configure
noticed station unavailable (111 KNTVDT), re-run --configure
noticed station unavailable (112 KNTVDT), re-run --configure
noticed station unavailable (351 KCBADT), re-run --configure
writing listings to /tmp/mythIE1zvn
skipping unavailable channel 81 KSBWDT
skipping unavailable channel 111 KNTVDT
skipping unavailable channel 112 KNTVDT
skipping unavailable channel 351 KCBADT

Ok.  I have to make these the same as what it has in its listing to
try it.  I should be seeing it in the listing mythfilldatabase pulls
from it from my profile.

That didn't work.  I am trying to log in again to see what cookie I
get again.  It changed.  I'm using the new values of the cookie I got.

got it!  This is looking better ...
noticed new station available (8 KSBWDT), re-run --configure
noticed new station available (11 KNTVDT), re-run --configure
noticed new station available (35 KCBADT), re-run --configure
let's see if we can compare mythconverg.channel stuff.

I changed column callsign from KSBW-DT, KNTV-HD, KCBA-DT, and KNTV-SD
to what you see below:

+--------+---------+--------+----------+--------------+----------+
| chanid | channum | freqid | callsign | name         | finetune |
+--------+---------+--------+----------+--------------+----------+
|   1008 | 8       | 10-1   | KSBWDT   | 10-1 KSBW-DT |        0 |
|   1011 | 11      | 12-4   | KNTVHD   | 12-4 KNTV-HD |        0 |
|   1035 | 35      | 13-2   | KCBADT   | 13-2 KCBA-DT |        0 |
|   1125 | 11      | 12-5   | KNTVDT   | 12-5 KNTV-SD |        0 |
+--------+---------+--------+----------+--------------+----------+

Next run shows this worked:

noticed new station available (8 KSBWDT), re-run --configure
noticed new station available (11 KNTVDT), re-run --configure
noticed new station available (35 KCBADT), re-run --configure
noticed station unavailable (81 KSBWDT), re-run --configure
noticed station unavailable (111 KNTVDT), re-run --configure
noticed station unavailable (112 KNTVDT), re-run --configure
noticed station unavailable (351 KCBADT), re-run --configure

Now, we see much clearer reason why it's not working, because we're so
much closer.

I fix my config file once again, finally looking like this:

retry limit: 4
retry delay: 1
postal code: 95003
provider: 172857 Local Broadcast Listings
# Air 10:
channel: 8 KSBWDT
# Air 11:
channel: 11 KNTVDT
#channel: 112 KNTVDT
# Air 32:
#channel: 461 KIONDT
# Air 13:
channel: 35 KCBADT

And it works:

Day 2004-1-24 schedule for station 8 KSBWDT has:29 programs
Day 2004-1-24 schedule for station 11 KNTVDT has:29 programs
Day 2004-1-24 schedule for station 35 KCBADT has:36 programs

Does it also do anything other than that one channel list, i.e., cable?:

noticed station unavailable (2 KTVU), re-run --configure
noticed station unavailable (3 KCBA), re-run --configure
noticed station unavailable (4 LOOR004), re-run --configure
noticed station unavailable (4 RECOV), re-run --configure
noticed station unavailable (5 KION), re-run --configure
noticed station unavailable (6 KSBW), re-run --configure
noticed station unavailable (7 ABC7), re-run --configure
noticed station unavailable (8 KTSF), re-run --configure

etc.

No.  So, your patch is limited; we have to find a way to make it
conditional.

I think I found a temporary solution: since I installed Myth on two
hosts, including mythfilldatabase, I am going to use the host with
modified ZapListings.pm for my account specific HDTV channels, and the
host with unmodified ZapListings.pm for the rest.  Luckily, this
allows me to choose either one for local broadcast listings that are
not HDTV, since they are in the same video source channel thing.

Running on the other host, we get:

noticed new station available (65 KKPX), re-run --configure
noticed new station available (66 KFSF), re-run --configure
noticed new station available (67 KSMS), re-run --configure
noticed station unavailable (8 KSBWDT), re-run --configure
noticed station unavailable (11 KNTVDT), re-run --configure
noticed station unavailable (35 KCBADT), re-run --configure
writing listings to /tmp/myth53uUSg
skipping unavailable channel 8 KSBWDT
skipping unavailable channel 11 KNTVDT
skipping unavailable channel 35 KCBADT

which is what we want, and everything else for cable ...
Day 2004-1-24 schedule for station 2 KTVU has:40 programs
Day 2004-1-24 schedule for station 3 KCBA has:36 programs
is fine.  (KCBA is antenna-35-NTSC antenna-13-ATSC cable-3-NTSC, so
that's correct.)

Is it ok to have two mythfilldatabases run on the same material, or
should I weed them out from running each others' areas?  I'll do that
now: one for broadcast & the other for cable.

How do we get the multiple programs for each channel?  Locally we have
this for one channel, but I don't know how to get it out of
Zap2It.Com (I didn't really try, though):

# CHANNEL 12 ###### signal = 176, lock = 43 (1b43)
entry {
#       input              = 0;
#       frequency table    = us-bcast;
       majorchannelnumber = 11;
       minorchannelnumber = 1;
       stationchannel     = 12;
       stationidentifier  = KNTV-HD;
       programnumber      = 4;
       disabled           = 0;
};
entry {
#       input              = 0;
#       frequency table    = us-bcast;
       majorchannelnumber = 11;
       minorchannelnumber = 2;
       stationchannel     = 12;
       stationidentifier  = KNTV-SD;
       programnumber      = 5;
       disabled           = 0;
};

I assumed they were doing KNTV-SD, but it could have been KNTV-HD.

> This is my first post, so please let me know what I can do in the
> future to make them better.

The myth list people don't follow traditional list post protocols
(e.g., being sure to cc all those who are concerned), and constantly
whine about standard practice in the most rude ways.  I would tread
very, very, very carefully if I were you.  I can't really give any
good examples since they just fall over themselves whenever I post
(either responding loudly or not at all, without really responding).
I'm beginning to wonder if I might feel out of place if they didn't,
but lately I've been using my rotating attention scheduler to handle
it.


More information about the mythtv-dev mailing list