[mythtv-users] tv_grab_na_dd

[Michael T. Dean]

Axel Thimm wrote:

>Did you try setting up a symlink from tv_grab_na to tv_grab_na_dd?
>tv_grab_na is considered retired by xmltv authors, I hope
>tv_grab_na_dd is a 100% functional drop-in, but I cannot easily test.
>  
>
It's not a drop-in replacement.  It uses an XML web service instead of 
screen scraping HTML pages (which makes it extremely fast--i.e. 5 
seconds for a day's listings instead of 2 1/2 minutes), so it was 
redesigned for the service.  Therefore, it has different command-line 
options.

Also, and more importantly for Myth users, it uses different channel 
ID's, so either a) the database needs to be updated to use the new 
channel ID's or b) the "--old-chan-id" command-line option must be 
specified (I don't know if the option will be supported indefinitely, 
though, and I don't know which is being used for the CVS version of 
MythTV).  Also, when configuring the grabber, you must specify a numeric 
timezone offset (i.e. -0400 for US Eastern Daylight Time), username, 
password, and lineup; and you don't have to specify a provider (that's 
done through the website when signing up for the account).

Doing a mythfilldatabase without "--old-chan-id" will create "near" 
duplicates of all channels (and the duplicates won't be associated with 
any sources/recordings).  Also, if the timezone offset is incorrect, the 
shows will appear in the EPG at the wrong times.

We should probably get something up that helps MythTV users who haven't 
upgraded to CVS (or who may decide not to immediately upgrade to 0.15 
when released) start using the new grabber because Tribune Media is 
"anxious" to stop the screen scraping that's eating up their bandwidth 
(and were kind enough to supply something better instead of shutting out 
XMLTV users).  They've already asked the XMLTV developers why people are 
still using the old grabber, and notified them that some changes that 
they need to make to the site in the very near future (new data) will 
break the old grabber.  The XMLTV developers will not fix the old 
grabber since it is no longer supported.  Therefore, depending on the 
timing of this change and the release of 0.15, many Myth users may get 
caught unaware.

To get things rolling, I'm using the below script to update my 
listings.  It's a good start, but there's plenty of room for 
improvement.  Note that the script does not take into account multiple 
DataDirect lineups or multiple MythTV source ID's and it always 
downloads/updates all the available data (which is about 2.5MB total for 
my 16 channels) instead of using "--days" and "--offset" options (like 
it should).  Oh, and it's vulnerable to a symlink attack thanks to the 
totally predictable temporary filenames.  :)

Finally, we'll want to tell everyone that the new service is extremely 
fast (WIIFM) and assure them that it is still available at no charge.  
The three-month subscription will be renewed by filling out another 
survey (similar to the one filled/fill out during registration).

Mike

#!/bin/sh

# temporary filenames
dirname='/tmp'
filename="tv-`date +%Y%m%d`"
dd_file=${dirname}/${filename}.dd
xmltv_file=${dirname}/${filename}.xmltv
# use card input 1
sourceid=1
# replace all data, up to 10 days
offset=-1

# Save the downloaded DataDirect listings to a file
#  - This file contains the data for all lineups and may be
#    "reprocessed" multiple times (so there is no need to re-download
#    for different lineups)
tv_grab_na_dd --dd-data ${dd_file} --download-only
# Process the downloaded DataDirect listings and output XMLTV listings
tv_grab_na_dd --dd-data ${dd_file} --reprocess \
              --output ${xmltv_file} --old-chan-id
# If using multiple lineups, create a separate configuration file for
# each one and specify the appropriate configuration file for the
# lineup such as:
#tv_grab_na_dd --config-file=lineup1.dat \
#              --dd-data ${dd_file} --reprocess \
#              --output ${xmltv_file} --old-chan-id

# Fill the database with the downloaded data
mythfilldatabase --file ${sourceid} ${offset} ${xmltv_file}

# Remove the temporary files
rm ${xmltv_file}
rm ${dd_file}