[mythtv-users] UK Radio Times XMLTV [was: Reliability of USB capture cards]

[Nick Morrott]

On 30 July 2010 06:46, Jason Chambers <lists at purplish-monkey.com> wrote:
> On 28/07/2010 19:28, Nick Morrott wrote:
>>
>> On 26 July 2010 01:27, Jason Chambers<lists at purplish-monkey.com>
>> wrote:
>>>
>>> Generally its very good.  There are a few minor problems which
>>> I'll get to later but none of them are annoying enough for me to
>>> start sending threatening letters to the BBC.
>>
>> Please don't! They provide the service gratis but do not create the
>> listings content themeselves - they pass them on from their listings
>>  provider after adding their own programme reviews where necessary.
>
> Fair enough. They would have probably sent Jonathan Ross round to sort me
> out anyways, so I'll keep quiet.
>
> Out of interest do you know /why/ the Radio Times publishes this data?

Historically, the old tv_grab_uk grabber used to scrape the Radio
Times website, placing additional load on the site and causing much
headache for the developers. The RT implemented the data service to
make it more straightforward to grab the data quickly.

> My googling just brings up various posts on mailing lists, forums and
> blogs about the XMLTV grabber itself.  However I cannot find anything from
> the Radio Times and/or BBC that mentions the fact that the raw data is
> available in the first place.

I think it's a service specifically for this project/grabber.
tv_grab_uk_rt was updated for it in September 2004 [1], a couple of
months before I started using it so its inception was before my time.

The BBC do publicise their own TV and Radio listings through the
Backstage site and services, but these only cover the BBC's channels.
There is a 3rd-party XMLTV grabber [2] that can provide listings from
this service.

> Is this something that they are required to do for some reason? Or some
> trial/beta service that they may scrap/charge for in the future?

It is a free service, but I don't think there's a specific
requirement. There's never been any talk of fees or of turning off the
service, but with BBC Worldwide's future not entirely certain, I
suppose anything is possible.

>>> 2. The descriptions and episode titles and not always consistent
>>> when shows are repeated.
>>
>> This can be - and is currently - improved if you provide details of
>> those shows where you see inconsistencies.
>
> I wasn't aware that the grabber did any changing of the data content beyond
> possibly dealing with overlapping times or dodgy character encoding
> problems.  XMLTV was one of those programs I just installed and let it get
> on with it without ever learning more about it.
>
> I've been reading through the program and the supplemental files on the
> website and I was pleasantly surprised at the amount of cleansing that is
> actually going on.

It didn't do any before I took it on back in 2007. Now it's pretty
good and the supplemental files make updating straightforward.

>> The current release of tv_grab_uk_rt allows for blanket category
>> updating on a per-title basis. At the time of writing, 493 different
>>  programme titles are updated.
>
> I run Debian's stable release, which has an older version of xmltv that
> didn't understand the category re-assigning codes. Which explains why I
> wasn't benefiting from those fixups.
>
>> The debug output of the grabber currently reports whenever the same
>> programme title is assigned to different categories within the
>> 14-day listings window.
>
> I notice that the --debug option gives a huge amount of output - it
> mixes pure debugging information (such as the mapping of channel ids and
> parsing supplement files) with the output of potential listing
> inconsistencies.

It started with nothing and has grown over time...

> Is there a reason for this, or would you be willing to accept a patch
> that adds an --inconsistencies option to separate out the two.

There is an effort to try to avoid non-standard grabber options in the
XMLTV project. The uk_rt grabber has several routines including the
inconsistency updates, bad char handling and season/episode numbering
detection which could conceivably have their own separate logging
output.

>> If you do notice inconsistencies in the data you see (and you're
>> running the current stable release of XMLTV), please post details to
>>  the ongoing uk_rt updates thread on the xmltv-users list. I update
>> the fixups frequently...
>
> I've installed the latest stable version now and subscribed to the mailing
> lists so I'll happily provide feedback on any future inconsistencies now
> that I know something can be done to correct them.

Thanks.

> Thanks for your hard work.

Thanks again,
Nick

[1] http://xmltv.cvs.sourceforge.net/viewvc/xmltv/xmltv/grab/uk_rt/tv_grab_uk_rt.in?revision=1.68&view=markup
[2] http://www.crustynet.org.uk/~chrisb/xmltv/tv_grab_uk_bbcbackstage

-- 
Nick Morrott

MythTV Official wiki: http://mythtv.org/wiki/
MythTV users list archive: http://www.gossamer-threads.com/lists/mythtv/users

"An investment in knowledge always pays the best interest." - Benjamin Franklin