[mythtv] Why MythTV didn't handle the UK Freeview lineup change

John Barberio barberio at lineone.net
Sun Jul 26 11:06:51 UTC 2009

Stuart Auchterlonie stuarta at squashedfrog.net wrote:
> John Barberio wrote:
> > Since none of the developers seemed interested in working out why  
> MythTV
> > couldn't cope with the lineup change moving 'Virgin 1' to a new  
> mux, and
> > a 'new' channel created for 'Virgin +1'. I've attempted to work  
> out what
> > the problem might be with black-box reverse engineering of how  
> MythTV
> > appears to have reacted to the lineup change. Developers, feel  
> free to
> > dispute this with your knowledge of how MythTV works.
> >
> We are interested. In fact we are busy trying to finish off the  
> rewrite
> of the channel scanner for the 0.22 release. We've been having a think
> about what you described in a effort to work out what went wrong.

It's been very hard to tell, as bug tracking tickets raised against  
this problem were constantly closed or locked by you with the cryptic  
comment of "Trac is not a discussion forum.", and you have not  
provided any information on any steps being taken on the bug.

> > MythTV, unlike most other DVB systems, does not store video and  
> audio
> > pids in its channel data. It only stores the NIT channel id  
> number, and
> > recovers the video and audio pid from the current broadcast NIT each
> > time the channel is accessed. However, it *does* store the channel's
> > name and the LCN for that channel.
> >
> The video and audio pids aren't guaranteed to be the same all the  
> time.
> Granted a lot of broadcasters don't change them to keep things simple.
> What is guaranteed to be unique is the ServiceID which is found in the
> PAT. This only changes when they go and rearrange the channels.
> However, as far as i can tell they cheated a bit when they "moved"
> virgin1. What happened was they aquired a 24hr slot on another mux
> and setup virgin1 there, and their existing 12hr virgin1 slot was
> renamed to virgin1+1

Again, here you make the incorrect assumption that the way the line up  
changed was 'invalid' or 'cheating'. It's not. It was within the  
accepted used on the network, ServiceID is unique only at time of  
transmission it may be reused if the same slot is changed to a new  
channel, as happened here.

> > Now, the problem is this... This design makes the assumption that
> > channel id numbers are UUIDs and will never be reused for a  
> different
> > channel.
> >
> There are 2 distinct "channel numbers". The LCN (logical channel  
> number)
> which is what puts BBC 1 = 1, BBC 2 = 2 etc etc, and the channel id
> assigned by myth which is based on a combination of the sourceid and  
> the
> actual channel number. Yes, MythTV uses this to uniquely identify that
> channel, however that isn't what it uses when making scheduling
> decisions, that uses the callsign. This allows us to select a channel
> to record from multiple different sources, for example, if you had  
> both
> dvb-t and dvb-s (for freesat), channels like BBC 1 can be found on
> multiple inputs and considered as valid candidate for a recording.
> Once it has picked a channel number to record, then the tuning info
> is retrieved from the database.
> Chanid gives us serviceid & mplexid
> mplexid gives us freq, code rates and all the other details for the  
> mux.
> We then tune to the mux based on the data we found from looking up the
> mplexid. Then we use the PAT to find the PID for the PMT, from the PMT
> we find the PIDs for the video and audio streams and then off we go
> on our merry way.
> So channelid's have to be unique so that myth can record what you  
> want.
> The underlying problem, seems to be more a case of the data wasn't
> correctly updated when a scan was done, rather than the uniqueness of
> the chanid.

Okay, so here we establish the different ways channel recording/live  
find the channel, and the scheduler/EIT scraper find the channel. This  
establishes that yes, you can have situations where MythTV can get  
information from two different ServiceIDs for the same channel.

> > This isn't true. What happened with the 'Virgin 1' line-up change  
> was
> > that 'Virgin +1' used the same channel id number as the old  
> 'Virgin 1'.
> > As far as mythtv was concerned, the channel it knew as 'Virgin 1'  
> was
> > still there, since it sees the channel id number, and tunes to  
> that not
> > knowing this is now the channel id for 'Virgin +1'. Oddly, the  
> program
> > information scraper doesn't seem to make this mistake, and manages  
> to
> > pick up the different program information for the new 'Virgin 1'.  
> This
> > leads to the apparent 'hour out of step' bug, since the program
> > information for the real 'Virgin 1' is showing, but 'Virgin +1' is  
> what
> > gets tuned to. (Alternatively, it might be the other way around.  
> With
> > the channel tuning doing it right, and program information doing it
> > wrong. Either way, one of them 'regenerates' the wrong channel
> > information.)
> Since you are talking about mythfilldatabase i'm making the assumption
> that you use RT as a program listing source via xmltv. This would also
> mean you have setup xmltvid's on your channels, and since setting them
> up is a pain, you've probably done a scan for channels without  
> deleting
> the old ones first.

Incorrect assumption. The only channel data on my system comes from  
EIT. I only use mythfilldatabase as it ensures the database is  
synchronised and all actions flushed to it.

> Personally i use, EIT so i tend to delete all my channels and rescan.
> What i did notice is that they did broadcast incorrect data for the
> changeover period, but they quickly sorted that out.

Did you notice this on your mythtv box only, or on other set top  
boxes? Doing a quick search, I can't find reports from people who've  
rescaned on a freeview STB having the 'one hour out of sync' problem.  
This problem seems to be specific to mythtv. The 'one hour out of  
sync' issue went away for me when I force deleted the channel  
information, flushed and synced the database with mythfilldatabase,  
and rescanned. This indicates to me that the issue was not in the  
broadcast data.

> >
> > Now, combine this with another characteristic of MythTV, that  
> database
> > deletes aren't active till a mythfilldatabase run. Then you have the
> > situation where the end user *can't delete* 'Virgin 1' and rescan,
> > because the deletion isn't flushed before they do a rescan, and the
> > channel scanner sees the old channel id, and makes the same  
> mistake again.
> >
> I'm not sure what you mean here? In the channel scanner if I tell it  
> to
> delete the channels, that's what it does, there and then.

This is an issue with your handling of the bug. It was reported to you  
*several times* that this isn't how the channel scanner works in  
practice. 'Virgin 1' with the old data would be recreated from cached  
information on the database, *unless* you deleted all channels, *and*  
ran mythfilldatabase to sync and flush out the database before making  
a rescan. Perhaps you have set up your mythtv database to be  
synchronous for debugging purposes, and did not notice this issue in  
production versions?

See http://www.gossamer-threads.com/lists/mythtv/users/383253 for user  
experiences of this.

I start to suspect this is a situation, where you are working on  
unrelated issues with the channel scanner, you see this issue come up  
and assume it is related to what you are working on, and close bug  
reports about it. You do not closely read the reports, and do not  
notice that it is a different issue relating to database 'ghosts' of  
deleted channels overwriting the 'new' channel found by a scan when  
they have the same ServiceID. You make some assumptions about what is  
being reported, *without asking the people reporting the problem*. You  
assume that it's related to what you are already working on, and close  
all the tickets raising the issue as 'duplicates'.

I hope you will review both this issue, and your own workflow in how  
you handle issues being raised.

  - John

More information about the mythtv-dev mailing list