[mythtv] backend from SVN 9761 bangs on mysqld

[Daniel Kristjansson]

On Mon, 2006-04-24 at 11:31 +0100, Stuart Auchterlonie wrote:
> On Sun, Apr 23, 2006 at 06:43:44PM -0400, Daniel Kristjansson wrote:
> There are two ways of doing this. Either do nothing and take
> the hit for the first 15min or so while the cache is build, or
> add some extra fields to the program table, and insert the cache
> key & value into those fields when the data is inserted in to the
> DB. On a new startup we can just read all this info back into the
> cache.
I was thinking of a separate table, so that those without EIT
don't need to store this data. Storing this by transport
wouldn't just save memory during the scan, but also when
recording. EIT collection is only going on on that single
transport in that case, so the rest of the cache data is
redundant. Also a smaller cache would also save some CPU,
but this probably isn't significant.

> These key & data values could also be put to other use. In the
> case where people like to use grabber data as well as EIT, this
> could be used to identify data sourced from the grabber.
There is a field in the program table now called listingsource,
it is initialized to '0' for DataDirect & XMLTV data and '1' for
EIT data. The reason for this field is to allow the data to be
sourced. My plan is to make this a bitmap so that data from
DataDirect/XMLTV would have a value of 0x1 and EIT data would
have a setting of 0x2 and, so combined data would have a value
of 0x3. This could be expanded so that say EIT with extended
tables would have a value of 0x4, so that you could assess the
overall quality of the data in the DB.

> I seriously doubt it is in the region of hundreds of megabytes...
I think the only way we'll know is if we implement the persistent
cache and monitor it's size. I don't really want to do this in
SVN-head, but I'm game for creating a branch with this in it.

We could also switch to another data structure as Janne suggests.
A hash_map would probably save some space, at least on 64 bit
systems. hash_map is not available on Microsoft Windows, but it
has the same interface as a regular STL map, so an ifdef on the
declaration could handle this in a WIN32 port.

> > > No, not really. I changed score_match in such a way that only a perfect 
> > > match can archieve a score of 12000. But Stuart Auchterlonie and I are 
> > > both not sure what you want to archieve with the second part of that 
> > > function. See attached diff.
> > So if the ending time of the program changed you could pick this up by
> > looking at the title or the description? :)
> If the program data changes it is MANDATORY to change the version
> number on the EIT data table.
My point was that this change is post EIT cache, so it would get
past the cache, and then possibly be ignored by this code if it
didn't check everything. I should probably point out that the
point of this code is not to find exact matches, but to find
partial matches with DataDirect/XMLTV data; so that the data
can be merged. At the moment Janne's extension for exact matching
code makes sense for EIT, but only because we aren't saving the
cache. If we were, any EIT update that made it through the cache
would be an improvement, and there would be no need to preserve
old EIT data. The listingsource field should allow the EIT only
case to be optimized in the future so that old EIT only data could
just be deleted instead of being updated (unless it was for an
in-progress recording).

The reason for partial matches is not only so that extra data
from each can be merged, I'd also like to give the user some
controls, such as always using the program title from DataDirect,
so that if a recording rule is set up using the DataDirect
data (which extends out to two weeks), one week EIT data
doesn't overwrite this with a slight variation which prevents
the recording from happening at all.

> There is also talk of rolling out Running Status Table (RST) support
> which is designed to indicate that a program starts early or late etc.
Handling this is a bit different though, it is more like monitoring
the XDS during a recording to determine if the program has changed,
right? i.e. it isn't announced until the program is in progress.
I should point out that there are two reasons I'd like to support
things like RST, XDS monitoring and last minute listings changes.
One is that this might help those poor folks in Australia where the
networks habitually go off schedule, and the other is that my wife
watches baseball, which often goes into extra innings. At the moment
I'm recording an extra hour on baseball games, and even then a rain
delay + extra innings = lost end of game on the more exciting games.

-- Daniel