[mythtv-users] Probable bug: Category search list

[Bruce Markey]

Before I'm deferred to, let's point out the 600 lb. Gorilla
in the room. The program.category field is a single string
containing the genre name that best describes the show. This
is true for any data source and is supported for XMLTV grabbers.
DataDirect provides multiple genre that are sorted by 'relevance'.
These are stored in 'programgenres' and are related by matching
the chanid and starttime:

mysql> select title, chanid, starttime, category from program where chanid=3732 and starttime='2007-12-31 20:00:00'\G
*************************** 1. row ***************************
    title: Shrek 2
   chanid: 3732
starttime: 2007-12-31 20:00:00
 category: Comedy
1 row in set (0.00 sec)

mysql> select * from programgenres where chanid=3732 and starttime='2007-12-31 20:00:00'\G
*************************** 1. row ***************************
   chanid: 3732
starttime: 2007-12-31 20:00:00
relevance: 0
    genre: Comedy
*************************** 2. row ***************************
   chanid: 3732
starttime: 2007-12-31 20:00:00
relevance: 1
    genre: Fantasy
*************************** 3. row ***************************
   chanid: 3732
starttime: 2007-12-31 20:00:00
relevance: 2
    genre: Animated
3 rows in set (0.00 sec)

mysql> select relevance, count(*) from programgenres group by relevance;
+-----------+----------+
| relevance | count(*) |
+-----------+----------+
| 0         |   117334 |
| 1         |    54668 |
| 2         |    13809 |
| 3         |     2788 |
| 4         |      392 |
+-----------+----------+


David Engel wrote:
> On Sun, Dec 30, 2007 at 09:40:45PM -0500, Mike Barnard wrote:
>>> It is not a bug, it is a missing feature as I know. I asked this
>>> question a few months ago. Mythfilldatabase takes only the first
>>> category, others are ignored.
>>     But the other categories are listed in mythfrontend's "program
>> details" screen, so mythfilldatabase must be storing them in the MySQL
>> tables. Maybe I'll look at writing a patch for this, but I don't know
>> jack about Qt, and my C++ is very rusty. Oh well - no better time to
>> sharpen the rusty skill-set, eh? Heh...
> 
> Setting the single category attribute from the most prominent of
> potentially several genres for a program is done on purpose.  I
> suspect this was done to match existing functionality in Myth before
> DataDirect support was added, but I'll defer to Bruce Markey on
> exactly why.

This was one of a dozen or more things quickly fixed the
night that David Shay's DataDirect patch went in. Someone
pointed out that categories were empty for this new grabber.
As I recall, less than an hour later, Isaac committed a fix
that stuffed the relevance=0 genre into 'program.category'
and this is how it's been done ever since. The "category" can
be found simply as an attribute in the program info. Looking
up the sub-genre involves (potentially nasty) JOINs.

Sometime later, I added code to append the sub-genre to the
category string on the Program Details page:

    if (category != "")
    {
        s = category;

        query.prepare("SELECT genre FROM programgenres "
                      "WHERE chanid = :CHANID AND starttime = :STARTTIME "
                      "AND relevance > 0 ORDER BY relevance;");
...
                s += ", " + query.value(0).toString();

For XMLTV. this table is empty so only the program.category is
included and life goes on. For DD, program.category is already
relevance=0 so this appends any other sub-genre in the order
determined by TMS.

Title: Shrek 2
...
Category: Comedy, Fantasy, Animated
-------cat--^  rel=1--^  rel=2--^


FWIW the Search Words->Advanced also uses genre and can add
a search for a single genre that you are interested in or
with other criteria. This is not as generally useful as David's
patch. Also, Custom Record includes examples to use genre as
well as category and with any other criteria. A search you like
can be stored then used again and again from Search Words->Stored
Searches. For example:

mysql> select * from customexample where rulename like 'HD S%'\G
*************************** 1. row ***************************
   rulename: HD Science
 fromclause: LEFT JOIN programgenres ON program.chanid = programgenres.chanid AND program.starttime = programgenres.starttime
whereclause: program.hdtv > 0
AND programgenres.genre = 'Science'
     search: 1

I built this by choosing "Only shows marked as HDTV" and "All
matches for a genre (Data Direct)", entering a rule name then
clicking "Store". Now whenever I choose "HD Science" from my
"Stored Searches", I see all the shows with any sub-genre of
'Science' that are marked as HDTV.

> That being said, I think a patch to add a genre search would probably
> be acceptable.  IMHO, I'd like to see it replace the existing category
> search.  That would be more involved and probably require changing
> mythfilldatabase to populate the genres table for non-DataDirect based
> grabbers.

I hadn't considered that but I supposed XMLTV grabs could stuff
the one category string into chanid, starttime with relevance=0.
However, I'd always assumed that anything relying on genre would
check to see if the programgenres was empty and have a different
code path. Populating the genre table needlessly sounds ugly but
it might simplify the code needed for proglist.

> I'm attaching an old, proof-of-concept patch I have to help you see
> what might need to be done.  The patch changed the category search
> into a two-level genre search.  I did this because I found it useful
> to break some searches down to an extra level.  It was also
> interesting to just browse and run across interesting programs which
> matched combinations of seemingly unrelated genres.

As I recall, the first idea was to have category/genre but
including any genre/genre in this patch is more useful because
Shrek 2 can not only be found as:

Comedy
   Comedy / Animated
   Comedy / Fantasy

but is also listed for:

Animated
   Animated / Comedy
   Animated / Fantasy

and:

Fantasy
   Fantasy / Animated
   Fantasy / Comedy

> This patch wasn't added because the large number of genre combinations
> makes navigating through the views unmanageable.  The solution is to
> change the view navigation and popup to explicitly suuport two-levels.
> That hasn't been done yet.

I've been using this patch for the past year or two or whenever
it was that this came up. It can take a long time to load too but I
I agree that the long popup list makes finding Western or Wrestling
difficult. I've taken a couple stabs at a popup with two lists
side by side. Getting the sub-genre list to update when scrolling
the first list and displaying the result for the selection are no
problem. I was stumped in the Qt goo to get the two lists side by
side on the popup, stretching to fit and navigation keys behaving
reasonably.

The other issue that I touched on above is; what would this look
like for XMLTV? Even if the genre table was stuffed with the one
category, there would be no pairs and the right column would always
be blank and useless. This is why I'd always assumed there would
need to be two code path for grabbers with and without genre.

I'm also in favor of this eventually going in but someone would
need to work on a better popup preferably something like this:

+------------------------------+
| Select Category              |
|+------------++--------------+|
|| Action     || <All Comedy> ||
|| Animated   || Animated     ||
|| Biography  || Bowling      ||
|| Bowling    ||:Fantasy::::::||
||:Comedy:::::|| Sitcom       ||
|| Fantasy    || Talk         ||
|| Medical    ||              ||
|| Sitcom     ||              ||
|| Wrestling  ||              ||
|+------------++--------------+|
+------------------------------+


--  bjm