[mythtv] Brainstorming a new idea for the program guide

[Jay Merrifield]

Damn syndicated mailing lists, you beat me to my response rob. I came up
with similar numbers on my end. If we wanted to go even further, we could
limit the previews to only prime time, I ran this query:

SELECT distinct(title) FROM program where category_type = 'series' and
hour(starttime) > 19 and hour(starttime) < 23;

and only got 951 shows on basic cable where I have about 129 channels. But I
completely understand if we don't want to abuse tvdb and get ourselves
banned. There's not many sources of tv information that we can go to. I
should point out that these days bandwidth is pretty cheap, but cpu power is
still a premium. We might overwhelm them not in bandwidth, but in terms of
raw queries that they would need to handle. A solution could be to package
up our own banner list of tv shows that we run on either the schedules
direct server or some other server that is zipped and downloaded on occasion
by the user. If we limit all the images to like 5-10k, x 3000 is 30 megs
(yes I realize that this is US only, and when you take into account global
shows and schedules it's much higher), but that's not taking into account
compression and the fact many tv shows wouldn't have any banners available.
If we date-stamped it, we could update it once a week during the new fall
shows, and the mid season winter shows, and once a month otherwise.

Jay

-------------------------------------
On Mon, Sep 21, 2009 at 16:28, Rob Smith <kormoc[at]gmail.com> wrote:
> On Mon, Sep 21, 2009 at 2:59 PM, Robert Johnston <anaerin[at]gmail.com>
wrote:
>> It could also be possible, if TVDB are willing...
>
> So, as you just said here, TVDB is not Myth, nor are we a part of
> TVDB. We have no control over their code, have no idea if their
> servers could handle it or not (and I'd venture to guess (yes guess)
> that they couldn't).
>
> All this is entirely meaningless on here until someone contacts TVDB
> and asks them permission to increase their (already overloaded)
> servers by a few thousand-fold.
>
> It's not even that your "Cache preload burst" would save that much
> over just a pipe-lined datadump, as the zipping wouldn't actually
> compress that well (these images are already compressed), so you're
> eating up a lot of cpu time, and then you can't really cache the
> results, as everyone's lineups are different. For example, I have
> FIOS, but I have all my sports and shopping channels out of my
> line-up, along with a majority of news channels. But not all of them.
> That's a (semi)unique setup for me, with a (semi)unique pile of
> images. It's a lot of work for very little gain.
>
> Also keep in mind how many users, let's do some math
>
> In the past month (Aug 21 to Sept 20th), we had 13,000 downloads of
> 0.21 from mythtv.org. Let's assume that 10,000 of them actually used
> the software and they have 5,000 shows in their guide data, at 5kb per
> show. (My guide listing has 5954 shows fyi, SELECT COUNT(DISTINCT
> TITLE) FROM program ).
>
> 10000 * 5000 * (5 kilobytes) = 238.418579 gigabytes
>
> and for the current 500kb art downloads
> 10000 * 5000 * (500 kilobytes) = 23.2830644 terabytes
>
> Just for the last month of downloads, from mythtv.org. Not counting
> the ones via package managers, shipped with the os, etc.
>
> That's a lot of data, and the potential is much much more. Sure,
> there's plenty of ways they might be able to change to make it work,
> but you'll need to talk with *them* to see if it could work and not
> argue with us about how trivial it is/could be/if they're even willing
> to allow us to do this.

There are some things to remember here.

1. Not every show will be eligible for images/data.

Here's the number I came up with: 2588 shows (
SELECT COUNT( DISTINCT TITLE )
FROM `program`
WHERE category_type != 'movie'
AND category != 'Paid Programming'
AND category != ''
)
down from 3235 shows (
SELECT COUNT(DISTINCT TITLE)
FROM program
)
That's on 71 channels of programming.

2. That presumes that every single show listed will have artwork for
it. This will most likely not be the case.

3. This is only the first download. Subsequent fetches will have
little of the data needed, as it will be cached client-side. The only
issue will be when the new TV seasons launch, and new programmes start
appearing.

4. P2P Technologies (libtorrent, for example) could be integrated to
make Myth "Play nice", sharing the data around, between
frontends/backends and systems, and taking a lot of the load off of
TVDB (Which I'm sure they'd appreciate).

So things aren't near as dire as they look at first glance.
-- 
Robert "Anaerin" Johnston
Sent from Regina, Saskatchewan, Canada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mythtv.org/pipermail/mythtv-dev/attachments/20090922/2e88c5a8/attachment.htm>