[mythtv-users] Mooting architecture for a DataDirect replacement

Jay R. Ashworth jra at baylink.com
Wed Jun 27 04:31:22 UTC 2007


On Tue, Jun 26, 2007 at 10:36:51PM -0400, Rod Smith wrote:
> On Tuesday 26 June 2007 21:59, ICMan wrote:
> 
> [re: Using the DNS protocol, but not the existing DNS hierarchy, for data 
> distribution:]
> 
> > The "name" of the station/server/service
> > bounds the data.  A server could be authoritative for more than one
> > domain (ie. station).  Perhaps the station names could be delegated by
> > the cable/feed company names (Rogers-Toronto, Star Choice-Chicago,
> > DirectTV-LosAngeles).
> 
> This could be an appealing aspect of this approach. It could make retrieving 
> data by station pretty painless. In the case of NNTP, you'd presumably need 
> to encode the station ID in the subject header (or possibly some other 
> header), which is a bit unaesthetic.

Sure, but who's *looking*.  :-)

> > I know Jay is flogging NNTP, but I just don't see how you get accurate
> > listings, a complete lineup,
> 
> These issues aren't any better or worse for NNTP vs. DNS (vs. most other 
> protocols); these issues relate more to the data collection end of things 
> than to the data distribution method used.

Well, it's a distributed problem.  Any non-centralized approach will
leave it distributed.

> > JIT listing changes, etc.
> 
> Usenet posts tend to propagate fairly quickly through the Usenet network, and 
> the load balancing is such that it shouldn't be a big strain on local NNTP 
> servers to have Myth boxes checking for new posts in a handful of newsgroups 
> every few minutes. To be sure, dissemination won't be instantaneous, but it 
> could be pretty quick -- on the order of a few minutes to an hour or so, if 
> users' boxes check their NNTP servers every few minutes.

I can forsee users' machines pinging every 15 to 30 minutes for new
postings -- unless they're recording or about to record something that
might be affected by changes to a sports or awards show event.  In
which case you could ping every minute.

This would be a *good* reason for separate newsgroups per station.

> > Would the  
> > proposed solution put an NNTP server at every station, or would they use
> > existing commercial NNTP servers to propogate changes?
> 
> Either method would work, as would our providing an NNTP server for this 
> specific purpose. Chances are most stations wouldn't want to install and run 
> an NNTP server on-site, so it'd probably be an off-site server -- if the 
> stations even contributed data directly (they might not, as some recent posts 
> have discussed).

I was proposing stations software servents posting the updates to a
pair of commercial servers; perhaps supernews and individual.net.

> > How do you 
> > ensure everyone with a public NNTP relay carries the appropriate news
> > groups?
> 
> Most Usenet news server administrators will add a new group or group
> hierarchy upon request, provided it's not something that'll overload
> their system or otherwise cause problems.

Yeah.

> > How do we tell people what news servers to point to?
>
> This could be a challenge, but not an insurmountable one. One approach
> would be to have the Myth system probe for appropriate servers,
> given the user's domain name (or ISP's domain name). For instance,
> if you say your ISP is example.net, it might probe nntp.example.net,
> news.example.net, and perhaps a few others. This will auto-detect the
> correct server a good portion of the time. The code might even include
> a lookup table for some common ISPs. Beyond that, you'd need to tell
> users to contact their ISP and/or subscribe to a commercial Usenet
> provider -- or if we run our own NNTP server, configure the system to
> use it.

Indeed.  And remember, this is hang on the side code; it neednt be part
of the Myth mainline.

> > Will NNTP server administrators freak over the increased load on the
> > servers that contain MythTV Listings Newsgroups?
>
> No. The extra load would be pretty small by Usenet standards. Somebody
> computed the data storage requirements for the whole project as being
> in the low hundreds of megabytes (per 2-week period, IIRC, which is
> all that would have to be cached). I've got a Leafnode server locally,
> which is a small cacheing NNTP client/server for personal use. It's
> currently got 22MB of files in its cache, and I only subscribe to
> one or two dozen text-only newsgroups. Much of Usenet is binary
> newsgroups, and some of those can easily run into the gigabytes of
> storage per day. Any news server that carries any but the tiniest
> binary newsgroups won't even notice the spike in traffic that TV
> listings would cause.

What Rod said.  :-)

> As a side note, with the MythEPG mailing lists now up and running
> (although I have yet to receive a post on them), how should we begin
> moving these discussions to that list (or should we)? I considred
> "cross-posting" (to use the Usenet term) this reply, but I'm not sure
> that would be considered good netiquette.

By all means.  There's no really good way to pinch it off here, and I
don't know whether a Reply-to would stick, but...

Cheers,
-- jra
-- 
Jay R. Ashworth                   Baylink                      jra at baylink.com
Designer                     The Things I Think                       RFC 2100
Ashworth & Associates     http://baylink.pitas.com                     '87 e24
St Petersburg FL USA      http://photo.imageinc.us             +1 727 647 1274


More information about the mythtv-users mailing list