[mythtv-users] Mooting architecture for a DataDirect replacement

[Rod Smith]

On Friday 22 June 2007 13:28, jedi at mishnet.org wrote:
> > At 11:49 AM -0400 6/22/07, Jay R. Ashworth wrote:
> >
> > I used net news back in the day.  My memories are of out-of-order
> > messages, replies to messages that never-ever came through, and other
> > general annoyances.

For the application under discussion, out-of-order postings don't matter; 
whether you get the update for HBO or Discovery Channel first won't matter to 
any even quarter-competently designed software on the MythTV box, and the 
MythTV developers are *MUCH* more than 1/4 competent! ;-)

Missing messages could be more of a problem, although my impression is that 
this is much less of an issue than it used to be. A couple of ideas occur to 
me to minimize this risk:

1) Some guidelines could be set to ensure that guide data is originally
   uploaded to a small set of known-reliable NNTP servers. Somebody in
   the community might even write software to monitor these servers and
   ensure that every valid post gets propagated to all the servers in this
   set. If an individual has problems because of an ISP's flaky server, that
   individual could obtain (possibly for money) a subscription to one of the
   known-reliable servers. In the event an extra NNTP server subscription is
   necessary, it shouldn't have to be an expensive one. Compared to something
   like one of the video binary newsgroups, the amount of data involved is
   tiny, so you wouldn't need an account with very fast access or huge
   download caps.

2) There's a tool known as par2 (http://www.par2.net) that's commonly used
   on binary newsgroups. It's an outgrowth of the protocols used in RAID
   arrays; it takes a group of files and generates recovery files, so that
   if a message is missing from a multi-message download, the missing data
   can be regenerated. Given appropriate grouping of data, something similar
   to this could be applied to guide data to guard against lost messages.
   The tricky thing would be figuring out precisely how to group data
   together to make this an effective strategy -- you wouldn't want to force
   people to download too much data they won't use in order to regenerate the
   one lost piece of data they do need. Automatically applying the par2 tools
   would also take some programming effort, but once the rules are set, I
   doubt if this would be a big challenge.

> I would expect usability would be the main hinderance with NNTP.

Why? The idea is not (I presume) to force people to use news readers like 
Leafnode, Knews, or Pan to download listings. The idea (as I understand it) 
is to incorporate a specialized NNTP client into MythTV, or to be called by 
MythTV. This client would access the newsgroup(s) that carry the data for an 
individual's system, download the headers, figure out which ones carry useful 
data (the subject lines would have to carry highly structured summary data), 
download the bodies of those messages, and then process those bodies much as 
MythTV already processes files from other sources. From a user's point of 
view, all that would be required would be pointing MythTV at an appropriate 
NNTP server and selecting or creating a lineup in some way. There might also 
be a possibility to tweak things -- say, by telling the system how often to 
check for updates. Overall, this wouldn't be much, if any, more complex than 
what we've got now, from a user's point of view. In fact, in some ways it 
would be simpler, since you wouldn't have to log onto zap2it to manage your 
lineups -- you'd do that from your Myth box!

-- 
Rod Smith
http://www.rodsbooks.com