[mythtv-users] Mooting architecture for a DataDirect replacement
mythtv at rodsbooks.com
Sat Jun 23 16:38:11 UTC 2007
On Saturday 23 June 2007 09:23, Peter Schachte wrote:
> Rod Smith wrote:
> >>> I used net news back in the day. My memories are of out-of-order
> >>> messages, replies to messages that never-ever came through, and other
> >>> general annoyances.
> > For the application under discussion, out-of-order postings don't matter;
> If there are multiple updates to the schedule in separate postings, I don't
> think this is true. If there's an update that says that station X is
> airing program Y at time Z, and another update sent an hour later that says
> no, it'll be at time Q instead, then you don't want to handle them in the
> wrong order.
This would only be a problem if the code were poorly written (you snipped my
original comment to that effect) or if the original data never arrived. Each
message could have a list of the original files and patches upon which it
relies, and if messages arrive out of order, the client-side software will
know to delay processing of the later data until the earlier data is
available and/or to not apply earlier changes that would overwrite more
recent ones. Alternatively, the software could cache all the original update
files, detect the out-of-order arrivals, and re-run the whole sequence from
the start. Obviously a lot of details would depend on the exact file formats
used, but I doubt if this problem would be all that difficult to solve.
> I believe the OP suggested we consider requirements for this system. As
> much as I like usenet for discussion forums, I think the requirements for
> our purposes here are rather different. One key requirement for us is that
> distribution be reliable; ie, at all times, every subscriber should have
> the correct schedule for their lineup as at the time of their last update.
I agree that this isn't the strongest point of Usenet, but the more reliable
Usenet providers are very reliable. In a worst-case scenario, a user with a
poor Usenet provider could switch to a more reliable one.
> Another is that if something goes wrong and somehow myth doesn't have the
> correct schedule, it should be able to tell, and be able to correct it.
> It's hard for me to see how to satisfy that with NNTP.
NNTP isn't the issue here; it's error detection and correction (or lack
thereof) in the encapsulated files.
> If postings get
> lost or come in out of order, the schedule winds up wrong. The problem
> could be detected if each posting contain a checksum for the whole lineup
> after the change in the posting is applied (not just for that posting).
> But if the checksum fails, what can you do? If you drop the whole schedule
> and reread all the postings from the same host, you'll just get back the
> same wrong schedule. You need an authority to go to to get the correct
I mentioned par2 files as a possible means of error correction in an earlier
post. Figuring out precisely how to apply this technology would require some
thought, but it would probably be do-able. Another possibility would be to
provide a central repository or "master" NNTP server to which Myth boxes
could connect as a last resort in the event of missing files. The code would
wait some period of time for missing files to appear on the user's primary
NNTP server and if they don't, connect to the master server to retrieve them.
> Someone else suggested bittorrent for this, and it actually has some
> properties that make it quite appropriate.
One problem with BitTorrent is that it's strongly associated with piracy. That
could make it a hard sell to potential content providers, such as TV stations
("you're going to be distributing the data to video pirates?! Get outta
here!"). I haven't looked into the issue in any detail, but I believe some
ISPs are blocking BitTorrent traffic, which if true is another big downside
for BitTorrent. Finally, I've heard stories of people getting DMCA takedown
notices for posting ANYTHING on BitTorrent, even family photos. Given the
frequency with which we'd need to post new data, this could become a very
serious problem if the RIAA (or whoever) is targeting BitTorrent users in a
wholesale way. Granted they'd have no legal leg to stand on, but defending
each and every takedown notice could become an unmanageable legal task.
More information about the mythtv-users