[mythtv-users] Mooting architecture for a DataDirect replacement

Dan Ritter dsr-myth at tao.merseine.nu
Fri Jun 22 19:32:49 UTC 2007

On Fri, Jun 22, 2007 at 11:27:55AM -0700, Yeechang Lee wrote:
> Jay R. Ashworth <jra at baylink.com> says:
> > Partially, yes.  I can see people like Yeechang Lee running servers
> > for the appropriate hierarchy to which they permit access by others
> > -- at least other Myth users.
> Sure. I have CPU cycles to spare on that quad-core Xeon and the kinds
> of newsgroups we'd be talking about would, I think, be pretty minimal
> in terms of disk space (compared to the typical 20,000-group newsfeed)
> and, assuming enough other upstream and downstream newsserver nodes,
> bandwidth. I obviously can't provide any meaningful uptime-reliability
> guarantees, but I do have 102 days' uptime and counting on said
> server. I'd need someone to provide DNS aliasing, though, so my box
> resolves to something sf.ca.us.feed.mythtv.org (and one of the boxes
> in the pool that resolves to ca.us.feed.mythtv.org and
> us.feed.mythtv.org), though.
> That said, I've never run INN or any other newsserver; my Usenet
> experience, although more than 15 years in duration, has been strictly
> as a consumer, not a provider. I am not worried about learning care
> and upkeep for a newsserver; however, I can't guarantee that my setup,
> as sophisticated as it is, will be sufficient to handle the kind of
> traffic necessary.

Let's suppose that you were going to be a central node for the US. How
much disk space and bandwidth? A quick estimate, all numbers made up.

A channel has 2 programs per hour all day long: 48 programs per
day. You'll store 14 days, or 672 programs, times 3000 channels
in the US. Just over 2 million entries. Each day 150,000 entries
get changed. That about 1.7 a second. We'll say 2 per second.

A program is a start time, a finish time, a channel id, a
program number, a title, a subtitle, a description, an episode
number, yadda yadda yadda. About a kilobyte, all told. So,
16Kbits/second on average, storage of 150MB. Let's say 300MB,
we're inefficient. Even if an XML representation triples it,
which it shouldn't, everyone has enough space to store all the
program information they will need.

People on dialup won't want to use this. People on cheaper DSL
lines will only be interested in being clients. But pretty much
anyone else can be at least a small server node, with 2 or 3
local neighbors just pulling down their regional groups and 2 or
3 big nodes feeding them. A big node should have more bandwidth,
but doesn't need much else.

NNTP is pretty efficient -- it's one of those old-style
telnettable protocols that a human can learn to speak in a few
minutes. Less than 5% overhead on the line.

Usenet was deep magic when a T1 and 10 GBs storage made you a
decent regional ISP. Now? Not so much, unless you want to carry
all the traffic in the world. Especially, not carrying binaries
reduces the traffic by a factor of, oh, a hundred. Maybe more, I
haven't checked lately.

The software got better, too, and so did the documentation.


.. .----. --   .-. . .- -.. .. -. --.   -.-- --- ..- .-.   -- .- .. .-.. .-.-.-   .-- .... ---   . .-.. ... .   .. ... ..--.. 
http://tao.merseine.nu/~dsr/eula.html is hereby incorporated by reference.

More information about the mythtv-users mailing list