[mythtv-users] Re: Bleb.org down??

[Rob Willett]

David,

I have looked at the Showshifter site and they have said that Digiguide won't
play ball with them at all. They and PureTelly are the only people I know doing
decent listings that are free(ish). I did look at trying to scrape PureTelly and
then gave up.

In light of this I have now written most of a very simple distribution service.
I offer this up as a sacrifical lamb...

It works on a three tier approach. 

Somebody, somewhere must create the master XML data files. This is the bit to
do... <cough>. I've written software to handle the distribution only. It will
work with a single server but clearly the more servers we have the wider the
distribution and the less load we each have to bear. It will also work around
the world by the user simply choosing a different Tier-2 server list when they
start. 

The software works as follows. A master server (bleb.org?) is used to seed a set
of Tier 2 servers. Seeding is simply copying the XML files off the master server
to the Tier 2 servers. I have assumed the XML files are appropriate for mythtv.
All the software does at the moment is distribute. The Tier 2 servers then make
all the XML files and the list of Tier 2 servers available to all and sundry.
Simple and easy, no databases, no hard work. All a tier-2 provider need do is
daily update their own files from either the master server or from another Tier
2 provider. They will need a web server and preferably a dedicated directory.
They will not need mySQL, PHP though they will need Perl, ping and wget (so far).

All the software is written in Perl. It consists of a perl script and a file
containing the list of Tier-2 servers and which directory the xml files are in.
This software will randomly choose one of the Tier 2 servers (never the master)
and try and download the XML software and the server list. If a server is down
it will choose another server until it either succeeds or runds out of servers. 

At the moment I have the core software written but am struggling a little with
wget when it isn't pulling back web pages, just files. I want the power of rsync
but on port 80. I'll crack that hopefully later today and will then send files out.

Rather than clogging up the list if there is enough interest I'll setup a
mailing list for anybody interested in helping out. If you are send me an e-mail
with to rob.mythtv.devlist.no.spam at robertwillett.com. Clearly remove the
.no.spam  to get this work.

Rob.

Quoting David <myth at dgreaves.com>:

> Rob Willett wrote:
> 
> >David,
> >
> >I have no ideas about the numbers of mythtv users in the UK. I would agree
> that
> >the numbers aren't high, but would also think that we want this feed to
> keep
> >going. I would happily pay DigiGuide (or PureTelly) their £10/year or
> whatever
> >to get a decent integrated EPG into mythTv. Since they won't play ball, I
> want
> >whatever we do to keep going for as long as possible so would *prefer* to
> try
> >and keep things above board and official. 
> >  
> >
> There was a comment on the list about approaching a company.
> A quick search failed - I'll research
> (I too would pay for the content)
> 
> >My design is complex only if we have to use a database and even then it's
> not
> >too much of a problem. If all we have is a daily updated XML file then
> simply
> >copy the file and make it available. I am a great believer in keeping it
> simple.
> >However I don't know what the process is for getting that XML file off
> bleb.org
> >and so was simply looking at the worse case. 
> >  
> >
> yes, sorry - I just rattled of a reply - wasn't being critical (well, I
> was, but in a nice way :) )
> What i described was more like a simple hack to make it work - then
> evolve it.
> 
> >Bittorrent is overkill but it has a lot of things already setup, like
> automated
> >distributed downloads and load balancing. Rather than programming websites
> for
> >mirrors, use an already existing program. The downside is that the user has
> to
> >have bittorrent setup on their system. It is more suitable for widespread
> peer
> >to peer distribution than for mirroring and then distribute. I think the
> biggest
> >problem here is not the actual download of data, the XNL file is tiny, but
> the
> >connection costs if too many people all hit a single server.
> >  
> >
> I wouldn't approach it like that - but I've been wrong before ;)
> I don't think we're doing p2p - we all want to be clients to a trusted
> resilient master.
> 
> Lets split the p2p clients into 2 groups - those with data to share
> (I'll call them servers) and those without (clients)
> Where does a p2p peer server get the data to share? running tv_grab_uk_rt?
> For what channels? All of them? and how are they distributed? As big
> xmltv's (ugh - I only want a few channels)
> 
> >I think what you're saying is that we should setup an independent TV XML
> file
> >distribution service independent of bleb.org. I have no problem with that
> but
> >was originally replying back to mirroring bleb.org. If we setup our own
> service
> >then what you say makes sense. 
> >  
> >
> I suppose I am - but only 'cos bleb doesn't do what I (and others) want,
> and Andrew's too busy having a life to reply (sensible chap) - and
> indeed you were - no offense ;)
> 

> 


o

-------------------------------------------------
This mail sent through IMP: http://horde.org/imp/