[mythtv-users] Mooting architecture for a DataDirect replacement
Michael Jones
mythlist at michaelandholly.com
Thu Jun 21 04:49:58 UTC 2007
Hmm.. good point.. MySQL with the potential number of hits, and
trying to generate the data every time a request was made could get
"expensive".
On Jun 21, 2007, at 12:33 AM, Peter Schachte wrote:
> Michael Jones wrote:
>> Ok.. if the SQL only is too "myth specific" try this on for
>> size... :-)
>>
>> If the data were in an SQL database, a frontend on the service could
>> EASILY export it to a file, if that's what the user/subscription/
>> preference whatever decided. The point is not the format that is
>> used.. since the contents of the SQL database can be output into
>> almost any format imaginable at the drop of a digital hat (usually
>> measured in milliseconds). Keep the format in most general,
>> manageable format and distribute it as needed.
>
> Sure, there's nothing wrong with using a database, but I think an
> XMLTV file
> would work just as well, and could be much cheaper on server
> resources to
> handle. The scheme you're suggesting requires the server to figure
> out what's
> changed since a certain date for every transaction. If the number
> of requests
> gets high, that could be expensive. It also requires a fair bit of
> coding. I
> would think rsync would be more efficient for the server.
>
> And if rsync is too expensive, a simple alternative requiring only
> a little
> coding and just an anonymous ftp or http server could lower the
> costs. This
> would be for the server to store for each market a set of
> difference files
> sufficient to bring the client up to date from whenever it last
> updated. Every
> 5 minutes, it could compute and store the changes in the last 5
> minutes. On
> the hour, it could delete all the 5-minute files and compute the
> changes over
> the last hour, and at midnight it could delete all the 1-hour files
> and compute
> the differences since midnight yesterday. This only requires the
> server to
> compute one diff every 5 minutes for each TV market it services,
> plus a service
> a bunch of (mostly small) ftp/http file requests.
>
> Then the client just needs to keep around the last midnight file
> and the last
> on-the-hour file in addition to the latest file. When it connects
> to the host,
> it just downloads all the files generated since it last connected.
> If it
> downloaded any on-the-hour diffs, it deletes its local 5-minute
> file; if it
> downoaded any midnight files, it also deletes its local on-the-hour
> file. Then
> it applies the diffs it downloaded in order to its latest file,
> bringing it up
> to date. This shouldn't require too much CPU resources. And the
> coding should
> just require a couple of reasonably simple shell or python scripts.
>
> The whole thing could be made more efficient, too, by using a
> binary difference
> format. It's not like anyone is going to be editing these XMLTV
> files by hand,
> or they're going to be changed by different parties independently,
> so the
> redundancy of diff isn't needed. A binary diff format could contain a
> cryptographic hash of the final file, so after application the
> result could be
> verified to be byte-correct. The file itself could just be a
> sequence of
> simple commands to move forward a number of bytes, delete a number
> of bytes,
> and insert a string. That would all take a bit more coding, but
> it's optional.
>
> --
> Peter Schachte We hang the petty thieves and appoint
> the great
> schachte at cs.mu.OZ.AU ones to public office.
> www.cs.mu.oz.au/~schachte/ -- Aesop
> Phone: +61 3 8344 1338
> _______________________________________________
> mythtv-users mailing list
> mythtv-users at mythtv.org
> http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
>
More information about the mythtv-users
mailing list