[mythtv-users] Idea for Listings Data

Matt Mossholder matt+mythTV-Users at mossholder.com
Mon Jun 25 23:02:46 UTC 2007


On Mon, 2007-06-25 at 10:58 -0400, Rod Smith wrote:
> Neither DNS nor NTP would do the job; neither was intended to transfer records 
> as large as we're talking about. For something "similar to" these protocols, 
> but designed to transfer files of the size we're discussing, it starts to 
> look a lot like Usenet or a P2P network, which is why the discussion on this 
> topic has focused on those two options so far.
> 

Rod,
        On first glance I was inclined to agree with the detractors of
using
DNS to host the data. However, a little research shows this may not be
as obvious as we all thought. Here's  a quote [1] from zeroconf.org
about DNS TXT records (the type we would use for a DNS based listings
service):

"General format rules for DNS TXT records
A DNS TXT record can be up to 65535 (0xFFFF) bytes long. The total
length is indicated by the length given in the resource record header in
the DNS message. There is no way to tell directly from the data alone
how long it is (e.g. there is no length count at the start, or
terminating NULL byte at the end).

The format of the data within a DNS TXT record is zero or more strings,
packed together in memory without any intervening gaps or padding bytes
for word alignment.

The format of each constituent string within the DNS TXT record is a
single length byte, followed by 0-255 bytes of text data.

These format rules are defined in Section 3.3.14 of RFC 1035, and are
not specific to DNS-SD. DNS-SD simply specifies a usage convention for
what data should be stored in those constituent strings."


        The first sentence is very telling. Each record can be up to
64K. I
seem to remember that the napkin-back calculations for the size of an
entry were along the lines of 1-2K per entry. Given this, it seems like
DNS TXT records may actually be an even better solution than NNTP...
consider the following:

        * DNS is pretty much a requirement for all ISPs, while NNTP
isn't as
well supported anymore (e.g. ISPs blocking access).

        * DNS records get cached by practically every ISP in the world,
NNTP is
fading away, if only because people have some many different
alternatives (My take, at least. Tell me if I am wrong).

        * DNS zone serial numbers and TTL could be used to handle record
updates. TTL could be used to minimize the need to re-request data on a
per-record basis, and the actual record TTL could be set based on the
historical rate of change for the given channel (e.g. changes to channel
X always occur more than 24 hours before broadcast, while channel Y is
changing the schedule up to the current hour). If we were really spiffy,
we could set the TTL of each returned request from the primary DNS
server, so that they expire within, say, 2 hours of the air time for the
show.




        Thoughts?


                --Matt




More information about the mythtv-users mailing list