[mythtv] Thoughts on encoder farms / distributed capturing and viewing

[Isaac Richards]

On Wednesday 30 October 2002 10:57 am, Robert Kulagowski wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Isaac, do you have thoughts on how you're going to proceed?  I've
> been thinking of "nice to have" features.  Stream of thought stuff
> here...
>
> o  Self-organizing encoder farms.  Every 30 seconds, mythtv sends out
> a broadcast packet to the local LAN that has information in it like:
> hostname, disk space free, number of encoder cards and encoder card
> type (or source, in case something like DirecTV is only available on
> one encoder).  This allows cooperative scheduling.  The other way is
> to simply do what Cisco's IP/TV does:  at startup, the server
> enumerates the encoder cards and the existing content and sends a
> message back to the Content Manager.  The Content Manager then
> changes the status of the server to "Managed" and maintains a table.
> The server and the content manager have to be manually configured to
> make them aware of each other.

I'll probably follow the master/slaves model, but yeah, something like that.  
Configure one box as a master node, have it run the DB and act on requests 
from client set-top boxes, etc.  Encoding nodes can automatically announce 
themselves, etc.

> o  A machine that doesn't have any encoder cards is still valuable,
> either as a transcode machine, or as a disk space repository /
> archive server.

Yup.

> o  Totally decoupling the encoder and the viewer.  Therefore, a local
> node that has an encoder _might not_ be using it to watch live TV;
> its "live" channel may actually be coming from a different encoder on
> a different node.  The ringbuffer file is getting filled using IP.

Should be quite easy to do, actually.  Someone's doing live-tv completely over 
a NFS mount right now, and this'd be half the amount of network traffic.

> o  A master scheduler is always looking a few minutes into the
> future.  If a program is supposed to be recorded, use the "free
> encoder card" broadcast messages to determine which node should do
> the recording, and the free space message to determine if we need to
> purge content.  If you use a master/slave relationship rather than a
> cooperative one, then this might not be as big an issue, since
> presumably the master will always know the status of each encoder
> card.

Right, the master box can know everything, and just tell a slave when to start 
and stop, no need to query, etc.

> o  A "stripe" based conflict resolution scheme.  I'm not sure that
> I'll be able to explain this, but I'll try.  One of the things that I
> don't like about my Tivo is the following:  "Everybody Loves Raymond"
> runs on CBS from 1900 to 1930 in Chicago.  Sometimes, due to clock
> issues, or the network being a little late, we miss the last bit of
> "zinger" before the final credit roll.  So I tell Tivo to go 2
> minutes long, and so now it records from 1900 to 1932.  The problem
> is that if there's a show on a different channel starting at 1930, it
> won't get recorded at all, because the Tivo isn't allocating smaller
> blocks.  So even if I'm willing to miss the initial credits on a
> show, I'm out of luck.  My thinking is that if you split a show into
> 1 minute blocks and overlap based on priority, the 32 minute show
> will overlap the first two minutes of the show on the other channel.
> Every minute, the encoder is checking to master schedule to see what
> channel it should be recording, so it will naturally "fall off" the
> end of the 32 minute show and onto the 28 minute one.  This scheme
> also handles "early start", since a higher priority show will overlap
> the end of a lower priority show.  Of course, if there are multiple
> encoder cards that are free then there's no conflict.

_Should_ be fairly easy to accomplish that.  I've already got over-time 
recording in there, it'd just need logic to decide to delay starting another 
recording while the overage is finished on the first one..  The current code 
doesn't handle early starts, though.

> o  Partial deletes.  I know that you mentioned commercial cutting,
> but I'm interested in "delete up to this point".  Say I have a 2 hour
> recording, and I've already watched 1 hour of it.  I want to free up
> the disk space by deleting the first hour.

Once the ability's there to remove segments of the video, this'd be easy..  
The hard part's rewriting the file, really.

> o  A "hint" system for shows that are on multiple times per day.  On
> my Tivo, if I tell it to record "The Daily Show", it's going to
> record it 4 times a day because lately, TMS hasn't been putting in
> show descriptions, so the Tivo doesn't know that the showing at 1400
> and 1900 is a rebroadcast of the previous days' 2200 broadcast.  When
> I do a manual record by time, I have to go in and delete the Friday
> "Comedy Central Presents" which is on at the same time because I
> can't program the Tivo to record M-Th.

The current timeslot recording stuff doesn't handle that?  It won't record 
unless the title of the program matches.

> o  Configurable "history".  Keep track of shows that were recorded,
> and when they were deleted.  That way, if I'm going to watch an
> episode of a show and I don't remember if I've seen the OSD can tell
> me, either at time of recording or in the "To Do" list.  The Tivo has
> a 28 day memory on this sort of stuff, but most syndicated shows take
> longer to cycle through than that.  Allow the user to specify how
> long until purge.  Disk space is fairly cheap.  Note:  this probably
> won't work unless we get the episode descriptions instead of just the
> show name.

It already keeps track of what you've recorded.

> How's that for a wish list?  Can you get all of that in before 0.7?

Can you? =)  Patches are, as always, accepted.

Isaac