[mythtv-users] Yeechang: Ticket #1660

[Steven Adeff]

On 4/5/07, Chris Pinkham <cpinkham at bc2va.org> wrote:
> First I have to comment on this... :)
>
> > In-Reply-To: <200704042200.l34M0Wdv025380 at freevo.ajlc.waterloo.on.ca>
>
> What's       the     deal      with    this ^^^^^^ on the Myth list?? :)
>
> * On Wed Apr 04, 2007 at 06:00:32PM -0400, Tony Lill wrote:
> > > Branching off the SavePositionMap() stuff into another thread is only
> > > putting a band-aid on the situation, anything else along the line could
> > > cause that reader to hang up too long and cause dropped data.
> >
> > I'm wondering what that "anything else" could be. When I was looking
> > at the mpegrecorder, the only interaction with the system I could see
> > was reading from the device and doing the database stuff. Everything
>
> I admit there isn't much in there, but think my comments below still
> hold true.  Who knows what might be (or need to be) added down the
> line that could make the processing waver in it's real-time ability
> while still being able to keep up over time.
>
> > buffering on the read side. If the data-processing can't keep up with
> > the data stream, eventually any buffer can be overrun.
>
> The object isn't to handle a situation where the processing can't keep
> up, it is to handle the situation where there are times along the way
> when the processing time involved may increase or decrease slightly
> while still maintaining real-time over the long haul.
> >
> > IMHO, the fundamental problem is that the path from the device to the
> > file has real-time contraints, and the database does not. Getting the
> > database out of that path is the correct solution, and throwing
> > buffering at is is the band-aid. But I could be biased !-)
>
> Actually, the path from the device to mythbackend has real-time
> constraints, but we could buffer the whole recording in memory before
> writing and it wouldn't cause ill-effects on the output file.  We need
> to get the data from the card as close to real-time as possible so that
> any internal buffer on the firewire card does not fill up, but our
> processing of that data doesn't need to be real-time as long as it can
> keep up over time and this processing can be pushed off to another
> thread.  Take a look at the software recorder in NuppelVideoRecorder.
> The main capture thread doesn't do the encoding, it just buffers raw
> frames and they are encoded/consumed in another thread.  The NVR
> recorder wasn't affected by the SQL updates because the capture of
> the data and the processing of the data are separated.
>
> If the real-time collection and the non-real-time processing are
> separated, then we don't need to separate out the DB updates further
> because they are already in the non-real-time section of the code.
>
> > I've wondered if all the read-side buffering in the other recorders
> > could be dropped if you used my patch, or something like it.
>
> I still think the buffering needs to be as close to the real-time
> component as makes sense.  Back in the day, I definitely preferred having
> a 16550A serial chip with the built-in buffering as opposed to the old
> 8250 or 16450 without buffering, because when anything happened that
> prevented me from reading data real-time I knew I had that safety
> buffer even it was only 16 bytes long.
>
> I liked the patch and the idea behind it, I think it's just a matter of
> which direction you look at the problem from and where you draw the
> (buffer) line at.  Processing occasionaly holds up collection, so
> there needs to be a buffer somewhere.  Do you put the buffer partway
> through the processing (ie, in the DB update), or do you put it in
> between processing and collection so that nothing else in processing
> can hold up collection either (assuming the buffer never fills up which
> is an even bigger issue).

I'm lost... (you crazy computer systems guys!)

But to me (silly little electric power engineer...) it would seem that
if the limiting factor to getting data from the firewire device (in
our case the cablebox) so as to not loose any data is the size of the
buffer on the firewire card/chip/bus(?), then wouldn't placing a
buffer on the collection side of things so that the limited hardware
buffer does not overrun and then let processing catch up after it
occasionally gets held up be the best option?

Another question, what is causing the processing to cause occasional
hold ups? Is it a processor speed thing, does it have to do with SQL?

thanks!

-- 
Steve
Before you ask, read the FAQ!
http://www.mythtv.org/wiki/index.php/Frequently_Asked_Questions
then search the Wiki, and this list,
http://www.gossamer-threads.com/lists/mythtv/
Mailinglist etiquette -
http://www.mythtv.org/wiki/index.php/Mailing_List_etiquette