[mythtv-users] Better listings for Schedules Direct users. Free! (was Re: another scheduling strangeness/question)

[Michael T. Dean]

  On 09/01/2010 01:39 AM, Ozzy Lash wrote:
> On Tue, Aug 31, 2010 at 6:50 PM, Michael T. Dean wrote:
>> And, now it's time to come clean.  What I hadn't said is that we actually
>> have full support for grabbing all days of listings data from TMS in a
>> single pull.  It's been this way for quite some time (including in
>> 0.23-fixes, and possibly even 0.22-fixes).
>>
>> After discussion with some of the Schedules Direct board and a lot of
>> effort, including testing, by people such as Robert Eden and Chris Petersen,
>> we've decided to "advertise" this approach.  Since it works in 0.23-fixes as
>> well as trunk, users may enable its use immediately.
>>
>> However, use of --dd-grab-all has not been optimized, so it can take
>> significantly more CPU and RAM than a "normal" run of mythfilldatabase.
>>   Users with resource-limited backend systems may not be able to use the
>> argument.
> I just tried it on a newish AMD quad core system with 4 Gig of RAM
> thinking "I don't have a resource limited system".  While I was doing
> a single recording over firewire and simultaneous commercial flagging,
> the run took a little over 45 minutes (only about 2.5 downloading the
> data).  I feel so humbled!  I have 2 source, one for clear QAM which
> only has 50 channels or so (maybe less) and one for firewire which has
> a few hundred (actually it says 502 in the mythfilldatabase output).
> Clearing the data for source 1 for all 14 days took about 3 minutes,
> and clearing the data for the firewire source took about 30 minutes.
> Does this point to something I need to tune in my database setup?  I
> have some tweaks to my mysql settings that were suggested a long time
> ago on this list if you had a lot of memory (probably 2 gig back then)
>   Here they are:

It's possible that the slow DB update was primarily due to DB locking 
caused by the fact that the database was in use while recording.  It's 
also possible that lineup--550 channels--may just be asking a lot of 
even your system.

If the run didn't cause any problems with your recordings, you can 
continue to use --dd-grab-all, even if it does take a long time to complete.

Regardless, if you decide not to use it with 0.23-fixes, you may want to 
try again when you upgrade to 0.24 (after it's released).  It will have 
some optimizations that may make it much less resource intensive, even 
for a 550-channel system.  In truth, I expect with 0.24, your 
--dd-grab-all run time will be almost the same as a run time without 
that argument.

As far as the DB optimizations go, I'll leave it to others to help.  The 
one thing I will say, however, is that having the database's binary data 
files on the same file system that you're using to record could have a 
huge impact on performance.  In truth, the best setup puts the database 
on a separate spindle.

> I looked at the listings at one point during the run through mythweb,
> and they were all cleared, which worried me about using this option
> all the time.  What would happen a recording was to start during this
> interval?

That I can't tell you.  If the scheduler runs without any listings in 
the database, it won't record anything.  In truth, though, if your 
system is taking 45mins to complete the run, you are probably better 
equipped to test that "what if" than anyone else, since you have such a 
large working time in which to put the recording start.  :)  I'd be very 
interested to see your results.

Mike