[mythtv-users] Better listings for Schedules Direct users. Free! (was Re: another scheduling strangeness/question)

[Ozzy Lash]

On Tue, Aug 31, 2010 at 6:50 PM, Michael T. Dean
<mtdean at thirdcontact.com> wrote:
>  On 08/31/2010 05:31 PM, Michael T. Dean wrote:
>>
>>  On 08/31/2010 05:24 PM, Yeechang Lee wrote:
>>>
>>> Michael T. Dean says:
>>>>
>>>> I'm planning to make changes to mythfilldatabase so that it always
>>>> retrieves all of the data (at minimum tomorrow through +13) for
>>>> Schedules Direct users.
>>>
>>> Would --refresh-all --refresh-today be sufficient to accomplish this
>>> today?
>>
>> No.  That would be /very/ bad.  The problem now is that mythfilldatabase
>> makes 2 separate requests--one to get each of today and +13.  (And, in
>> truth, it can make additional requests to get +12 and +11 and ... if it
>> detects significant holes in the listings.)  Using --refresh-all makes /13/
>> requests (+1 through +13) and adding --refresh-today adds a 14th request.
>>
>> AIUI, Robert has said that pulling all the data most likely woudn't be a
>> problem (more testing still required, so please bear with us :) if it was
>> done as a single request for all 14 days of listings.  To handle this
>> properly and reliably on all our users' systems, we need some changes to the
>> code and, possibly, to the MythTV database schema.  We're planning these
>> changes, but they will take some time.
>
> And, now it's time to come clean.  What I hadn't said is that we actually
> have full support for grabbing all days of listings data from TMS in a
> single pull.  It's been this way for quite some time (including in
> 0.23-fixes, and possibly even 0.22-fixes).
>
> After discussion with some of the Schedules Direct board and a lot of
> effort, including testing, by people such as Robert Eden and Chris Petersen,
> we've decided to "advertise" this approach.  Since it works in 0.23-fixes as
> well as trunk, users may enable its use immediately.
>
> However, use of --dd-grab-all has not been optimized, so it can take
> significantly more CPU and RAM than a "normal" run of mythfilldatabase.
>  Users with resource-limited backend systems may not be able to use the
> argument.

I just tried it on a newish AMD quad core system with 4 Gig of RAM
thinking "I don't have a resource limited system".  While I was doing
a single recording over firewire and simultaneous commercial flagging,
the run took a little over 45 minutes (only about 2.5 downloading the
data).  I feel so humbled!  I have 2 source, one for clear QAM which
only has 50 channels or so (maybe less) and one for firewire which has
a few hundred (actually it says 502 in the mythfilldatabase output).
Clearing the data for source 1 for all 14 days took about 3 minutes,
and clearing the data for the firewire source took about 30 minutes.
Does this point to something I need to tune in my database setup?  I
have some tweaks to my mysql settings that were suggested a long time
ago on this list if you had a lot of memory (probably 2 gig back then)
 Here they are:

key_buffer_size         = 192M

query_cache_limit       = 2M
query_cache_size        = 64M
query_cache_type        = 1

table_cache             = 128
myisam_sort_buffer_size = 192M
sort_buffer_size        = 2M
read_buffer_size        = 2M
join_buffer_size        = 2M
read_rnd_buffer_size    = 2M


I looked at the listings at one point during the run through mythweb,
and they were all cleared, which worried me about using this option
all the time.  What would happen a recording was to start during this
interval?

Bill