[mythtv] Ticket #2194: Patch to randomise mythfilldatabase times
Carl Reynolds
mythtv-dev at hyperbole-software.com
Sat Sep 16 20:37:44 UTC 2006
William Uther wrote:
>Alo,
>
>On 15/09/2006, at 2:55 PM, MythTV wrote:
>
>
>
>>#2194: Patch to randomise mythfilldatabase times
>>------------------------------------------------
>>+---------------------------
>> Reporter: willu.mailingLists at cse.unsw.edu.au | Owner:
>>cpinkham
>> Type: enhancement | Status: new
>> Priority: minor | Milestone: 0.21
>>Component: mythtv | Version: head
>> Severity: medium | Resolution:
>>------------------------------------------------
>>+---------------------------
>>Comment (by cpinkham):
>>
>> Doesn't this patch have the adverse side effect that jobs we want
>>to run
>> once a day could potentially run every 12 hours because of the
>>following
>> line:
>>
>> longEnough = ((period * oneday) - oneday/2);
>>
>>
>
>Yes. It does have that side-effect. This is because the old logic
>was "don't run more frequently than every 24 hours", NOT "run once
>per calendar day".
>
>I was after an effect that was something like "grab once per day, at
>a random time". If I left the old logic, then when the system
>grabbed late one day it would be forced to grab late the next day too
>(else it would have grabbed more than once in 24 contiguous hours).
>The only way it could get back to grabbing early would be to ratchet
>back in 5 minute intervals, or skip a day entirely.
>
>I decided that I did't want to fill the database too frequently, so I
>allow it to ratchet back in 12 hour increments rather than 5 minute
>increments. You could adjust that 6 hour increments if you want.
>I'd suggest leaving it as 12 hours - it isn't going to ratchet back
>easily because of the random delays.
>
>There is another issue that I thought about after submitting the
>patch... the patch currently assumes that your machine is running all
>the time. I haven't played with the auto shutdown-startup stuff in
>myth. It might not play nice with this patch.
>
>
>
>> That alone would be a bad thing, but is probably prevented by what
>>appears
>> to be flawed logic in the random time check:
>>
>> ((random()%((maxhour-minhour+1)*6)) == 0)
>>
>> So if maxhour is 24 and minhour is 0, then you're checking if the
>>random
>> number is evenly divisible by (24 - 0 + 1) * 6 which works out to
>>150.
>> Those would seem to be very very low odds which would mean we'd
>>always be
>> running in the last half hour of our window when the "(hour ==
>>maxhour &&
>> minute > 30)" check kicked in. Doesn't seem very randomized.
>>
>>
>
>Myth is making that check every 5 minutes. So in your example you
>have 24*12 = 288 chances to succeed, each of which, individually, has
>probability 1/150. The probability you don't run at all is (149/150)
>^288 = 0.15. So you are right in that it is slightly biased.
>Exactly how biased depends upon the range of time you give it to run.
>
>The probability currently isn't uniform because I didn't think it was
>worth the hassle. If you want to make it uniform then you need to
>make the probability 1/(# remaining chances to run today) as opposed
>to the current probability of 2/(# total chances to run today).
>
>I think this does it:
>
>((random()%((maxhour-hour)*12 + (60-minute)/5 - 6) == 0))
>
>(maxhour-hour)*12 = the number of attempts in the remaining full hours
>(60-minute)/5 = the number of attempts in the remaining partial hour
>- 6 = we don't want to include the last 30 minutes where a run is forced
>
>Of course, I've tested the other code, whereas I haven't tested
>this. Although the new formula plays more nicely with the auto-
>shutdown/wakeup stuff, in that if you wake up half-way through the
>"ok period" then you have boosted chance of running.
>
>
>
>> Can you check the logs to see when your mythfilldatabase runs were
>> actually occuring to show this is really random and not semi-
>>random, but
>> mostly in the last 30 minutes of the window?
>>
>>
>
>I checked this before posting the patch. But here is the data from
>my current logs. I set my grabber to run between 2 and 4 pm daily.
>
>% grep "mythbackend: Running mythfilldatabase" *
>mythbackend.log:2006-09-15 15:00:36.423 mythbackend: Running
>mythfilldatabase
>mythbackend.log.1:2006-09-14 16:04:14.974 mythbackend: Running
>mythfilldatabase
>mythbackend.log.10:2006-09-05 16:34:44.054 mythbackend: Running
>mythfilldatabase
>mythbackend.log.11:2006-09-04 16:32:59.660 mythbackend: Running
>mythfilldatabase
>mythbackend.log.12:2006-09-03 15:44:55.887 mythbackend: Running
>mythfilldatabase
>mythbackend.log.13:2006-09-02 14:42:09.719 mythbackend: Running
>mythfilldatabase
>mythbackend.log.14:2006-09-01 14:05:05.174 mythbackend: Running
>mythfilldatabase
>mythbackend.log.2:2006-09-13 14:56:30.569 mythbackend: Running
>mythfilldatabase
>mythbackend.log.3:2006-09-12 14:13:42.043 mythbackend: Running
>mythfilldatabase
>mythbackend.log.4:2006-09-11 14:31:46.024 mythbackend: Running
>mythfilldatabase
>mythbackend.log.5:2006-09-10 15:22:02.131 mythbackend: Running
>mythfilldatabase
>mythbackend.log.6:2006-09-09 14:04:50.952 mythbackend: Running
>mythfilldatabase
>mythbackend.log.7:2006-09-08 15:19:28.773 mythbackend: Running
>mythfilldatabase
>mythbackend.log.8:2006-09-07 14:26:42.468 mythbackend: Running
>mythfilldatabase
>mythbackend.log.9:2006-09-06 15:10:30.554 mythbackend: Running
>mythfilldatabase
>
>Hrm, maybe between 2 and 5pm... ahh - I remember now: MaxHour is
>inclusive :). And that answers your followup mail :) Although that
>means that the 24 on lines 174 & 178 really should change to 23, as
>should the 24s in programs/mythfrontend/globalsettings.cpp
>
>Or you could fix it so that maxhr is non-inclusive (it was inclusive
>before my patch in the code, but not the preference comments).
>That's probably a better fix.
>
>So here is the complete set of fixes - same as before but with a
>uniform distribution and non-inclusive max hour.
>
>Index: housekeeper.cpp
>===================================================================
>--- housekeeper.cpp (revision 10995)
>+++ housekeeper.cpp (working copy)
>@@ -48,7 +48,7 @@
> unsigned int longEnough = 0;
> if (period)
>- longEnough = ((period * oneday) - 600);
>+ longEnough = ((period * oneday) - oneday/2);
> else
> longEnough = oneday / 8;
>@@ -70,8 +70,13 @@
> if ((now.toTime_t() - lastrun.toTime_t()) > longEnough)
> {
> int hour = now.toString(QString("h")).toInt();
>- if ((hour >= minhour) && (hour <= maxhour))
>- runOK = true;
>+ if ((hour >= minhour) && (hour < maxhour))
>+ {
>+ int minute = now.toString(QString("m")).toInt();
>+ if ((hour == maxhour-1 && minute > 30)
>+ || ((random()%((maxhour-hour-1)*12+(60-
>minute)/5 - 6) == 0))
>+ runOK = true;
>+ }
> }
> }
> else
>@@ -202,7 +207,7 @@
> if ((nextRun < now) &&
> (lastRun.secsTo(now) > (3 * 60 * 60)) &&
>- ((minhr <= hour) && (hour <= maxhr)))
>+ ((minhr <= hour) && (hour < maxhr)))
> runMythFill = true;
> }
> else if (wantToRun("MythFillDB", period, minhr,
>maxhr))
>@@ -241,7 +246,7 @@
> updateLastrun(dbTag);
> }
>- sleep(300);
>+ sleep(300 + (random()%8));
> }
>}
>
>But that is totally untested.
>
>Be well,
>
>Will :-}
>
>--
>Dr William Uther National ICT Australia
>Phone: +61 2 8306 0424 Computer Science and Engineering
>Email: william.uther at nicta.com.au University of New South Wales
>Email: willu at cse.unsw.edu.au Sydney, Australia
>
>Web: http://www.cse.unsw.edu.au/~willu/ or http://www.nicta.com.au/
>
>NICTA email Disclaimer:
>http://www.cse.unsw.edu.au/~willu/NICTAEmailDisclaimer.html
>UNSW email Disclaimer:
>http://www.eng.unsw.edu.au/emaildis.htm
>
>
>_______________________________________________
>mythtv-dev mailing list
>mythtv-dev at mythtv.org
>http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-dev
>
>
>
>
>
I thought randomization of the request was taken care of by use of the
acknowledge function which requests from DataDirect a time to make the
next download.
Am I wrong about that?
Carl.
More information about the mythtv-dev
mailing list