[mythtv] [Patch] Logging to database PLUS automatic mythfilldatabase

[Joseph A. Caputo]

On Thursday 26 February 2004 19:40, Matt White wrote:
> Joseph A. Caputo wrote:
> > Nice work... I'm glad to see you've incorporated the log flushing
> > into the backend.  However, I'm still concerned.  I've had my /var
> > partition fill up very quickly in the past when something went
> > wonky on the backend.  The problem is/was that a condition occurred
> > that caused the backend to spit out a seemingly infinite stream of
> > errors that filled several gigs before I caught it.  When something
> > happens that causes that kind of log activity, an age-based log
> > expiration algorithm is insufficient.  I'd really like to see a
> > limit on the *number* of total log entries allowed in the table. 
> > Also, it would probably help to have a mechanism to filter out
> > repetitive log entries, like so (warning: crude hack follows, but
> > you'll get the idea):
>
> [snip fixed code]
>
> Fantastic idea!  I haven't had a runaway like that with myth for
> quite a while, so I hadn't thought of that.  I've added your code -
> it pretty much worked out of the box :-)  I've also added two things
> that I forgot to add to globalsettings.cpp in mythfrontend for the
> log flush routine - maximum age of acknowledged messages to keep, and
> maximum age of all messages to keep.
>
> I'm just going to do some testing here to make sure that the logging
> stuff doesn't do anything I don't expect, and then I'll post another
> diff.
>
> While I'm testing, I'd like some opinions on another idea, from a
> discussion with Mark Edwards (thanks, Mark).  I'm thinking of adding
> another config item ("Print logged messages higher than this
> priority") and a bit of code in the log function to print the module
> and message for matching events to stdout as well as logging them.
>
> That would give us a single logging function that could output both
> to the database and optionally to stdout.  The VERBOSE macro is
> probably still important for debugging stuff, but this way you
> wouldn't need two calls where you want to log to both the database
> and stdout. It would also give the advantage of applying Joseph's "no
> multiple message" logic to printed messages as well.
>
> Thoughts?


Glad I could help.  Would you be able to add a size limit to the log 
table as well?  Either per-module or overall.  After (or before) 
inserting a log message (assuming it's not a repeat), you can do a:

SELECT COUNT(*) from mythlog WHERE module = <module name>

to get the current number of entries for a module; if it's greater than 
some maximum, enter a loop where you delete the oldest entry for that 
module until it's below the max number of entries.  Actually, there's 
probably a clever way to delete the <n> oldest entries, but my SQL 
isn't good enough...

In any event, the simple case is that you should only ever have 1 entry 
to delete, unless you've lowered the max number (which would be 
configurable) since the last time the housekeeping thread ran.

One last thing... (and I haven't looked at the UI for the logs, so maybe 
you already filter this out of the UI...)  Does it really make sense to 
keep logs in the database after they've been acknowledged?  At the very 
least they should be filtered out of the UI view, but I don't know that 
there's really any point to keeping them in the DB once they've been 
acknowledged.  Just something to chew on.

Keep up the good work!

-JAC