[mythtv] [Patch] Logging to database PLUS automatic

[Mike Jenkins - SupportMe]

>Date: Fri, 27 Feb 2004 12:00:14 -0500
>From: "Joseph A. Caputo" <jcaputo1 at comcast.net>
>Subject: Re: [mythtv] [Patch] Logging to database PLUS automatic
>mythfilldatabase
>To: Development of mythtv <mythtv-dev at mythtv.org>
>Message-ID: <200402271200.15882.jcaputo1 at comcast.net>
>Content-Type: text/plain;  charset="iso-8859-1"
>
>On Thursday 26 February 2004 19:40, Matt White wrote:
> > > > Joseph A. Caputo wrote:
> > > Nice work... I'm glad to see you've incorporated the log flushing
> > > into the backend.  However, I'm still concerned.  I've had my /var
> > > partition fill up very quickly in the past when something went
> > > wonky on the backend.  The problem is/was that a condition occurred
> > > that caused the backend to spit out a seemingly infinite stream of
> > > errors that filled several gigs before I caught it.  When something
> > > happens that causes that kind of log activity, an age-based log
> > > expiration algorithm is insufficient.  I'd really like to see a
> > > limit on the *number* of total log entries allowed in the table.
> > > Also, it would probably help to have a mechanism to filter out
> > > repetitive log entries, like so (warning: crude hack follows, but
> > > you'll get the idea):
> >
> > [snip fixed code]
> >
> > Fantastic idea!  I haven't had a runaway like that with myth for
> > quite a while, so I hadn't thought of that.  I've added your code -
> > it pretty much worked out of the box :-)  I've also added two things
> > that I forgot to add to globalsettings.cpp in mythfrontend for the
> > log flush routine - maximum age of acknowledged messages to keep, and
> > maximum age of all messages to keep.
> >
> > I'm just going to do some testing here to make sure that the logging
> > stuff doesn't do anything I don't expect, and then I'll post another
> > diff.
> >
> > While I'm testing, I'd like some opinions on another idea, from a
> > discussion with Mark Edwards (thanks, Mark).  I'm thinking of adding
> > another config item ("Print logged messages higher than this
> > priority") and a bit of code in the log function to print the module
> > and message for matching events to stdout as well as logging them.
> >
> > That would give us a single logging function that could output both
> > to the database and optionally to stdout.  The VERBOSE macro is
> > probably still important for debugging stuff, but this way you
> > wouldn't need two calls where you want to log to both the database
> > and stdout. It would also give the advantage of applying Joseph's "no
> > multiple message" logic to printed messages as well.
> >
> > Thoughts?
>
>
> Glad I could help.  Would you be able to add a size limit to the log
> table as well?  Either per-module or overall.  After (or before)
> inserting a log message (assuming it's not a repeat), you can do a:
>
> SELECT COUNT(*) from mythlog WHERE module = <module name>
>
> to get the current number of entries for a module; if it's greater than
> some maximum, enter a loop where you delete the oldest entry for that
> module until it's below the max number of entries.  Actually, there's
> probably a clever way to delete the <n> oldest entries, but my SQL
> isn't good enough...
>
> In any event, the simple case is that you should only ever have 1 entry
> to delete, unless you've lowered the max number (which would be
> configurable) since the last time the housekeeping thread ran.
>
> One last thing... (and I haven't looked at the UI for the logs, so maybe
> you already filter this out of the UI...)  Does it really make sense to
> keep logs in the database after they've been acknowledged?  At the very
> least they should be filtered out of the UI view, but I don't know that
> there's really any point to keeping them in the DB once they've been
> acknowledged.  Just something to chew on.
>
> Keep up the good work!
>
> -JAC
>

>From MySql ver 4.0.0 onwards, the ORDER BY syntax was added to DELETE, so
this code should do what you're looking for . . .
DELETE FROM mythlog WHERE module = <module name> ORDER BY ldate ASC LIMIT x;
which will delete the oldest x entries (by ldate) where the row matches the
criteria


Pre 4.0.0 are restricted to . . .
DELETE FROM mythlog WHERE module = <module name> LIMIT x;


Cheers

Mike