[mythtv] [Patch] Logging to database PLUS automatic
mythfilldatabase
Joseph A. Caputo
jcaputo1 at comcast.net
Thu Feb 26 16:13:58 EST 2004
On Wednesday 25 February 2004 20:27, Matt White wrote:
> OK - here's the latest version of my db logging patch. The changes
> are:
>
> * uses the same numbering as unix syslog() priorities (make sure you
> update to the newest mythlog - see below for URL).
>
> * on Isaac's suggestion, I've incorporated the 'mythflushlog.pl'
> script functionality into mythbackend. Go into Settings->General,
> and on the third page, you will be able to enable/disable database
> logging, and also enable/disable the log cleaning functionality and
> set the frequency of the cleaning.
>
> * While I was doing that, I figured I'd add a generic "Housekeeping"
> thread. Currently, the housekeeping thread has two tasks -
> cleaning the log table, and running mythfilldatabase. Configuration
> is done in Settings->General. For running mythfilldatabase,
> configuration includes:
> * enable/disable automatic execution
> * frequency of execution (# days)
> * limits on when to run it (ie. only between certain hours)
> * path and arguments for mythfilldatabase
> * path for logfile for mythfilldatabase output
>
> I figured as long as I was doing a housekeeping thread,
> mythfilldatabase was a big one. It's got some advantages - easier to
> configure, no worries about QTDIR being set (it has to be set to run
> mythbackend, so it'll be inherited).
>
> I've also added a table to store the timestamps of the last run of
> each of the housekeeping items. Both this table and the logging
> table have been added to dbcheck.cpp.
>
> Everything I've added is turned off by default. Other defaults are:
>
> Log Cleaning frequency: 14 days
> Mythfilldatabase frequency: 1 day
> Mythfilldatabase hour restriction: between 2 AM and 5 AM
> Mythfilldatabase path: /usr/local/bin/mythfilldatabase
> Mythfilldatabase args: none
> Mythfilldatabase logfile: /var/log/mythfilldatabase.log
>
> As mentioned above, the mythlog module has been updated as well to
> make the priorities correct. It can be downloaded from:
>
> http://borris.usask.ca/mythtv/mythlog-0.2.tar.gz
>
> Attached are two patches. The big one is what I've described above.
> The second is a patch to the mainmenu.xml and main_settings.xml
> to add the MythLog module to the menus.
Nice work... I'm glad to see you've incorporated the log flushing into
the backend. However, I'm still concerned. I've had my /var partition
fill up very quickly in the past when something went wonky on the
backend. The problem is/was that a condition occurred that caused the
backend to spit out a seemingly infinite stream of errors that filled
several gigs before I caught it. When something happens that causes
that kind of log activity, an age-based log expiration algorithm is
insufficient. I'd really like to see a limit on the *number* of total
log entries allowed in the table. Also, it would probably help to have
a mechanism to filter out repetitive log entries, like so (warning:
crude hack follows, but you'll get the idea):
void MythContext::LogEntry(const QString &module, int priority,
const QString &message, const QString &details)
{
static QMap<QString,int> lastLogCounts;
static QMap<QString,QString> lastLogStrings;
if (gContext->GetNumSetting("LogEnabled", 0) == 1)
{
if (message == lastLogStrings[module])
{
lastLogCount[module] += 1;
}
else
{
if (0 < lastLogCounts[module])
{
LogEntry(module, priority, QString("Last message repeated %1
times").arg(lastLogCounts[module]);
}
lastLogCount[module] = 0;
lastLogStrings[module] = message;
}
d->dbLock.lock();
if (d->m_db->isOpen())
{
KickDatabase(d->m_db);
QString querystr = QString("INSERT INTO mythlog ( module,
priority, "
"logdate, host, message, details)
"
"values ( '%1', %2, now(), '%3',
"
"'%4','%5' );") . arg(module) .
arg(priority) .
arg(d->m_localhostname) .
arg(message) . arg(details);
QSqlQuery result = d->m_db->exec(querystr);
if (!result.isActive())
MythContext::DBError("LogEntry", querystr);
}
d->dbLock.unlock();
}
}
Anyway, that's my USD$0.02
-JAC
More information about the mythtv-dev
mailing list