[mythtv] [Patch] Logging to database PLUS automatic mythfilldatabase

Joseph A. Caputo jcaputo1 at comcast.net
Thu Feb 26 16:13:58 EST 2004


On Wednesday 25 February 2004 20:27, Matt White wrote:
> OK - here's the latest version of my db logging patch.  The changes
> are:
>
> * uses the same numbering as unix syslog() priorities (make sure you
>    update to the newest mythlog - see below for URL).
>
> * on Isaac's suggestion, I've incorporated the 'mythflushlog.pl'
> script functionality into mythbackend.  Go into Settings->General,
> and on the third page, you will be able to enable/disable database
> logging, and also enable/disable the log cleaning functionality and
> set the frequency of the cleaning.
>
> * While I was doing that, I figured I'd add a generic "Housekeeping"
>    thread.  Currently, the housekeeping thread has two tasks -
> cleaning the log table, and running mythfilldatabase.  Configuration
> is done in Settings->General.  For running mythfilldatabase,
> configuration includes:
>      * enable/disable automatic execution
>      * frequency of execution (# days)
>      * limits on when to run it (ie. only between certain hours)
>      * path and arguments for mythfilldatabase
>      * path for logfile for mythfilldatabase output
>
> I figured as long as I was doing a housekeeping thread,
> mythfilldatabase was a big one.  It's got some advantages - easier to
> configure, no worries about QTDIR being set (it has to be set to run
> mythbackend, so it'll be inherited).
>
> I've also added a table to store the timestamps of the last run of
> each of the housekeeping items.  Both this table and the logging
> table have been added to dbcheck.cpp.
>
> Everything I've added is turned off by default.  Other defaults are:
>
>    Log Cleaning frequency: 14 days
>    Mythfilldatabase frequency: 1 day
>    Mythfilldatabase hour restriction: between 2 AM and 5 AM
>    Mythfilldatabase path: /usr/local/bin/mythfilldatabase
>    Mythfilldatabase args: none
>    Mythfilldatabase logfile: /var/log/mythfilldatabase.log
>
> As mentioned above, the mythlog module has been updated as well to
> make the priorities correct.  It can be downloaded from:
>
> http://borris.usask.ca/mythtv/mythlog-0.2.tar.gz
>
> Attached are two patches.  The big one is what I've described above.
> The second is a patch to the mainmenu.xml and main_settings.xml
> to add the MythLog module to the menus.


Nice work... I'm glad to see you've incorporated the log flushing into 
the backend.  However, I'm still concerned.  I've had my /var partition 
fill up very quickly in the past when something went wonky on the 
backend.  The problem is/was that a condition occurred that caused the 
backend to spit out a seemingly infinite stream of errors that filled 
several gigs before I caught it.  When something happens that causes 
that kind of log activity, an age-based log expiration algorithm is 
insufficient.  I'd really like to see a limit on the *number* of total 
log entries allowed in the table.  Also, it would probably help to have 
a mechanism to filter out repetitive log entries, like so (warning: 
crude hack follows, but you'll get the idea):


void MythContext::LogEntry(const QString &module, int priority,
                     const QString &message, const QString &details)
{
    static QMap<QString,int> lastLogCounts;
    static QMap<QString,QString> lastLogStrings;

    if (gContext->GetNumSetting("LogEnabled", 0) == 1)
    {
	if (message == lastLogStrings[module])
	{
		lastLogCount[module] += 1;
	}
	else
	{
		if (0 < lastLogCounts[module])
		{
			LogEntry(module, priority, QString("Last message repeated %1 
times").arg(lastLogCounts[module]);
		}

		lastLogCount[module] = 0;
		lastLogStrings[module] = message;
	}

        d->dbLock.lock();
    
        if (d->m_db->isOpen())
        {
            KickDatabase(d->m_db);
    
            QString querystr = QString("INSERT INTO mythlog ( module, 
priority, "
                                       "logdate, host, message, details) 
"
                                       "values ( '%1', %2, now(), '%3', 
"
                                       "'%4','%5' );") . arg(module) .
                                       arg(priority) . 
arg(d->m_localhostname) .
                                       arg(message) . arg(details);
    
            QSqlQuery result = d->m_db->exec(querystr);
            if (!result.isActive())
                MythContext::DBError("LogEntry", querystr);
        }
    
        d->dbLock.unlock();
    }
}


Anyway, that's my USD$0.02

-JAC



More information about the mythtv-dev mailing list