[mythtv-users] High cpu usage with mythfilldatabase on 0.27 night fixes

Thu Nov 6 07:56:02 UTC 2014

On 2014-10-22 13:56, jason at jadrallypix.me.uk wrote:
> mythfilldatabase currently takes around 10 mins to complete and Perl
> uses high cpu on one of the cores - anywhere from 80-100% whilst it's
> in progress.
> 
> Looking at the log, it appears to sit at "2014-10-22 14:13:30.106195 I
> [4661/4661] CoreContext filldata.cpp:378 (GrabData) - XMLTV config
> file is: /home/mythtv/.mythtv/Freesat.xmltv" for 10 mins before moving
> on to complete the task
> 
> 
> 2014-10-22 14:13:20.074534 C [4661/4661] thread_unknown
> mythcommandlineparser.cpp:2595 (ConfigureLogging) - mythfilldatabase
> version: fixes/0.27 [v0.27.4-4-gb305eb5] www.mythtv.org
> 2014-10-22 14:13:20.074558 C [4661/4661] thread_unknown
> mythcommandlineparser.cpp:2597 (ConfigureLogging) - Qt version:
> compile: 4.8.1, runtime: 4.8.1
> 2014-10-22 14:13:20.074565 N [4661/4661] thread_unknown
> mythcommandlineparser.cpp:2599 (ConfigureLogging) - Enabled verbose
> msgs:  general
> 2014-10-22 14:13:20.074656 N [4661/4661] thread_unknown
> logging.cpp:907 (logStart) - Setting Log Level to LOG_INFO
> 2014-10-22 14:13:20.085487 I [4661/4664] Logger logging.cpp:308 (run)
> - Added logging to the console
> 2014-10-22 14:13:20.086385 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Interrupt handler
> 2014-10-22 14:13:20.086399 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Terminated handler
> 2014-10-22 14:13:20.086410 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Segmentation fault
> handler
> 2014-10-22 14:13:20.086420 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Aborted handler
> 2014-10-22 14:13:20.086431 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Bus error handler
> 2014-10-22 14:13:20.086441 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Floating point
> exception handler
> 2014-10-22 14:13:20.086452 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Illegal instruction
> handler
> 2014-10-22 14:13:20.086467 I [4661/4661] thread_unknown
> signalhandling.cpp:194 (SetHandlerPrivate) - Setup Real-time signal 0
> handler
> 2014-10-22 14:13:20.086515 N [4661/4661] thread_unknown
> mythdirs.cpp:55 (InitializeMythDirs) - Using runtime prefix = /usr
> 2014-10-22 14:13:20.086532 N [4661/4661] thread_unknown
> mythdirs.cpp:68 (InitializeMythDirs) - Using configuration directory =
> /home/mythtv/.mythtv
> 2014-10-22 14:13:20.086619 I [4661/4661] CoreContext
> mythcorecontext.cpp:257 (Init) - Assumed character encoding:
> en_GB.UTF-8
> 2014-10-22 14:13:20.086998 N [4661/4661] CoreContext
> mythcontext.cpp:504 (LoadDatabaseSettings) - Empty LocalHostName.
> 2014-10-22 14:13:20.087008 I [4661/4661] CoreContext
> mythcontext.cpp:512 (LoadDatabaseSettings) - Using localhost value of
> jos-desktop
> 2014-10-22 14:13:20.087046 I [4661/4661] CoreContext
> mythcontext.cpp:693 (TestDBconnection) - Testing network connectivity
> to '192.168.1.93'
> 2014-10-22 14:13:20.087619 I [4661/4665] SystemManager
> mythsystemunix.cpp:275 (run) - Starting process manager
> 2014-10-22 14:13:20.087713 I [4661/4667] SystemIOHandlerR
> mythsystemunix.cpp:91 (run) - Starting IO manager (read)
> 2014-10-22 14:13:20.087678 I [4661/4666] SystemSignalManager
> mythsystemunix.cpp:510 (run) - Starting process signal handler
> 2014-10-22 14:13:20.089501 I [4661/4668] SystemIOHandlerW
> mythsystemunix.cpp:91 (run) - Starting IO manager (write)
> 2014-10-22 14:13:20.190990 I [4661/4663] LogForward
> loggingserver.cpp:1373 (forwardMessage) - New Client:  (#1)
> 2014-10-22 14:13:20.191185 I [4661/4663] LogForward
> loggingserver.cpp:142 (FileLogger) - Added logging to
> /home/mythtv/.mythtv/mythfilldatabase.20141022131320.4661.log
> 2014-10-22 14:13:20.271987 N [4661/4661] CoreContext
> mythcorecontext.cpp:1634 (InitLocale) - Setting QT default locale to
> en_GB
> 2014-10-22 14:13:20.272075 I [4661/4661] CoreContext
> mythcorecontext.cpp:1667 (SaveLocaleDefaults) - Current locale en_GB
> 2014-10-22 14:13:20.272147 N [4661/4661] CoreContext
> mythlocale.cpp:121 (LoadDefaultsFromXML) - Reading locale defaults
> from /usr/share/mythtv//locales/en_gb.xml
> 2014-10-22 14:13:20.277922 I [4661/4661] CoreContext
> mythtranslation.cpp:65 (load) - Loading en_gb translation for module
> mythfrontend
> 2014-10-22 14:13:20.280218 I [4661/4661] CoreContext
> schemawizard.cpp:118 (Compare) - Current MythTV Schema Version
> (DBSchemaVer): 1317
> 2014-10-22 14:13:20.282608 I [4661/4661] CoreContext
> mythcorecontext.cpp:426 (ConnectCommandSocket) - MythCoreContext:
> Connecting to backend server: 192.168.1.93:6543 (try 1 of 1)
> 2014-10-22 14:13:20.285343 I [4661/4661] CoreContext
> mythcorecontext.cpp:1547 (CheckProtoVersion) - Using protocol version
> 77
> 2014-10-22 14:13:20.287541 I [4661/4661] CoreContext main.cpp:327
> (main) - Opening blocking connection to master backend
> 2014-10-22 14:13:26.393120 I [4661/4661] CoreContext filldata.cpp:569
> (Run) - Updating source #1 (Freesat) with grabber tv_grab_uk_rt
> 2014-10-22 14:13:26.394745 I [4661/4661] CoreContext filldata.cpp:583
> (Run) - Found 53 channels for source 1 which use grabber
> 2014-10-22 14:13:29.504763 I [4661/4661] CoreContext filldata.cpp:641
> (Run) - Grabber has capabilities: baseline manualconfig cache
> preferredmethod tkconfig apiconfig lineups
> 2014-10-22 14:13:30.105481 I [4661/4661] CoreContext filldata.cpp:664
> (Run) - Grabber prefers method: allatonce
> 2014-10-22 14:13:30.106195 I [4661/4661] CoreContext filldata.cpp:378
> (GrabData) - XMLTV config file is: /home/mythtv/.mythtv/Freesat.xmltv
> 2014-10-22 14:23:31.715345 I [4661/4661] CoreContext
> programdata.cpp:1113 (HandlePrograms) - Updated programs: 1094
> Unchanged programs: 13516
> 2014-10-22 14:23:32.023827 N [4661/4661] CoreContext main.cpp:450
> (main) - Data fetching complete.
> 2014-10-22 14:23:32.023857 I [4661/4661] CoreContext main.cpp:458
> (main) - Adjusting program database end times.
> 2014-10-22 14:23:32.024429 I [4661/4661] CoreContext main.cpp:464
> (main) -     0 replacements made
> 2014-10-22 14:23:32.024434 I [4661/4661] CoreContext main.cpp:466
> (main) - Marking generic episodes.
> 2014-10-22 14:23:35.290811 I [4661/4661] CoreContext main.cpp:478
> (main) -     Found 50
> 2014-10-22 14:23:35.290837 I [4661/4661] CoreContext main.cpp:481
> (main) - Extending non-unique programids with multiple parts.
> 2014-10-22 14:23:35.405734 I [4661/4661] CoreContext main.cpp:532
> (main) -     Found 0
> 2014-10-22 14:23:35.405746 I [4661/4661] CoreContext main.cpp:534
> (main) - Fixing missing original airdates.
> 2014-10-22 14:23:35.467366 I [4661/4661] CoreContext main.cpp:549
> (main) -     Found 0 with programids
> 2014-10-22 14:23:35.513715 I [4661/4661] CoreContext main.cpp:569
> (main) -     Found 0 without programids
> 2014-10-22 14:23:35.513727 I [4661/4661] CoreContext main.cpp:573
> (main) - Marking repeats.
> 2014-10-22 14:23:35.635796 I [4661/4661] CoreContext main.cpp:587
> (main) -     Found 0
> 2014-10-22 14:23:35.635810 I [4661/4661] CoreContext main.cpp:589
> (main) - Unmarking new episode rebroadcast repeats.
> 2014-10-22 14:23:35.755502 I [4661/4661] CoreContext main.cpp:599
> (main) -     Found 0
> 2014-10-22 14:23:36.490099 I [4661/4661] CoreContext main.cpp:608
> (main) - Marking episode first showings.
> 2014-10-22 14:23:41.233235 I [4661/4661] CoreContext main.cpp:638
> (main) -     Found 12562
> 2014-10-22 14:23:41.233248 I [4661/4661] CoreContext main.cpp:640
> (main) - Marking episode last showings.
> 2014-10-22 14:23:50.237623 I [4661/4661] CoreContext main.cpp:670
> (main) -     Found 12240
> 2014-10-22 14:23:50.242293 I [4661/4661] CoreContext main.cpp:697 
> (main) -
> ===============================================================
> | Attempting to contact the master backend for rescheduling.  |
> | If the master is not running, rescheduling will happen when |
> | the master backend is restarted.                            |
> ===============================================================
> 2014-10-22 14:23:50.243590 N [4661/4661] CoreContext main.cpp:707
> (main) - mythfilldatabase run complete.
> 2014-10-22 14:23:50.243974 I [4661/4661] CoreContext
> mythcontext.cpp:1194 (~MythContext) - Waiting for threads to exit.
> 
> I cannot see anything obvious within the log to suggest what is
> causing the issue. Hopefully somebody more experienced can shed some
> light on it for me
> 
> Thanks.

Bumping it to see if anyone can help.