[mythtv-users] slow mythfilldatabase

David Newman dnewman at networktest.com
Sat Jan 9 01:25:42 UTC 2021



On 1/6/21 7:49 AM, Bill Meek wrote:
> On 1/6/21 2:50 AM, Stephen Worthington wrote:
>> On Tue, 5 Jan 2021 22:03:16 -0800, you wrote:
>>
>>> MythTV Version : v31.0-v31.0 on Ubuntu 20 LTS, mysql 8.0.22
>>>
>>> Greetings. After upgrading from Ubuntu 18 LTS to 20 LTS,
>>> mythfilldatabase takes 45-60 minutes to run, sometimes more. It
>>> previously took less than three minutes, and the system didn't have this
>>> problem when upgrading from Ubuntu 16 LTS to 18 LTS.
>>
>> Where do you get your EPG data from?  Previous reports of similar
>> problems turned out to be caused by getting a huge data set from
>> Schedules Direct - all of the EPG rather than just the new data.  As I
>> understand it (not being a Schedules Direct user), you are supposed to
>> only get the full EPG data on the first download, and then after that
>> limit how much you get to only the new data or maybe only one week of
>> data.  If you have recently changed to Schedules Direct or to the new
>> way of downloading the Schedules Direct data, that might be your
>> problem.
> 
> @dn, please mention that you completed the XMLTV conversion and that exactly one
> copy of the data exists. 

Thanks, Bill. I did no conversion of any kind. There's one
mythfilldatabase.sql database. It's the same one that updated quickly
when the system previously ran Ubuntu 18 LTS.

If I missed some steps in upgrading to Ubuntu 20 LTS, please let me know.

(tl;dr: It seems to work better now; details below.)


Here's the SQLite example, which should be similar to
> the other method:
> 
>   https://forum.mythtv.org/viewtopic.php?f=36&t=3480&start=15#p19508

That forum post seems to be mainly about SQLite. The grabber on this
system is:

tv_grab_zz_sdjson

> 
> With an up-to-date* MythTV version, you can get more info by running
> mythfilldatabase -v xmltv. 

 mythfilldatabase version: fixes/31 [v31.0-v31.0] www.mythtv.org


Do that for the backend by using: mythbackend --setverbose xmltv
> and waiting for the backend to run MFDB or just running: mythfilldatabase -v xmltv
> from the command line *as user mythtv*.
> 
> *I don't remember when the commit went in that improves the MFDB/XMLTV logging,
> but if you do this,  you'll get it:
> 
>   sudo add-apt-repository ppa:mythbuntu/31
>   sudo apt update
>   sudo apt dist-upgrade

This upgraded 13 packages:

  libmyth libmyth-python libmythtv-perl mythtv mythtv-backend mythtv-common
  mythtv-database mythtv-dbg mythtv-doc mythtv-frontend
mythtv-transcode-utils
  mythweb php-mythtv

> 
> Also, verify that this exists (should have been installed automatically):
> 
> $ cat /etc/mysql/conf.d/mythconverg.cnf
> # Disable Binary logging for the mythconverg database. Binary logging
> # for all databases was off by default prior to MySQL V8.
> [mysqld]
> binlog-ignore-db=mythconverg

yup


> 
> If it doesn't, add it and restart the DB. I'd stop the backend first.
> 
> FYI, I don't use any of the innodb_... or log_... changes in my .cnf file.

Yeah, I don't know that any of this made a difference. That .cnf file is
all stuff that were suggested optimizations from the MythTV wiki. I had
the same issue before using this version of the .cnf file.

But...it looks like these changes made a difference!

I ran mythfilldatabase manually after these steps, and it was *much*
faster. I didn't prepend a time command, but it was something like two
minutes or less. Let's see how it goes when mythfilldatabase runs
programmatically.

Thanks again for your help!

dn



> 


More information about the mythtv-users mailing list