[mythtv-users] slow mythfilldatabase
David Newman
dnewman at networktest.com
Sat Jan 9 01:25:42 UTC 2021
On 1/6/21 7:49 AM, Bill Meek wrote:
> On 1/6/21 2:50 AM, Stephen Worthington wrote:
>> On Tue, 5 Jan 2021 22:03:16 -0800, you wrote:
>>
>>> MythTV Version : v31.0-v31.0 on Ubuntu 20 LTS, mysql 8.0.22
>>>
>>> Greetings. After upgrading from Ubuntu 18 LTS to 20 LTS,
>>> mythfilldatabase takes 45-60 minutes to run, sometimes more. It
>>> previously took less than three minutes, and the system didn't have this
>>> problem when upgrading from Ubuntu 16 LTS to 18 LTS.
>>
>> Where do you get your EPG data from? Previous reports of similar
>> problems turned out to be caused by getting a huge data set from
>> Schedules Direct - all of the EPG rather than just the new data. As I
>> understand it (not being a Schedules Direct user), you are supposed to
>> only get the full EPG data on the first download, and then after that
>> limit how much you get to only the new data or maybe only one week of
>> data. If you have recently changed to Schedules Direct or to the new
>> way of downloading the Schedules Direct data, that might be your
>> problem.
>
> @dn, please mention that you completed the XMLTV conversion and that exactly one
> copy of the data exists.
Thanks, Bill. I did no conversion of any kind. There's one
mythfilldatabase.sql database. It's the same one that updated quickly
when the system previously ran Ubuntu 18 LTS.
If I missed some steps in upgrading to Ubuntu 20 LTS, please let me know.
(tl;dr: It seems to work better now; details below.)
Here's the SQLite example, which should be similar to
> the other method:
>
> https://forum.mythtv.org/viewtopic.php?f=36&t=3480&start=15#p19508
That forum post seems to be mainly about SQLite. The grabber on this
system is:
tv_grab_zz_sdjson
>
> With an up-to-date* MythTV version, you can get more info by running
> mythfilldatabase -v xmltv.
mythfilldatabase version: fixes/31 [v31.0-v31.0] www.mythtv.org
Do that for the backend by using: mythbackend --setverbose xmltv
> and waiting for the backend to run MFDB or just running: mythfilldatabase -v xmltv
> from the command line *as user mythtv*.
>
> *I don't remember when the commit went in that improves the MFDB/XMLTV logging,
> but if you do this, you'll get it:
>
> sudo add-apt-repository ppa:mythbuntu/31
> sudo apt update
> sudo apt dist-upgrade
This upgraded 13 packages:
libmyth libmyth-python libmythtv-perl mythtv mythtv-backend mythtv-common
mythtv-database mythtv-dbg mythtv-doc mythtv-frontend
mythtv-transcode-utils
mythweb php-mythtv
>
> Also, verify that this exists (should have been installed automatically):
>
> $ cat /etc/mysql/conf.d/mythconverg.cnf
> # Disable Binary logging for the mythconverg database. Binary logging
> # for all databases was off by default prior to MySQL V8.
> [mysqld]
> binlog-ignore-db=mythconverg
yup
>
> If it doesn't, add it and restart the DB. I'd stop the backend first.
>
> FYI, I don't use any of the innodb_... or log_... changes in my .cnf file.
Yeah, I don't know that any of this made a difference. That .cnf file is
all stuff that were suggested optimizations from the MythTV wiki. I had
the same issue before using this version of the .cnf file.
But...it looks like these changes made a difference!
I ran mythfilldatabase manually after these steps, and it was *much*
faster. I didn't prepend a time command, but it was something like two
minutes or less. Let's see how it goes when mythfilldatabase runs
programmatically.
Thanks again for your help!
dn
>
More information about the mythtv-users
mailing list