[mythtv-users] Populating the .xmltv file for my grabber from mythtv database?

Stephen Worthington stephen_agent at jsw.gen.nz
Wed May 5 04:01:38 UTC 2021


On Tue, 4 May 2021 13:32:55 -0400, you wrote:

>On Tue, May 4, 2021 at 12:58 PM David Watkins <watkinshome at gmail.com> wrote:
>
>>
>>
>> On Sun, 2 May 2021 at 14:55, Larry Kennedy <lunchtimelarry at gmail.com>
>> wrote:
>>
>>> I am just now attempting the upgrade to version 31, and the cutover from
>>> DD to XMLTV.
>>>
>>> I successfully configured the sqlite grabber to pull from SD, and noticed
>>> that it downloaded all of the channels on my SD lineup, even though most
>>> are not selected.  I guess this is intended behavior?
>>>
>>> Seeing how tedious it is to mark channels as "selected" in the Sqlite
>>> database, I came up with a slightly different way to transfer the channels
>>> I currently care about.
>>>
>>> I run this query in my existing mythconverg database:
>>>
>>> SELECT CONCAT('update channels set selected = 1 where channum = ',
>>> channum ,";") AS combined FROM channel where visible=1
>>>
>>> Pipe this output to a file named selected.sql which creates the sqlite
>>> update statements to set the correct channels to "selected" in the new
>>> grabber database to match what you now have as visible channels in mythtv.
>>>
>>> Then, just use the grabber tool to make all the channels as "not
>>> selected" by default, and then pipe the file from above into the sqlite
>>> query as seen on the xmltv wiki:
>>>
>>> sqlite3 $HOME/.xmltv/SchedulesDirect.DB < selected.sql
>>>
>>> This seems simpler than some of the other methods I've seen on the wiki
>>> and elsewhere...
>>>
>>> Moving on....I need some advice before I go past the rubicon on this v31
>>> + xmltv upgrade:
>>>
>>> After I do the 31 upgrade, and since I only use an HDHR, I plan to just
>>> blow away my channels and repopulate.  I think all the channel ids will
>>> line up the same as before.
>>>
>>> Currently, in v30, I run mythfilldatabase as a cron job.  Should I
>>> continue to do that, or switch to another method?  I've never let mythtv
>>> natively run the mythfilldatabase process.  I'm having a hard time
>>> following the intent of the setup-video-sources wiki page on this topic.
>>> Advice appreciated!
>>>
>>> Thanks,
>>> Larry
>>>
>>>
>> I have a combined BE/FE which shuts down when not being used and wakes up
>> for recordings.  I run this script from crondaily and it seems to work out
>> OK.
>>
>> #!/usr/bin/bash
>> /usr/local/bin/mythshutdown --lock
>>
>> tv_grab_zz_sdjson --days 10 --config-file ~/.xmltv/tv_grab_zz_sdjson.conf
>> --output ~/sd_listing.xml 2>/dev/null
>>
>> /usr/local/bin/mythfilldatabase --only-update-guide --max-days 10 --file
>> --sourceid 2 --xmlfile ~/sd_listing.xml 2>/dev/null
>>
>> /usr/local/bin/mythshutdown --unlock
>>
>> It's not an ideal solution because I have a low powered ION motherboard
>> with only 4GB RAM and  this more or less guarantees that MFD will run while
>> a recording is in progress when I'd prefer to ensure it ran when the box
>> wasn't recording.  Also, if  I ever go more than 10 days without a
>> scheduled recording the whole thing will come to a stop because it will run
>> out of guide data and never wake up to load any more.  In practice neither
>> of those things causes a problem.
>>
>>
>> Although SD returns 18 days of guide data I've found that many channels
>> just have 'boilerplate'  programme information after 10 days or so, so I
>> limit both SD and MFD to 10.
>>
>>
>I attempted an upgrade over the weekend, but ended up rolling it back.
>Kudos to the myth devs that created the backup and restore scripts!
>
>I'm still on v30 with Schedules Direct DataDirect and a cron job to run
>mythfilldatabase.   To get to v31 and the new xmltv grabber, I decided to
>do the XMLTV upgrade now, followed by an upgrade to v31, say, a week
>later.  I chose this path since it appears that v31 would force the same
>upgrade of XMLTV, and I wanted to decouple these two things.  If I
>approached this incorrectly, please let me know.
>
>I ran the xmltv install, then followed the wiki to configure the sqlite
>grabber.  This all appeared to work, as I was able to see like 1,000 rows
>of channel data  in the sqlite database.  As indicated in this thread, I
>created a simple way to set the right channels as "selected".
>
>After that, I went into mythtv-setup, channel editor, and deleted all my
>channels.  I only have one source that's by HDHR.
>
>Then, I added a new video source that matches the name of the grabber
>config, set this up as Multinational, and done.
>
>Then I went into the other setup screen where I map the video source to my
>HDHR tuners.  Forget the name of that one.  Capture cards?
>
>After this, I ran mythfilldatabase.  Looking at mythweb, I could see it
>populated the channels back into the database, but I am pretty sure it put
>in more than the ones I marked as "selected" in the Sqlite database
>Strange.  The listing started to appear, but VERY slowly.
>
>Mytfilldatabase ran for like 4+ hours, and since I was getting close to the
>time I needed to finish or bail out, I killed it.  I then tried running the
>script that does 3 days at a time, and this ran to full completion right
>away, but never fully populated my listings.  I still had major gaps in the
>listings.  Some were there and some never showed up.
>
>Any advice before I give this another go this coming weekend?  Is my
>strategy flawed?  Did I miss a step?  How best to run mythfilldatabase so
>that it doesn't take all day?  Is it the 3 days at a time script?
>
>Larry

As I understand it (not being an SD user), the first time you run
mythfilldatabase after installing the new setup, it does take hours to
run.  Subsequent updates need to be limited to only a few days to make
the time reasonable.  This has been discussed here several times, and
also on the forum, so a search here:

https://lists.archive.carbon60.com/mythtv/users/

and

here:

https://forum.mythtv.org/

should help.

Unfortunately, it seems that Google searches these days are not
finding the mailing list references much, unless you have a very
specific keyword.

Looking on as a disinterested party, I have never understood why the
SD json EPG requires such long times to run.  The actual process of
putting the EPG data from my XMLTV generated sources into the database
takes less than a minute.  I have two sources, 141 channels and one
week of EPG data.


More information about the mythtv-users mailing list