[mythtv-users] find_orphans.py broken on 32 bit x86?

Stephen Worthington stephen_agent at jsw.gen.nz
Sun Nov 4 08:26:50 UTC 2018


On Sun, 04 Nov 2018 16:13:08 +1300, you wrote:

>On Sat, 3 Nov 2018 15:39:15 -0400, you wrote:
>
>>On 10/31/18, Tom Dexter <digitalaudiorock at gmail.com> wrote:
>>> On 10/31/18, Jonatan Lindblad <lindbladjonatan at gmail.com> wrote:
>>>>
>>>> I'm pretty sure it's the bookmarkupdate value that's the problem,
>>>> https://forum.mythtv.org/viewtopic.php?f=36&t=1716&start=15.
>>>>
>>>> Jonatan
>>>
>>> Bingo...that was it. Something's clearly trying to use that 0000-00-00
>>> 00:00:00. When I manually updated that to the CURRENT_TIMESTAMP
>>> find_orphans.py worked. As noted in that forum topic, it's not a
>>> solution...but still good to know.
>>>
>>> Thanks!
>>> Tom
>>>
>>
>>Interesting. I've been able to narrow this failure down. The
>>getProgram() method of the Recorded class will always fail in this
>>same manner for any recorded entry that still has the default
>>0000-00-00 00:00:00 value for bookmarkupdate. Here's a simple script
>>where I can reproduce this using any such recording (by chanid and
>>starttime):
>>
>>#!/usr/bin/env python2
>>from MythTV import MythDB, MythBE, Recorded, MythError
>>import datetime, pytz
>>DB = MythDB()
>>kwargs = {'chanid': 1111, 'starttime': datetime.datetime(2018, 11, 3,
>>1, 0, 0, 0, pytz.utc)}
>>recs = list(DB.searchRecorded(**kwargs))
>>
>>for rec in recs:
>>    print rec.title
>>    p = rec.getProgram()
>>
>>If the specified recording has ever been bookmarked that works fine.
>>However for anything that hasn't I bet this:
>>
>>python test.py
>>Crazy Ex-Girlfriend
>>Traceback (most recent call last):
>>  File "test.py", line 11, in <module>
>>    p = rec.getProgram()
>>  File "/usr/lib/python2.7/site-packages/MythTV/dataheap.py", line
>>387, in getProgram
>>    return Program.fromRecorded(self)
>>  File "/usr/lib/python2.7/site-packages/MythTV/mythproto.py", line
>>946, in fromRecorded
>>    return be.getRecording(rec.chanid, rec.starttime)
>>  File "/usr/lib/python2.7/site-packages/MythTV/mythproto.py", line
>>647, in getRecording
>>    return Program(res[1:], db=self.db)
>>  File "/usr/lib/python2.7/site-packages/MythTV/mythproto.py", line
>>875, in __init__
>>    DictData.__init__(self, raw)
>>  File "/usr/lib/python2.7/site-packages/MythTV/altdict.py", line 148,
>>in __init__
>>    data = self._process(data)
>>  File "/usr/lib/python2.7/site-packages/MythTV/altdict.py", line 164,
>>in _process
>>    data[i] = self._trans[self._field_type[i]](v)
>>  File "/usr/lib/python2.7/site-packages/MythTV/altdict.py", line 104,
>>in <lambda>
>>    lambda x: datetime.fromtimestamp(x, datetime.UTCTZ())\
>>  File "/usr/lib/python2.7/site-packages/MythTV/utility/dt.py", line
>>304, in fromtimestamp
>>    obj = super(datetime, cls).fromtimestamp(float(timestamp), tz)
>>ValueError: timestamp out of range for platform time_t
>>
>>I'm not yet totally clear, but I think there are conversions happening
>>there automatically based on the database column data type, and that
>>one fails.
>>
>>Tom
>
>I am running Ubuntu 18.04, and I tried your test program.  I had to
>install the python-tz package first to get it to run.  I selected a
>suitable recording using:
>
>select * from recorded where bookmarkupdate='0000-00-00 00:00:00'
>limit 1\G
>
>But it ran without errors.
>
>My recollection is that when I was writing my new version of
>mythimport, I had problems with 'all zeroes' timestamps, and I changed
>one or two settings in MariaDB to fix that.  I am guessing that one of
>those changes fixes this problem.  Or it may be simply that I am using
>MariaDB and it has different defaults.  The thing I think that may
>control this is the SQL_MODE='NO_ZERO_DATES' option.
>
>This command should give you a list of your default SQL_MODE settings:
>
>SELECT REPLACE(@@SQL_MODE, ',', '\n');
>
>If you have any of the STRICT_ options on, that could cause this
>problem.

And I have found what I did in mythimport:

local_config = _ConfigXml(LOCAL_CONFDIR)
db = MythTV.MythDB(
    DBHostName = local_config.hostname,
    DBUsername = local_config.username,
    DBPassword = local_config.password,
    DBName =     local_config.database,
    DBPort =     local_config.port
    )
dbc = db.cursor()

# Change the SQL mode settings for the session to allow zeroes in
# timestamps, as required by MythTV.
#rows = dbc.execute("SELECT @@SESSION.sql_mode;")
#if rows != 1:
#    raise MythDBError
#sql_mode = dbc.fetchone()[0]
#logger.debug("original sql_mode = " + str(sql_mode))
#sql_mode = sql_mode.replace('NO_ZERO_IN_DATE,', '')
#sql_mode = sql_mode.replace('NO_ZERO_DATE,', '')
#sql_mode = sql_mode.replace('NO_ZERO_IN_DATE', '')
#sql_mode = sql_mode.replace('NO_ZERO_DATE', '')
#sql_mode = 'STRICT_ALL_TABLES,' + sql_mode
rows = dbc.execute('SET sql_notes=0;')
logger.debug('SET sql_notes=0; rows=' + str(rows))
sql_mode =
'STRICT_ALL_TABLES,ONLY_FULL_GROUP_BY,NO_ENGINE_SUBSTITUTION'
logger.debug("sql_mode setting = " + str(sql_mode))
dbc.execute("SET SESSION sql_mode = '" + sql_mode + "';")
rows = dbc.execute("SELECT @@SESSION.sql_mode;")
if rows != 1:
    raise MythDBError
sql_mode = dbc.fetchone()[0]
logger.debug("new sql_mode = " + str(sql_mode))
dbc.execute('SET sql_notes=1;')

And this is the debug output produced by the above code:

2018-11-04 00:26:33 DEBUG SET sql_notes=0; rows=0
2018-11-04 00:26:33 DEBUG sql_mode setting =
STRICT_ALL_TABLES,ONLY_FULL_GROUP_BY,NO_ENGINE_SUBSTITUTION
2018-11-04 00:26:33 DEBUG new sql_mode =
ONLY_FULL_GROUP_BY,STRICT_ALL_TABLES,NO_ENGINE_SUBSTITUTION


More information about the mythtv-users mailing list