[mythtv] Python: collecting searchRecorded into list() has side effects
Jan
mythtv at jd67.de
Thu Jul 25 15:56:26 UTC 2013
Am 25.07.2013 15:07, schrieb Raymond Wagner:
> I can't take a closer look currently, but initial guess is that your database connections weren't being properly reused, and you maxed out on allowed connections.
>
>
>
> On Jul 25, 2013, at 7:22, Jan <mythtv at jd67.de> wrote:
>
>> Hi,
>>
>> I try to get a list of Recorded via searchRecorded and print it in given order.
>>
>> As far as I understand searchRecored don't support order by query (ORDER BY).
>>
>> So I collect all Recorded into a list and sort the list ...
>>
>> *** pseudo code **************************************************************
>>
>> # get the
>> rs = list( db.searchRecoreded(recgroup='Default')
>>
>> # do some sorting
>> rs.sort(key=operator.itemgetter('starttime'), reverse=reverse)
>>
>> for r in rs:
>> print r.title
>>
>> ******************************************************************************
>> at this point everything work as expected.
>>
>> But if I try to get to get the cutpoints from each recorded, if will fail after
>> round about 40 recorded
>>
>> *** more pseudo **************************************************************
>> for i,r in enumerate(rs):
>> # get cutlist also
>> print i, r.title, r.markup.getuncutlist()
>>
>> ---
>> 0 Cold Creek Manor []
>> 1 Good Will Hunting []
>> 2 Hairspray []
>> ... snip ...
>> 39 Bad Boys []
>> 40 Bad Boys II---------------------------------------------------------------------------
>> MythDBError Traceback (most recent call last)
>> <ipython-input-12-301408f356a5> in <module>()
>> 1 for i,r in enumerate(rs):
>> ----> 2 print i, r.title, r.markup.getuncutlist()
>> 3
>>
>> /usr/lib/python2.7/dist-packages/MythTV/dataheap.pyc in getuncutlist(self)
>> 274 return self._buildlist(self.MARK_CUT_START, self.MARK_CUT_END)
>> 275 def getuncutlist(self):
>> --> 276 return self._buildlist(self.MARK_CUT_END, self.MARK_CUT_START)
>> 277
>> 278 class _Rating( DBDataRef ):
>>
>> /usr/lib/python2.7/dist-packages/MythTV/utility/other.pyc in _buildlist(self, ms, me)
>> 401 start = []
>> 402 stop = []
>> --> 403 for mark in sorted(self, key=lambda m: m.mark):
>> 404 if mark.type == ms:
>> 405 if len(start) == len(stop):
>>
>> /usr/lib/python2.7/dist-packages/MythTV/database.pyc in __iter__(self)
>> 483
>> 484 def __iter__(self):
>> --> 485 self._populate()
>> 486 return list.__iter__(self)
>> 487
>>
>> /usr/lib/python2.7/dist-packages/MythTV/database.pyc in _populate(self, force, data)
>> 554 list.append(self, self.SubData(zip(self._datfields, row)))
>> 555 self._populated = True
>> --> 556 self._origdata = self.deepcopy()
>> 557
>> 558 @classmethod
>>
>> /usr/lib/python2.7/dist-packages/MythTV/database.pyc in deepcopy(self)
>> 576 def deepcopy(self):
>> 577 if not self._populated: return []
>> --> 578 return self.fromCopy([dat.copy() for dat in self])
>> 579
>> 580 def revert(self):
>>
>> /usr/lib/python2.7/dist-packages/MythTV/database.pyc in fromCopy(cls, data)
>> 566 @classmethod
>> 567 def fromCopy(cls, data):
>> --> 568 c = cls('', bypass=True)
>> 569 c._populated = True
>> 570 for dat in data: list.append(c, dat)
>>
>> /usr/lib/python2.7/dist-packages/MythTV/database.pyc in __init__(self, where, db, bypass)
>> 526 def __init__(self, where, db=None, bypass=False):
>> 527 list.__init__(self)
>> --> 528 self._db = DBCache(db)
>> 529 self._setClassDefs(self._db)
>> 530 if bypass: return
>>
>> /usr/lib/python2.7/dist-packages/MythTV/database.pyc in __init__(self, db, args, **dbconn)
>> 1261 break
>> 1262 else:
>> -> 1263 raise MythDBError(MythError.DB_CREDENTIALS)
>> 1264
>> 1265 # write configuration back into file, ignored if not old format
>>
>> MythDBError: Could not find database login credentials
>>
>> ******************************************************************************
>>
>> If I iterate over searchRecorded (without sorting) everything works be fine:
>>
>> *** more pseudo **************************************************************
>>
>> for r in DB.searchRecorded(recgroup='Default'):
>> print r.title, r.markup.getuncutlist()
>>
>> ---
>>
>> Cold Creek Manor []
>> ... snip ~ 350 other recorded
>> Tetsuo - The Bullet Man []
>> ******************************************************************************
>>
>> It looks like, if I store the Recorded into a list, a connection will hold
>> for each Recorded and after ~40 Recorded it will run out of connections.
>>
>> QUESTIONS:
>> Is it in general, a bad idea to store many (>40) Recorded objects into a list()?
>>
>> Is there any way to iterate a lot of Recorded in a sorted manner?
>>
>> Help is welcome
>> Jan
>>
>>
>>
I did some debugging and it looks like that the new connections made and
STORED into DBDataRef._origdata by by deepcopy.
Just for testing I cleared the _origdata and it works perfect.
*** pseudo code
**************************************************************
for i,r in enumerate( list(DB.searchRecorded(recgroup='Default')) ):
print i, r.title, r.markup.getuncutlist()
r.markup._origdata = None # WARNING: THIS IS NOT A FIX OR WORKAROUND
****
More information about the mythtv-dev
mailing list