[mythtv-commits] Ticket #5070: Database schema upgrade error from 1216 to 1217.

MythTV mythtv at cvs.mythtv.org
Wed Jan 14 21:45:52 UTC 2009


#5070: Database schema upgrade error from 1216 to 1217.
--------------------------------+-------------------------------------------
 Reporter:  nlabadie at gmail.com  |        Owner:  janne   
     Type:  defect              |       Status:  assigned
 Priority:  major               |    Milestone:  0.22    
Component:  mythtv              |      Version:  head    
 Severity:  medium              |   Resolution:          
  Mlocked:  0                   |  
--------------------------------+-------------------------------------------

Comment(by sphery <mtdean at thirdcontact.com>):

 The attached patch, mythtv-5070-detect_db_charset_corruption.patch ,
 detects database corruption before performing the character set conversion
 and stops the database upgrade at the 0.21-fixes DB schema.

 The users experiencing duplicate keys had misconfigured databases that
 caused the DB connection to use the utf8 character set rather than the
 latin1 character set that MythTV 0.21-fixes and below required, as
 described at
 http://www.mythtv.org/wiki/index.php/Fixing_Corrupt_Database_Encoding.
 Since the connection character set differed from the column character set,
 MySQL performed a conversion of all the UTF8 data MythTV sent to the DB
 into latin1 before storing it in the DB.  Therefore, the data in the
 users' databases was latin1, not utf8, so when the DB upgrade attempted to
 interpret the data as UTF8, it found cases where the data was improperly
 encoded and truncated the data at the first invalid character (i.e. 8-bit
 latin1 characters), causing duplicate entries.

 The patch only allows the DB upgrade to proceed past 0.21-fixes DB schema
 version if the data in people.name and oldprogram.oldtitle (2 columns
 which caused the most problems) can successfully be converted to UTF8
 without any truncations (which are shown by MySQL as warnings). In the
 event that corruption is detected, the DB upgrade failes and users are
 directed to the wiki page
 http://www.mythtv.org/wiki/index.php/Fixing_Corrupt_Database_Encoding ,
 which provides a means of fixing the corruption.  The patch does not,
 however, fix the corruption.

 I wrote the patch to loop over the tables on which to perform trial
 conversions, rather than on the list of SQL commands to execute because
 the approach allows for easily adding additional test conversions.

 Some users above mentioned that skipping the 1215-1216 upgrade (conversion
 to varbinary) caused the upgrade to succeed.  Note that this will only
 succeed for users whose data contain only latin1 characters.  For users
 with misconfigured databases whose data contained non-latin1 characters,
 data was corrupted upon MySQL's conversion from utf8 to latin1 (i.e. the
 latin1 representation is wrong), so skipping the varbinary step will cause
 issues.

-- 
Ticket URL: <http://svn.mythtv.org/trac/ticket/5070#comment:20>
MythTV <http://www.mythtv.org/>
MythTV


More information about the mythtv-commits mailing list