[mythtv-commits] Ticket #5070: Database schema upgrade error from 1216 to 1217.
MythTV
mythtv at cvs.mythtv.org
Wed Jan 14 21:45:52 UTC 2009
#5070: Database schema upgrade error from 1216 to 1217.
--------------------------------+-------------------------------------------
Reporter: nlabadie at gmail.com | Owner: janne
Type: defect | Status: assigned
Priority: major | Milestone: 0.22
Component: mythtv | Version: head
Severity: medium | Resolution:
Mlocked: 0 |
--------------------------------+-------------------------------------------
Comment(by sphery <mtdean at thirdcontact.com>):
The attached patch, mythtv-5070-detect_db_charset_corruption.patch ,
detects database corruption before performing the character set conversion
and stops the database upgrade at the 0.21-fixes DB schema.
The users experiencing duplicate keys had misconfigured databases that
caused the DB connection to use the utf8 character set rather than the
latin1 character set that MythTV 0.21-fixes and below required, as
described at
http://www.mythtv.org/wiki/index.php/Fixing_Corrupt_Database_Encoding.
Since the connection character set differed from the column character set,
MySQL performed a conversion of all the UTF8 data MythTV sent to the DB
into latin1 before storing it in the DB. Therefore, the data in the
users' databases was latin1, not utf8, so when the DB upgrade attempted to
interpret the data as UTF8, it found cases where the data was improperly
encoded and truncated the data at the first invalid character (i.e. 8-bit
latin1 characters), causing duplicate entries.
The patch only allows the DB upgrade to proceed past 0.21-fixes DB schema
version if the data in people.name and oldprogram.oldtitle (2 columns
which caused the most problems) can successfully be converted to UTF8
without any truncations (which are shown by MySQL as warnings). In the
event that corruption is detected, the DB upgrade failes and users are
directed to the wiki page
http://www.mythtv.org/wiki/index.php/Fixing_Corrupt_Database_Encoding ,
which provides a means of fixing the corruption. The patch does not,
however, fix the corruption.
I wrote the patch to loop over the tables on which to perform trial
conversions, rather than on the list of SQL commands to execute because
the approach allows for easily adding additional test conversions.
Some users above mentioned that skipping the 1215-1216 upgrade (conversion
to varbinary) caused the upgrade to succeed. Note that this will only
succeed for users whose data contain only latin1 characters. For users
with misconfigured databases whose data contained non-latin1 characters,
data was corrupted upon MySQL's conversion from utf8 to latin1 (i.e. the
latin1 representation is wrong), so skipping the varbinary step will cause
issues.
--
Ticket URL: <http://svn.mythtv.org/trac/ticket/5070#comment:20>
MythTV <http://www.mythtv.org/>
MythTV
More information about the mythtv-commits
mailing list