[mythtv-commits] Ticket #5946: Fix (unimportant) data corruption in database character set conversion
MythTV
mythtv at cvs.mythtv.org
Sat Nov 29 18:01:39 UTC 2008
#5946: Fix (unimportant) data corruption in database character set conversion
----------------------------------------------+-----------------------------
Reporter: sphery <mtdean at thirdcontact.com> | Owner: ijr
Type: patch | Status: new
Priority: minor | Milestone: unknown
Component: mythtv | Version: head
Severity: medium | Mlocked: 0
----------------------------------------------+-----------------------------
The conversion from CHAR(latin1)->BINARY->CHAR(utf8) performed for some
fields in database upgrades 1216 and 1217 results in null-padded data in
the CHAR field. MySQL expects CHAR field data to be space-padded, so it
never strips the null characters from the data when returning query
results or when manipulating data. The null characters then break
comparisons (meaning we get new "uncorrupted" values placed in the DB in
addition to the corrupted values) and can be displayed, for example, in
MythWeb.
The attached patch changes the conversion of all multi-character CHAR
fields to use CHAR(latin1)->VARBINARY->CHAR(utf8) to prevent the null-
padding corruption from occurring. Specifically, the affected fields are
people.name, programgenres.genre, programrating.{system,rating},
recordedrating.{system.rating} .
There are still columns of type CHAR(1) for which I didn't change the
intermediate BINARY data type, and it's possible that in some databases
values for fields which are not defined as "NOT NULL" may be converted to
a single null character when they are meant to be blank (which would be
represented internally as a single space). The additional CHAR(1) fields
include cardinput.sharable,
dtv_multiplex.{inversion,polarity,bandwidth,transmission_mode}, and
programgenres.relevance ; however, in my database, none include empty
values, so none are corrupted.
Though this will change the database upgrade history, it will simply
prevent corruption of data for users who have not yet upgraded their
databases. I will be submitting a patch that fixes the corrupt data for
those who have already converted to trunk.
Users should not attempt to fix the data as it's really unimportant data
and most likely there exist uncorrupted data for any current listings data
and the upcoming patch will fix it properly. Fixing it properly is not
trivial because of key uniqueness constraints which will be violated due
to the existence of duplicates whose only difference is the null-padding
corruption. The new, uncorrupted values will have been inserted by
mythfilldatabase/EIT, so are likely to exist if MythTV
(mythbackend/mythfilldatabase) has ever been run since the database
conversion.
--
Ticket URL: <http://svn.mythtv.org/trac/ticket/5946>
MythTV <http://www.mythtv.org/>
MythTV
More information about the mythtv-commits
mailing list