[mythtv-commits] Ticket #9188: UPnP browsing "By Title" fails for shows with non-Latin1 characters [+PATCH]

[MythTV]

#9188: UPnP browsing "By Title" fails for shows with non-Latin1 characters
[+PATCH]
------------------------------+---------------------------------------------
 Reporter:  foceni@…          |           Owner:  beirdo    
     Type:  patch             |          Status:  assigned  
 Priority:  minor             |       Milestone:  unknown   
Component:  MythTV - UPnP     |         Version:  0.23-fixes
 Severity:  medium            |      Resolution:            
 Keywords:  upnp, utf8        |   Ticket locked:  0         
------------------------------+---------------------------------------------

Comment (by David Kubicek <foceni@…>):

 '''UPDATE:''' all changes from '''.toLatin1()''' to '''.toUtf8()''' in the
 patch are essential. Without them, the most frequent UPnP/DLNA browsing -
 "By Title" - doesn't work for any locale with characters outside of Latin1
 (ISO-8859-1), that's all other 15 ISO/8859 code pages and even Unicode
 (despite QT4 being internally Unicode).

 The problem is in how QStrings are converted into byte streams for
 '''QUrl::fromPercentEncoding()'''. This method '''requires''' %-encoded
 '''UTF8 stream'''. It decodes %-encoding and then decodes the resulting
 UTF8 byte stream into Unicode. When passed regular (non-%-encoded) UTF8,
 like the payload is, it just decodes it as UTF8.

 All mentioned QStrings are pieces of the UPnP payload, which is correctly
 imported from client requests in '''HTTPRequest::ParseRequest()''' using
 '''QString::fromUtf8()'''. They contain actual Unicode strings.

 The original coder expected all values to be '''only''' %-encoded and thus
 '''.toLatin1()''' conversion for '''QUrl::fromPercentEncoding()''' would
 work, because %-encoded string is pure ASCII. Actually, though, UPnP
 payload is hardly (if ever) %-encoded and '''.toLatin1()''' applied to a
 Unicode string strips all non-Latin1 characters and the result, of course,
 isn't UTF8 byte array either. That's what's wrong.

 Because '''QUrl::fromPercentEncoding()''' requires UTF8 byte array, we
 must use '''.toUtf8()''' and not '''.toLatin1()''' or any other local
 8-bit code page. Otherwise, we feed it byte encoding not in accordance
 with the docs '''and''' every non-Latin1 character from the payload is
 stripped (in QT4 replaced by a "?"). It is these values returned by
 '''QUrl::fromPercentEncoding()''' that are ultimately used by Myth (for DB
 queries, etc).

 Because this bug is apparent, without possible side effect and breaks UPnP
 for many locales including UTF8, I'd like to ask to commit the fix even to
 the *-fixes branches...

-- 
Ticket URL: <http://svn.mythtv.org/trac/ticket/9188#comment:3>
MythTV <http://www.mythtv.org/>
MythTV Media Center