[mythtv-users] mythtv dropping mysql???

Michael Watson michael at thewatsonfamily.id.au
Wed Oct 22 08:24:17 UTC 2014


On 22/10/2014 12:19 AM, Michael T. Dean wrote:
> On 10/21/2014 02:47 AM, Michael Watson wrote:
>> On 21/10/2014 12:55 PM, Michael T. Dean wrote:
>>> And it will be possible for you to access the "full raw data".  It's 
>>> still a MySQL database file.  We're just saying we're not going to 
>>> make people install the MySQL server to use it. 
>>
>> If its just going to be an MySQL database file sitting in a folder 
>> somewhere, then someone could simply point a MySQL Server to that 
>> file, and really fubar the database if they left the MythDB engine 
>> running at the same time, maybe file locks will prevent this, but 
>> where there is a will, there's a way....
>
> Right.  It's your data, so you're more than welcome to break it as 
> much as you like using whatever process you like.  We're not trying to 
> stop that, we're just trying to make it easier to set up and use and 
> maintain MythTV, make it less likely for accidental data breakage 
> (especially when some user is trying to fix a problem and sees an 
> ancient or incorrect-for-their-specific-situation post with some SQL 
> hack/workaround that seems like it would help), and make it so that 
> everyone--developers included (especially!)--have to use the database 
> properly when writing code for MythTV.
>
> It's actually unbelievable how many people are complaining about the 
> idea, especially as some of these people are the same people who have 
> complained in the past that, "It's just terrible design that every 
> single MythTV application (backends, frontends, jobqueues, MythWeb, 
> scripts, 3rd-party client, ...) needs direct database access.  Any 
> sane design keeps clients from directly accessing the database."  An 
> embedded database is just a good way of enforcing that sane 
> design--since embedded MySQL is not a networked/multi-client database 
> engine, only one process (the one in which the database is embedded) 
> can ever read from it.  We could do a good design with a 
> client/server, networked, multi-client database engine--but the only 
> way to enforce the design with that approach is policy.  (And if you 
> have much experience with tech/techies, you'll know how well policy 
> does at enforcing behaviors.)
>
>> Recently there has been a little bit of chatter about optimising 
>> MySQL for MythTV, and how much of a difficult task it is, I wonder 
>> how Myth (with embedded MySQL/MariaDB) will achieve this, or will 
>> there be more config variables to cater for this.  If more config 
>> variables are added, brings me back to wondering why the DB needs to 
>> be embedded.  Is it merely to prevent the odd user from destroying 
>> there database, or simply to prevent the odd user difficulty in 
>> setting up MythTV
>
> More than just set up--as you say below, distros like Mythbuntu or 
> LinHES have done an excellent job of making setup nearly foolproof.  
> However, even just maintaining the database is difficult enough for 
> non-DBA's (even those using distros like Mythbuntu/LinHES that do 
> their best to help maintain the DB). See, for example, the tip of the 
> iceberg at:
>
> https://code.mythtv.org/trac/ticket/12299
> https://code.mythtv.org/trac/ticket/10831
> https://code.mythtv.org/trac/ticket/10348
> https://code.mythtv.org/trac/ticket/10177
> https://code.mythtv.org/trac/ticket/9978
> https://code.mythtv.org/trac/ticket/9300
> https://code.mythtv.org/trac/ticket/8404
> https://code.mythtv.org/trac/ticket/8473
> https://code.mythtv.org/trac/ticket/8089
> https://code.mythtv.org/trac/ticket/6429
> https://code.mythtv.org/trac/ticket/5415
> https://code.mythtv.org/trac/ticket/3804
> (That being just a sampling of the ones that were actually reported as 
> bugs that doesn't even touch the significantly greater number of posts 
> on list with the same issue, let alone the questions in IRC, or the 
> people who didn't know what was going on and also never asked on a 
> MythTV list/forum/irc channel and eventually either figured it out 
> themselves or deleted their DB and started over or waited long enough 
> for the scheduled optimize_mythdb.pl their distro had set up to fix it 
> or ...)
Does MySQL need to be embedded for the MBE to regularly run 
optimize_mythdb.pl or a backup by itself?????
>
> There's also the issue of backing up and restoring the DB--something 
> that's very difficult to do properly if you don't know much about 
> databases.  As a matter of fact, there's a significant portion of 
> users (even SQL-knowledgeable (especially SQL-knowledgeable?) users) 
> who have corrupt schemas because of improperly backing up/restoring 
> their DBs.
>
> And you would be surprised just how much those who think they know 
> MySQL/SQL/MythTV's schema/... still don't know about it.  There are a 
> significant number of bugs that were put into MythTV by us developers 
> (who, arguably, understand more about MythTV's database usage than a 
> majority of users) that came down to a misunderstanding of some little 
> MySQL nuance or even a misinterpretation of the MythTV data.  For 
> example, take the whole store "UTF data in columns that MySQL 
> considers Latin-1 so we don't triple the size of the text data" as a 
> major example--a significant percentage of users had corrupt data 
> because they/their distros/scripts they used with the database 
> (including mysqldump)/approaches they used to restore data/... were 
> not configured to allow MythTV to use the columns like that--and no 
> one (MythTV devs or otherwise) realized the extent of the 
> configuration requirements until we tried to switch to UTF-8 columns.  
> Now, if there were only one configuration used by every MythTV 
> user--even if it was a wrong configuration--it would have been 
> straightforward to make that conversions because the approach would 
> have been the same for everyone (even if it had to fix data corruption 
> caused by previous misconfiguration).
>
> Now, you may be saying, "If someone can't understand an error message 
> that says, 'Table <whatever> is marked as crashed and should be 
> repaired,' they shouldn't be running MySQL in the first place."  If 
> so, we completely agree with you.  :)
>
>> - A Firstime Myth User, is far better to try installing MythBuntu (or 
>> one of the other dedicated distro's) than to try compiling/installing 
>> themselves, but then if a user can compile Myth by themselves, surely 
>> they can configure a DB server by themself.
>>
>> Is it that, it has come a time that MythTV needs a consistent DB 
>> engine, much like the need for a consistent ffmpeg version.  Are the 
>> differences in DB engines becoming to great across distros/platforms?
>
> That's not really a concern at this point.  We only support "MySQL" 
> (which includes--at least until there are significant differences--at 
> least some MariaDB allowance) and we don't need any modifications to 
> the MySQL code and only need to enforce a minimum DB version.
>
>> Lets say, I want to enable/disable User Job 1 on all existing 
>> recording rules, currently through the frontend or mythweb this is a 
>> painfully slow process with more than a handful of rules.  Does the 
>> framwork currently exist in Perl/Python Bindings or Services API to 
>> achieve this operation?    An example of how something like this can 
>> be scripted using the "proper" tools, could help me and others to use 
>> the proper tools rather than resorting to "lazy" SQL commands.
>
> While I don't have an exact example, there's 
> http://www.mythtv.org/wiki/0.26_Python_bindings/Data_Handlers#Record 
> and 
> https://github.com/MythTV/mythtv/blob/master/mythtv/programs/mythbackend/services/dvr.h#L148 
> .  The main reason there aren't many examples of how to use those 
> things is because for years people have been getting used to doing it 
> the wrong way (through direct database editing using SQL) so even 
> though those approaches have existed for years, people haven't taken 
> the time to learn a new/better approach.
>
> Imagine if, instead of writing some SQL to just stick values into 
> columns in the DB (where you have to figure out which columns and 
> either assume or look up the specific values to insert into those 
> columns), you had a generic script you could run to manipulate your 
> recording rules en masse--for example, it first lets you select a 
> group of rules (either one by one or using a "query"--some criteria 
> such as "user job 1 is enabled") and then apply a modification to 
> those selected rules (such as "enable user job 2").  Such a script 
> would be straightforward to write--the hardest part being the generic 
> UI.  Now imagine if that functionality were available directly from 
> the backend web server (the new "MythWeb" so to speak).  It seems that 
> bulk changes to recording rules is a common-enough requirement that we 
> /should/ have such a capability, but no one (dev or otherwise) has 
> been motivated enough to write a UI for it since there's a lazy way out.
I have written a couple of scripts that make use of the python bindings, 
some of which would have been far easier/quicker to do simple sql 
queries/updates.

For all your reasons for the change, I am still not seeing a large 
benefit.  Now, if it was said that "we need this to enable the creation 
of a frontend for xxxx platform, or simpler/better integration with 
plex/kodi (xbmc)" then I and many others (I am guessing) would not have 
uttered a word.

In the end, its Myth DVR (Developers Video Recorder) not a (Digital 
Video Recorder), and us users must put up with what the developers wish 
to do.  The current MythMusic plugin is a classic example of how this 
attitude can go wrong.  I used the old MythMusic plug in heavily, now I 
use Kodi for all my Music needs, and using Kodi (with cMyth Plugin) more 
and more for general frontend use.

>
> Mike
> _______________________________________________
> mythtv-users mailing list
> mythtv-users at mythtv.org
> http://www.mythtv.org/mailman/listinfo/mythtv-users
> http://wiki.mythtv.org/Mailing_List_etiquette
> MythTV Forums: https://forum.mythtv.org



More information about the mythtv-users mailing list