I agree that a condescending tone is unnecessary in this forum. It also appears that some participants attempt to obfuscate the situation rather than offer clear, direct advice. All I (we) need is a little guidance from those who understand how this
thing is designed (and therefore understand its flaws). <br><br>Assuming the BUSQ is executed by the BE and not the FE, I don't understand why a FE action (i.e., my attempt to navigate through the recordings menu following a delete) would hang
if a BE action (i.e., rescheduler) is triggered to perform a task (i.e.,
BUSQ). Aren't these things running independently and asynchronously? Unless they are
somehow locking each other inside the database, I wonder why they are affecting each other. I come from a Sybase/Oracle point of view on this, but I suppose it is possible that the BUSQ could be holding share locks that prevent another database transaction from acquiring exclusive locks necessary for an update/delete. But I am not sure what this could be given the use-case in question here. Is it possible that something else is blocking the FE action apart from the BUSQ? Could the BUSQ be a red herring?
<br><br>FYI, I enabled slow deletes and so far I am not convinced it has helped any. This was a useful suggestion that perhaps I dismissed prematurely. <br><br>Several people have suggested we convert the recordedseek
table to innoDB, but that table does not participate in the BUSQ. Further, this table has never shown up in my slow-sql log, nor have I seen any evidence that the BUSQ (or any other query) is blocked by another database activity using the recordedseek
table. The BUSQ consistently runs in 7 seconds, even if I run it ad-hoc when nothing else is going on. I remain puzzled as to how the recordedseek
table could be a factor. Additionally, I know that making changes to the schema *can* break future upgrades, so I am reluctant to do this unless it is absolutely necessary. Is this innoDB change one that could break the next upgrade? I suppose if it is, I could just convert it back and then do the upgrade....
<br><br>Larry<br><br><div><span class="gmail_quote">On 11/30/07, <b class="gmail_sendername"><a href="mailto:f-myth-users@media.mit.edu">f-myth-users@media.mit.edu</a></b> <<a href="mailto:f-myth-users@media.mit.edu">f-myth-users@media.mit.edu
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Date: Thu, 29 Nov 2007 21:38:39 -0500<br> From: "Michael T. Dean" <
<a href="mailto:mtdean@thirdcontact.com">mtdean@thirdcontact.com</a>><br><br> > On 11/29/2007 07:55 PM, David Rees wrote:<br> > > On Nov 29, 2007 4:41 PM, Michael T. Dean <<a href="mailto:mtdean@thirdcontact.com">
mtdean@thirdcontact.com</a>> wrote:<br> > ><br> > >>> It is very frustrating to<br> > >>> sit and wait up to 10 seconds after every delete (from either the frontend<br> > >>> or mythweb).
<br> > >>><br> > >>> I am open to suggestions as to how I can improve this situation.<br> > >>><br> > >> Do you have your MySQL database on the same drive as your recording
<br> > >> disk? If so, that's the first thing you should consider fixing.<br> > >><br> > > It is completely unreasonable to expect someone to have a multiple<br> > > disk system to get decent usability from MythTV.
<br><br> > IMHO, it is completely unreasonable to expect Myth to be able to make<br> > your kernel/filesystem driver/hardware be able to do something that your<br> > kernel/filesystem driver/hardware is unable to do.
<br><br>It is also completely unreasonable for you to ignore and dismiss all<br>the evidence people are handing you that RESCHEDULES ARE SLOW even<br>if one has ARLEADY DONE all of the recommended screwing around.<br><br>
(Ignore the red herring of "deletions" for the moment, just to<br>simplify the discussion---the only relevant point there is that<br>deletions force a reschedule.)<br><br> > > I would venture to<br> > > guess that the number of single disk systems far outnumber the number
<br> > > of multi-disk systems<br><br> > You do realize that Myth uses a /lot/ of storage space, right?<br><br>You do realize that an increasing number of people are starting to<br>call you on your condescending attitude, right?
<br><br>(You're not always that way, and you're often helpful. But you're<br>also quite often a determined Pollyanna and/or apologist for the<br>status quo, insisting that -all- user complaints -must- be because the
<br>-user- is somehow confused and that Myth itself must be perfect. Just<br>because many users -are- confused is no reason to assume (or insist)<br>that they -all- are. I've gritted my teeth, grumbled to myself, and
<br>kept quiet when you've done this, but recent traffic in other threads<br>shows that others are starting to come out and say it.)<br><br> > Do you
<br> > really think someone buys a new 750GB hard drive, then throws away the<br> > 300GB hard drive she was using for Myth? Or buys a 300GB hard drive and<br> > throws away the old 80GB hard drive? You do realize that most (all?)
<br> > motherboards come with more than one disk connector, right?<br><br>Every additional disk---even if the disk itself is "free"---adds heat,<br>noise, and power cost; reduces reliability and the ability to add any
<br>-more- disks (due to using up both space and bus connectors); and<br>complicates the machine's configuration.<br><br>Many people are unwilling to make those sorts of sacrifices for what<br>looks, at the heart of it, to be poor implementation; many others
<br>-cannot- make that sacrifice if their backend is limited in, e.g.,<br>physical size.<br><br>And besides, as my evidence has shown, putting the DB on a separate<br>spindle doesn't help significantly. It is NOT, repeat NOT, any sort
<br>of contention for where the disk head is hanging out, unless you count<br>the contention that MySQL is imposing on ITSELF by executing its queries.<br><br> > Also, you do realize that MySQL is a /network-capable/ database
<br> > management system, right? You don't even have to put MySQL on either<br> > backend. It could be on the dedicated frontend. It could be on another<br> > computer somewhere in the house. ...<br>
<br>...adding even -more- heat, power, space, and unreliability, unless<br>the machine was already in use---in which case it just adds unreliability<br>and makes it more complicated to take that machine---or any intervening
<br>piece of network hardware--- down for any reason (lest you break Myth's<br>ability to do -anything- while it's down).<br><br> > Fine. I really couldn't care less what he does first. But my point is
<br> > that he should be trying to fix the MySQL configuration,<br><br>And how -exactly- do you recommend that he do that, besides putting it<br>on a second spindle that he may not have? And how would you recommend
<br>I fix -my- configuration, given all the info I've given you in previous<br>threads about all the things I've tried with marginal or zero benefit?<br>Replies of the form "don't keep so much history", "have fewer channels",
<br>"you can't use even one any-channel rule", and so forth are not going<br>to get a good reception, since all of those are things Myth is supposed<br>to handle well.<br><br>There are three fundamental problems here. Solving any of them would
<br>solve the whole mess:<br>(a) The BUSQ has very poor performance for some people.<br>(b) Large portions of Myth hang completely waiting for the BUSQ.<br>(c) Common UI tasks, such as deletion or scheduling, run the BUSQ.
<br><br>I frankly think that (b) might be the most tractable, because it<br>-might- be easier to separate out the BUSQ into another thread and/or<br>refactor the schema to avoid weird crosslocking of tables, than it<br>might be to change either (a) or (c). But that's just a guess.
<br><br>[If it were up to me, I'd have solved (a) by implementing a<br>truth-maintenance system to do scheduling, rather than by building<br>unwieldy DB queries, but since the chances of -that- making it into<br>Myth are zero I'm not going to waste any time on it. The current DB
<br>approach is something of a travesty because it has to redo -all- of<br>its work on -every- reschedule, as if it's coming into the world<br>brand-new with no prior knowledge; a TMS has the great advantage that<br>reasonable changes cause small propagations and are very fast; they
<br>even have the advantage that explaining -why- something was or wasn't<br>scheduled in some way can be derived by traversing the structure and<br>generating the explanation, which is something that can only be<br>crudely approximated with the current DB-based approach.]
<br><br>And of course there's also:<br>(d) The BUSQ seems to increase some sort of window of vulnerability<br> to timing races that can crash the backend if it's accessed by<br> something else<br>...which isn't a performance problem so much as evidence that there's
<br>something screwy going on in a mutex somewhere.<br><br> > not blaming the<br> > BUSQ/arguing with me that the scheduler query is a problem when his<br>
<br>But others, like myself, are arguing that the BUSQ is a big problem<br>all by itself, and EVEN IF everything else about deletion was instantaneous,<br>we'd still be seeing UI lockups and generally bad performance, because we
<br>-also- see them in contexts where no deletion is happening, e.g,. when<br>scheduling new shows via the UI, or when the BUSQ is running autonomously<br>because some recording has just finished on its own.<br><br> > system executes the query in the same amount of time as mine and my
<br> > system does /not/ have the "entire system locks up when I try to delete<br> > a recording" issue his has. Also, it doesn't really help that he's<br> > refusing to try slow deletes because he has a super-fast XFS filesystem
<br> > and he's smart enough to know that a slow delete (that will cause the<br> > removal of recordedseek/recordedmarkup entries to be delayed about 2min<br> > 10sec per GiB of recording) will have no effect on performance of his
<br> > deletes.<br><br>...or maybe he just can't believe that deleting a few thousand items<br>from a DB can take tens of seconds. I can't. If you're saying that<br>he has to defer that to some random time by using code designed to
<br>work around filesystems that have their own problems, just to cause<br>the DB update to happen at some indeterminate time in the future, I<br>can see why he might view this dubiously as a kluge (and, for that<br>matter, as something that could bite him when he least expects it,
<br>depending on when that actual DB update finally -does- happen).<br><br> > In other words, my system seems--to me--to be proof that a system that<br> > takes 6-7 seconds to execute the scheduling run does not lock up when a
<br> > recording is deleted. Perhaps I'm just naive, though, in thinking my<br> > system acts like a properly configured Myth system. Maybe I've<br> > misconfigured my system and gotten unreasonably good performance because
<br> > of it.<br><br>Various devs commented that they, too, were seeing reschedule times<br>upwards of 30 seconds; there was a long thread with stats about that<br>back when the DB-vs-recording-glitches thing was playing out. I guess
<br>the devs' systems are misconfigured, too. Maybe you should explain to<br>them the error of their ways, and perhaps it will enlighten us unwashed<br>masses, too.<br>_______________________________________________<br>
mythtv-users mailing list<br><a href="mailto:mythtv-users@mythtv.org">mythtv-users@mythtv.org</a><br><a href="http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users">http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users
</a><br></blockquote></div><br>