[mythtv-users] How to make a good bug report?

[John Goerzen]

On Thu, Jul 08, 2004 at 11:50:36PM -0400, Isaac Richards wrote:
> On Thursday 08 July 2004 11:18 pm, John Goerzen wrote:
> > How can you possibly justify such a statement?
> >
> > Our general policy at Debian is to ask for clarification in these
> > situations, and if the submitter refuses to clarify, the bug is fair
> > game to close.  That seems to work well.
> 
> And it's my policy to mark a bug that doesn't contain enough information as 
> invalid.  If the user wants to reopen it and provide useable information, 
> great.  If they don't, there's no more work on my part to close it later on.

Which makes some sense *IF YOU SPECIFY WHAT INFORMATION YOU NEED*.

You did not.  How are people supposed to know what information you need?
It's not in the submission guidelines.  I couldn't find it in a FAQ
anywhere.  I *did* look.

> And if someone get all pissy about it being closed, tough.  
> 
> >
> > What exactly is the problem with "watch live tv, press M, press Enter,
> > and the resulting screen demonstrates the flickering"?
> >
> > Just because a bug does not manifest itself on every system does not
> > mean it doesn't exist.
> 
> If it only manifests itself on _your_ system, you need to provide information 
> on how to reproduce it on everyone's system.

Let me clue you in on a little principle of bug hunting:

  Some bugs may not manifest themselves on every system, and may not
  even be reproducible on every system.

For instance, there was a bug in the Linux kernel that caused panics on
my Alpha.  The same code worked fine on many other people's Alphas and
on PCs.  It turned out that there *was* a bug in the kernel that only
showed up under very particular circumstances, relating to the specific
hardware setup I had, and was literally not possible to reproduce on
other systems.

Other examples of this principle could relate to deadlock situations,
where bugs may not even be reliable reproducible on any system, or may
manifest themselves only on lightly-loaded systems, SMP systems, etc.

With MythTV, I can easily imagine bugs that manifest themselves only on
machines with certain types of video capture cards, motherboards,
networking, etc.  There are a huge number of dependencies in the system.

The more correct statement would be:

  If it only manifests itself on your system, you need to help figure
  out why.

And that is something I have explicitly offered to do.  I could send you
strace output, set breakpoints and relevant places and send you info
frmo gdb, send you ltrace output, send you a core file, etc, etc.  But
you have not said what it is that would help you, other than "make it
happen everywhere" which may well not even be possible.

> > The effect is blindingly obvious.
> 
> It doesn't allow me to reproduce the bug, so it's not a valid step in
> reproducing the bug.

It allows *me* to reproduce the bug, and since I'm the one noticing it,
it seems obvious.  In any case, at the time I submitted it, I had no
idea that nobody else was seeing the problem.

> > > What I _don't_ want is random people who don't know what they're
> > > doing from
> >
> > I described *exactly* how the bug occured on an otherwise working
> > system.  I also described environments where it occured, and gave
> > the precise actions that lead to it.
> 
> No, you didn't describe the environment _at all_ the first time.

Sure I did.  I said it was MythTV 0.15.1.  Your submission form and
instructions asked for nothing else.  You could have had a box for OS, X
version, etc, etc.  I had no idea what would help you out on this,
especially since it occured on two different machines.

As I write that, it occurs to me that both machines with the problem use
AMD CPUs (one a AthlonXP 2800+, the other an Athlon64 running in 32-bit
mode), and the one without the problem is a Pentium M.

Now, if the problem is indeed relating to AMD vs. Intel CPUs, it could
well be that you would *never* be able to reproduce the problem unless
you have the same CPU I do.

So it really makes no sense to insist that bugs be universally
reproducible in order to be legitimate bugs.

> > I hope you are not saying "MythTV developers are too 31337 to accept
> > bug reports from mere users."
> 
> No, I'm saying that I just set up the bugzilla last week, and I want
> to get all the bugs worked out of the process before opening it up to
> random users.

That makes sense.  Might I suggest adding a note to main screen then?
Since the URL has already been posted to mythtv-users, and I've posted
it to my blog [1] along with a long list of links relating to MythTV,
Google will likely find it soon if it hasn't yet.  I can remove the link
from my site but it's in the archives already.

-- John

[1] http://changelog.complete.org/