This web page is designed to provide an honest look at what makes a good bug report. If you are plagued by a bug in Helix and you want to see it fixed as soon as possible (and who doesn’t?) you owe it to yourself to read this page.

Much of what is found here is stated in similar fashion in an article by Brett Simmons of Ranchero Software, the creator of NetNewsWire and MarsEdit. But Brett leaves a few things unsaid that I want to address here.

So you read that and think: why does any of that matter? Isn’t it enough that I reported it? Sadly, no. Resources are never limitless — an adage that is doubly true here at QSA ToolWorks — and there are always more than enough bugs to fix. Of course, critical flaws are given top priority, but beyond that, we have to prioritize. One thing that raises the priority level is a demonstration of the bug that makes it so obvious that our time is immediately spent fixing it, instead of trying to understand it.

Of course, any bug we see we try to fix. (If we can’t fix it quickly, we document it as a Known Problem; a list of these is found in techdb.) So, getting us to see the bug is your top priority. How do you do that?

When it comes to bug reports, this is pretty much how it works: when a bug report is read, the first reaction is to parse it for ‘coherence’ — is it clear and concise? Write as though you have 30 seconds or less to explain the problem. (We have short attention spans.)

Using our requested channels is important. Describing the bug as directly as possible is also highly desirable. Writing in a non-adversarial tone helps. Brevity (where possible) is critical. Accessibility to the information is also important. Simple demonstrable examples are a huge bonus. And lastly, avoiding speculation is big plus too. Do all that and you’ve got a pretty good chance of getting an engineer to stop and look for a fix.

Providing a concise reproducible sample demonstrating the problem is crucial. When a bug is encountered, try reproducing it until you’ve got a 1-2-3 step process to trigger the bug. Then try to reproduce the bug in a new collection. If that fails, make a copy of your existing collection and remove everything that is not required — data and structure both — to see the bug. (The new tools in Helix 6.2 (and later) make it easy to remove data and unrelated structure for your collection.)

There are often many ways to accomplish a task in Helix and, if you are reliably seeing a bug, there is almost certainly something in the specific steps you take that causes it to surface. Explicit step by step directions are essential.

Let’s consider the desired characteristics of a bug report one by one:

1. Channels. Our recommended procedure is that you start by logging in to techdb — our online Helix database — and searching the Bug Report list to see if others have already reported this bug. You can participate in the discussion and may even find a workaround. If you don’t find your bug there, you can enter a new bug report, but we request that you first contact technical support and discuss the issue. We may have already identified the bug, or we may determine that your issue is something else altogether — maybe a confusing spec in Helix.
2. Coherence. Can we make sense of what is written? Not using accurate terminology makes it harder to follow what you are trying to say. Because Helix is a visual language, many people have invented their own words for the elements within a collection. But that is your language, not ours. If you take the time to ‘translate’ into the terms we understand, you make it much easier for us to follow your description of the bug. Likewise, avoid references to code names. (As in: “I’m running Pele on a Blackbird under Puma.”)
   Another serious issue is pronouns. Avoid them. Today I read a non-Helix bug report that said, "When I transfer a file between my Mac and my PC, it crashes." OK, what crashes? The Mac? The PC? The desk both computers are sitting on? It may be obvious to you which it you are referring to, but keep in mind that we have no prior experience in trying to understand your problem.
3. Accessibility: Getting a report as an attachment in an email is no fun; techdb (see above) is our primary channel for attachments. Emailing an attachment puts it on one person’s desk. Sending it via techdb puts it in the hands of everybody here at QSA. If an engineer is assigned to “work on report R7023” and everything he needs is attached as support documents for R7023 in techdb, that’s a formula for quick action.
4. Tone: This one is difficult, but when sending a bug report be mindful not to ruffle feathers unnecessarily. When you report a bug, you are essentially saying to an engineer ‘you screwed up’ so you should try to soften the blow as much as possible. Phrases like ‘This is unacceptable’ and ‘This is costing me money’ often carry the connotation that we’re just being negligent or callous. Right or wrong, a personal reaction ensues on our end which has the net effect of making us less inclined to devote immediate attention to your report. The old adage ‘You can catch more flies with honey than you can with vinegar’ is still around because it is true.
5. Directness: Give us the important facts right up front. Don’t wait until the 8th paragraph to say ‘This bug only happens after running the collection for hours.’ Having used no such qualifiers up to that point, you’ve led us to believe you have found a straightforward logic bug. Bugs that appear only after extended use are in a wholly different category and require much more one-on-one work with you to help us identify the source. Leading us through a complex series of steps and then saying, in effect, you probably won’t see the bug anyway is frustrating.
6. Brevity: Short and to the point is best. Your personal Helix history is usually irrelevant. The exception to this rule is when you can point to a specific version where the bug first appeared. Using a “1-2-3” outline of the steps to reproduce is always better than a narrative rendition.
7. Speculation: Unless you have demonstrable skills in programming in C/C++, don’t offer suggestions as to where in the code the problem lies. (But if you do have programming skills, please contact us! We might give you an opportunity to work in the code and fix these bugs yourself!) History has proven that speculation of this sort is usually far wide of the mark. At best it distracts from the engineer’s ability to focus on the true source of the problem.
8. Demonstrability: If you’ve got a well-documented sample that shows the bug, your bug immediately moves up the priority ladder two or three rungs. This does not mean you can just send us your existing collection with a ‘the bug is in here’ (somewhere) note. At the very least it means adding text labels (or comments, in the case of icons) that include everything that was also said outside of the collection. It may be logical to assume that the engineer can always refer to the other source material but even if they can, it isn’t the most efficient way to work. Bug swatting is a demanding task that requires significant focus. Every possible distraction needs to be eliminated. By putting the documentation right in the collection — right on the template itself even — you keep the focus directly on your bug. If you send a sample collection, the tighter the example, the better. By removing irrelevant rectangles from the template it becomes easier to see the bug, and to avoid chasing red herrings. Keeping the engineer focused on the code always makes for faster bug fixing.
9. Reducibility: Although Helix is capable of producing incredibly complex structures, It is amazing how many bugs can be reduced to a very simple core. A sample collection should be as small and uncluttered as possible. Does the bug require data? If not, deleting all of the data (there’s a handy script for that on our AppleScripts page) provides the greatest amount of size reduction. Does this bug happen in Design Mode in RADE? If so, reducing it is greatly simplified. If a particular view is triggering a crash or wrong behavior, remove the rectangles from the template that are not involved. (This is typically a trial and error process.) Remove all of the user icons; if the bug involves a Username tile, create a new user icon using the same name, but without the (unnecessary) views and sequences added to menus. Now you can delete most of the unrelated views, sequences, and many of the supporting icons that have nothing to do with the bug itself. Deleting those will enable to you delete other icons, and you can repeat the cycle until what is left is only the structure needed to see the bug. Complete the process by using Helix Utility to make a compressed copy.

It’s not that we don’t want to fix bugs. It’s just that chasing down bugs is hard work. (If it were not so, they would be fixed already!) We choose which tasks we work on by weighing the amount of time required vs. the severity of the bug. All in all, one fact remains: the easier you make it for us to see your bug, the faster we will be able to fix it.

Don’t let any of that scare you away from contacting us with your bug reports. If you want to just send an email that says ‘Hey, I think I found a problem…,’ that is fine. We take all reports seriously, but we can only do so much, and some bugs simply have to go on the back burner. The purpose of this page is to give you an insight on how to get your bug on the fast track to resolution.

To the Bug Report page.