|How To Create Incredible Bug Reports|
Essential to helping us fix bugs is the ability to not just notice bugs, but to identify them in a way that makes it as easy as possible for us to see it. The adage here is “we can’t fix what we can’t see.”
This page contains guidelines on how to create a useful sample collection that document the bug you have found. If you follow these guidelines, you’ll make it easier for us to see your bug and therefore to fix it quickly. For us, the first rule is: bugs that are easy to reproduce get top priority. (Well, bugs that impact data integrity get higher priority, outside of beta testing, those are rare.)
Keep in mind that there are some changes when switching from Classic to macOS. These are not necessarily bugs. We tried to maintain the familiar Helix functions as much as possible, but some changes are forced upon us by macOS. If something is different, check the Release Notes to make sure it isn’t a Specification Change or a Known Problem.
|Crafting Superb Reports||
The very best reports should:
When writing instructions, be sure to use accurate terminology. Helix is a visual language and we have learned that many people have invented their own words to describe the objects in a Helix collection, and that is fine for your personal use, but telling us something like ‘the rejector is wrong’ is infinitely less helpful than saying ‘validations are failing.’
If you’ve found a bug, your first response should not be to send us your 200MB collection that contains 50 relations and 500 views with a note that says, in effect, “there’s a bug in here somewhere.” When you see a bug, your first thought should be “Can I reproduce this in a small collection built from scratch?” If you can demonstrate the bug in a collection created from scratch, you’ve got a bug we can fix in hours, not days.
If you can reproduce the bug in small sample collection that shows the bug and contains instructions right in the collection itself, you have guaranteed that you will get our immediate attention.
But sometimes a new collection built from scratch doesn’t show the bug, or building such a sample is impractical. We still need a sample that shows us the bug as simply as possible, so the next step is to reduce your collection until it is as simple as possible and the bug is neatly isolated.
To some, the prospect of ‘disassembling’ a complex collection is a daunting task, but we hove found that even the most complex collections can be reduced in just a couple of hours, particularly by leveraging the power of AppleScript.
If you are about to take on the task of reducing a collection to show us a bug, these guidelines can help you reduce a collection, and to present us with the best possible chance of fixing your bug quickly. The following steps are written from the perspective of isolating a bug that is seen when working in ‘User Mode’ (as opposed to Design Mode bugs seen in the Inspector, etc.) but either way, the principles are the same.
Yes, it can be a lot of work to reduce a bug to its smallest possible size, but remember that the more you do to make it easy for us to see the bug, the faster we can fix it. And if anecdotal reports are of any value, our users seem to find great satisfaction in helping in this way, and sometimes they even learn things about their collection they can use to make it better.
If attempts to trim it down fail, then by all means send us the whole collection. We’d prefer that you contact us first so we can help you reduce it further, but either way it is critically important that you accurately document te steps needed for us to see the bug. When documenting a collection, be sure to refer to items by exact name, not by menu position or other loose criteria: “Choose ‘Sales Summary’ in the ‘Reports’ menu,” is far better than “Open the first report.”
|Uploading Sample Collections||
Once you’ve got the perfect sample collection, how do you get it to us? One of two ways:
Start by making a .zip archive of the collection, along with any relevant files (import files, etc.). Name this archive in a way that will help us match it to your report. If you’ve crated a report in techdb, name the zip archive ‘Rnnnn.zip’ where ‘nnnn’ is the report number. Otherwise, name it ‘Cnnn.zip’ where ‘nnn’ is your customer number. (You can also add a word or two to help identify is, such as ‘R9999-Abacus Error.zip’)
|Screenshots of Crashes? No!||
When Helix crashes, it should display a dialog telling you ‘A nnnn error occurred’ — please do not take a screenshot of that dialog and upload it! Write down the code and click the Quit button. Helix should then go into a spin and then crash. That is where the useful information might be gathered. Follow the instructions on the How to Report macOS Crashes page to get the crash log and attach that as a supporting document to your beta report. Again: No Screenshots!
|Screenshots of Anomalies? Maybe.||
A picture is worth a thousand words, so include macOS vs Classic screen shots if there is a difference worth illustrating. If you do this, be absolutely certain that you include a screenshot of the Classic version so we have something to compare to. And please try to describe the problem as you see it; don’t just upload a picture with a report that says ‘this doesn’t look right.’
Of course, not every bug can be isolated down to a simple test case. We know that, and we don’t want to discourage you from sending your reports because you can’t seem to isolate the bug. Just keep in mind that the more you do to help us see the bug, the faster we can fix it. And that helps everybody.
|For Further Study|