|Rounding in Helix (An Introduction)|
Rounding is generally a pretty straightforward topic: when you round, round to the nearest number for the level of precision specified. Confusion is introduced when the value to be rounded is exactly half way between the two possible values. This technote explains and documents how rounding is done in Helix.
For the sake of illustration, this technote discusses rounding to the nearest integer, typically done with Helix’s Round ◊ tile. The same rules apply to the Round ◊ to nearest ◊ tile, which allows you to specify the rounding precision.
Rounding always seeks the nearest value to round to, but when the original value is exactly half way between the target values, Helix employs one of two distinct sets of rounding rules, depending on the data type being rounded.
The rules below only apply to values that fall exactly half way between the two possible values. All other values round to the nearest number.
|Rounding Numbers: Scientific Rounding||
Number data (technically, floating point numbers) were introduced in the very first version of Helix and have always used the scientific rounding method. Scientific rounding seeks to minimize cumulative errors, so it rounds to the nearest even number, thereby reducing the possibility that the result are skewed.
To see scientific rounding in another setting, open Apple’s ScriptEditor and run this simple AppleScript (make sure the event log is open so you can see the results).
repeat with i from -10 to 10 get round (i + 0.5) rounding to nearest end repeat
In AppleScript, to nearest uses scientific rounding, and you get the same answers as Helix produces when rounding data in number format.
|Rounding Fixed Points: Financial Rounding||
Fixed Point data (technically, integers) were introduced later in Helix’s history and have always used the financial rounding method. Financial rounding seeks to make rounding more predictable, so it rounds away from zero, thereby guaranteeing consistent rounding.
Financial rounding is also known as “symmetric arithmetic rounding,” “banker’s rounding,” and a few others names. There are also variations on the basic rules for financial rounding — always rounding toward zero is one common variation — so be aware that not everybody has the same rules in mind when speaking about financial rounding.
To see financial rounding in another setting, open Apple’s ScriptEditor and run this AppleScript (make sure the event log is open so you can see the results).
repeat with i from -10 to 10 get round (i + 0.5) rounding as taught in school end repeat
In AppleScript, as taught in school uses financial rounding, and you get the exact answers that Helix produces when rounding data in Fixed Point format.
|Number Datatype Conversion Errors (Decimal to Binary Errors)||
Certain numbers can be seen to violate the specification for scientific rounding. This problem is introduced because computers typically convert decimal (base 10) numbers into their binary (base 2) equivalents before doing mathematic operations. In decimal math, we have many fractional values (e.g. 1/3) that we understand to be ‘infinitely repeating decimal numbers.’ Attempting to divide 100 identical items evenly between three people is impossible. Doing mathematical operations along this line introduces rounding errors that we naturally understand and for which we compensate. A human being can look at ((100/3)*3) and understand that the answer is 100, but if you work it out, the answer comes out as 99.999… and we simply ‘round it off’ to 100. But it is important to keep in mind that 100 is an approximation, the ‘real’ answer is 99.999...
When examining binary numbers, you find that a whole different series of fractional numbers turn out to be infinitely repeating. For example 1/10 is an infinitely repeating binary number.
Now consider how this applies to rounding. Given the number .235 and being asked to round to the nearest .01, you would apply the scientific rounding rules and arrive at the (correct) answer of .24. However, Helix rounds this to .23.
Why? The answer is binary conversion error. 235/1000 is, when expressed as a binary number, an infinitely repeating number. Converting 235/1000 to binary and then back to decimal yields (approximately) .23499999… Because this value is not exactly half way between the two numbers we are potentially rounding to, it is rounded to the nearest value (down, in this case) and the result appears incorrect.
Why does Helix make this mistake? The math routines that Helix uses are part of macOS: they are part of the common system code that most programs use.
In macOS, open Spotlight (or the Calculator Dashboard Widget for older versions) and type 55/3*3 and you will see that the answer is not what it should be. In both these cases, the value has to be approximated, and the minute error is seen when math operations are performed.
If you do not need more than two decimal places of precision, you can avoid these rounding errors by converting the number into a Fixed Point data type before performing math operations, converting the result back to a number data type, if necessary.
It should be remembered that the value that shown on a Helix view is the display value. Helix always stores numeric data (Numbers and Fixed Point datatypes) with full precision. (15 decimal places for Number, 2 for Fixed Point.)
Many problems which are thought to be rounding errors are actually perception errors, caused by display values that do not show the stored value in full precision. If a series of values is not adding up as expected, increase the display precision (number of places after the decimal) by one or two places and recheck the data. You may find that the stored values do not match the displayed values, and that Helix is accurately performing the calculation on the stored values.
One place this can be confusing is with sales tax calculations: If your tax rate is 6.25% and you have a taxable sale of $39.95 the sales tax calculates out to $2.496875. If you store that in a Number field, that is what Helix stores regardless of the display formatting (unless you use a rounding tile). In this example, if the tax field is formatted to display 2 decimal places it will appear as $2.50, but that is a rounded up value. Add two of those together and the answer is not $5.00, it is $4.99375, which when displayed using 2 decimal places appears as $4.99.
|Converting Data from Numbers to Fixed Point|
|Converting Data from Number to Fixed Point||
After reading the above, the obvious answer for most cases is to use Fixed Point datatype for numeric data that is never more precise than 2 decimal places. Unfortunately, doing this conversion is not as straightforward as one would hope.
First is the issue that Helix will not let you change the datatype of a field that contains data in even one record. Of course, you could select the field and use the ‘Delete Data’ command in the ‘Icon’ menu to remove the data, after which you can switch the datatype, but then you’ve lost your data.
Add to that the complication that Helix is a “strongly typecast” language so, for example, you can’t have an abacus that adds a Number and a Fixed Point field: any such abacus becomes invalid until the issue is corrected. The same is true for posting, defaulting values into rectangles, etc.
The answer to this dilemma is to create a new Fixed Point datatype field, migrate the data from the old Number field to the new Fixed Point field, and then migrate the uses of the field from the old one to the new one. When the migration is complete, delete the old field to reclaim its storage space. Here is a step-by-step outline to doing this migration:
Once you’ve done this a time or two, the process becomes obvious. Not all numeric data is suitable for Fixed Point storage, but for those where it does, you resolve the rounding issues mentioned above, and your data requires less space for storage, as Fixed Point uses a more efficient encoding method.