FloatDebugUpdate01Well it might not be as old as "Ye Olde English" but every once in a while you have to brush the cobwebs off of binary numbers and hexadecimal formats. A coworker had a problem converting numerical constants from a PLC editor and for some reason the programmers decided to print floating-point constants in the editor as integer numbers instead of sensible real numbers. For example, the software would list 1092616192 instead of 10.0! My co-worker had found a web site tool to convert the integer numbers into floating-point but then hit a stumbling block when he encountered large negative integers, such as -1088841318 (which is -0.6). My first thought was to take the two's complement of the number to convert it back to a positive integer so he could continue using the web site tool he had found.

Fortunately, the old floating-point debugger tool I wrote a long time ago (1999) came to the rescue. It was quicker to enter the numbers into the application rather than the web site, but the only problem was that the entry area to enter the numbers only accepted floating-point and hexadecimal. So I created an integer entry text box and also placed a 1's and 2's complement button on the form for ease of use. He was able to use this tool to finish the conversion.

After discussing the solution with another co-worker, we delved into the difference of 1's versus 2's complement. My initial contention since the difference was either +0 or +1, it only affected the final bit and would have no real impact to the result. My co-worker then reminded me of the special cases where adding 1 to the result would cause the 1 to be carried up in the number and in some cases even into the exponent. In order to debug these cases, I took the core code from the VB application and created an Excel spreadsheet that could duplicate the functionality and serve as a test bed for the fringe cases.

The test cases that I generated showed that the carry was indeed negligible as I originally postulated. Here are some examples:

Decimal Sample Hex 1's comp 2's comp 1's float 2's float %Diff
-1098907648 0xBE800000 0x417FFFFF 0x41800000 15.99999905 16 0.000006%
-1132462080 0xBC800000 0x437FFFFF 0x43800000 255.9999847 256 0.000006%
-1140850688 0xBC000000 0x43FFFFFF 0x44000000 511.9999695 512 0.000006%

FloatConvert The bottom line is that the nature of the IEEE floating-point number minimizes the effect of the 2's complement carry. In other words, adding one to the value does not cause a big change in the number specifically because the mantissa is really the sum of the fractional powers of 2 (2^-2 + 2^-3 + 2^-4 + ... + 2^-23). In the case where the mantissa is 0x3FFFFF (2^-3 + ... + 2^-23) adding 1 to the mantissa results in 0x400000 (2^-2). The actual difference in 0x3FFFFF and 0x40000 when summing the inverse powers of 2 is (4194302 / 8388608) compared to (4194304 / 8388608) which is 0.4999998 versus 0.5!

Technorati tags: