Re: adding up decimal fractions in the spreadsheet

From: Carsten Dominik <dominik@uva.nl>
To: r.thiel@uni-jena.de
Cc: emacs-orgmode@gnu.org
Subject: Re: adding up decimal fractions in the spreadsheet
Date: Sat, 27 Sep 2008 14:19:29 +0200	[thread overview]
Message-ID: <2550EE81-2105-41DC-B0A3-40D3D62B788C@uva.nl> (raw)
In-Reply-To: <d6b6c7740809270019s5b2d6a83k7ee7cb2ad8e165ae@mail.gmail.com>

Hi Rainer,

it is a common mis-conception that a number that has a limited number  
of digits in base 10 can be represented exactly on a computer.  This  
is correct for integers, but not for fractional numbers.

Example:

1. Open the emacs calc with M-x calc RET
2. Type 12130.68 (first number from your example)
3. Press `p 200 RET' to switch to 200 digit precision.
4. press `d r 2 RET' o display the number in Base 2.

You'll see

10111101100010.1010111000010100011110101110000101000111101011100001010001111010111000010100011110101110000101000111101011100001010001111010111000010100011110101110000 
................

It works the other way round as well, for example  
0.33333333............. has an exact representation in base 3 (0.1).

This means that even simple additions like this do produce small  
errors on computers, and the displayed result will depend upon how he  
conversion is done.

(format "%s" x) just spits out digits, as many as possible.  That is  
why the fix I have pushed to the git repo uses

   (number-to-string res)

and not

   (format "%s" res)

number-to-string is an internal function that tries very hard to do  
the conversion in the way you intend it.

- Carsten

On Sep 27, 2008, at 9:19 AM, Rainer Thiel wrote:

> Following a hint by Bernt Hansen, I have now changed org-table-sum to
> (format "%s" res) until I'll be able to use the corrected version
> Carsten Dominik uploaded to the git repository (which I am not sure
> how to access).
>
> Even so, org-table-sum at times shows a strange behaviour.
>
> When I try to sum up the following table:
>
> |  12130.68 |
> |  -1444.19 |
> | -12744.90 |
> |   -186.00 |
> |   7000.00 |
> |  -7056.00 |
> |   -335.58 |
> |   -277.00 |
>
> I get -2912.9899999999993 (-2912.99 being the exact result).  This is
> something I'd expect when square roots or logarithms are implied, but
> with such a simple addition and subtraction of decimal numbers I'd be
> inclined to call it a bug.  Is there a way to improve org-mode in this
> respect?
>
> Many thanks again for this great mode
> -- 
> Prof. Dr. Rainer Thiel
> Institut für Altertumswissenschaften
> 07737 Jena, Germany (EU)
> r.thiel@uni-jena.de
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode