Possible Bug in Table Calculations?

Possible Bug in Table Calculations?

[Peter Jones]

I'm probably just missing something, but...

Org Table:
|  Values |
|---------|
|  568.68 |
|  599.32 |
|  140.16 |
| 1308.20 |
#+TBLFM: @5$1=vsum(@2..@-1);%.2f

Results from calc:
     568.68
     599.32
   + 1168.
     140.16
   + 1308.16

Is this some sort of rounding error?  What am I doing wrong?

-- 
Peter Jones, pmade inc.
http://pmade.com

Re: Possible Bug in Table Calculations?

[Carsten Dominik]

Hi Peter,

org tables work with a limited numerical accuracy of results to avoid  
fields that result from operations like divisions to get too wide.   
Maybe I really should change this and make people specify a format in  
the formula if they want it narrower.

Currently, Org uses cals with only 5 significant digits, this is the  
cause for this accuracy.  You can change this per equation by  
including something like "n8" or "f8" into the mode string, like

   @5$1=vsum(@2..@-1);n8%.2f

Or you can customize the variable org-calc-default-formats and change  
the format there from (float 5) to something like (float 8), which  
will solve you problem for good.

Maybe I should actually do this by default.  Yes, I wil, do so.   
Better questions about too wide fields than inaccurate calculations.

HTH

- Carsten

On May 15, 2008, at 8:57 PM, Peter Jones wrote:

> I'm probably just missing something, but...
>
> Org Table:
> |  Values |
> |---------|
> |  568.68 |
> |  599.32 |
> |  140.16 |
> | 1308.20 |
> #+TBLFM: @5$1=vsum(@2..@-1);%.2f
>
> Results from calc:
>     568.68
>     599.32
>   + 1168.
>     140.16
>   + 1308.16
>
> Is this some sort of rounding error?  What am I doing wrong?
>
> -- 
> Peter Jones, pmade inc.
> http://pmade.com
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

Re: Possible Bug in Table Calculations?

[Peter Jones]

Carsten Dominik <C.Dominik@uva.nl> writes:
> Currently, Org uses cals with only 5 significant digits, this is the
> cause for this accuracy.  You can change this per equation by
> including something like "n8" or "f8" into the mode string, like
>
>   @5$1=vsum(@2..@-1);n8%.2f

Ah, I totally glossed over the n8 format.  Thanks.

> Maybe I should actually do this by default.  Yes, I wil, do so.
> Better questions about too wide fields than inaccurate calculations.

I agree, and thanks!

-- 
Peter Jones, pmade inc.
http://pmade.com