emacs-orgmode@gnu.org archives
 help / color / mirror / code / Atom feed
* Possible Bug in Table Calculations?
@ 2008-05-15 18:57 Peter Jones
  2008-05-16  7:39 ` Carsten Dominik
  0 siblings, 1 reply; 3+ messages in thread
From: Peter Jones @ 2008-05-15 18:57 UTC (permalink / raw)
  To: emacs-orgmode

I'm probably just missing something, but...

Org Table:
|  Values |
|---------|
|  568.68 |
|  599.32 |
|  140.16 |
| 1308.20 |
#+TBLFM: @5$1=vsum(@2..@-1);%.2f

Results from calc:
     568.68
     599.32
   + 1168.
     140.16
   + 1308.16

Is this some sort of rounding error?  What am I doing wrong?

-- 
Peter Jones, pmade inc.
http://pmade.com

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Possible Bug in Table Calculations?
  2008-05-15 18:57 Possible Bug in Table Calculations? Peter Jones
@ 2008-05-16  7:39 ` Carsten Dominik
  2008-05-16 14:35   ` Peter Jones
  0 siblings, 1 reply; 3+ messages in thread
From: Carsten Dominik @ 2008-05-16  7:39 UTC (permalink / raw)
  To: Peter Jones; +Cc: emacs-orgmode

Hi Peter,

org tables work with a limited numerical accuracy of results to avoid  
fields that result from operations like divisions to get too wide.   
Maybe I really should change this and make people specify a format in  
the formula if they want it narrower.

Currently, Org uses cals with only 5 significant digits, this is the  
cause for this accuracy.  You can change this per equation by  
including something like "n8" or "f8" into the mode string, like

   @5$1=vsum(@2..@-1);n8%.2f

Or you can customize the variable org-calc-default-formats and change  
the format there from (float 5) to something like (float 8), which  
will solve you problem for good.

Maybe I should actually do this by default.  Yes, I wil, do so.   
Better questions about too wide fields than inaccurate calculations.

HTH

- Carsten

On May 15, 2008, at 8:57 PM, Peter Jones wrote:

> I'm probably just missing something, but...
>
> Org Table:
> |  Values |
> |---------|
> |  568.68 |
> |  599.32 |
> |  140.16 |
> | 1308.20 |
> #+TBLFM: @5$1=vsum(@2..@-1);%.2f
>
> Results from calc:
>     568.68
>     599.32
>   + 1168.
>     140.16
>   + 1308.16
>
> Is this some sort of rounding error?  What am I doing wrong?
>
> -- 
> Peter Jones, pmade inc.
> http://pmade.com
>
>
> _______________________________________________
> Emacs-orgmode mailing list
> Remember: use `Reply All' to send replies to the list.
> Emacs-orgmode@gnu.org
> http://lists.gnu.org/mailman/listinfo/emacs-orgmode

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Possible Bug in Table Calculations?
  2008-05-16  7:39 ` Carsten Dominik
@ 2008-05-16 14:35   ` Peter Jones
  0 siblings, 0 replies; 3+ messages in thread
From: Peter Jones @ 2008-05-16 14:35 UTC (permalink / raw)
  To: emacs-orgmode

Carsten Dominik <C.Dominik@uva.nl> writes:
> Currently, Org uses cals with only 5 significant digits, this is the
> cause for this accuracy.  You can change this per equation by
> including something like "n8" or "f8" into the mode string, like
>
>   @5$1=vsum(@2..@-1);n8%.2f

Ah, I totally glossed over the n8 format.  Thanks.

> Maybe I should actually do this by default.  Yes, I wil, do so.
> Better questions about too wide fields than inaccurate calculations.

I agree, and thanks!

-- 
Peter Jones, pmade inc.
http://pmade.com

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2008-05-16 14:36 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-05-15 18:57 Possible Bug in Table Calculations? Peter Jones
2008-05-16  7:39 ` Carsten Dominik
2008-05-16 14:35   ` Peter Jones

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).