[BUG] hline references on left side of table formula

[BUG] hline references on left side of table formula

[Rick Frankel]

Hi-

I don't know if this is a bug or feature :), but if an hline reference
(@I, etc) is used on the left side of a calculation, it applies to ALL
columns in the row even if the column is specfied.

Here are some examples to show the results. I would expect all three
versions to generate the same results as the first example.

#+BEGIN_ORG
   * Absolute reference (expected results)
     | a | b |
     |---+---|
     | x | 1 |
     | y | 2 |
     |---+---|
     |   | 3 |
   #+TBLFM: @4$2=vsum(@I..@II)

   * hline reference
     | a     | b |
     |-------+---|
     | x     | 1 |
     | y     | 2 |
     |-------+---|
     | x + y | 3 |
   #+TBLFM: @II$2=vsum(@I..@II)

   * hline reference with full cell specification in sum
     | a | b |
     |---+---|
     | x | 1 |
     | y | 2 |
     |---+---|
     | 3 | 3 |
   #+TBLFM: @II$2=vsum(@I$2..@II$2)
   #+END_ORG

FWIW, I believe the problem is that `org-table-recalculate' is
matching lhs cell references explicitly against pure numeric
references ("@[0-9]+$[0-9]+") and therefore expands the lhs via
`org-expand-lhs-ranges' instead of expanding it with
`org-table-get-descriptor-line'

rick

Re: [BUG] hline references on left side of table formula

[Carsten Dominik]

Hi Rick,

hline-relative references on the left side of a table are currently not supported.  The fact that this is expanded is a bug.  A patch catching this case would be very welcome.

Regards

- Carsten

On May 1, 2013, at 7:27 PM, Rick Frankel <rick@rickster.com> wrote:

> Hi-
> 
> I don't know if this is a bug or feature :), but if an hline reference
> (@I, etc) is used on the left side of a calculation, it applies to ALL
> columns in the row even if the column is specfied.
> 
> Here are some examples to show the results. I would expect all three
> versions to generate the same results as the first example.
> 
> #+BEGIN_ORG
>  * Absolute reference (expected results)
>    | a | b |
>    |---+---|
>    | x | 1 |
>    | y | 2 |
>    |---+---|
>    |   | 3 |
>  #+TBLFM: @4$2=vsum(@I..@II)
> 
>  * hline reference
>    | a     | b |
>    |-------+---|
>    | x     | 1 |
>    | y     | 2 |
>    |-------+---|
>    | x + y | 3 |
>  #+TBLFM: @II$2=vsum(@I..@II)
> 
>  * hline reference with full cell specification in sum
>    | a | b |
>    |---+---|
>    | x | 1 |
>    | y | 2 |
>    |---+---|
>    | 3 | 3 |
>  #+TBLFM: @II$2=vsum(@I$2..@II$2)
>  #+END_ORG
> 
> FWIW, I believe the problem is that `org-table-recalculate' is
> matching lhs cell references explicitly against pure numeric
> references ("@[0-9]+$[0-9]+") and therefore expands the lhs via
> `org-expand-lhs-ranges' instead of expanding it with
> `org-table-get-descriptor-line'
> 
> rick
> 
> 

Re: [BUG] hline references on left side of table formula

[lompik]

Hi,

I was interested to use hline references on the left side. However I found out that it
is not completely unsupported.
When the left side consist of:
* single cells references, it does not work
* ranged cells, it works for me (org 8.0.3)

The patch below tranforms the single cell references to ranged cells .

However it still does not work for references like @+ or @- I$1.. , which can be solve 
using references like @I+1#1.

Thanks,


*** a/org-table.el Sat Jun 29 16:20:30 2013
--- b/org-table.el Tue Oct 1 00:23:35 2013
***************
*** 3227,3232 ****
--- 3227,3236 ----
(put-text-property 0 (length (caar res))
:orig-eqn e (caar res))))
(t
+ ;; if contains hline-reference and refers to fixed cell..make it range
+ (if (string-match "^@[+-]?I*[^.]*$" lhs )
+ (setq lhs (format "%s..%s" lhs lhs )))
+ 
(setq range (org-table-get-range lhs org-table-current-begin-pos
1 nil 'corners))
(setq r1 (nth 0 range) c1 (nth 1 range)


However, fields like @-I or @+II 

> 
> Hi Rick,
> 
> hline-relative references on the left side of a table are currently not 
> supported. The fact that this is expanded is a bug. A patch catching this 
> case would be very welcome.
> 
> Regards
> 
> - Carsten
> 
> On May 1, 2013, at 7:27 PM, Rick Frankel  wrote:
> 
> > Hi-
> > 
> > I don't know if this is a bug or feature :), but if an hline reference
> > (@I, etc) is used on the left side of a calculation, it applies to ALL
> > columns in the row even if the column is specfied.
> > 
> > Here are some examples to show the results. I would expect all three
> > versions to generate the same results as the first example.
> > 
> > #+BEGIN_ORG
> > * Absolute reference (expected results)
> > | a | b |
> > |---+---|
> > | x | 1 |
> > | y | 2 |
> > |---+---|
> > | | 3 |
> > #+TBLFM: @4$2=vsum(@address@hidden)
> > 
> > * hline reference
> > | a | b |
> > |-------+---|
> > | x | 1 |
> > | y | 2 |
> > |-------+---|
> > | x + y | 3 |
> > #+TBLFM: @II$2=vsum(@address@hidden)
> > 
> > * hline reference with full cell specification in sum
> > | a | b |
> > |---+---|
> > | x | 1 |
> > | y | 2 |
> > |---+---|
> > | 3 | 3 |
> > #+TBLFM: @II$2=vsum(@address@hidden)
> > #+END_ORG
> > 
> > FWIW, I believe the problem is that `org-table-recalculate' is
> > matching lhs cell references explicitly against pure numeric
> > references ("@[0-9]+$[0-9]+") and therefore expands the lhs via
> > `org-expand-lhs-ranges' instead of expanding it with
> > `org-table-get-descriptor-line'
> > 
> > rick
> > 
> > 
___________________________________________________________
Qu'y a-t-il ce soir à la télé ? D'un coup d'œil, visualisez le programme sur Voila.fr http://tv.voila.fr/programmes/chaines-tnt/ce-soir.html

Re: [BUG] hline references on left side of table formula

[Bastien]

Hi,

lompik@voila.fr writes:

> I was interested to use hline references on the left side. However I found out that it
> is not completely unsupported.
> When the left side consist of:
> * single cells references, it does not work
> * ranged cells, it works for me (org 8.0.3)

(I'm not sure I fully understand the issue.)

> The patch below tranforms the single cell references to ranged cells .
>
> However it still does not work for references like @+ or @- I$1.. , which can be solve 
> using references like @I+1#1.

If you come up with a complete patch, please submit it by following
the guidelines here: http://orgmode.org/worg/org-contribute.html

Thanks!

-- 
 Bastien

Re: [BUG] hline references on left side of table formula

[Rick Frankel]

On 2013-11-06 03:23, Bastien wrote:
> lompik@voila.fr writes:
> 
> I was interested to use hline references on the left side. However I 
> found out that it
> is not completely unsupported.
> When the left side consist of:
> * single cells references, it does not work
> * ranged cells, it works for me (org 8.0.3)
> 
> (I'm not sure I fully understand the issue.)
> 
> The patch below tranforms the single cell references to ranged cells .
> 
> However it still does not work for references like @+ or @- I$1.. ,
> which can be solve using references like @I+1#1.
> 
> If you come up with a complete patch, please submit it by following
> the guidelines here: http://orgmode.org/worg/org-contribute.html
> 
What he is saying, is that references like =@II$2=, or =@II+1$2= do not
work correctly on the left-hand side of a table format (verified by
carsten in a previous thread) which is why i created the patch to
disallow hline-relative references on th LHS.

However, he has found that ranged references line as =@II$2..@II+2$2=
do in fact work to reference a single cell on the left hand side on a
formula.

For example:

Given the input table:

| a | b |
|---+---|
| x | 1 |
| y | 2 |
|---+---|
|   |   |
|   |   |

The following results occur:

| a     | b |
|-------+---|
| x     | 1 |
| y     | 2 |
|-------+---|
| x + y | 3 |
| x + y | 3 |
#+TBLFM: @II$2=vsum(@I..@II)

| a | b |
|---+---|
| x | 1 |
| y | 2 |
|---+---|
|   | 3 |
|   |   |
#+TBLFM: @II$2..@II+1$2=vsum(@I..@II)

| a | b |
|---+---|
| x | 1 |
| y | 2 |
|---+---|
| 3 | 3 |
| 3 | 3 |
#+TBLFM: @II$2=vsum(@I$2..@II$2)

| a | b |
|---+---|
| x | 1 |
| y | 2 |
|---+---|
|   | 3 |
|   |   |
#+TBLFM: @II$2..@II+1$2=vsum(@I$2..@II$2)

So, perhaps my patch was premature, but should instead should check
for hline refs w/o ranges on the lhs. Here's a new patch to only
error-out if there is no range spec. Note that this also fixes Achim's
issue w/ the colnames test failing.


rick

-------------------- 8< --------------------
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -3016,8 +3016,9 @@ known that the table will be realigned a little 
later anyway."
;; Insert constants in all formulas
(setq eqlist
(mapcar (lambda (x)
-             (if (string-match "^@-?I+" (car x))
-             (user-error "Can't assign to hline relative reference"))
+             (if (and (string-match "^@-?I+" (car x))
+                  (not (string-match "\.\.@" (car x))))
+             (user-error "Can't assign to hline relative reference 
without a range specification."))
(when (string-match "\\`$[<>]" (car x))
(setq lhs1 (car x))
(setq x (cons (substring

Re: [BUG] hline references on left side of table formula

[Achim Gratz]

Rick Frankel writes:
> What he is saying, is that references like =@II$2=, or =@II+1$2= do not
> work correctly on the left-hand side of a table format (verified by
> carsten in a previous thread) which is why i created the patch to
> disallow hline-relative references on th LHS.

They are working, sort of, by constructing an LHS range implicitly with
the cursor at the end of the table in the first column (equivalent of
@$).  This is hard to explain, but try a slightly more advance table:

| a | b |
|---+---|
| w | 1 |
| x | 2 |
|---+---|
|   |   |
|   |   |
|---+---|
| y | 3 |
| z | 4 |
|---+---|
|   |   |

with these formulas:
#+TBLFM: $=vsum(@-II..@-I)


> However, he has found that ranged references line as =@II$2..@II+2$2=
> do in fact work to reference a single cell on the left hand side on a
> formula.
>
> For example:
>
> Given the input table:
>
> | a | b |
> |---+---|
> | x | 1 |
> | y | 2 |
> |---+---|
> |   |   |
> |   |   |
>
> The following results occur:
>
> | a     | b |
> |-------+---|
> | x     | 1 |
> | y     | 2 |
> |-------+---|
> | x + y | 3 |
> | x + y | 3 |
> #+TBLFM: @II$2=vsum(@I..@II)
>
> | a | b |
> |---+---|
> | x | 1 |
> | y | 2 |
> |---+---|
> |   | 3 |
> |   |   |
> #+TBLFM: @II$2..@II+1$2=vsum(@I..@II)
>
> | a | b |
> |---+---|
> | x | 1 |
> | y | 2 |
> |---+---|
> | 3 | 3 |
> | 3 | 3 |
> #+TBLFM: @II$2=vsum(@I$2..@II$2)
>
> | a | b |
> |---+---|
> | x | 1 |
> | y | 2 |
> |---+---|
> |   | 3 |
> |   |   |
> #+TBLFM: @II$2..@II+1$2=vsum(@I$2..@II$2)
>
> So, perhaps my patch was premature, but should instead should check
> for hline refs w/o ranges on the lhs. Here's a new patch to only
> error-out if there is no range spec. Note that this also fixes Achim's
> issue w/ the colnames test failing.
>
>
> rick
>
> -------------------- 8< --------------------
> --- a/lisp/org-table.el
> +++ b/lisp/org-table.el
> @@ -3016,8 +3016,9 @@ known that the table will be realigned a little 
> later anyway."
> ;; Insert constants in all formulas
> (setq eqlist
> (mapcar (lambda (x)
> -             (if (string-match "^@-?I+" (car x))
> -             (user-error "Can't assign to hline relative reference"))
> +             (if (and (string-match "^@-?I+" (car x))
> +                  (not (string-match "\.\.@" (car x))))
> +             (user-error "Can't assign to hline relative reference 
> without a range specification."))
> (when (string-match "\\`$[<>]" (car x))
> (setq lhs1 (car x))
> (setq x (cons (substring
>
>
>
>
-- 

Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: [BUG] hline references on left side of table formula

[Achim Gratz]

Achim Gratz writes:
[...]
> with these formulas:
> #+TBLFM: $=vsum(@-II..@-I)

whoops, press C-c C-c in the wrong buffer.

I meant these formulas:

#+TBLFM: @$=vsum(@-II..@-I)
#+TBLFM: @II=vsum(@-II..@-I)
#+TBLFM: @III=vsum(@-II..@-I)
#+TBLFM: @IIII=vsum(@-II..@-I)
#+TBLFM: @II..$2=vsum(@-II..@-I)
#+TBLFM: @III..$2=vsum(@-II..@-I)
#+TBLFM: @IIII..$2=vsum(@-II..@-I)
#+TBLFM: @II+1..@II$2=vsum(@-II..@-I)
#+TBLFM: @II$2..@II+1$2=vsum(@-II..@-I)

Some of the results are useful when your table has a certain, even
though in general this does the wrong thing as Carsten said.  I'm not
really having an opinion on whether this should be an error (as your
previous patch does, which should then add a correction to the test that
is now failing) or if some / all of this should stay allowed until
somebody musters the time to fix it properly (I think this would amount
to re-implementing a good part of what is org-table).


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

Re: [BUG] hline references on left side of table formula

[Michael Brand]

Hi Rick

On Wed, Nov 6, 2013 at 7:50 PM, Rick Frankel <rick@rickster.com> wrote:
> +                  (not (string-match "\.\.@" (car x))))

This would be the same as

> +                  (not (string-match "..@" (car x))))

but you meant

> +                  (not (string-match "\\.\\.@" (car x))))

Michael

Re: [BUG] hline references on left side of table formula

[Rick Frankel]

On 2013-11-07 00:59, Michael Brand wrote:
> On Wed, Nov 6, 2013 at 7:50 PM, Rick Frankel <rick@rickster.com> wrote:
> +                  (not (string-match "\.\.@" (car x))))
> 
> This would be the same as
> 
> +                  (not (string-match "..@" (car x))))
> 
> but you meant
> 
> +                  (not (string-match "\\.\\.@" (car x))))
> 

Of course, you are right.

rick

Re: [BUG] hline references on left side of table formula

[Rick Frankel]

[-- Attachment #1: Type: text/plain, Size: 2336 bytes --]

On 2013-11-06 15:14, Achim Gratz wrote:
> Achim Gratz writes:
> [...]
> with these formulas:
> #+TBLFM: $=vsum(@-II..@-I)
> 
> whoops, press C-c C-c in the wrong buffer.
> 
> I meant these formulas:
> 
> #+TBLFM: @$=vsum(@-II..@-I)
> #+TBLFM: @II=vsum(@-II..@-I)
> #+TBLFM: @III=vsum(@-II..@-I)
> #+TBLFM: @IIII=vsum(@-II..@-I)
> #+TBLFM: @II..$2=vsum(@-II..@-I)
> #+TBLFM: @III..$2=vsum(@-II..@-I)
> #+TBLFM: @IIII..$2=vsum(@-II..@-I)
> #+TBLFM: @II+1..@II$2=vsum(@-II..@-I)
> #+TBLFM: @II$2..@II+1$2=vsum(@-II..@-I)
> 
> Some of the results are useful when your table has a certain, even
> though in general this does the wrong thing as Carsten said.  I'm not
> really having an opinion on whether this should be an error (as your
> previous patch does, which should then add a correction to the test 
> that
> is now failing) or if some / all of this should stay allowed until
> somebody musters the time to fix it properly (I think this would amount
> to re-implementing a good part of what is org-table).


Ok, i see what's happening in your examples (a testing org file
attached), though i question the usefullness of most of the results ;).

The updated patch attached to the previous email (fixed as pointed out
by michael):

--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -3016,8 +3016,9 @@ known that the table will be realigned a little 
later anyway."
;; Insert constants in all formulas
(setq eqlist
(mapcar (lambda (x)
-             (if (string-match "^@-?I+" (car x))
-             (user-error "Can't assign to hline relative reference"))
+             (if (and (string-match "^@-?I+" (car x))
+                  (not (string-match "\\.\\.@" (car x))))
+             (user-error "Can't assign to hline relative reference 
without a range specification."))
(when (string-match "\\`$[<>]" (car x))
(setq lhs1 (car x))
(setq x (cons (substring

Both fixes the testing issue and allows what seems to me to be the
most relevant use-case.

I have no real position on whether the best solution is to back out
the change, or modify it as above to make the test pass and allow the
logical usecase.

If the first, I think it would be useful for the documentation to
(somehow) explain what happens if an explicit column specificaton is
made without a corresponding range.

Please chime in on which patch should be applied.

rick

[-- Attachment #2: tblref.org --]
[-- Type: text/plain, Size: 3758 bytes --]

* Code
:PROPERTIES:
:eval:     never
:END:
#+name: table
#+BEGIN_SRC emacs-lisp :results silent :eval yes
"| a | b |
|---+---|
| w | 1 |
| x | 2 |
|---+---|
|   |   |
|   |   |
|---+---|
| y | 3 |
| z | 4 |
|---+---|
|   |   |"
#+END_SRC
#+name: expand-tables
#+BEGIN_SRC emacs-lisp
  (save-excursion
      (while (re-search-forward org-table-TBLFM-begin-regexp)
        (org-table-calc-current-TBLFM)))
#+END_SRC
#+name: generate-tables
#+BEGIN_SRC emacs-lisp :var table=table :results wrap
  (mapconcat
   (lambda (fm)
     (mapconcat
      'identity (list (concat "**" (substring fm 8)) table fm) "\n"))
   (list "#+TBLFM: @$=vsum(@-II..@-I)"
         "#+TBLFM: @II=vsum(@-II..@-I)"
         "#+TBLFM: @III=vsum(@-II..@-I)"
         "#+TBLFM: @IIII=vsum(@-II..@-I)"
         "#+TBLFM: @II..$2=vsum(@-II..@-I)"
         "#+TBLFM: @III..$2=vsum(@-II..@-I)"
         "#+TBLFM: @IIII..$2=vsum(@-II..@-I)"
         "#+TBLFM: @II+1..@II$2=vsum(@-II..@-I)"
         "#+TBLFM: @II$2..@II+1$2=vsum(@-II..@-I)") "\n")
#+END_SRC

* Eval me! (=\C-c\C-v\C-s=)
:PROPERTIES:
:ID: EVAL-ME
:END:
#+name: call-generate
#+call: generate-tables[:eval yes](table=table) :results wrap
#+call: expand-tables[:eval yes]() :results silent

#+RESULTS: call-generate
:RESULTS:
** @$=vsum(@-II..@-I)
| a     | b |
|-------+---|
| w     | 1 |
| x     | 2 |
|-------+---|
|       |   |
|       |   |
|-------+---|
| y     | 3 |
| z     | 4 |
|-------+---|
| y + z |   |
#+TBLFM: @$=vsum(@-II..@-I)
** @II=vsum(@-II..@-I)
| a                             | b |
|-------------------------------+---|
| w                             | 1 |
| x                             | 2 |
|-------------------------------+---|
| w + x                         |   |
| w + x                         |   |
|-------------------------------+---|
| w + x + w + x                 | 3 |
| w + x + w + x                 | 4 |
|-------------------------------+---|
| w + x + w + x + w + x + w + x |   |
#+TBLFM: @II=vsum(@-II..@-I)
** @III=vsum(@-II..@-I)
| a | b |
|---+---|
| w | 1 |
| x | 2 |
|---+---|
|   |   |
|   |   |
|---+---|
| 0 | 3 |
| 0 | 4 |
|---+---|
| 0 |   |
#+TBLFM: @III=vsum(@-II..@-I)
** @IIII=vsum(@-II..@-I)
| a     | b |
|-------+---|
| w     | 1 |
| x     | 2 |
|-------+---|
|       |   |
|       |   |
|-------+---|
| y     | 3 |
| z     | 4 |
|-------+---|
| y + z |   |
#+TBLFM: @IIII=vsum(@-II..@-I)
** @II..$2=vsum(@-II..@-I)
| a                             |  b |
|-------------------------------+----|
| w                             |  1 |
| x                             |  2 |
|-------------------------------+----|
| w + x                         |  3 |
| w + x                         |  3 |
|-------------------------------+----|
| w + x + w + x                 |  6 |
| w + x + w + x                 |  6 |
|-------------------------------+----|
| w + x + w + x + w + x + w + x | 12 |
#+TBLFM: @II..$2=vsum(@-II..@-I)
** @III..$2=vsum(@-II..@-I)
| a | b |
|---+---|
| w | 1 |
| x | 2 |
|---+---|
|   |   |
|   |   |
|---+---|
| 0 | 0 |
| 0 | 0 |
|---+---|
| 0 | 0 |
#+TBLFM: @III..$2=vsum(@-II..@-I)
** @IIII..$2=vsum(@-II..@-I)
| a     | b |
|-------+---|
| w     | 1 |
| x     | 2 |
|-------+---|
|       |   |
|       |   |
|-------+---|
| y     | 3 |
| z     | 4 |
|-------+---|
| y + z | 7 |
#+TBLFM: @IIII..$2=vsum(@-II..@-I)
** @II+1..@II$2=vsum(@-II..@-I)
| a     | b |
|-------+---|
| w     | 1 |
| x     | 2 |
|-------+---|
| w + x | 3 |
|       |   |
|-------+---|
| y     | 3 |
| z     | 4 |
|-------+---|
|       |   |
#+TBLFM: @II+1..@II$2=vsum(@-II..@-I)
** @II$2..@II+1$2=vsum(@-II..@-I)
| a | b |
|---+---|
| w | 1 |
| x | 2 |
|---+---|
|   | 3 |
|   |   |
|---+---|
| y | 3 |
| z | 4 |
|---+---|
|   |   |
#+TBLFM: @II$2..@II+1$2=vsum(@-II..@-I)
:END:

Re: [BUG] hline references on left side of table formula

[Achim Gratz]

Rick Frankel writes:
> Ok, i see what's happening in your examples (a testing org file
> attached), though i question the usefullness of most of the results ;).

I tend to agree, but as the test case shows anything that does work and
produces the intended results will eventually have at least one user.
Three useful formulas that don't follow the pattern your patch is
checking for are:

#+TBLFM: @-I=vsum(@I$2..@II$2)
#+TBLFM: @-I$>=vsum(@I$2..@II$2)
#+TBLFM: @-I$>..$>=vsum(@I$2..@II$2)

i.e. when your table is structured to put the results under the last
hline.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves