[BUG] org-table-eval-formula mishandles semicolons in formulas [9.7-pre (release_9.6.14-1019-g5092ec @ /home/nick/src/emacs/org/org-mode/lisp/)]

[BUG] org-table-eval-formula mishandles semicolons in formulas [9.7-pre (release_9.6.14-1019-g5092ec @ /home/nick/src/emacs/org/org-mode/lisp/)]

[Nick Dokos]

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

     https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.
------------------------------------------------------------------------

`org-table-eval-formula' interprets *any* semicolon in a formula as the
beginning of a format specifier. So if the formula contains a literal
semicolon that is not a format specifier

    #+TBLFM: $3='(concat "COMMENT ON public.layout_forms." $1 " IS " (rc-sq $2) ";")

The problem is that `org-table-eval-formula' decides that the semicolon
is the beginning of a format specifier very early in the function and
splits off the "specifier" from the rest of the formula, leading to
errors when the truncated formula is evaluated later.

ThHere is a semi-reasonable workaround in that the regexp matches the
*last* semicolon, so we can add a semicolon at the end of the formula,
specifying an empty format, but the formula is not truncated any longer.
If it is decided that the code must not be changed, then maybe the
workaround could be added to the documentation (perhaps as a footnote to
section "Emacs Lisp forms as formulas").

This was reported by user `vfclists' on Emacs SE:

    https://emacs.stackexchange.com/questions/80062/is-it-because-this-org-table-formula-contains-double-quotes-it-doesnt-work
    

Emacs  : GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.17.8)
 of 2023-12-28
Package: Org mode version 9.7-pre (release_9.6.14-1019-g5092ec @ /home/nick/src/emacs/org/org-mode/lisp/)
-- 
Nick

Re: [BUG] org-table-eval-formula mishandles semicolons in formulas [9.7-pre (release_9.6.14-1019-g5092ec @ /home/nick/src/emacs/org/org-mode/lisp/)]

[Ihor Radchenko]

Nick Dokos <ndokos@gmail.com> writes:

> `org-table-eval-formula' interprets *any* semicolon in a formula as the
> beginning of a format specifier. So if the formula contains a literal
> semicolon that is not a format specifier
>
>     #+TBLFM: $3='(concat "COMMENT ON public.layout_forms." $1 " IS " (rc-sq $2) ";")
>
> The problem is that `org-table-eval-formula' decides that the semicolon
> is the beginning of a format specifier very early in the function and
> splits off the "specifier" from the rest of the formula, leading to
> errors when the truncated formula is evaluated later.

Confirmed.
Important.

> ThHere is a semi-reasonable workaround in that the regexp matches the
> *last* semicolon, so we can add a semicolon at the end of the formula,
> specifying an empty format, but the formula is not truncated any longer.
> If it is decided that the code must not be changed, then maybe the
> workaround could be added to the documentation (perhaps as a footnote to
> section "Emacs Lisp forms as formulas").
>
> This was reported by user `vfclists' on Emacs SE:
>     https://emacs.stackexchange.com/questions/80062/is-it-because-this-org-table-formula-contains-double-quotes-it-doesnt-work

We should eventually fix this bug. I do not think that we should add
workarounds into the manual.

The problem is that org-table.el code is fragile. A proper fix would
require rewriting formula parsing code to something more reliable.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-table-eval-formula mishandles semicolons in formulas [9.7-pre (release_9.6.14-1019-g5092ec @ /home/nick/src/emacs/org/org-mode/lisp/)]

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Nick Dokos <ndokos@gmail.com> writes:
>
>> `org-table-eval-formula' interprets *any* semicolon in a formula as the
>> beginning of a format specifier. So if the formula contains a literal
>> semicolon that is not a format specifier
>>
>>     #+TBLFM: $3='(concat "COMMENT ON public.layout_forms." $1 " IS " (rc-sq $2) ";")
>>
>> The problem is that `org-table-eval-formula' decides that the semicolon
>> is the beginning of a format specifier very early in the function and
>> splits off the "specifier" from the rest of the formula, leading to
>> errors when the truncated formula is evaluated later.
>
> Confirmed.
> Important.

Fixed, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=5af062511

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-table-eval-formula mishandles semicolons in formulas [9.7-pre (release_9.6.14-1019-g5092ec @ /home/nick/src/emacs/org/org-mode/lisp/)]

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

>>>     #+TBLFM: $3='(concat "COMMENT ON public.layout_forms." $1 " IS " (rc-sq $2) ";")
> ...
> Fixed, on main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=5af062511

Alas. We cannot restrict what to put after ";" in formulas, because it
can be interpreted as format spec for `format':

|  a |  b | percent of a in b |
|----+----+-------------------|
| 10 | 20 | #ERROR            |
| 20 | 30 | #ERROR            |
#+TBLFM: $3=($1/$2)*100;%.2f%%

So, your example 
#+TBLFM: $3='(concat "COMMENT ON public.layout_forms." $1 " IS " (rc-sq $2) ";")

is actually ambiguous - it may also be interpreted as ") format specifier.

You can solve the problem by putting an extra ; at the end of the
formula:
#+TBLFM: $3='(concat "COMMENT ON public.layout_forms." $1 " IS " (rc-sq $2) ";");

I added an example explaining this caveat to the manual.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=6799350a8

| 1 | 1; |
#+tblfm: $2='(concat $1 ";");

| 1 | ") |
#+tblfm: $2='(concat $1 ";")

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>