quotation marks in table cell vs. org-babel-ref-resolve

quotation marks in table cell vs. org-babel-ref-resolve

[Greg Minshall]

hi.  i'm trying to use =org-babel-ref-resolve= to access the values of a
column in a table.  it all works fine for many cases.  however, if the
contents of an entry in the column starts with quotation marks, those
quotation marks disappear, and anything outside the quoted part of the
cell "disappears" (in the result).  see below.

would this be intentional?  a bug?

or, if there's a better way to retrieve the contents of a table column,
i'd love to hear.

cheers, Greg

ps -- it doesn't seem to matter that "35" can be considered an integer;
i.e., "a35" seems to behave the same.

----
#+name: foo
| this | is |  a | header                           |
|------+----+----+----------------------------------|
|   33 | 42 | 32 | "35"                             |
|   33 | 42 | 32 | "35" thirtyfive                  |
|   33 | 42 | 32 | 35 thirtyfive                    |
|   33 | 42 | 32 | thirtyfive "35"                  |
|   33 | 42 | 32 | thirtyfive "35" post-thirty-five |

#+begin_src elisp
  (org-babel-ref-resolve "foo")
#+end_src

#+RESULTS:
| this | is |  a | header                           |
|------+----+----+----------------------------------|
|   33 | 42 | 32 | 35                               |
|   33 | 42 | 32 | 35                               |
|   33 | 42 | 32 | 35 thirtyfive                    |
|   33 | 42 | 32 | thirtyfive "35"                  |
|   33 | 42 | 32 | thirtyfive "35" post-thirty-five |

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Greg Minshall]

fwiw, tracing, the problem appears to be this line
----
((eq (string-to-char cell) ?\") (read cell))
----
in =org-babel-read=.

presumably there are many cases where this is the right thing to do.

but, maybe look for a simple =^"[^"]*"$= (i.e., a quotation mark, some
other stuff, a quotation mark, and *nothing* else in the cell)?

or (heaven help us), something more complicated, with random escaped
quotation marks inside the "cell"?

cheers, Greg

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Tim Cross]

Greg Minshall <minshall@umich.edu> writes:

> fwiw, tracing, the problem appears to be this line
> ----
> ((eq (string-to-char cell) ?\") (read cell))
> ----
> in =org-babel-read=.
>
> presumably there are many cases where this is the right thing to do.
>
> but, maybe look for a simple =^"[^"]*"$= (i.e., a quotation mark, some
> other stuff, a quotation mark, and *nothing* else in the cell)?
>
> or (heaven help us), something more complicated, with random escaped
> quotation marks inside the "cell"?
>
> cheers, Greg

The key question is what is the use case for having this 'mixed' content
in a table cell?

I really don't like the idea of adding even more regexp processing of
contents in order to resolve complexities in data formats which are
rare.  the benefit of the line you identified is that it is simple and
easy to maintain. There may be edge cases where it does not work
correctly, but perhaps the right solution in those situations is fix the
format of the data in the table rather than complicate the functions
used to extract the data from a table.

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Greg Minshall]

hi, Tim,

> The key question is what is the use case for having this 'mixed' content
> in a table cell?

in my case, i am putting RFC822('ish) e-mail addresses in a column of an
org-mode table.  and, i want to extract them.
----
| oxymoron@example.com                      |
| Greg Oxymoron <oxymoron@example.com>      |
| "Greg G. Oxymoron" <oxymoron@example.com> |
----
for the third row returns =Greg G. Oxymoron=, rather than my desired
="Greg G. Oxymoron" <oxymoron@example.com>=.

by the way, do you know the use case for the current behavior for
strings that start with a ="=?  i couldn't find anything in the manual.

i wonder if maybe the existing parameter =inhibit-lisp-eval= (which, in
the path i am exercising, is non-nil) could also be used to not do the
check for a ="=?  (maybe that's also a hack, but i think it would solve
my problem. :)

cheers, Greg

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Tim Cross]

Greg Minshall <minshall@umich.edu> writes:

> hi, Tim,
>
>> The key question is what is the use case for having this 'mixed' content
>> in a table cell?
>
> in my case, i am putting RFC822('ish) e-mail addresses in a column of an
> org-mode table.  and, i want to extract them.
> ----
> | oxymoron@example.com                      |
> | Greg Oxymoron <oxymoron@example.com>      |
> | "Greg G. Oxymoron" <oxymoron@example.com> |
> ----
> for the third row returns =Greg G. Oxymoron=, rather than my desired
> ="Greg G. Oxymoron" <oxymoron@example.com>=.
>
> by the way, do you know the use case for the current behavior for
> strings that start with a ="=?  i couldn't find anything in the manual.
>
> i wonder if maybe the existing parameter =inhibit-lisp-eval= (which, in
> the path i am exercising, is non-nil) could also be used to not do the
> check for a ="=?  (maybe that's also a hack, but i think it would solve
> my problem. :)
>

I don't know. It could be related to the spreadsheet capabilities or it
could simply be an oversight in how the code extracts values from
tables. 

I tend to use the function org-table-to-list to extract the data from a
table. It gives me a nested list which I can then process with elisp in
any way I want. I don't know if that would help or how it will interpret
a cell whic contains both quoted and unquoted data.

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Greg Minshall]

sigh.

the following is pretty much a lie.  apologies.

> =org-babel-ref-resolve= is convenient to use (as you provide a REF,
> rather than create a [temp] buffer, visit the file, whatever).

the creating a temp buffer, visiting the file, are *not* provided by
=org-babel-ref-resolve=.

cheers (or, cheerless's :), Greg

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Greg Minshall]

hi, Tim,

thanks for the reply.

> I don't know. It could be related to the spreadsheet capabilities or it
> could simply be an oversight in how the code extracts values from
> tables.

if anyone has any knowledge in this area, i'd be curious to hear.

> I tend to use the function org-table-to-list to extract the data from a
> table. It gives me a nested list which I can then process with elisp in
> any way I want. I don't know if that would help or how it will interpret
> a cell whic contains both quoted and unquoted data.

for the record, things -- lists and lisps -- being equal, the function
you presumably meant to write was =org-table-to-lisp=.

=org-babel-ref-resolve= is convenient to use (as you provide a REF,
rather than create a [temp] buffer, visit the file, whatever).  but, it
isn't clear to me what functions (=org-babel-ref-resolve= and/or
=org-table-to-lisp=) should be considered part of the "Emacs Org Mode
API". :)

(i've gotten "around" the issue by prepending my e-mail addresses with
[: SPACE].)

cheers, Greg

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Greg Minshall]

i have a problem similar to this from a few months ago.  in this case,
it seems to be when trying to pass a column with a quoted-string inside
of it.  (this is on the "bugfix" branch.)  my e-lisp code seems to get
just the initial part of the string from the cell with "odd" contents.

----
#+name: xl
| this, "he said" | is where, | "you go in" there |

#+begin_src elisp :var xl=xl
  (message "%S" xl)
#+end_src
----

when i execute this, *Messages* says
----
(("this, \"he said\"" "is where," "you go in"))
Code block evaluation complete.
----
(my complaint: there's no there there. :)

in December, when i reported something similar, Tim pushed back, and
maybe i should elaborate:

> The key question is what is the use case for having this 'mixed'
> content in a table cell?

for better or worse, i most often use org-mode tables as (storage and
data entry mechanisms for) simple data bases.  other than the pipe
symbol ('|'), i assume i can put "anything" in a cell, and extract it
later.  i find this very valuable.

normally, i pass such tables to some R code (this example, btw, breaks
passing to R code).  but occasionally i do other things with them.

the behavior i'm seeing seems to me inconsistent, indicating it is maybe
not intentional.  exporting, for example, at least to html. i see the
full contents of the third column.

cheers, Greg

Re: quotation marks in table cell vs. org-babel-ref-resolve

[Ihor Radchenko]

Greg Minshall <minshall@umich.edu> writes:

> hi, Tim,
>
>> The key question is what is the use case for having this 'mixed' content
>> in a table cell?
>
> in my case, i am putting RFC822('ish) e-mail addresses in a column of an
> org-mode table.  and, i want to extract them.
> ----
> | oxymoron@example.com                      |
> | Greg Oxymoron <oxymoron@example.com>      |
> | "Greg G. Oxymoron" <oxymoron@example.com> |
> ----
> for the third row returns =Greg G. Oxymoron=, rather than my desired
> ="Greg G. Oxymoron" <oxymoron@example.com>=.

Fixed.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=633ca9e69e57e68a42c893999488e9c67dac49b8

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>