[bug] Macro in citation not expanded

[bug] Macro in citation not expanded

[M. ‘quintus’ Gülker]

Dear list,

it turns out that macros in citation commands are not expanded. Minimal
working example:

/tmp/mwe.org:

    #+TITLE: Test
    #+LANGUAGE: de
    #+bibliography: /tmp/mwe.bib
    #+cite_export: csl /tmp/juristische-schulung.csl
    #+MACRO: name @@latex:\textsc{$1}@@
    
    Dies {{{name(Foo)}}}  ist ein Test. [cite:@doe2022nothing p. 55,, zum vorgenannten Argument {{{name(Doe)}}} aaO.]

/tmp/mwe.bib is:

    @Book{doe2022nothing,
    author       = {John Doe},
    title        = {Nothing Important},
    year      = {2022},
    edition   = {2},
    publisher = {Some Publisher},
    location  = {Nowhere},
    langid    = {english}}

/tmp/juristische-schulung.csl is <https://github.com/citation-style-language/styles/blob/e22b8a566bad9b4c7f52720f60dd875057a5d210/juristische-schulung.csl>.

Exporting this to LaTeX yields:

    Dies \textsc{Foo}  ist ein Test.\footnote{\textit{Doe}, Nothing important, 2. Aufl. (2022), 55, zum vorgenannten Argument \{\{\{name(Doe)\}\}\} aaO.}

This replaces the first call to the `name' macro properly, but it does
not replace the call to the `name' macro inside the `cite:' construct.
Instead, it copies the macro construct verbatim into the LaTeX footnote.
The correct output should have been:

    Dies \textsc{Foo}  ist ein Test.\footnote{\textit{Doe}, Nothing important, 2. Aufl. (2022), 55, zum vorgenannten Argument \textsc{Doe} aaO.}

Version information:

Org mode version 9.6-pre (release_9.5-1162-g15b3aa @ /home/quintus/.emacs.d/org-mode/lisp/)
citeproc.el @ ba49516265fa24b138346c4918d39d19b4de8a62
GNU Emacs 27.2 (build 1, x86_64-suse-linux-gnu, GTK+ Version 3.24.31, cairo version 1.16.0)

  -quintus

-- 
Dipl.-Jur. M. Gülker | https://mg.guelker.eu | PGP: Siehe Webseite
Passau, Deutschland  | kontakt@guelker.eu    | O<

Re: [bug] Macro in citation not expanded

[Ihor Radchenko]

M. ‘quintus’ Gülker <post+orgmodeml@guelker.eu> writes:

> it turns out that macros in citation commands are not expanded.

Confirmed.

They are not.
Only bold, code, entity, italic, latex-fragment, strike-through,
subscript, superscript, underline, and verbatim objects are allowed
inside citations.
This is hard-coded inside `org-element-object-restrictions'.

It sounds reasonable to allow macros inside citation references.
Maybe we also want to allow other things. Maybe simply allow all objects
but other citations, line-breaks, and table-cells?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [bug] Macro in citation not expanded

[M. ‘quintus’ Gülker]

Am Sonntag, dem 16. Oktober 2022 schrieb Ihor Radchenko:
> Only bold, code, entity, italic, latex-fragment, strike-through,
> subscript, superscript, underline, and verbatim objects are allowed
> inside citations.
> This is hard-coded inside `org-element-object-restrictions'.

This is a bit unfortunate, because I like to use macros for inserting
semantic formatting which is not part of org’s syntax. In the example
given in my OP, I use it to mark up names with small caps. The exact
example given includes a short note by me talking about the cited author
a little further. Another example where I need this is when I cite some
statement made in an interview, which may come out as something like:

    [cite:{{{name(Interviewee)}}} in @interviewer p. 5]

Indirect statements by persons mentioned in news articles are another
example where I would use such a construct.

I define the `name' macro in such a way that it will format
`Interviewee` in the same way as the citation style formats the author
name for @interviewer, ensuring uniformness in the created footnote.

I also use the very same `name' macro inside the actual article text, so
that uniform formatting is guaranteed there as well.

> It sounds reasonable to allow macros inside citation references.
> Maybe we also want to allow other things. Maybe simply allow all objects
> but other citations, line-breaks, and table-cells?

I have no strong opinion on this. I certainly do not need tables inside
citations. If it is easy to just permit everything except what you
named, I would say just allow it. Maybe someone has use for it. Maybe
footnotes inside [cite:] construct should also not be allowed, at least
for footnote-based styles.

  -quintus

-- 
Dipl.-Jur. M. Gülker | https://mg.guelker.eu | PGP: Siehe Webseite
Passau, Deutschland  | kontakt@guelker.eu    | O<

[RFC PATCH] Re: [bug] Macro in citation not expanded

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 903 bytes --]

M. ‘quintus’ Gülker <post+orgmodeml@guelker.eu> writes:

>> It sounds reasonable to allow macros inside citation references.
>> Maybe we also want to allow other things. Maybe simply allow all objects
>> but other citations, line-breaks, and table-cells?
>
> I have no strong opinion on this. I certainly do not need tables inside
> citations. If it is easy to just permit everything except what you
> named, I would say just allow it. Maybe someone has use for it. Maybe
> footnotes inside [cite:] construct should also not be allowed, at least
> for footnote-based styles.

See the attached WIP patch.
Note that I also removed link as our tests explicitly test against
having links inside citation references.

However, I can see how links can be useful in prefix/suffix...
I am wondering what was the rationale behind disallowing so many objects
inside citation references.


[-- Attachment #2: 0001-org-element-Allow-more-objects-inside-citations.patch --]
[-- Type: text/x-patch, Size: 1887 bytes --]

From aa3fac7248aa1201bbf92d492dcd5c79f8d0a544 Mon Sep 17 00:00:00 2001
Message-Id: <aa3fac7248aa1201bbf92d492dcd5c79f8d0a544.1666067320.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Tue, 18 Oct 2022 12:22:56 +0800
Subject: [PATCH] org-element: Allow more objects inside citations
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/org-element.el (org-element-object-restrictions): Allow all but
citation, citation-reference, line-break, table-cell, link and
footnote-reference objects inside citations.

Reported-by: M. ‘quintus’ Gülker <post+orgmodeml@guelker.eu>
Link: https://orgmode.org/list/87tu425pla.fsf@guelker.eu
---
 lisp/org-element.el | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index 7b26e877e..7aac00087 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -401,10 +401,14 @@ (defconst org-element-object-restrictions
 			     subscript superscript underline verbatim))
 	 (standard-set
 	  (remq 'citation-reference (remq 'table-cell org-element-all-objects)))
-	 (standard-set-no-line-break (remq 'line-break standard-set)))
+	 (standard-set-no-line-break (remq 'line-break standard-set))
+         (standard-set-for-citations (seq-difference
+                                      standard-set-no-line-break
+                                      '( citation citation-reference
+                                         footnote-reference link))))
     `((bold ,@standard-set)
       (citation citation-reference)
-      (citation-reference ,@minimal-set)
+      (citation-reference ,@standard-set-for-citations)
       (footnote-reference ,@standard-set)
       (headline ,@standard-set-no-line-break)
       (inlinetask ,@standard-set-no-line-break)
-- 
2.35.1


[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [RFC PATCH] Re: [bug] Macro in citation not expanded

[M. ‘quintus’ Gülker]

Dear Ihor,

thank for the WIP patch. I have applied it to current main
(f83e45526b5ec4627d601673be3680e2dece1b33), but when I now export the
document described in the OP to LaTeX, the macro is erased entirely. I
receive:

    1 Doe, Nothing important, 2. Aufl. (2022), 55, zum vorgenannten Argument aaO.

The same for HTML export.

  -quintus

-- 
Dipl.-Jur. M. Gülker | https://mg.guelker.eu | PGP: Siehe Webseite
Passau, Deutschland  | kontakt@guelker.eu    | O<

Re: [RFC PATCH] Re: [bug] Macro in citation not expanded

[Ihor Radchenko]

M. ‘quintus’ Gülker <post+orgmodeml@guelker.eu> writes:

> thank for the WIP patch. I have applied it to current main
> (f83e45526b5ec4627d601673be3680e2dece1b33), but when I now export the
> document described in the OP to LaTeX, the macro is erased entirely. I
> receive:
>
>     1 Doe, Nothing important, 2. Aufl. (2022), 55, zum vorgenannten Argument aaO.

This is to be expected. That's one of the issues we are discussing in
https://orgmode.org/list/87zgd87di9.fsf@localhost. citeproc.el (external
library) requires some trickery to handle formatting correctly, and we
need to find a way to pass all possible Org markup yet making use of the
necessary markup transformations dictated by CSL styles.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [RFC PATCH] Re: [bug] Macro in citation not expanded

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Subject: [PATCH] org-element: Allow more objects inside citations

Applied, onto main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=9ef18956b

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>