org-latex-classes with functions, incomplete doc

org-latex-classes with functions, incomplete doc

[Florian Beck]

Hi,

the docstring for `org-latex-classes' says:

"Instead of a list of sectioning commands, you can also specify
a function name.  That function will be called with two
parameters, the (reduced) level of the headline, and a predicate
non-nil when the headline should be numbered.  It must return
a format string in which the section title will be added."

This is wrong. The way this function is called in `org-latex-headline'
requires it to return a string with TWO format specifiers, e.g.
"\section{%%s}%%s\n", the second where the CONTENT of the section is
being added. Maybe `org-latex-headline' should add "%%s\n" itself – as
it does for other cases?

Also, I'm using this to add an optional argument to my sections. Can I
expect this to work? (i.e. being called in a context where the variables
`info' and `headline' are defined?)

#+BEGIN_SRC emacs-lisp
(defun fb/latex-sections (level numbered)
    (let* ((level (1- level))
           (sec-name (nth level fb/latex-section-names))
           (sec (when sec-name
                  (format "\\%s%s%s{%%s}\n%%s"
                          sec-name
                          (if numbered "" "*")
                          ;; ""
                          (or (when (plist-get info :toc-title)
                                (let ((toc-title (org-element-property :toc-title headline)))
                                  (when toc-title (format "[%s]" toc-title))))
                                   "")
                          ))))
      sec))
#+END_SRC


Org-mode version 7.9.3e (7.9.3e-961-g521d47 @ /home/flo/.emacs.d/org-mode/lisp/)
-- 
Florian Beck

Re: org-latex-classes with functions, incomplete doc

[Nicolas Goaziou]

Hello,

Florian Beck <fb@miszellen.de> writes:

> the docstring for `org-latex-classes' says:
>
> "Instead of a list of sectioning commands, you can also specify
> a function name.  That function will be called with two
> parameters, the (reduced) level of the headline, and a predicate
> non-nil when the headline should be numbered.  It must return
> a format string in which the section title will be added."
>
> This is wrong. The way this function is called in `org-latex-headline'
> requires it to return a string with TWO format specifiers, e.g.
> "\section{%%s}%%s\n", the second where the CONTENT of the section is
> being added. Maybe `org-latex-headline' should add "%%s\n" itself – as
> it does for other cases?

Indeed. It's now the case. Thanks for reporting this.

> Also, I'm using this to add an optional argument to my sections. Can I
> expect this to work? (i.e. being called in a context where the variables
> `info' and `headline' are defined?)
>
> #+BEGIN_SRC emacs-lisp
> (defun fb/latex-sections (level numbered)
>     (let* ((level (1- level))
>            (sec-name (nth level fb/latex-section-names))
>            (sec (when sec-name
>                   (format "\\%s%s%s{%%s}\n%%s"
>                           sec-name
>                           (if numbered "" "*")
>                           ;; ""
>                           (or (when (plist-get info :toc-title)
>                                 (let ((toc-title (org-element-property :toc-title headline)))
>                                   (when toc-title (format "[%s]" toc-title))))
>                                    "")
>                           ))))
>       sec))
> #+END_SRC

Actually, the proper way to do this is to define a derived back-end with
a custom headline translation function.

#+begin_src emacs-lisp
(org-export-define-derived-backend my-latex latex
  :translate-alist ((headline . fb/my-latex-headline)))

(defun fb/my-latex-headline (headline contents info)
  ...
  Do whatever you want here)
#+end_src

Also, you can use `org-export-with-backend' as a fallback case for your
custom function.

From there you can use:

  (org-export-to-buffer 'my-latex "*My own export*")

or,

  (org-export-to-file 'my-latex "some-file.tex")

You may wrap the previous calls into an interactive command (just copy
and adapt from those in ox-latex.el). For example:

#+begin_src emacs-lisp
(defun fb/my-latex-export-to-latex
  (&optional async subtreep visible-only body-only ext-plist)
  (interactive)
  (let ((outfile (org-export-output-file-name ".tex" subtreep)))
    (if async
	(org-export-async-start
	    (lambda (f) (org-export-add-to-stack f 'my-latex))
	  `(expand-file-name
	    (org-export-to-file
	     'my-latex ,outfile ,subtreep ,visible-only ,body-only ',ext-plist)))
      (org-export-to-file
       'my-latex outfile subtreep visible-only body-only ext-plist))))
#+end_src

Optionally, you can add an entry in the dispatcher for your new command:

#+begin_src emacs-lisp
(org-export-define-derived-backend my-latex latex
  :translate-alist ((headline . fb/my-latex-headline))
  :menu-entry
  (?l 2 ((?m "With my special extension" fb/my-latex-export-to-latex))))
#+end_src


Regards,

-- 
Nicolas Goaziou

Re: org-latex-classes with functions, incomplete doc

[Florian Beck]

Thanks for your explanations, very much appreciated.

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> the proper way to do this is to define a derived back-end with
> a custom headline translation function.

Ok, I tried this. There is a problem, however. This is what I came up
with:

#+BEGIN_SRC emacs-lisp
(defun fb/org-latex-headline (headline contents info)
  (let* ((full-section (org-latex-headline headline contents info))
	 (toc-title (if (plist-get info :toc-title)
			(org-element-property :toc-title headline)))
	 (section-regex "\\`\\\\\\(sub\\)*\\(section\\|paragraph\\){")
	 (new-section
	  (when (and toc-title
		     (string-match section-regex full-section))
	    (let ((subs (match-string 1 full-section))
		  (section (match-string 2 full-section))
		  (rest (substring full-section (match-end 0))))
	      (concat
	       "\\" subs section "["
	       ;; replace brackets (from `org-latex-headline')
	       (replace-regexp-in-string
		"\\[" "("
		(replace-regexp-in-string
		 "\\]" ")"
		 toc-title))
	       "]{" rest)))))
    (or new-section
	full-section)))
#+END_SRC

As you can see, the solution is much more convoluted.

The reason is that I have to parse the string returned by
`org-latex-headline' or am I missing something? I ran into a similar
problem while adding padding ("\\\\\n" -> "\\\\[0.4em]\n") to table
rows.

IMO, the probem is this: the translation is (mostly) application of
content to a template (a format string), but these templates are build
(mostly, sectioning is actually an exception) inside the default
translation functions. It would be much easier, when this template would
be accessible from "outside", like this:

#+BEGIN_SRC emacs-lisp
(defun my-org-latex-headline (headline contents info)
  (let ((sec-format 
         (plist-get info :sec-format))) ;; or something like that
    [modify sec-format]
    (plist-put info :sec-format sec-format)
    (org-latex-headline)))
#+END_SRC

The same goes for other functions.

--
Florian Beck

Re: org-latex-classes with functions, incomplete doc

[Nicolas Goaziou]

Florian Beck <fb@miszellen.de> writes:

> Ok, I tried this. There is a problem, however. This is what I came up
> with:
>
> #+BEGIN_SRC emacs-lisp
> (defun fb/org-latex-headline (headline contents info)
>   (let* ((full-section (org-latex-headline headline contents info))

I suggest (org-export-with-backend 'latex headline contents info) to not
depend on the actual name of the translator.

> 	 (toc-title (if (plist-get info :toc-title)
> 			(org-element-property :toc-title headline)))

There's no :toc-title property in the communication channel. The
exhaustive list of its properties is written in ox.el, at "The
Communication Channel" section.

> 	 (section-regex "\\`\\\\\\(sub\\)*\\(section\\|paragraph\\){")
> 	 (new-section
> 	  (when (and toc-title
> 		     (string-match section-regex full-section))
> 	    (let ((subs (match-string 1 full-section))
> 		  (section (match-string 2 full-section))
> 		  (rest (substring full-section (match-end 0))))
> 	      (concat
> 	       "\\" subs section "["
> 	       ;; replace brackets (from `org-latex-headline')
> 	       (replace-regexp-in-string
> 		"\\[" "("
> 		(replace-regexp-in-string
> 		 "\\]" ")"
> 		 toc-title))
> 	       "]{" rest)))))
>     (or new-section
> 	full-section)))
> #+END_SRC
>
> As you can see, the solution is much more convoluted.

Because you're not using the proper tool. If you just want to modify the
string returned by the `latex' back-end, use a filter. You will have
access to the transcoded headline (in LaTeX format, as a string) and the
communication channel.

#+begin_src emacs-lisp
(defun fb/my-headline-transformation (headline backend info)
  (when (eq backend 'latex)
    ;; Here HEADLINE is the output from `latex' back-end, as a string.
    ...
    ))

(add-to-list 'org-export-filter-headline-functions
             'fb/my-headline-transformation)
#+end_src

What I suggest gives you access to the headline as parsed data. This is
much more powerful, but a completely different task.

> IMO, the probem is this: the translation is (mostly) application of
> content to a template (a format string), but these templates are build
> (mostly, sectioning is actually an exception) inside the default
> translation functions. It would be much easier, when this template would
> be accessible from "outside", like this:

There are already many ways to alter output from a back-end. It's just
a matter of using the right tool.


Regards,

-- 
Nicolas Goaziou

Re: org-latex-classes with functions, incomplete doc

[Florian Beck]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

>> 	 (toc-title (if (plist-get info :toc-title)
>> 			(org-element-property :toc-title headline)))
>
> There's no :toc-title property in the communication channel. The
> exhaustive list of its properties is written in ox.el, at "The
> Communication Channel" section.

Obviouly, I defined it, otherwise it wouldn't work.

  :options-alist ((:toc-title "TOC_TITLE" nil nil t) ... )

>> As you can see, the solution is much more convoluted.
>
> Because you're not using the proper tool. If you just want to modify the
> string returned by the `latex' back-end, use a filter. You will have
> access to the transcoded headline (in LaTeX format, as a string) and the
> communication channel.

But not to the element properties, which is what I need.

> There are already many ways to alter output from a back-end. It's just
> a matter of using the right tool.

So, which is it? I'm a bit confused right now.

-- 
Florian Beck

Re: org-latex-classes with functions, incomplete doc

[Nicolas Goaziou]

Florian Beck <fb@fbeck.net> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>>> 	 (toc-title (if (plist-get info :toc-title)
>>> 			(org-element-property :toc-title headline)))
>>
>> There's no :toc-title property in the communication channel. The
>> exhaustive list of its properties is written in ox.el, at "The
>> Communication Channel" section.
>
> Obviouly, I defined it, otherwise it wouldn't work.
>
>   :options-alist ((:toc-title "TOC_TITLE" nil nil t) ... )
>
>>> As you can see, the solution is much more convoluted.
>>
>> Because you're not using the proper tool. If you just want to modify the
>> string returned by the `latex' back-end, use a filter. You will have
>> access to the transcoded headline (in LaTeX format, as a string) and the
>> communication channel.
>
> But not to the element properties, which is what I need.
>
>> There are already many ways to alter output from a back-end. It's just
>> a matter of using the right tool.
>
> So, which is it? I'm a bit confused right now.

I now get what you intend to do (or so I think).

I didn't implement this feature in ox-latex.el, mainly because a proper
implementation needs to be done at the ox.el level.

Anyway, we're back to step one: if you want to handle headlines
differently (i.e. by adding your own properties), you need to fork
`latex' back-end, as explained before. If you encounter problems, you
can post back here.


Regards,

-- 
Nicolas Goaziou

Modifying the exporter (was: org-latex-classes with functions, incomplete doc)

[Florian Beck]

Nicolas Goaziou <n.goaziou@gmail.com> writes:


> Anyway, we're back to step one: if you want to handle headlines
> differently (i.e. by adding your own properties), you need to fork
> `latex' back-end, as explained before. If you encounter problems, you
> can post back here.

Ok, I took some time to extract a minimal example. It works fine, but
on a very low level (see below).

Again, the goal is to add an optional argument to sectioning command.
The best way I could come up with is this (I omit the
`fb/org-export-pdf' function):

#+BEGIN_SRC emacs-lisp
(defun fb/org-export-modify-headline (headline string)
  (if (string-match
       (rx
	string-start "\\"
	(group-n 1 (0+ "sub"))
	(group-n 2 (or "part" "chapter" "section" "paragraph"))
	(group-n 3 (zero-or-one "\*"))
	"{" (group-n 4 (minimal-match (0+ (not (any "}"))))) "}")
       string)
      (let* ((level (match-string 1 string))
	     (type (match-string 2 string))
	     (stars (match-string 3 string))
	     (title (match-string 4 string))
	     (toc-title (org-element-property :toc-title headline))
	     (new-hl
	      (format "\\%s%s%s%s{%s}"
		      (or level "")
		      type
		      (or stars "")
		      (if toc-title (format "[%s]" toc-title) "")
		      title)))
	(replace-match new-hl t t string 0))
    string))

(defun fb/org-latex-headline (headline contents info)
  (fb/org-export-modify-headline
   headline
   (org-export-with-backend 'latex headline contents info)))

(org-export-define-derived-backend fb/org-export-pdf latex
  :translate-alist ((headline . fb/org-latex-headline))
  :options-alist ((:toc-title "TOC_TITLE" nil nil t))
  :menu-entry (?l 99 ((?d "Export PDF file" fb/org-export-pdf))))
#+END_SRC

As you can see, I pull apart the string and then put it back together.
(Relatively straightforward in this case, much more involved for, say,
links.)

In a perfect world, I would have access to these elements and the format
string, so I could either modify them before calling
`org-export-with-backend' or assemble the string myself.

-- 
Florian Beck

Re: Modifying the exporter

[Nicolas Goaziou]

Florian Beck <fb@miszellen.de> writes:

> Ok, I took some time to extract a minimal example. It works fine, but
> on a very low level (see below).
>
> Again, the goal is to add an optional argument to sectioning command.
> The best way I could come up with is this (I omit the
> `fb/org-export-pdf' function):
>
> #+BEGIN_SRC emacs-lisp
> (defun fb/org-export-modify-headline (headline string)
>   (if (string-match
>        (rx
> 	string-start "\\"
> 	(group-n 1 (0+ "sub"))
> 	(group-n 2 (or "part" "chapter" "section" "paragraph"))
> 	(group-n 3 (zero-or-one "\*"))
> 	"{" (group-n 4 (minimal-match (0+ (not (any "}"))))) "}")
>        string)
>       (let* ((level (match-string 1 string))
> 	     (type (match-string 2 string))
> 	     (stars (match-string 3 string))
> 	     (title (match-string 4 string))
> 	     (toc-title (org-element-property :toc-title headline))
> 	     (new-hl
> 	      (format "\\%s%s%s%s{%s}"
> 		      (or level "")
> 		      type
> 		      (or stars "")
> 		      (if toc-title (format "[%s]" toc-title) "")
> 		      title)))
> 	(replace-match new-hl t t string 0))
>     string))

Why don't you simply replace (or add, if needed) optional argument from
sectioning command with `replace-regexp-in-string' instead of building
the whole string again? Like the following (untested):

#+begin_src emacs-lisp
(defun fb/org-latex-headline (headline contents info)
    (let ((original-hl (org-export-with-backend 'latex headline contents info))
          (toc-title (org-element-property :toc-title headline)))
      (cond ((not toc-title) original-hl)
            ((string-match "\\`\\.*?\\*?\\[\\([^]]\\)\\]" original-hl)
             (replace-match toc-title nil nil original-hl 1))
            (t (replace-regexp-in-string
                "\\`\\.*?\\*?" (concat "\\&" toc-title) original-hl)))))
#+end_src

> As you can see, I pull apart the string and then put it back together.
> (Relatively straightforward in this case, much more involved for, say,
> links.)

Writing translators for headlines links and tables is usually a major
undertaking when creating a new back-end. That's why there are so many
tools to operate on these elements in ox.el.

Anyway, this is not bad, as you forked a 3k locs back-end with about 30
locs.

> In a perfect world, I would have access to these elements and the format
> string, so I could either modify them before calling
> `org-export-with-backend' or assemble the string myself.

In your perfect world, there would be billions of variables, depending
on the element, the back-end and the task. Customization is still
important, so I offer generic tools when variables are not enough.

Note that I might implement this feature at one point, but I still have
to think about its specifications.


Regards,

-- 
Nicolas Goaziou