Help with export filter?

Help with export filter?

[Thomas S. Dye]

Aloha all,

Inspired by discussions and code on the mailing list, I managed to
cobble together the headline filter below.  It "works" in that the
pdf output from LaTeX export is exactly what I want.  I'm thrilled!

It has one unwanted side effect.  In the tex file, a headline tagged
with either :newpage: or :clearpage: includes some extra baggage, like
this:

  \newpage
  \section*{Introduction\hfill{}\textsc{}}
  \label{sec-5}

I tried setting the option tags:nil, but then my export tags had no
effect.  Can someone suggest how I can avoid the \hfill etc.?  Or, am I
picking nits here?

***** Filter headline tags

#+name: filter-headline-tags
#+BEGIN_SRC emacs-lisp :results silent
  (defun tsd-filter-headline-tags (contents backend info)
    "Ignore headlines with tag `ignoreheading' and/or start LaTeX
  section with `newpage' or `clearpage' command."
    (cond ((and (org-export-derived-backend-p backend 'latex)
                (string-match "\\`.*newpage.*\n" (downcase contents))
                (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
           (replace-match "\\\\newpage\n" nil nil contents))
          ((and (org-export-derived-backend-p backend 'latex)
                (string-match "\\`.*clearpage.*\n" (downcase contents))
                (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
           (replace-match "\\\\clearpage\n" nil nil contents))
          ((and (org-export-derived-backend-p backend 'latex 'html 'ascii)
                (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
           (replace-match "" nil nil contents))
          ((and (org-export-derived-backend-p backend 'latex)
                (string-match "\\(\\`.*\\)newpage\\(.*\n\\)" (downcase contents)))
           (replace-match "\\\\newpage\n\\1\\2"  nil nil contents))
          ((and (org-export-derived-backend-p backend 'latex)
                (string-match "\\(\\`.*\\)clearpage\\(.*\n\\)" (downcase contents)))
           (replace-match "\\\\clearpage\n\\1\\2"  nil nil contents))))
#+END_SRC

All the best,
Tom


-- 
T.S. Dye & Colleagues, Archaeologists
735 Bishop St, Suite 315, Honolulu, HI 96813
Tel: 808-529-0866, Fax: 808-529-0884
http://www.tsdye.com

Re: Help with export filter?

[Rasmus]

Hi Tom,

tsd@tsdye.com (Thomas S. Dye) writes:

> Aloha all,
>
> Inspired by discussions and code on the mailing list, I managed to
> cobble together the headline filter below.  It "works" in that the
> pdf output from LaTeX export is exactly what I want.  I'm thrilled!
>
> It has one unwanted side effect.  In the tex file, a headline tagged
> with either :newpage: or :clearpage: includes some extra baggage, like
> this:
>
>   \newpage
>   \section*{Introduction\hfill{}\textsc{}}
>   \label{sec-5}
>
> I tried setting the option tags:nil, but then my export tags had no
> effect.  Can someone suggest how I can avoid the \hfill etc.?  Or, am I
> picking nits here?
>
> ***** Filter headline tags
>
> #+name: filter-headline-tags
> #+BEGIN_SRC emacs-lisp :results silent
>
>   (defun tsd-filter-headline-tags (contents backend info)
>     "Ignore headlines with tag `ignoreheading' and/or start LaTeX
>   section with `newpage' or `clearpage' command."
>     (cond ((and (org-export-derived-backend-p backend 'latex)
>                 (string-match "\\`.*newpage.*\n" (downcase contents))
>                 (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
>            (replace-match "\\\\newpage\n" nil nil contents))
>           ((and (org-export-derived-backend-p backend 'latex)
>                 (string-match "\\`.*clearpage.*\n" (downcase contents))
>                 (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
>            (replace-match "\\\\clearpage\n" nil nil contents))
>           ((and (org-export-derived-backend-p backend 'latex 'html 'ascii)
>                 (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
>            (replace-match "" nil nil contents))
>           ((and (org-export-derived-backend-p backend 'latex)
>                 (string-match "\\(\\`.*\\)newpage\\(.*\n\\)" (downcase contents)))
>            (replace-match "\\\\newpage\n\\1\\2"  nil nil contents))
>           ((and (org-export-derived-backend-p backend 'latex)
>                 (string-match "\\(\\`.*\\)clearpage\\(.*\n\\)" (downcase contents)))
>            (replace-match "\\\\clearpage\n\\1\\2"  nil nil contents))))
> #+END_SRC

Here's an edited filter that works with the enclosed test document.
Note that you could split this into different filters, but it may be
non-trivial with this identification-scheme since things would be
moving around.

(defun tsd-filter-headline-tags (contents backend info)
  "Ignore headlines with tag `ignoreheading' and/or start LaTeX
  section with `newpage' or `clearpage' command."
  (cond ((and (org-export-derived-backend-p backend 'latex)
	      (string-match "\\`.*newpage.*\n" (downcase contents))
	      ;; if you want to get rid of labels use the string
	      ;; "\\`.*ignoreheading.*\n.*\n"
	      (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
	 (replace-match "\\\\newpage\n" nil nil contents))
	((and (org-export-derived-backend-p backend 'latex)
	      (string-match "\\`.*clearpage.*\n" (downcase contents))
	      (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
	 (replace-match "\\\\clearpage\n" nil nil contents))
	((and (org-export-derived-backend-p backend 'latex 'html 'ascii)
	      (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
	 (replace-match "" nil nil contents))
	((and (org-export-derived-backend-p backend 'latex)
	      (string-match "\\(\\`.*?\\)\\(?:\\\\hfill{}\\)?\\\\textsc{.*?newpage.*?}\\(.*\n\\)"
			    (downcase contents)))
	 (replace-match "\\\\newpage\n\\1\\2"  nil nil contents))
	((and (org-export-derived-backend-p backend 'latex)
	      (string-match "\\(\\`.*?\\)\\(?:\\\\hfill{}\\)?\\\\textsc{.*?clearpage.*?}\\(.*\n\\)" (downcase contents)))
	 (replace-match "\\\\clearpage\n\\1\\2"  nil nil contents))))

(add-to-list 'org-export-filter-headline-functions 'tsd-filter-headline-tags)

* my first headline
  cont0
* ignored headline					      :ignoreheading:
  cont1 (ignored headline)
* heading with newpage						    :newpage:
  newline before *here*
* heading with clearpage					  :clearpage:
  clearpage before *here*
* ignored heading with newpage				       :newpage:tag2:
  newline before *here*.  =tag2= is lost.
* ignore heading with clearpage			    :ignoreheading:clearpage:
  clearpage before *here*

—Rasmus

-- 
🐉

Re: Help with export filter?

[Thomas S. Dye]

Aloha Rasmus,

Rasmus <rasmus@gmx.us> writes:

> Hi Tom,
>
>
> Here's an edited filter that works with the enclosed test document.
> Note that you could split this into different filters, but it may be
> non-trivial with this identification-scheme since things would be
> moving around.
>
> (defun tsd-filter-headline-tags (contents backend info)
>   "Ignore headlines with tag `ignoreheading' and/or start LaTeX
>   section with `newpage' or `clearpage' command."
>   (cond ((and (org-export-derived-backend-p backend 'latex)
> 	      (string-match "\\`.*newpage.*\n" (downcase contents))
> 	      ;; if you want to get rid of labels use the string
> 	      ;; "\\`.*ignoreheading.*\n.*\n"
> 	      (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
> 	 (replace-match "\\\\newpage\n" nil nil contents))
> 	((and (org-export-derived-backend-p backend 'latex)
> 	      (string-match "\\`.*clearpage.*\n" (downcase contents))
> 	      (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
> 	 (replace-match "\\\\clearpage\n" nil nil contents))
> 	((and (org-export-derived-backend-p backend 'latex 'html 'ascii)
> 	      (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
> 	 (replace-match "" nil nil contents))
> 	((and (org-export-derived-backend-p backend 'latex)
> 	      (string-match "\\(\\`.*?\\)\\(?:\\\\hfill{}\\)?\\\\textsc{.*?newpage.*?}\\(.*\n\\)"
> 			    (downcase contents)))
> 	 (replace-match "\\\\newpage\n\\1\\2"  nil nil contents))
> 	((and (org-export-derived-backend-p backend 'latex)
> 	      (string-match "\\(\\`.*?\\)\\(?:\\\\hfill{}\\)?\\\\textsc{.*?clearpage.*?}\\(.*\n\\)" (downcase contents)))
> 	 (replace-match "\\\\clearpage\n\\1\\2"  nil nil contents))))
>
> (add-to-list 'org-export-filter-headline-functions 'tsd-filter-headline-tags)
>
> * my first headline
>   cont0
> * ignored headline					      :ignoreheading:
>   cont1 (ignored headline)
> * heading with newpage						    :newpage:
>   newline before *here*
> * heading with clearpage					  :clearpage:
>   clearpage before *here*
> * ignored heading with newpage				       :newpage:tag2:
>   newline before *here*.  =tag2= is lost.
> * ignore heading with clearpage			    :ignoreheading:clearpage:
>   clearpage before *here*
>
> —Rasmus

Thanks very much for your help.  The new filter does just what I want.

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: Help with export filter?

[Samuel Loury]

[-- Attachment #1: Type: text/plain, Size: 2607 bytes --]

Hi all,
Rasmus <rasmus@gmx.us> writes:

> Hi Tom,
>
> tsd@tsdye.com (Thomas S. Dye) writes:
>
>> Aloha all,
>>
>> Inspired by discussions and code on the mailing list, I managed to
>> cobble together the headline filter below.  It "works" in that the
>> pdf output from LaTeX export is exactly what I want.  I'm thrilled!
>>
>> It has one unwanted side effect.  In the tex file, a headline tagged
>> with either :newpage: or :clearpage: includes some extra baggage, like
>> this:
>>
>>   \newpage
>>   \section*{Introduction\hfill{}\textsc{}}
>>   \label{sec-5}
>>
>> I tried setting the option tags:nil, but then my export tags had no
>> effect.  Can someone suggest how I can avoid the \hfill etc.?  Or, am I
>> picking nits here?
>>
>> ***** Filter headline tags
>>
>> #+name: filter-headline-tags
>> #+BEGIN_SRC emacs-lisp :results silent
>>
>>   (defun tsd-filter-headline-tags (contents backend info)
>>     "Ignore headlines with tag `ignoreheading' and/or start LaTeX
>>   section with `newpage' or `clearpage' command."
>>     (cond ((and (org-export-derived-backend-p backend 'latex)
>>                 (string-match "\\`.*newpage.*\n" (downcase contents))
>>                 (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
>>            (replace-match "\\\\newpage\n" nil nil contents))
>>           ((and (org-export-derived-backend-p backend 'latex)
>>                 (string-match "\\`.*clearpage.*\n" (downcase contents))
>>                 (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
>>            (replace-match "\\\\clearpage\n" nil nil contents))
>>           ((and (org-export-derived-backend-p backend 'latex 'html 'ascii)
>>                 (string-match "\\`.*ignoreheading.*\n" (downcase contents)))
>>            (replace-match "" nil nil contents))
>>           ((and (org-export-derived-backend-p backend 'latex)
>>                 (string-match "\\(\\`.*\\)newpage\\(.*\n\\)" (downcase contents)))
>>            (replace-match "\\\\newpage\n\\1\\2"  nil nil contents))
>>           ((and (org-export-derived-backend-p backend 'latex)
>>                 (string-match "\\(\\`.*\\)clearpage\\(.*\n\\)" (downcase contents)))
>>            (replace-match "\\\\clearpage\n\\1\\2"  nil nil contents))))
>> #+END_SRC

This I just the feature I need today. Thank you for sharing. Do you know
what is missing to add the feature to org-mode instead of adding in each
user's .emacs file?

My best,
-- 
Konubinix
GPG Key    : 7439106A
Fingerprint: 5993 BE7A DA65 E2D9 06CE  5C36 75D2 3CED 7439 106A

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: Help with export filter?

[Nicolas Goaziou]

Hello

Samuel Loury <konubinix@gmail.com> writes:

> This I just the feature I need today. Thank you for sharing. Do you know
> what is missing to add the feature to org-mode instead of adding in each
> user's .emacs file?

This function apparently includes many features. Which one are you
talking about?

The :ignoreheading: part has been discussed already on this ML (and it's
a FAQ, IIRC). The major problem is that it is not clearly defined as
a parse tree operation.


Regards,

-- 
Nicolas Goaziou

Re: Help with export filter?

[Samuel Loury]

[-- Attachment #1: Type: text/plain, Size: 1608 bytes --]

Dear Nicolas,

Thank you for the prompt reply.
Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Samuel Loury <konubinix@gmail.com> writes:
>
>> This I just the feature I need today. Thank you for sharing. Do you know
>> what is missing to add the feature to org-mode instead of adding in each
>> user's .emacs file?
>
> This function apparently includes many features. Which one are you
> talking about?

I used the clearpage and newpage features. The only other way I found
out was to add a bit of latex just before the heading.

--8<---------------cut here---------------start------------->8---
* My heading
  Some content with images.
@@latex:\clearpage@@
* My other heading, totally cleared
  some other content
--8<---------------cut here---------------end--------------->8---

What I dislike with this way of doing is that the latex fragment is
inside the heading above while I would prefer to associate it with the
second heading. The semantic in my case is "I am starting a brand new
topic, clear the floats from the last topic so that they won't pollute
my new topic".

I find the solution with the code of Thomas S. Dye very elegant:
--8<---------------cut here---------------start------------->8---
* My heading
  Some content with images.
* My other heading, totally cleared             :clearpage:
  some other content
--8<---------------cut here---------------end--------------->8---

I hope I clarified the use and the what for.

Best regards,
-- 
Konubinix
GPG Key    : 7439106A
Fingerprint: 5993 BE7A DA65 E2D9 06CE  5C36 75D2 3CED 7439 106A

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: Help with export filter?

[Nicolas Goaziou]

Samuel Loury <konubinix@gmail.com> writes:

> I find the solution with the code of Thomas S. Dye very elegant:
> * My heading
>   Some content with images.
> * My other heading, totally cleared             :clearpage:
>   some other content
>
> I hope I clarified the use and the what for.

AFAICT, this is a very specific, cosmetic, feature targeting a single
family of export back-ends (i.e., LaTeX and derivatives).

I see nothing wrong to letting users implement it themselves, as they
see fit (e.g., one could prefer using a less visible node property for
that).

Regards,

Re: Help with export filter?

[Samuel Loury]

[-- Attachment #1: Type: text/plain, Size: 829 bytes --]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Samuel Loury <konubinix@gmail.com> writes:
>
>> I find the solution with the code of Thomas S. Dye very elegant:
>> * My heading
>>   Some content with images.
>> * My other heading, totally cleared             :clearpage:
>>   some other content
>>
>> I hope I clarified the use and the what for.
>
> AFAICT, this is a very specific, cosmetic, feature targeting a single
> family of export back-ends (i.e., LaTeX and derivatives).
>
> I see nothing wrong to letting users implement it themselves, as they
> see fit (e.g., one could prefer using a less visible node property for
> that).

I agree with you.

Thank you for taking the time to answer.

-- 
Konubinix
GPG Key    : 7439106A
Fingerprint: 5993 BE7A DA65 E2D9 06CE  5C36 75D2 3CED 7439 106A

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: Help with export filter?

[Rasmus]

Samuel Loury <konubinix@gmail.com> writes:

> Dear Nicolas,
>
> Thank you for the prompt reply.
> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> Samuel Loury <konubinix@gmail.com> writes:
>>
>>> This I just the feature I need today. Thank you for sharing. Do you know
>>> what is missing to add the feature to org-mode instead of adding in each
>>> user's .emacs file?
>>
>> This function apparently includes many features. Which one are you
>> talking about?
>
> I used the clearpage and newpage features. The only other way I found
> out was to add a bit of latex just before the heading.

As Nicolas says, we'd need to think carefully as to how to incorporate
such things across backends for it to be worthwhile to include.  For
clearpage you'd have something like "\f" in ascii and there's an odt
equivalent as well.  For html it's less clear.

New features should be implemented like headline properties.

I would still like to see a proper ignoreheading, but it's a proper
definition of this behavior is hard.

Note: For all Tom's examples it would be enough to have a way to insert
arbitrary code just before (and maybe just after) a headline.  An
"export_before_BACKEND" property, say.

Rasmus

-- 
Vote for Dick Taid in an election near you!