New worg page: Publishing beamer slideshows and articles from one source

New worg page: Publishing beamer slideshows and articles from one source

[James Harkins]

Hi all,

Some weeks ago, I wrote here about a large project (training materials for 
the SuperCollider audio/music programming language), written entirely in 
org using the Beamer exporter to produce both slideshows and an 
article-format book for the students to use as a reference.

I just put a page up on worg describing the implementation details:

http://orgmode.org/worg/exporters/beamer/beamer-dual-format.html

The project lives at https://github.com/jamshark70/scweek2014 -- note that 
the material itself is under a Creative Commons license, CC-BY-SA. As far 
as I'm concerned, if you want to use any of the Emacs or LaTeX code, knock 
yourself out, but please don't republish the written material in any way 
that's contrary to CC-BY-SA.

Hope this is of use,
hjh

Re: New worg page: Publishing beamer slideshows and articles from one source

[Suvayu Ali]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

Hi James,

On Sat, May 10, 2014 at 10:57:11AM +0800, James Harkins wrote:
> 
> http://orgmode.org/worg/exporters/beamer/beamer-dual-format.html

It is a great resource.  A couple of comments about formatting: the
first two footnotes might be better as links from the text.  I propose
the attached patch.  If you think this is fine, I'll push it.

BTW, the ignoreheading thing has been asked so many times, maybe I
should put it in an FAQ.  But then, the content is more appropriate for
org-hacks ... I'm undecided.  Any thoughts?

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

[-- Attachment #2: 0001-Minor-formatting-change.patch --]
[-- Type: text/plain, Size: 2436 bytes --]

From 514178abdba0522ab4c44b6daa2cdbb80ddad9c5 Mon Sep 17 00:00:00 2001
From: Suvayu Ali <fatkasuvayu+linux@gmail.com>
Date: Mon, 12 May 2014 21:15:05 +0200
Subject: [PATCH] Minor formatting change

---
 exporters/beamer/beamer-dual-format.org | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/exporters/beamer/beamer-dual-format.org b/exporters/beamer/beamer-dual-format.org
index 3750035..e339088 100644
--- a/exporters/beamer/beamer-dual-format.org
+++ b/exporters/beamer/beamer-dual-format.org
@@ -495,8 +495,8 @@ Rather than create a document class to turn top-level headings into =\part=
 commands, I embedded the LaTeX code for it directly into the full-article 
 template. The trick is closing the environments for the previous sections. 
 This requires a top-level heading, which should not start a new section. I 
-found that =:B_ignoreheading:= did not work for this, but an export filter 
-by Suvayu Ali[fn:54e951f5] did exactly what I needed.
+found that =:B_ignoreheading:= did not work for this, but an [[http://stackoverflow.com/questions/10295177/is-there-an-equivalent-of-org-modes-b-ignoreheading-for-non-beamer-documents][export filter]]
+by Suvayu Ali did exactly what I needed.
 
 #+name: articleParts
 #+caption: Part of the full-article export document, with embedded LaTeX \part syntax.
@@ -535,7 +535,7 @@ are put into an =\mbox=, to suppress hyphenation.)
 Org's formatting markup is visual: asterisks for bold, slashes for
 italics and so on.
 
-I think export macros[fn:5c80275b] could support semantic markup that could export
+I think [[http://orgmode.org/manual/Macro-replacement.html#Macro-replacement][export macros]] could support semantic markup that could export
 to LaTeX or HTML equally well, but I didn't investigate that in this
 project. Here, I just embedded LaTeX commands directly into the org
 files: free standing for simple uses, and using export snippets[fn:448d1164] for
@@ -544,10 +544,6 @@ which LaTeX export treats specially).
 
 * Footnotes
 
-[fn:54e951f5] http://stackoverflow.com/questions/10295177/is-there-an-equivalent-of-org-modes-b-ignoreheading-for-non-beamer-documents
-
-[fn:5c80275b] http://orgmode.org/manual/Macro-replacement.html#Macro-replacement
-
 [fn:448d1164] Export snippets look like this:
 =@@backendname:text...@@=. They will export only to that backend. You
 can write several of them in a row for different backends:
-- 
1.9.0

Re: New worg page: Publishing beamer slideshows and articles from one source

[James Harkins]

Suvayu Ali <fatkasuvayu+linux <at> gmail.com> writes:

> A couple of comments about formatting: the
> first two footnotes might be better as links from the text.  I propose
> the attached patch.  If you think this is fine, I'll push it.

Sure, go ahead.

hjh

Re: New worg page: Publishing beamer slideshows and articles from one source

[Suvayu Ali]

On Wed, May 14, 2014 at 12:56:35AM +0000, James Harkins wrote:
> Suvayu Ali <fatkasuvayu+linux <at> gmail.com> writes:
> 
> > A couple of comments about formatting: the
> > first two footnotes might be better as links from the text.  I propose
> > the attached patch.  If you think this is fine, I'll push it.
> 
> Sure, go ahead.

Done :)

-- 
Suvayu

Open source is the future. It sets us free.

Re: New worg page: Publishing beamer slideshows and articles from one source

[Bastien]

Hi Suvayu,

Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:

> BTW, the ignoreheading thing has been asked so many times, maybe I
> should put it in an FAQ.  But then, the content is more appropriate for
> org-hacks ... I'm undecided.  Any thoughts?

It's both a FAQ and a hack, perhaps put the answer in the FAQ and the
details in org-hacks?

-- 
 Bastien

Re: New worg page: Publishing beamer slideshows and articles from one source

[Suvayu Ali]

Hi Bastien,

On Wed, May 21, 2014 at 03:28:28PM +0200, Bastien wrote:
> Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:
> 
> > BTW, the ignoreheading thing has been asked so many times, maybe I
> > should put it in an FAQ.  But then, the content is more appropriate for
> > org-hacks ... I'm undecided.  Any thoughts?
> 
> It's both a FAQ and a hack, perhaps put the answer in the FAQ and the
> details in org-hacks?

Done!

<http://orgmode.org/worg/org-faq.html#ignoreheadline>
<http://orgmode.org/worg/org-hacks.html#ignoreheadline>

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: New worg page: Publishing beamer slideshows and articles from one source

[Rasmus]

Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:

> Hi Bastien,
>
> On Wed, May 21, 2014 at 03:28:28PM +0200, Bastien wrote:
>> Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:
>> 
>> > BTW, the ignoreheading thing has been asked so many times, maybe I
>> > should put it in an FAQ.  But then, the content is more appropriate for
>> > org-hacks ... I'm undecided.  Any thoughts?
>> 
>> It's both a FAQ and a hack, perhaps put the answer in the FAQ and the
>> details in org-hacks?
>
> Done!
>
> <http://orgmode.org/worg/org-faq.html#ignoreheadline>

Fine.

> <http://orgmode.org/worg/org-hacks.html#ignoreheadline>

The intention is nice, but this entry is flawed to the point that it
should be removed or undergo some heavy editing IMO. . .

Take a quick example, after initializing emacs -q as in the footnote¹:

* my heading
  p1
* ignoreheading
  p2
* ignore this heading 					      :ignoreheading:
  p3


When you export this with the proposed filter the result is
*completely broken* in plain text and html².  In html it doesn't even
manage to remove the offending headlines, much less fix the toc.  In
LaTeX it removes a headline too much, but of course it's kind of
artificial in the example above.

So if we *really* want to document this in the FAQ let us be honest:
all of our attempts so far suck badly!  The code posted only works in
LaTeX and this needs to be reflected in backend check.

Depending on the reader's use case there a better solutions.

   1. If you do not have :ignoreheading: after :noexport: a hook
      removing headlines is the best option.  You fully leverage the
      power of ox and it thus is the only trivial solution for
      non-LaTeX exporters.³
   2. If you only want to do LaTeX and you want to use the section
      filter and you do not have heading that consist only of
      =verbatim= then you can recover the ox-object representation
      from the text properties and check the actual tags.⁴
   3. If you do have =verbatim= only headings you can use a regexp.⁵

If you want to remove headings in html and text via a section filters
it's going to be pretty nontrivial as content such as the TOC is
generated by Org.

—Rasmus

Footnotes: 
¹   
(add-to-list 'load-path "/usr/share/emacs/site-lisp/org")
(require 'ox)

(defun sa-ignore-headline (contents backend info)
  "Ignore headlines with tag `ignoreheading'."
  (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii)
                  (string-match "\\`.*ignoreheading.*\n"
                                (downcase contents)))
    (replace-match "" nil nil contents)))

(add-to-list 'org-export-filter-headline-functions 'sa-ignore-headline)
(progn
  (switch-to-buffer "test.org")
  (org-mode)
  (insert-string
   "* my heading
  p1
* ignoreheading
  p2
* ignore this heading 					      :ignoreheading:
  p3"))

²
			       ━━━━━━━━━━
				TEST.ORG


				 Rasmus
			       ━━━━━━━━━━


Table of Contents
─────────────────

1 my heading
2 ignoreheading
3 ignore this heading                                    :ignoreheading:


1 my heading
════════════

  p1


═══════════════

  p2


═════════════════════

  p3


³   https://lists.gnu.org/archive/html/emacs-orgmode/2014-03/msg01459.html

⁴   https://lists.gnu.org/archive/html/emacs-orgmode/2014-03/msg01480.html

⁵   https://stackoverflow.com/questions/10295177/is-there-an-equivalent-of-org-modes-b-ignoreheading-for-non-beamer-documents

-- 
Lasciate ogni speranza, voi che leggete questo.

Re: New worg page: Publishing beamer slideshows and articles from one source

[Bastien]

Hi Rasmus,

Rasmus <rasmus@gmx.us> writes:

> The intention is nice, but this entry is flawed to the point that it
> should be removed or undergo some heavy editing IMO. . .

Yes, the sa-ignore-headline function should be updated.

Can you give it a go?

I don't see why this should not be easy: you can simply try to match
different regexps depending on the backend, and remove the correct
stuff.  The only tricky part is to remove the heading from the table
of contents, but this is optional and the limitation can be advertized
as such.

Worg is a wiki, it's all yours!

Thanks,

-- 
 Bastien

Re: New worg page: Publishing beamer slideshows and articles from one source

[Rasmus]

Bastien <bzg@gnu.org> writes:

> Hi Rasmus,
>
> Rasmus <rasmus@gmx.us> writes:
>
>> The intention is nice, but this entry is flawed to the point that it
>> should be removed or undergo some heavy editing IMO. . .
>
> Yes, the sa-ignore-headline function should be updated.
>
> Can you give it a go?

In two weeks.

> I don't see why this should not be easy: you can simply try to match
> different regexps depending on the backend, and remove the correct
> stuff.

Sure.  But the regexp that was posted fails in this regard.

> The only tricky part is to remove the heading from the table
> of contents, but this is optional and the limitation can be advertized
> as such.

Come on, that's not really acceptable for an advice on org-mode.org.

To deal with the tricky part you need an additional filter.  Or you
need a filter on the final output and you'd probably need to make some
accounting of what's present and what is not so that you can update
counters and TOCs.

—Rasmus

-- 
Together we'll stand, divided we'll fall

Re: New worg page: Publishing beamer slideshows and articles from one source

[Bastien]

Hi Rasmus,

Rasmus <rasmus@gmx.us> writes:

>> The only tricky part is to remove the heading from the table
>> of contents, but this is optional and the limitation can be advertized
>> as such.
>
> Come on, that's not really acceptable for an advice on org-mode.org.

It is better than the current situation.

> To deal with the tricky part you need an additional filter.  Or you
> need a filter on the final output and you'd probably need to make some
> accounting of what's present and what is not so that you can update
> counters and TOCs.

That's why I say it's tricky.

Let's keep this constructive and move forward -- maybe that was not
your intention, but the tone of your first email was a bit harsh.
I'm sure Suvayu is open to suggestions on how to improve the filter.

-- 
 Bastien

Re: New worg page: Publishing beamer slideshows and articles from one source

[Rasmus]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

Bastien <bzg@gnu.org> writes:

> Hi Rasmus,
>
> Rasmus <rasmus@gmx.us> writes:
>
>>> The only tricky part is to remove the heading from the table
>>> of contents, but this is optional and the limitation can be advertized
>>> as such.
>>
>> Come on, that's not really acceptable for an advice on org-mode.org.
>
> It is better than the current situation.

I disagree. Let's try to keep the quality on Worg high.  Let's not put
broken code there, especially without explaining why.

I have attached a quick patch with a text that I think is more
accurate and that tries to explain the issue.  I think the tone may be
too "academic"/dry.

Since Suvayu put the initial effort into the answer I will wait for
his approval before submitting it.

>> To deal with the tricky part you need an additional filter.  Or you
>> need a filter on the final output and you'd probably need to make some
>> accounting of what's present and what is not so that you can update
>> counters and TOCs.
>
> That's why I say it's tricky.
>
> Let's keep this constructive and move forward -- maybe that was not
> your intention, but the tone of your first email was a bit harsh.
> I'm sure Suvayu is open to suggestions on how to improve the filter.

That was not the intention.  For that I appoligize.

–Rasmus

-- 
Summon the Mothership!

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: ignore-headlines.patch --]
[-- Type: text/x-diff, Size: 4868 bytes --]

diff --git a/org-hacks.org b/org-hacks.org
index 63dfa62..2679a00 100644
--- a/org-hacks.org
+++ b/org-hacks.org
@@ -2195,30 +2195,52 @@ before you proceed.
     :CUSTOM_ID: ignoreheadline
     :END:
 #+index: Export!ignore headlines
-Sometimes users want to ignore the headline text during export.  In
-the old exporter one would use a export hook.  From Org 8.0 onwards,
-the export filter mechanism is used (see: =org-export-filters-alist=).
-
-To ignore a headline during export using special tags, you can use the
-following filter.
-#+begin_src emacs-lisp
-  (defun sa-ignore-headline (contents backend info)
-    "Ignore headlines with tag `ignoreheading'."
-    (when (and (org-export-derived-backend-p backend 'latex 'html 'ascii)
-                    (string-match "\\`.*ignoreheading.*\n"
-                                  (downcase contents)))
-      (replace-match "" nil nil contents)))
-
-  (add-to-list 'org-export-filter-headline-functions 'sa-ignore-headline)
-#+end_src
-Adding this to your configuration, you can ignore headlines by tagging
-them with =ignoreheading=.  In the above snippet, this works for
-LaTeX, HTML, and ASCII backends.  You can add more backends to the
-list by just appending the appropriate name like this: =\'backend=.
-If you would rather have this feature in all backends, then simply
-remove the =when= condition.
+Sometimes users want to ignore the headline text during export like in
+the Beamer exporter (=ox-beamer=).  In the [[http://orgmode.org/manual/Beamer-export.html#Beamer-export][Beamer exporter]] one can use
+the tag =ignoreheading= to disable the export of a certain headline,
+whilst still retaining the content of the headline.  We can imitate
+this feature in other export backends.  Note that this is not a
+particularly easy problem, as the Org exporter creates a static
+representation of section numbers, table of contents etc.
+
+Consider the following document
+#+BEGIN_SRC org
+  ,* head 1                                                           :noexport:
+  ,* head 2                                                      :ignoreheading:
+  ,* head 3
+  ,* =head 4=                                                    :ignoreheading:
 
-You can find the latest version of the filter in [[https://github.com/suvayu/.emacs.d/blob/master/org-mode-config.el][my setup on github]].
+#+END_SRC
+We want to remove heading 2 and 4.
+
+There are different strategies to accomplish this:
+1. The best option is to remove headings tagged with =ignoreheading=
+   before export starts.  This can be accomplished with the hook
+   =org-export-before-parsing-hook= that runs before the buffer has
+   been parsed.  In the example above, however, =head 2= would not be
+   exported as it becomes part of =head 1= which is not exporter.  To
+   overcome this move perhaps =head 1= can be moved to the end of the
+   buffer.  An example of a hook that removes headings is before
+   parsing is available [[https://lists.gnu.org/archive/html/emacs-orgmode/2014-03/msg01459.html][here]].  Note, this solution is compatible with
+   /all/ export formats!
+2. The problem is simple when exporting to LaTeX, as the LaTeX
+   compiler determines numbers.  We can thus use
+   =org-export-filter-headline-functions= to remove the offending
+   headlines.  One regexp-based solution that looks for the word
+   =ignoreheading= is available on [[https://stackoverflow.com/questions/10295177/is-there-an-equivalent-of-org-modes-b-ignoreheading-for-non-beamer-documents][StackOverflow]] for both the legacy
+   exporter Org v7 exporter and the current Org v8 exporter.  Note,
+   however, that this filter will only work with LaTeX (numbering and
+   the table of content may break in other exporters).  In the example
+   above, this filer will work flawlessly in LaTeX, it will not work
+   at all in HTML and it will fail to update section numbers, TOC and
+   leave some auxiliary lines behind when exporting to plain text.
+3. Another solution that tries to recover the Org element
+   representation is available [[https://lists.gnu.org/archive/html/emacs-orgmode/2014-03/msg01480.html][here]].  In the example above this filter
+   will not remove =head 4= exporting to any backend, since verbatim
+   strings do not retain the Org element representation.  It will
+   remove the extra heading line when exporting to plain text, but
+   will also fail to update section numbers.  It should be fairly
+   simple to also make it work with HTML.
 
 *** Export Org to Org and handle includes.
 #+index: Export!handle includes
@@ -3846,4 +3868,3 @@ hydrometer correction, abv calculation, priming sugar for a given CO_2
 volume, etc. More integration with org-mode should be possible: for
 instance it would be nice to be able to use a lookup table (of ingredients)
 to calculate target original gravity, IBUs, etc.
-

Re: New worg page: Publishing beamer slideshows and articles from one source

[Suvayu Ali]

Hi Rasmus, Bastien,

On Mon, May 26, 2014 at 05:56:38PM +0200, Rasmus wrote:
> 
> I have attached a quick patch with a text that I think is more
> accurate and that tries to explain the issue.  I think the tone may be
> too "academic"/dry.
> 
> Since Suvayu put the initial effort into the answer I will wait for
> his approval before submitting it.

Sorry I was a bit sloppy.  The origin of the hack is from LaTeX export.
I did not test it extensively for other backends.  Now I realise why it
worked for me; I always export with no toc for html and ascii.  and
LaTeX takes care of toc.

Your patch looks good, however I think the language is indeed too
academic and could be simplified and made more accesible.  It is Worg
after all :).

> >> To deal with the tricky part you need an additional filter.  Or you
> >> need a filter on the final output and you'd probably need to make some
> >> accounting of what's present and what is not so that you can update
> >> counters and TOCs.
> >
> > That's why I say it's tricky.
> >
> > Let's keep this constructive and move forward -- maybe that was not
> > your intention, but the tone of your first email was a bit harsh.
> > I'm sure Suvayu is open to suggestions on how to improve the filter.
> 
> That was not the intention.  For that I appoligize.

In fact, I'm always of the opinion, if the change is minor, or corrects
a mistake, or simplifies things, just go ahead with it.  It is a
community resource!

I won't have time to work on this any time soon (now I'm at a workshop,
review deadline next week, holiday the week after).

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: New worg page: Publishing beamer slideshows and articles from one source

[Bastien]

Hi,

Suvayu Ali <fatkasuvayu+linux@gmail.com> writes:

> In fact, I'm always of the opinion, if the change is minor, or corrects
> a mistake, or simplifies things, just go ahead with it.

My thoughts exactly -- Rasmus, please go ahead by apply your change
and Suvayu will complete it when he has time.

Thanks both,

-- 
 Bastien