[BUG] New exporter exports TOC twice

[BUG] New exporter exports TOC twice

[Thorsten Jolitz]

[double-post to bring this issue to the attention of Nicolas]

Exporting a Worg file with this header (#+TOC: headlines 2) 

,-----------------------------------------------------------------------------------------
| #+OPTIONS: H:3 num:nil \n:nil @:t ::t |:t ^:{} -:t f:t *:t TeX:t LaTeX:t
|             skip:nil d:(HIDE) tags:not-in-toc
| #+TOC:        headlines 2
| #+STARTUP:    align fold nodlcheck hidestars oddeven lognotestate hideblocks
| #+SEQ_TODO:   TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
| #+TAGS:       Write(w) Update(u) Fix(f) Check(c) noexport(n)
| #+TITLE:      Header arguments and result types in Org Babel
| #+AUTHOR:     Thorsten Jolitz, Eric Schulte
| #+EMAIL:      tj[at]data-driven[dot]de
| #+LANGUAGE:   en
| #+LINK_UP:    index.php
| #+LINK_HOME:  http://orgmode.org/worg/
| #+EXPORT_EXCLUDE_TAGS: noexport
| 
| For a complete header argument reference see the Org-mode manual's page
| which lists all
| [[http://orgmode.org/manual/Specific-header-arguments.html][Specific-header-arguments]].
| This page holds ancillary notes and tricks which have not made it into
| the manual.
| 
| * Generally use =verbatim= when using =drawer=, =raw= or =org=
| We often want to add =verbatim= (which inhibits interpretation as a
| value, which can often result in a list or table result), when
| inserting results directly into the buffer using =drawer=, =raw= or
| =org= which don't do tabular interpretation. [...]
`-----------------------------------------------------------------------------------------

exports the TOC twice in HTML export and ASCII export. 

HTML:
,--------------------------------------------------------------------------------
| Header arguments and result types in Org Babel
| 
| Table of Contents
| 
|   * Generally use verbatim when using drawer, raw or org
|   * Common combinations of header-args and result types
|   * Setting language and file specific default header argument values
| 
| Table of Contents
| 
|   * Generally use verbatim when using drawer, raw or org
|   * Common combinations of header-args and result types
|   * Setting language and file specific default header argument values
| 
| For a complete header argument reference see the Org-mode manual's page which
| lists all Specific-header-arguments. This page holds ancillary notes and tricks
| which have not made it into the manual.
`--------------------------------------------------------------------------------

ASCII:
,------------------------------------------------------------------------
| 
|             ________________________________________________
| 
|              HEADER ARGUMENTS AND RESULT TYPES IN ORG BABEL
| 
|                      Thorsten Jolitz, Eric Schulte
|             ________________________________________________
| 
| 
| Table of Contents
| _________________
| 
| Generally use `verbatim' when using `drawer', `raw' or `org'
| Common combinations of header-args and result types
| Setting language and file specific default header argument values
| 
| 
| Table of Contents
| _________________
| 
| Generally use `verbatim' when using `drawer', `raw' or `org'
| Common combinations of header-args and result types
| Setting language and file specific default header argument values
| For a complete header argument reference see the Org-mode manual's page
| which lists all [Specific-header-arguments]. This page holds ancillary
| notes and tricks which have not made it into the manual.
`------------------------------------------------------------------------


-- 
cheers,
Thorsten

Re: [BUG] New exporter exports TOC twice

[Nicolas Goaziou]

Hello,

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Exporting a Worg file with this header (#+TOC: headlines 2) 
>
> ,-----------------------------------------------------------------------------------------
> | #+OPTIONS: H:3 num:nil \n:nil @:t ::t |:t ^:{} -:t f:t *:t TeX:t LaTeX:t
> |             skip:nil d:(HIDE) tags:not-in-toc
> | #+TOC:        headlines 2
> | #+STARTUP:    align fold nodlcheck hidestars oddeven lognotestate hideblocks
> | #+SEQ_TODO:   TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
> | #+TAGS:       Write(w) Update(u) Fix(f) Check(c) noexport(n)
> | #+TITLE:      Header arguments and result types in Org Babel
> | #+AUTHOR:     Thorsten Jolitz, Eric Schulte
> | #+EMAIL:      tj[at]data-driven[dot]de
> | #+LANGUAGE:   en
> | #+LINK_UP:    index.php
> | #+LINK_HOME:  http://orgmode.org/worg/
> | #+EXPORT_EXCLUDE_TAGS: noexport
> | 
> | For a complete header argument reference see the Org-mode manual's page
> | which lists all
> | [[http://orgmode.org/manual/Specific-header-arguments.html][Specific-header-arguments]].
> | This page holds ancillary notes and tricks which have not made it into
> | the manual.
> | 
> | * Generally use =verbatim= when using =drawer=, =raw= or =org=
> | We often want to add =verbatim= (which inhibits interpretation as a
> | value, which can often result in a list or table result), when
> | inserting results directly into the buffer using =drawer=, =raw= or
> | =org= which don't do tabular interpretation. [...]
> `-----------------------------------------------------------------------------------------
>
> exports the TOC twice in HTML export and ASCII export. 

If you don't specify a toc item in the OPTIONS line, Org will use the
value of `org-export-with-toc', which is non-nil by default.

So, your example is equivalent to:

  #+OPTIONS: toc:t
  #+TOC: headline 2

Hence you get two tables of contents.


Regards,

-- 
Nicolas Goaziou

Re: [BUG] New exporter exports TOC twice

[Thorsten Jolitz]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> If you don't specify a toc item in the OPTIONS line, Org will use the
> value of `org-export-with-toc', which is non-nil by default.
>
> So, your example is equivalent to:
>
>   #+OPTIONS: toc:t
>   #+TOC: headline 2
>
> Hence you get two tables of contents.

Ok, but then with 

,----------------
| #+OPTIONS: toc:2
`----------------

the 

,-----------------
| #+TOC: headline 2
`-----------------

is actually redundant - I would need it only if I want to export other
things than headlines, right?

-- 
cheers,
Thorsten

Re: [BUG] New exporter exports TOC twice

[Nicolas Goaziou]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> If you don't specify a toc item in the OPTIONS line, Org will use the
>> value of `org-export-with-toc', which is non-nil by default.
>>
>> So, your example is equivalent to:
>>
>>   #+OPTIONS: toc:t
>>   #+TOC: headline 2
>>
>> Hence you get two tables of contents.
>
> Ok, but then with 
>
> ,----------------
> | #+OPTIONS: toc:2
> `----------------
>
> the 
>
> ,-----------------
> | #+TOC: headline 2
> `-----------------
>
> is actually redundant - I would need it only if I want to export other
> things than headlines, right?

They are not equivalent, but are indeed redundant.


Regards,

-- 
Nicolas Goaziou

Re: [BUG] New exporter exports TOC twice

[Carsten Dominik]

On 24.4.2013, at 13:50, Nicolas Goaziou <n.goaziou@gmail.com> wrote:

> Hello,
> 
> Thorsten Jolitz <tjolitz@gmail.com> writes:
> 
>> Exporting a Worg file with this header (#+TOC: headlines 2) 
>> 
>> ,-----------------------------------------------------------------------------------------
>> | #+OPTIONS: H:3 num:nil \n:nil @:t ::t |:t ^:{} -:t f:t *:t TeX:t LaTeX:t
>> |             skip:nil d:(HIDE) tags:not-in-toc
>> | #+TOC:        headlines 2
>> | #+STARTUP:    align fold nodlcheck hidestars oddeven lognotestate hideblocks
>> | #+SEQ_TODO:   TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
>> | #+TAGS:       Write(w) Update(u) Fix(f) Check(c) noexport(n)
>> | #+TITLE:      Header arguments and result types in Org Babel
>> | #+AUTHOR:     Thorsten Jolitz, Eric Schulte
>> | #+EMAIL:      tj[at]data-driven[dot]de
>> | #+LANGUAGE:   en
>> | #+LINK_UP:    index.php
>> | #+LINK_HOME:  http://orgmode.org/worg/
>> | #+EXPORT_EXCLUDE_TAGS: noexport
>> | 
>> | For a complete header argument reference see the Org-mode manual's page
>> | which lists all
>> | [[http://orgmode.org/manual/Specific-header-arguments.html][Specific-header-arguments]].
>> | This page holds ancillary notes and tricks which have not made it into
>> | the manual.
>> | 
>> | * Generally use =verbatim= when using =drawer=, =raw= or =org=
>> | We often want to add =verbatim= (which inhibits interpretation as a
>> | value, which can often result in a list or table result), when
>> | inserting results directly into the buffer using =drawer=, =raw= or
>> | =org= which don't do tabular interpretation. [...]
>> `-----------------------------------------------------------------------------------------
>> 
>> exports the TOC twice in HTML export and ASCII export. 
> 
> If you don't specify a toc item in the OPTIONS line, Org will use the
> value of `org-export-with-toc', which is non-nil by default.
> 
> So, your example is equivalent to:
> 
>  #+OPTIONS: toc:t
>  #+TOC: headline 2
> 
> Hence you get two tables of contents.


Hmm, I understand the reasoning here - but my feeling says that
the presence of one or more #+TOC lines in a file should probably
overrule both #+OPTIONS: toc: and the content of org-export-with-toc.
So in that case, your would then only get one TOC, at the
location of that line.

What do you think?  Are there good reasons for not doing it
as I propose?

- Carsten

Re: [BUG] New exporter exports TOC twice

[Sebastien Vauban]

Hi Carsten,

Carsten Dominik wrote:
> On 24.4.2013, at 13:50, Nicolas Goaziou <n.goaziou-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> Thorsten Jolitz <tjolitz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>> 
>>> Exporting a Worg file with this header (#+TOC: headlines 2) 
>>> exports the TOC twice in HTML export and ASCII export. 
>> 
>> If you don't specify a toc item in the OPTIONS line, Org will use the
>> value of `org-export-with-toc', which is non-nil by default.
>> 
>> So, your example is equivalent to:
>> 
>>  #+OPTIONS: toc:t
>>  #+TOC: headline 2
>> 
>> Hence you get two tables of contents.
>
> Hmm, I understand the reasoning here - but my feeling says that
> the presence of one or more #+TOC lines in a file should probably
> overrule both #+OPTIONS: toc: and the content of org-export-with-toc.
> So in that case, your would then only get one TOC, at the
> location of that line.
>
> What do you think?  Are there good reasons for not doing it
> as I propose?

Maybe what I'll say is stupid, but, in LaTeX, there's the minitoc package for
having a big TOC at the beginning of the document (default) + TOC per chapter,
which can be limited to less (or more) sublevels.

Maybe allowing to insert extra TOC here and there would allow one to make
something like that available to other backends?  Though, as of today, I don't
think the TOC is limitable per org-level-1 headline...

Best regards,
  Seb

-- 
Sebastien Vauban

Re: [BUG] New exporter exports TOC twice

[Carsten Dominik]

On 27.4.2013, at 10:52, Sebastien Vauban <sva-news@mygooglest.com> wrote:

> Hi Carsten,
> 
> Carsten Dominik wrote:
>> On 24.4.2013, at 13:50, Nicolas Goaziou <n.goaziou@gmail.com> wrote:
>>> Thorsten Jolitz <tjolitz@gmail.com> writes:
>>> 
>>>> Exporting a Worg file with this header (#+TOC: headlines 2) 
>>>> exports the TOC twice in HTML export and ASCII export. 
>>> 
>>> If you don't specify a toc item in the OPTIONS line, Org will use the
>>> value of `org-export-with-toc', which is non-nil by default.
>>> 
>>> So, your example is equivalent to:
>>> 
>>> #+OPTIONS: toc:t
>>> #+TOC: headline 2
>>> 
>>> Hence you get two tables of contents.
>> 
>> Hmm, I understand the reasoning here - but my feeling says that
>> the presence of one or more #+TOC lines in a file should probably
>> overrule both #+OPTIONS: toc: and the content of org-export-with-toc.
>> So in that case, your would then only get one TOC, at the
>> location of that line.
>> 
>> What do you think?  Are there good reasons for not doing it
>> as I propose?
> 
> Maybe what I'll say is stupid, but, in LaTeX, there's the minitoc package for
> having a big TOC at the beginning of the document (default) + TOC per chapter,
> which can be limited to less (or more) sublevels.
> 
> Maybe allowing to insert extra TOC here and there would allow one to make
> something like that available to other backends?  Though, as of today, I don't
> think the TOC is limitable per org-level-1 headline...
> 

Hi Sebastien,

I am not saying multiple tocs should not be allowed.  I am all for that.
However, I think that by inserting a #+TOC line, the user indicates
desire for local control.  Therefore, org-export-with-toc should be ignored,
and, by extension, also #+OPTIONS: toc (because this is really a local way
to set org-export-with-toc).

- Carsten

Re: [BUG] New exporter exports TOC twice

[Sebastien Vauban]

Hi Carsten,

Carsten Dominik wrote:
> On 27.4.2013, at 10:52, Sebastien Vauban <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org> wrote:
>> Carsten Dominik wrote:
>>> On 24.4.2013, at 13:50, Nicolas Goaziou <n.goaziou-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>> Thorsten Jolitz <tjolitz-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:
>>>> 
>>>>> Exporting a Worg file with this header (#+TOC: headlines 2) 
>>>>> exports the TOC twice in HTML export and ASCII export. 
>>>> 
>>>> If you don't specify a toc item in the OPTIONS line, Org will use the
>>>> value of `org-export-with-toc', which is non-nil by default.
>>>> 
>>>> So, your example is equivalent to:
>>>> 
>>>> #+OPTIONS: toc:t
>>>> #+TOC: headline 2
>>>> 
>>>> Hence you get two tables of contents.
>>> 
>>> Hmm, I understand the reasoning here - but my feeling says that
>>> the presence of one or more #+TOC lines in a file should probably
>>> overrule both #+OPTIONS: toc: and the content of org-export-with-toc.
>>> So in that case, your would then only get one TOC, at the
>>> location of that line.
>>> 
>>> What do you think?  Are there good reasons for not doing it
>>> as I propose?
>> 
>> Maybe what I'll say is stupid, but, in LaTeX, there's the minitoc package for
>> having a big TOC at the beginning of the document (default) + TOC per chapter,
>> which can be limited to less (or more) sublevels.
>> 
>> Maybe allowing to insert extra TOC here and there would allow one to make
>> something like that available to other backends?  Though, as of today, I don't
>> think the TOC is limitable per org-level-1 headline...
>
> I am not saying multiple tocs should not be allowed.  I am all for that.
> However, I think that by inserting a #+TOC line, the user indicates
> desire for local control.  Therefore, org-export-with-toc should be ignored,
> and, by extension, also #+OPTIONS: toc (because this is really a local way
> to set org-export-with-toc).

OK, that's another point of view. I clearly have no objection for that
behavior.

Best regards,
  Seb

-- 
Sebastien Vauban

Re: [BUG] New exporter exports TOC twice

[Nicolas Goaziou]

Hello,

Carsten Dominik <carsten.dominik@gmail.com> writes:

> I am not saying multiple tocs should not be allowed.  I am all for that.
> However, I think that by inserting a #+TOC line, the user indicates
> desire for local control.  Therefore, org-export-with-toc should be ignored,
> and, by extension, also #+OPTIONS: toc (because this is really a local way
> to set org-export-with-toc).

The problem is that #+TOC cannot be a strict equivalent to
`org-export-with-toc', since the former cannot be introduced in the
document template.

Also, this change would require each user back-end developer to check
for the presence of a TOC keyword with "headlines" value in the parse
tree when handling :with-toc property. This is not complicated, but
there are already many uncomplicated issues to think about when writing
a back-end.

In a nutshell, I don't think we should try to outsmart the user by
ignoring his setup here. I suggest to improve the manual, if needed,
instead.


Regards,

-- 
Nicolas Goaziou

Re: [BUG] New exporter exports TOC twice

[Carsten Dominik]

Hi Nicolas,

On 28.4.2013, at 09:28, Nicolas Goaziou <n.goaziou@gmail.com> wrote:

> Hello,
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>> I am not saying multiple tocs should not be allowed.  I am all for that.
>> However, I think that by inserting a #+TOC line, the user indicates
>> desire for local control.  Therefore, org-export-with-toc should be ignored,
>> and, by extension, also #+OPTIONS: toc (because this is really a local way
>> to set org-export-with-toc).
> 
> The problem is that #+TOC cannot be a strict equivalent to
> `org-export-with-toc', since the former cannot be introduced in the
> document template.

I am not sure I understand.  What do you mean?

> Also, this change would require each user back-end developer to check
> for the presence of a TOC keyword with "headlines" value in the parse
> tree when handling :with-toc property. This is not complicated, but
> there are already many uncomplicated issues to think about when writing
> a back-end.

An alternative would be that the parser already makes this change.  Upon
finding #+TOC, it would change the OPTION value in the parse tree.

> 
> In a nutshell, I don't think we should try to outsmart the user by
> ignoring his setup here. I suggest to improve the manual, if needed,
> instead.

That is certainly an alternative, once I have understood the issues.

Thanks for your patience.

- Carsten

Re: [BUG] New exporter exports TOC twice

[Nicolas Goaziou]

Hello,

Carsten Dominik <carsten.dominik@gmail.com> writes:

>> The problem is that #+TOC cannot be a strict equivalent to
>> `org-export-with-toc', since the former cannot be introduced in the
>> document template.
>
> I am not sure I understand.  What do you mean?

A TOC keyword belongs to the contents of the document. On the other
hand, :with-toc is handled in a "template" function, which has access to
both the contents and the meta-data around it. Therefore, :with-toc can
insert a table of contents in more places than TOC.

For example, in latex back-end, TOC cannot insert a \tableofcontents
before \begin{document}, but toc:t can. Of course, this example doesn't
make sense as per LaTeX syntax, but it is possible that some other
back-end wants to allow this.

>> Also, this change would require each user back-end developer to check
>> for the presence of a TOC keyword with "headlines" value in the parse
>> tree when handling :with-toc property. This is not complicated, but
>> there are already many uncomplicated issues to think about when writing
>> a back-end.
>
> An alternative would be that the parser already makes this change.  Upon
> finding #+TOC, it would change the OPTION value in the parse tree.

The parser doesn't handle the OPTION keyword (it's just another
keyword), and neither should it (export options mustn't influence how
the parsing is done).

It belongs to the export framework to read export options and handle
them. Upon reading the with-toc value, it is indeed possible to check
for the presence of a TOC keyword in the parse tree.

However, in this case, I don't see the need to go out of our way just to
interpret differently what the user specified. Getting two tables of
contents is not surprising when you understand that "toc:t" and "#+TOC:
headlines" are independent ways of requesting a table of contents.


Regards,

-- 
Nicolas Goaziou

Re: [BUG] New exporter exports TOC twice

[Carsten Dominik]

Hi Nicolas,

On 2.5.2013, at 00:08, Nicolas Goaziou <n.goaziou@gmail.com> wrote:

> Hello,
> 
> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
>>> The problem is that #+TOC cannot be a strict equivalent to
>>> `org-export-with-toc', since the former cannot be introduced in the
>>> document template.
>> 
>> I am not sure I understand.  What do you mean?
> 
> A TOC keyword belongs to the contents of the document. On the other
> hand, :with-toc is handled in a "template" function, which has access to
> both the contents and the meta-data around it. Therefore, :with-toc can
> insert a table of contents in more places than TOC.
> 
> For example, in latex back-end, TOC cannot insert a \tableofcontents
> before \begin{document}, but toc:t can. Of course, this example doesn't
> make sense as per LaTeX syntax, but it is possible that some other
> back-end wants to allow this.

OK.

> 
>>> Also, this change would require each user back-end developer to check
>>> for the presence of a TOC keyword with "headlines" value in the parse
>>> tree when handling :with-toc property. This is not complicated, but
>>> there are already many uncomplicated issues to think about when writing
>>> a back-end.
>> 
>> An alternative would be that the parser already makes this change.  Upon
>> finding #+TOC, it would change the OPTION value in the parse tree.
> 
> The parser doesn't handle the OPTION keyword (it's just another
> keyword), and neither should it (export options mustn't influence how
> the parsing is done).
> 
> It belongs to the export framework to read export options and handle
> them. Upon reading the with-toc value, it is indeed possible to check
> for the presence of a TOC keyword in the parse tree.
> 
> However, in this case, I don't see the need to go out of our way just to
> interpret differently what the user specified. Getting two tables of
> contents is not surprising when you understand that "toc:t" and "#+TOC:
> headlines" are independent ways of requesting a table of contents.


This is clear enough.  I have pushed a fix to the manual which should
make this clearer.

Regards

- Carsten