Should comments break paragraphs?

Should comments break paragraphs?

[Kodi Arfer]

If you export

some text
# a comment
more text

then you get two paragraphs, not one. Is this intentional? The old 
export engine gave you one paragraph. I liked that behavior better, 
since it allowed me to insert notes in the middle of paragraphs. The 
manual still says "Paragraphs are separated by at least one empty line." 
On the other hand, it's probably a little cleaner to treat a comment 
just like a blank linke.

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Kodi Arfer <kodi@arfer.net> writes:

> If you export
>
> some text
> # a comment
> more text
>
> then you get two paragraphs, not one. Is this intentional? 

Yes. Comments are elements. Elements cannot be found in paragraphs.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Christian Wittern]

On 2013-07-16 03:54, Nicolas Goaziou wrote:
> Yes. Comments are elements. Elements cannot be found in paragraphs.
Hmm.  In my book, the concept of comments implies that they should be able 
to appear on any line in the text, without altering the meaning of the 
context it appears in.  So to me, it would seem necessary to treat comments 
as special, different from other elements.

Just my 2 yen,

Christian

-- 
Christian Wittern, Kyoto

Re: Should comments break paragraphs?

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

Christian Wittern <cwittern@gmail.com> writes:

> On 2013-07-16 03:54, Nicolas Goaziou wrote:
>> Yes. Comments are elements. Elements cannot be found in paragraphs.
> Hmm.  In my book, the concept of comments implies that they should be
> able to appear on any line in the text, without altering the meaning
> of the context it appears in.  So to me, it would seem necessary to
> treat comments as special, different from other elements.
>
> Just my 2 yen,

I have to agree with you here: a comment should *not* impact on the
result, because if it does, it is not a comment anymore, but a command.

It should be exported in the exporter as either a comment in the
resulting document (e.g %% in LaTeX, a comment or note in odt, ...) or
ignored in the exported document, but definitely not have an impact on
the *layout* of the document.

Cheers,

Rainer

>
> Christian


-- 
Rainer M. Krug

email: RMKrug<at>gmail<dot>com

[-- Attachment #2: Type: application/pgp-signature, Size: 486 bytes --]

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Christian Wittern <cwittern@gmail.com> writes:

> Hmm.  In my book, the concept of comments implies that they should be
> able to appear on any line in the text, without altering the meaning
> of the context it appears in.  So to me, it would seem necessary to
> treat comments as special, different from other elements.

I don't think comments make much sense anywhere in Org (e.g., in an
example block or in a source block). Also Org has no inline comment
syntax (and I don't think it needs one: it is no programming language).
So comments do not really fit in paragraphs.

Last, but not least, it is also easier to parse it that way.

Now, if you want to improve comments anyway, you can always provide
patches. There's some non-trivial work involved, though.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 1500 bytes --]


Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Christian Wittern <cwittern@gmail.com> writes:
>
>> Hmm.  In my book, the concept of comments implies that they should be
>> able to appear on any line in the text, without altering the meaning
>> of the context it appears in.  So to me, it would seem necessary to
>> treat comments as special, different from other elements.
>
> I don't think comments make much sense anywhere in Org (e.g., in an
> example block or in a source block). Also Org has no inline comment
> syntax (and I don't think it needs one: it is no programming language).
> So comments do not really fit in paragraphs.

I see comments as "entities which do not have any impact on the final
product". 

If you regard the org file as the final product, then I
completely agree (I use org mainly for literate programming, and I don't
use any comments unless in linger code blocks, but that is not in
org). But if I use org for writing e.g. presentations in beamer, I might want
to add comments which should not be on the product (slides, article,
handout, ...) but which contain info in the org file.

So we are talking different levels here.

Cheers,

Rainer

>
> Last, but not least, it is also easier to parse it that way.
>
> Now, if you want to improve comments anyway, you can always provide
> patches. There's some non-trivial work involved, though.
>
>
> Regards,


-- 
Rainer M. Krug

email: RMKrug<at>gmail<dot>com

[-- Attachment #2: Type: application/pgp-signature, Size: 486 bytes --]

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Rainer M Krug <Rainer@krugs.de> writes:

> I see comments as "entities which do not have any impact on the final
> product". 
>
> If you regard the org file as the final product, then I
> completely agree (I use org mainly for literate programming, and I don't
> use any comments unless in linger code blocks, but that is not in
> org). But if I use org for writing e.g. presentations in beamer, I might want
> to add comments which should not be on the product (slides, article,
> handout, ...) but which contain info in the org file.
>
> So we are talking different levels here.

Comments do not alter the final product: they are ignored during export.
But there are places where you cannot have comments, at all. If we look
at the following example, similar to OP's:

  Some text
  # comment
  Some other text

comment do not split the paragraph: there were two paragraphs since the
beginning (but still no spoon). It's only disturbing if you think
comments can be inserted within paragraphs, which is not the case.

This may not be perfect, but, as for me, it's good enough.

Another option would be to remove every comment line before parsing the
buffer (which is, IIRC, what previous exporter did), but it would only
hide the situation for a while. After all I could expect to have
comments anywhere, including some places like

  * Headline
  # Between an headline and planning info. Update org-agenda.el.
  SCHEDULED: <2013-07-16 Tue>

  - item 1
  # Some comment between items. Update org-list.el
  - item 2

  #+HEADER: var=2
  # A comment between affiliated keywords and true data. Update ob-core.el.
  #+RESULTS: some-table
  | a |

  | a | b |
  # Comments within rows. Update org-table.el
  | c | d |

And then, the situation would appear again: it's not just about the parser
or the exporter. Every part of Org would have to handle comments differently.

Again, I think comments are acceptable, once you are aware about their
limitations. But, I don't prevent you from opening that can of worms.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 1987 bytes --]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Rainer M Krug <Rainer@krugs.de> writes:
>
>> I see comments as "entities which do not have any impact on the final
>> product". 
>>
>> If you regard the org file as the final product, then I
>> completely agree (I use org mainly for literate programming, and I don't
>> use any comments unless in linger code blocks, but that is not in
>> org). But if I use org for writing e.g. presentations in beamer, I might want
>> to add comments which should not be on the product (slides, article,
>> handout, ...) but which contain info in the org file.
>>
>> So we are talking different levels here.
>
> Comments do not alter the final product: they are ignored during export.
> But there are places where you cannot have comments, at all. If we look
> at the following example, similar to OP's:
>
>   Some text
>   # comment
>   Some other text
>
> comment do not split the paragraph: there were two paragraphs since the
> beginning (but still no spoon). It's only disturbing if you think
> comments can be inserted within paragraphs, which is not the case.

I think I see what you mean: if the comment would not be there, the text
would look as follow:

,----
| Some text
| 
| Some other text
`----

for the exporter. So the comment here

,----
| Some text
| # comment
| Some other text
`----

is ignored, but not the empty line resulting - is this correct? I think
the confusion is that comments in org are between comment lines and
inline comments in this regard. 

>
> This may not be perfect, but, as for me, it's good enough.
>
[snip: examples (24 lines)]
>
> Again, I think comments are acceptable, once you are aware about their
> limitations. But, I don't prevent you from opening that can of worms.

Don't worry - I am happy as it stands now and have no complains.

>
>
> Regards,

Cheers,

Rainer

-- 
Rainer M. Krug

email: RMKrug<at>gmail<dot>com

[-- Attachment #2: Type: application/pgp-signature, Size: 486 bytes --]

Re: Should comments break paragraphs?

[Andreas Leha]

Rainer M Krug <Rainer@krugs.de> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> Hello,
>>
>> Rainer M Krug <Rainer@krugs.de> writes:
>>
>>> I see comments as "entities which do not have any impact on the final
>>> product". 
>>>
>>> If you regard the org file as the final product, then I
>>> completely agree (I use org mainly for literate programming, and I don't
>>> use any comments unless in linger code blocks, but that is not in
>>> org). But if I use org for writing e.g. presentations in beamer, I might want
>>> to add comments which should not be on the product (slides, article,
>>> handout, ...) but which contain info in the org file.
>>>
>>> So we are talking different levels here.
>>
>> Comments do not alter the final product: they are ignored during export.
>> But there are places where you cannot have comments, at all. If we look
>> at the following example, similar to OP's:
>>
>>   Some text
>>   # comment
>>   Some other text
>>
>> comment do not split the paragraph: there were two paragraphs since the
>> beginning (but still no spoon). It's only disturbing if you think
>> comments can be inserted within paragraphs, which is not the case.
>
> I think I see what you mean: if the comment would not be there, the text
> would look as follow:
>
> ,----
> | Some text
> | 
> | Some other text
> `----
>
> for the exporter. So the comment here
>
> ,----
> | Some text
> | # comment
> | Some other text
> `----
>
> is ignored, but not the empty line resulting - is this correct? I think
> the confusion is that comments in org are between comment lines and
> inline comments in this regard. 
>

In that case, I guess some confusion might come from LaTeX, for example,
where also the newline is ignored by the comment, s.t. in LaTeX 
,----
| Some text
| % comment
| Some other text
`----
is interpreted as
,----
| Some text
| Some other text
`----

[...]

IMO, there is no 'feature' in the 'paragraph breaking' comments in
org.  But I understand that it is quite hard to change that behaviour.


Regards,
Andreas

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:

> In that case, I guess some confusion might come from LaTeX, for example,
> where also the newline is ignored by the comment, s.t. in LaTeX 
> ,----
> | Some text
> | % comment
> | Some other text
> `----
> is interpreted as
> ,----
> | Some text
> | Some other text
> `----

Comparing Org and LaTeX is like comparing apples and oranges: it can
only lead to false assumptions and cloud thinking.

> IMO, there is no 'feature' in the 'paragraph breaking' comments in
> org.  But I understand that it is quite hard to change that behaviour.

Just to be clear. I didn't talk about a "feature". I merely described
the state of affairs, and added that, albeit not perfect, it seemed good
enough to me.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Andreas Leha]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:
>
>> In that case, I guess some confusion might come from LaTeX, for example,
>> where also the newline is ignored by the comment, s.t. in LaTeX 
>> ,----
>> | Some text
>> | % comment
>> | Some other text
>> `----
>> is interpreted as
>> ,----
>> | Some text
>> | Some other text
>> `----
>
> Comparing Org and LaTeX is like comparing apples and oranges: it can
> only lead to false assumptions and cloud thinking.
>

Agreed.  I just wanted to mention one possible source of such false
assumptions.


>> IMO, there is no 'feature' in the 'paragraph breaking' comments in
>> org.  But I understand that it is quite hard to change that behaviour.
>
> Just to be clear. I didn't talk about a "feature". I merely described
> the state of affairs, and added that, albeit not perfect, it seemed good
> enough to me.

I hope I did not offend you with that paragraph.  I did not want to say
that you called it a feature.  If was merely a summary of the thread, as
I understood it.

I actually never ran into the problem myself, as I do not comment too
much in my org files.  So I am in the 'good enough' group as well.

Regards,
Andreas

Re: Should comments break paragraphs?

[Eric S Fraga]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:
>
>> In that case, I guess some confusion might come from LaTeX, for example,
>> where also the newline is ignored by the comment, s.t. in LaTeX 
>> ,----
>> | Some text
>> | % comment
>> | Some other text
>> `----
>> is interpreted as
>> ,----
>> | Some text
>> | Some other text
>> `----
>
> Comparing Org and LaTeX is like comparing apples and oranges: it can
> only lead to false assumptions and cloud thinking.

True but the original three lines (text, comment, text) gets exported
differently to LaTeX and to HTML.  The former results in one paragraph;
the latter in two.  So even though you may treat the two text lines as
separate elements in the parser, the LaTeX output is as if they were one
element as there is no blank line between them (LaTeX's paragraph
separator).

But I'm happy with how things are!  I export to LaTeX mostly and
comments basically act as inline comments when they sit between two
normal paragraphs. 

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.3.50.1, Org release_8.0.5-337-g9f3bed

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Rainer M Krug <Rainer@krugs.de> writes:

> I think I see what you mean: if the comment would not be there, the text
> would look as follow:
>
> ,----
> | Some text
> | 
> | Some other text
> `----
>
> for the exporter. So the comment here
>
> ,----
> | Some text
> | # comment
> | Some other text
> `----
>
> is ignored, but not the empty line resulting - is this correct? I think
> the confusion is that comments in org are between comment lines and
> inline comments in this regard.

You can replace the comment by a blank line, or a fixed-width area, or
a table... It may be better to think it that way: "# " syntax introduces
a "commented paragraph", not a "comment line".


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Fabrice Popineau]

[-- Attachment #1: Type: text/plain, Size: 354 bytes --]

> Another option would be to remove every comment line before parsing the
> buffer (which is, IIRC, what previous exporter did), but it would only
> hide the situation for a while. After all I could expect to have
> comments anywhere, including some places like
>


On your example, why wouldn't it work if we remove comments before
exporting ?

Fabrice

[-- Attachment #2: Type: text/html, Size: 627 bytes --]

Re: Should comments break paragraphs?

[Fabrice Popineau]

[-- Attachment #1: Type: text/plain, Size: 730 bytes --]

2013/7/16 Fabrice Popineau <fabrice.popineau@supelec.fr>

>
> Another option would be to remove every comment line before parsing the
>> buffer (which is, IIRC, what previous exporter did), but it would only
>> hide the situation for a while. After all I could expect to have
>> comments anywhere, including some places like
>>
>
>
> On your example, why wouldn't it work if we remove comments before
> exporting ?
>
>
Ok. Understood thanks to Rainer's message.

However, it would still be useful to be able to comment some line of text
in the middle of a paragraph.
This is only an example. I agree that the usual meaning of comments is that
they shouldn't have an impact
on the output. But Org is almost a 2D language.

Fabrice

[-- Attachment #2: Type: text/html, Size: 1498 bytes --]

Re: Should comments break paragraphs?

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 1435 bytes --]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Christian Wittern <cwittern@gmail.com> writes:
>
>> Hmm.  In my book, the concept of comments implies that they should be
>> able to appear on any line in the text, without altering the meaning
>> of the context it appears in.  So to me, it would seem necessary to
>> treat comments as special, different from other elements.
>
> I don't think comments make much sense anywhere in Org (e.g., in an
> example block or in a source block). Also Org has no inline comment
> syntax (and I don't think it needs one: it is no programming language).
> So comments do not really fit in paragraphs.
>

When Org-mode is used as a document preparation language inline comments
are very useful.  The use case being notes on the surrounding material
which are not to be published.  Both HTML and LaTeX support comments and
at least in LaTeX they are extensively used (in my experience) and can
be very helpful.

>
> Last, but not least, it is also easier to parse it that way.
>

Stripping lines starting with "^ *#[^+]" is a trivial pre-processing
step, and would support the traditional Org-mode comments which (as I
recall) could previously appear mid paragraph without causing problems.

>
> Now, if you want to improve comments anyway, you can always provide
> patches. There's some non-trivial work involved, though.
>

The attached patch worked on some small example files for me.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-support-inline-comments-w-o-breaking-paragraphs.patch --]
[-- Type: text/x-patch, Size: 1661 bytes --]

From ce4c30ebe56d4cd66810bca48824d8841e7b130d Mon Sep 17 00:00:00 2001
From: Eric Schulte <schulte.eric@gmail.com>
Date: Tue, 16 Jul 2013 09:44:59 -0600
Subject: [PATCH] support inline comments w/o breaking paragraphs

* lisp/org-element.el (org-element-parse-buffer): Strip inline comments
  as a pre-processing step before exporting.
---
 lisp/org-element.el | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index 8d64657..d69d9ba 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -4092,13 +4092,19 @@ or objects within the parse tree.
 
 This function assumes that current major mode is `org-mode'."
   (save-excursion
-    (goto-char (point-min))
-    (org-skip-whitespace)
-    (org-element--parse-elements
-     (point-at-bol) (point-max)
-     ;; Start in `first-section' mode so text before the first
-     ;; headline belongs to a section.
-     'first-section nil granularity visible-only (list 'org-data nil))))
+    (let ((save (buffer-string)))
+      (unwind-protect
+	  (progn
+	    (goto-char (point-min))
+	    (delete-matching-lines "^ *# .[^\n]*\n")
+	    (org-skip-whitespace)
+	    (org-element--parse-elements
+	     (point-at-bol) (point-max)
+	     ;; Start in `first-section' mode so text before the first
+	     ;; headline belongs to a section.
+	     'first-section nil granularity visible-only (list 'org-data nil)))
+	(delete-region (point-min) (point-max))
+	(insert save)))))
 
 (defun org-element-parse-secondary-string (string restriction &optional parent)
   "Recursively parse objects in STRING and return structure.
-- 
1.8.3.2


[-- Attachment #3: Type: text/plain, Size: 71 bytes --]


Cheers,

>
>
> Regards,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> When Org-mode is used as a document preparation language inline comments
> are very useful.  The use case being notes on the surrounding material
> which are not to be published.  Both HTML and LaTeX support comments and
> at least in LaTeX they are extensively used (in my experience) and can
> be very helpful.

Again, Org is neither LaTeX nor HTML.

> Stripping lines starting with "^ *#[^+]" is a trivial pre-processing
> step, and would support the traditional Org-mode comments which (as I
> recall) could previously appear mid paragraph without causing
> problems.

> The attached patch worked on some small example files for me.

As I explained in a previous post, it's only a kludge hiding the real
problem: comments cannot appear anywhere, even if we don't take export
into account. Actually, handling comments during export is the easiest
part of the problem.

> From ce4c30ebe56d4cd66810bca48824d8841e7b130d Mon Sep 17 00:00:00 2001
> From: Eric Schulte <schulte.eric@gmail.com>
> Date: Tue, 16 Jul 2013 09:44:59 -0600
> Subject: [PATCH] support inline comments w/o breaking paragraphs
>
> * lisp/org-element.el (org-element-parse-buffer): Strip inline comments
>   as a pre-processing step before exporting.

The parser is unrelated to the export process, so export pre-processing
shouldn't happen in org-element.el. Also, parsing a buffer mustn't
modify it in the process.

Even if you move it to ox.el, this is not an acceptable solution. Think
about the following example

 - item 1

   # with a comment

 - item 2

If you simply delete matching lines, you break the list.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> When Org-mode is used as a document preparation language inline comments
>> are very useful.  The use case being notes on the surrounding material
>> which are not to be published.  Both HTML and LaTeX support comments and
>> at least in LaTeX they are extensively used (in my experience) and can
>> be very helpful.
>
> Again, Org is neither LaTeX nor HTML.
>

I did not intend to suggest that Org was either LaTeX or HTML.

Org-mode is a document preparation system, and I think there is a strong
motivation for the inline comment use case in any document preparation
system.

>
>> Stripping lines starting with "^ *#[^+]" is a trivial pre-processing
>> step, and would support the traditional Org-mode comments which (as I
>> recall) could previously appear mid paragraph without causing
>> problems.
>
>> The attached patch worked on some small example files for me.
>
> As I explained in a previous post, it's only a kludge hiding the real
> problem: comments cannot appear anywhere, even if we don't take export
> into account.

So are you proposing to remove comments from Org-mode entirely?  I think
this would be a significant regression.

> Actually, handling comments during export is the easiest part of the
> problem.
>

Yes, I can see how teaching e.g., the new list parser to ignore comments
could be tricky.

>
>> From ce4c30ebe56d4cd66810bca48824d8841e7b130d Mon Sep 17 00:00:00 2001
>> From: Eric Schulte <schulte.eric@gmail.com>
>> Date: Tue, 16 Jul 2013 09:44:59 -0600
>> Subject: [PATCH] support inline comments w/o breaking paragraphs
>>
>> * lisp/org-element.el (org-element-parse-buffer): Strip inline comments
>>   as a pre-processing step before exporting.
>
> The parser is unrelated to the export process, so export pre-processing
> shouldn't happen in org-element.el. Also, parsing a buffer mustn't
> modify it in the process.
>

But if we treat comments as semantically empty, then maybe they should
be ignored by the parser?

>
> Even if you move it to ox.el, this is not an acceptable solution. Think
> about the following example
>
>  - item 1
>
>    # with a comment
>
>  - item 2
>
> If you simply delete matching lines, you break the list.
>

Yes, I would argue that this list should be broken, because with or
without the comment the items are separated by two blank lines.  In my
opinion comments as semantically empty lines is a simple rule for both
users and parsers to understand.

Cheers,

>
>
> Regards,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Eric Schulte <schulte.eric@gmail.com> writes:

> So are you proposing to remove comments from Org-mode entirely? 

Certainly not.

I'm on the "good enough" side and I simply suggest to live with it.
Also, if you badly need inline comments, there's the empty "comment"
macro:

  #+MACRO: comment 

  Some text {{{comment(This is my comment)}}} and some more.

> But if we treat comments as semantically empty, then maybe they should
> be ignored by the parser?

Yes, we can ignore them at the parser level. But what about the rest of
Org? There are many places that do not rely on the parser and do not
ignore comments (lists being an example, agenda being another one, and
I'm pretty sure comments at some locations can break Babel too).

Something could be done, but again, it would certainly not be
a one-liner in org-element.el or ox.el.

>> Even if you move it to ox.el, this is not an acceptable solution. Think
>> about the following example
>>
>>  - item 1
>>
>>    # with a comment
>>
>>  - item 2
>>
>> If you simply delete matching lines, you break the list.
>>
>
> Yes, I would argue that this list should be broken, because with or
> without the comment the items are separated by two blank lines.

On the other hand, visually, this list looks perfectly correct.

> In my opinion comments as semantically empty lines is a simple rule
> for both users and parsers to understand.

Comments are already almost semantically empty lines. That's why:

  Text
  # Comment
  Other Text

generates two paragraphs in Org (and in every back-end excepted `latex',
but that's a different problem). They basically act as a paragraph
separator, much like blank lines.

Also, note that if they really are empty lines, they cannot be inlined
within paragraphs either.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Bastien]

Hi all,

I think everyone agrees that comments are an important part of Org.

Nicolas, apart from the current code, what theoretically prevents Org
from considering them as *empty* lines (not blank lines)?

Where should one start if someone wants to propose a patch for this?

Thanks,

-- 
 Bastien

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

> I think everyone agrees that comments are an important part of Org.

Then I, for one, am not in the "everyone" part. FWIW, I think comments
are a minor part of Org and that we're just bikeshedding.

> Nicolas, apart from the current code, what theoretically prevents Org
> from considering them as *empty* lines (not blank lines)?

This question is too vague.

What is the difference between an empty line and the way they are
treated actually? More importantly, where, precisely, comments should be
allowed to appear in an Org buffer?

IOW, you need to define exactly what a comment should be in Org, first,
if the current definition doesn't suit you (or someone else willing to
change it). Then, we'll know where the pitfalls are.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

>> I think everyone agrees that comments are an important part of Org.
>
> Then I, for one, am not in the "everyone" part. FWIW, I think comments
> are a minor part of Org and that we're just bikeshedding.

Perhaps "part" is not the right word here.

The way to know whether comments are important or not is to poll
users.  My guess is that many of them use comments (I do a lot).
That's what I call "important".

And I don't think we not bikeshedding: several users explain why
they think the current behavior does not match what they expect.
If the conversation goes on, it's just that the answer is not
what they expect either.

>> Nicolas, apart from the current code, what theoretically prevents Org
>> from considering them as *empty* lines (not blank lines)?
>
> This question is too vague.
>
> What is the difference between an empty line and the way they are
> treated actually?

(I used the "empty vs. blank" line terminology from the thread.)

By considered an empty line, I mean this:

This is
# a comment
within a paragraph.

If "# a comment" is considered to be an empty line (i.e. a no-line),
then the above is treated as one single paragraph.  If it is treated
as a blank line, the above is treated as three paragraphs: a first
paragraph, a commented paragraph, another paragraph.

> More importantly, where, precisely, comments should be
> allowed to appear in an Org buffer?

I think "no-line" comments they should be allowed anywhere expect in
src code blocks, where the comment syntax is different.

For example this list:

- an item
# a comment
- another item

would be treated as

- an item
- another item

etc.

> IOW, you need to define exactly what a comment should be in Org, first,
> if the current definition doesn't suit you (or someone else willing to
> change it). Then, we'll know where the pitfalls are.

I think we have two options: either define comments in Org syntax,
either treat them as normal lines and delete them before parsing a
buffer for export purposes.

A simple (setq org-export-ignore-comments t) would put the user in
the second situation, where comments are deleted before parsing and
exporting, and treated as standard citizens when manipulating or
buffers.  (Eric's patch goes into that direction.)

Then (setq org-export-ignore-comments nil) would put us in the first
situation, which is the current one, where comments are defines as
elements within Org syntax, with some constraints when parsing or
exporting them (such as separating a paragraph.)

What do you think?

-- 
 Bastien

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

> By considered an empty line, I mean this:
>
> This is
> # a comment
> within a paragraph.
>
> If "# a comment" is considered to be an empty line (i.e. a no-line),
> then the above is treated as one single paragraph.  If it is treated
> as a blank line, the above is treated as three paragraphs: a first
> paragraph, a commented paragraph, another paragraph.

Why is it an issue on the Org side? So far, it only seems to be an
annoyance when exporting to HTML (and maybe ODT).

>> More importantly, where, precisely, comments should be
>> allowed to appear in an Org buffer?
>
> I think "no-line" comments they should be allowed anywhere expect in
> src code blocks, where the comment syntax is different.
>
> For example this list:
>
> - an item
> # a comment
> - another item
>
> would be treated as
>
> - an item
> - another item
>
> etc.

Then it requires serious coding in many parts of Org, as I already
explained in this thread.

> I think we have two options: either define comments in Org syntax,
> either treat them as normal lines and delete them before parsing a
> buffer for export purposes.

I already explained why the second "option" wasn't one. Look again at
your example:

  1. an item
  # a normal line breaking the list
  1. an item in another list

but, upon exporting, both items will belong to the same list. This is
just nonsensical.

> A simple (setq org-export-ignore-comments t) would put the user in
> the second situation, where comments are deleted before parsing and
> exporting, and treated as standard citizens when manipulating or
> buffers.  (Eric's patch goes into that direction.)

And the direction is wrong... Parsing shouldn't modify the buffer being
parsed, ever. But you can use a hook for that purpose.

> Then (setq org-export-ignore-comments nil) would put us in the first
> situation, which is the current one, where comments are defines as
> elements within Org syntax, with some constraints when parsing or
> exporting them (such as separating a paragraph.)
>
> What do you think?

I still think the same. Comments belong to Org syntax (if they don't,
you can't even fill them correctly, for example). If you redefine them,
there's no easy workaround. You have to change every part of Org that
assumes there will be no comment in its way (lists, agenda, babel,
parser and probably more I can't think of).

If it's an HTML/ODT export issue, it's far easier to patch the export
back-ends instead. 10 lines of code in each one, maybe.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Bastien]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

>   1. an item
>   # a normal line breaking the list
>   1. an item in another list
>
> but, upon exporting, both items will belong to the same list. This is
> just nonsensical.

Users who want comments to be equivalent to empty lines will not write
the above.  If they do, it's their responsability.

>> A simple (setq org-export-ignore-comments t) would put the user in
>> the second situation, where comments are deleted before parsing and
>> exporting, and treated as standard citizens when manipulating or
>> buffers.  (Eric's patch goes into that direction.)
>
> And the direction is wrong... Parsing shouldn't modify the buffer being
> parsed, ever. But you can use a hook for that purpose.

I didn't suggest that parsing should modify the before: I said "where
comments are deleted before parsing and exporting".

There should be an easy solution for that.

>> Then (setq org-export-ignore-comments nil) would put us in the first
>> situation, which is the current one, where comments are defines as
>> elements within Org syntax, with some constraints when parsing or
>> exporting them (such as separating a paragraph.)
>>
>> What do you think?
>
> I still think the same. Comments belong to Org syntax (if they don't,
> you can't even fill them correctly, for example). If you redefine them,
> there's no easy workaround.

I didn't suggest to redefine comments.

> You have to change every part of Org that
> assumes there will be no comment in its way (lists, agenda, babel,
> parser and probably more I can't think of).
>
> If it's an HTML/ODT export issue, it's far easier to patch the export
> back-ends instead. 10 lines of code in each one, maybe.

This is a general pre-export issue, it does not depend on the
exporters themselves.

So again, what prevents us to make it easy for users to treat comments
as no-line before parsing and exporting?

-- 
 Bastien

Re: Should comments break paragraphs?

[Rainer M Krug]

Bastien <bzg@gnu.org> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>>   1. an item
>>   # a normal line breaking the list
>>   1. an item in another list
>>
>> but, upon exporting, both items will belong to the same list. This is
>> just nonsensical.
>
> Users who want comments to be equivalent to empty lines will not write
> the above.  If they do, it's their responsability.
>
>>> A simple (setq org-export-ignore-comments t) would put the user in
>>> the second situation, where comments are deleted before parsing and
>>> exporting, and treated as standard citizens when manipulating or
>>> buffers.  (Eric's patch goes into that direction.)
>>
>> And the direction is wrong... Parsing shouldn't modify the buffer being
>> parsed, ever. But you can use a hook for that purpose.
>
> I didn't suggest that parsing should modify the before: I said "where
> comments are deleted before parsing and exporting".
>
> There should be an easy solution for that.
>
>>> Then (setq org-export-ignore-comments nil) would put us in the first
>>> situation, which is the current one, where comments are defines as
>>> elements within Org syntax, with some constraints when parsing or
>>> exporting them (such as separating a paragraph.)
>>>
>>> What do you think?
>>
>> I still think the same. Comments belong to Org syntax (if they don't,
>> you can't even fill them correctly, for example). If you redefine them,
>> there's no easy workaround.
>
> I didn't suggest to redefine comments.
>
>> You have to change every part of Org that
>> assumes there will be no comment in its way (lists, agenda, babel,
>> parser and probably more I can't think of).
>>
>> If it's an HTML/ODT export issue, it's far easier to patch the export
>> back-ends instead. 10 lines of code in each one, maybe.
>
> This is a general pre-export issue, it does not depend on the
> exporters themselves.
>
> So again, what prevents us to make it easy for users to treat comments
> as no-line before parsing and exporting?

OK - I can't comment on the inner workings of org and the exporters (so
this might not make any sense at all), but I think that
comments have their place in exports as well:

LaTeX: has comments
odt format: has notes

So at least in these two formats, it would make sense to provide the
exports to the exporter, so that they can then be converted to
comments. But in general, I agree that they should be treated as being
non existent, rather then as "\n", unless the exporter deals with them
on it's own.

Cheers,

Rainer

-- 
Rainer M. Krug

email: RMKrug<at>gmail<dot>com

Re: Should comments break paragraphs?

[Rasmus]

Rainer M Krug <Rainer@krugs.de> writes:


> OK - I can't comment on the inner workings of org and the exporters (so
> this might not make any sense at all), but I think that
> comments have their place in exports as well:
>
> LaTeX: has comments

Do you mean '%'?  If so it's not part of the output and I do not agree
that LaTeX has notes.  There exists several packages enabling some
type of notes, or exporting pdf notes.  The right approach (IMO) to
add e.g. pdf notes is inlinetasks and altering
org-latex-format-inlinetask-function.  It would be interesting to add
support for pdfcomment.sty, I think.

> odt format: has notes

I assume you mean the comments that in recent version of Libreoffice
show up on the side of the documents.  Again, I don't agree that they
are comments in the sense that '^#.*' is a comment.  Again, I think
the correct approach for adding such notes is inlinetasks.

As for how to handle comments: I don't care.  For me comments are a
temporary state before moving to a separate "* trash :noexport:" or
ARCHIVE heading.

–Rasmus

-- 
The Kids call him Billy the Saint

Re: Should comments break paragraphs?

[Eric S Fraga]

Rasmus <rasmus@gmx.us> writes:

[...]

> show up on the side of the documents.  Again, I don't agree that they
> are comments in the sense that '^#.*' is a comment.  Again, I think
> the correct approach for adding such notes is inlinetasks.

+1

Comments are internal to the org file and should not be
exported.  Inlinetasks work very well for "notes" for exporting.
-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.3.50.1, Org release_8.0.5-337-g9f3bed

Re: Should comments break paragraphs?

[Christian Moe]

Eric S Fraga writes:

> Rasmus <rasmus@gmx.us> writes:
>
> [...]
>
>> show up on the side of the documents.  Again, I don't agree that they
>> are comments in the sense that '^#.*' is a comment.  Again, I think
>> the correct approach for adding such notes is inlinetasks.
>
> +1
>
> Comments are internal to the org file and should not be
> exported.

+1

There is also a #+begin_annotation block special to ODT export, and
there was some talk about devising a backend-agnostic solution along
those lines for comments that one /does/ want exported.

Yours,
Christian

Re: Should comments break paragraphs?

[Eric Abrahamsen]

Christian Moe <mail@christianmoe.com> writes:

> Eric S Fraga writes:
>
>> Rasmus <rasmus@gmx.us> writes:
>>
>> [...]
>>
>>> show up on the side of the documents.  Again, I don't agree that they
>>> are comments in the sense that '^#.*' is a comment.  Again, I think
>>> the correct approach for adding such notes is inlinetasks.
>>
>> +1
>>
>> Comments are internal to the org file and should not be
>> exported.
>
> +1
>
> There is also a #+begin_annotation block special to ODT export, and
> there was some talk about devising a backend-agnostic solution along
> those lines for comments that one /does/ want exported.

Latex could use a marginpar, and HTML a specially-styled div (an <aside>
tag in HTML5) -- seems plausible!

Re: Should comments break paragraphs?

[Rasmus]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

>> There is also a #+begin_annotation block special to ODT export, and
>> there was some talk about devising a backend-agnostic solution along
>> those lines for comments that one /does/ want exported.
>
> Latex could use a marginpar, and HTML a specially-styled div (an <aside>
> tag in HTML5) -- seems plausible!

There's already support for inlinetasks in ox-latex and they create
minipages.  It can easily be altered and different styles could be
provided.  Users could then select how to translate inlinetasks.
E.g. marginpars would presumably not work for beamer (although I have
no idea).

–Rasmus

-- 
This is the kind of tedious nonsense up with which I will not put

Re: Should comments break paragraphs?

[Eric Abrahamsen]

Rasmus <rasmus@gmx.us> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>>> There is also a #+begin_annotation block special to ODT export, and
>>> there was some talk about devising a backend-agnostic solution along
>>> those lines for comments that one /does/ want exported.
>>
>> Latex could use a marginpar, and HTML a specially-styled div (an <aside>
>> tag in HTML5) -- seems plausible!
>
> There's already support for inlinetasks in ox-latex and they create
> minipages.  It can easily be altered and different styles could be
> provided.  Users could then select how to translate inlinetasks.
> E.g. marginpars would presumably not work for beamer (although I have
> no idea).

I also thought, glancing at this whole discussion, that inlinetasks were
kind of what everyone was talking about: something that is stuck in the
middle of everything else, that doesn't semantically affect its
surroundings, that can be exported or not. I do think minipages and
margipars are quite semantically distinct, though.

Anyway, inlinetasks have always struck me as a gross hack (how many
stars, again?), and I wasn't actually prepared to code anything myself,
so I thought I'd restrain myself...

Re: Should comments break paragraphs?

[Rasmus]

Christian Moe <mail@christianmoe.com> writes:

>>> show up on the side of the documents.  Again, I don't agree that they
>>> are comments in the sense that '^#.*' is a comment.  Again, I think
>>> the correct approach for adding such notes is inlinetasks.
>>
>> +1
>>
>> Comments are internal to the org file and should not be
>> exported.
>
> +1
>
> There is also a #+begin_annotation block special to ODT export, and
> there was some talk about devising a backend-agnostic solution along
> those lines for comments that one /does/ want exported.

I wasn't aware of begin_annotation.  That's cool.  Yeah, a backed
agnostic note syntax would be cool.  I'd go with inlinetasks, but I'm
also happy with begin_annotation.  

A good thing about inlinetasks is that they can be turned off (from
export) easily.

pdfcomment for LaTeX is cool, but poorly supported by Evince, but
works great in the Mozilla Aurora.  And there's already Nicolas'
boxes.

I don't know how it could be added meaningfully to html, but it should
be pretty doable.

–Rasmus

-- 
Er du tosset for noge' lårt!

Re: Should comments break paragraphs?

[Eric Schulte]

[-- Attachment #1: Type: text/plain, Size: 324 bytes --]

>
> This is a general pre-export issue, it does not depend on the
> exporters themselves.
>
> So again, what prevents us to make it easy for users to treat comments
> as no-line before parsing and exporting?

Unless I'm mistaken, I think this is exactly what my previous patch does
(reproduced here for ease of reference).


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-support-inline-comments-w-o-breaking-paragraphs.patch --]
[-- Type: text/x-patch, Size: 1661 bytes --]

From ce4c30ebe56d4cd66810bca48824d8841e7b130d Mon Sep 17 00:00:00 2001
From: Eric Schulte <schulte.eric@gmail.com>
Date: Tue, 16 Jul 2013 09:44:59 -0600
Subject: [PATCH] support inline comments w/o breaking paragraphs

* lisp/org-element.el (org-element-parse-buffer): Strip inline comments
  as a pre-processing step before exporting.
---
 lisp/org-element.el | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index 8d64657..d69d9ba 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -4092,13 +4092,19 @@ or objects within the parse tree.
 
 This function assumes that current major mode is `org-mode'."
   (save-excursion
-    (goto-char (point-min))
-    (org-skip-whitespace)
-    (org-element--parse-elements
-     (point-at-bol) (point-max)
-     ;; Start in `first-section' mode so text before the first
-     ;; headline belongs to a section.
-     'first-section nil granularity visible-only (list 'org-data nil))))
+    (let ((save (buffer-string)))
+      (unwind-protect
+	  (progn
+	    (goto-char (point-min))
+	    (delete-matching-lines "^ *# .[^\n]*\n")
+	    (org-skip-whitespace)
+	    (org-element--parse-elements
+	     (point-at-bol) (point-max)
+	     ;; Start in `first-section' mode so text before the first
+	     ;; headline belongs to a section.
+	     'first-section nil granularity visible-only (list 'org-data nil)))
+	(delete-region (point-min) (point-max))
+	(insert save)))))
 
 (defun org-element-parse-secondary-string (string restriction &optional parent)
   "Recursively parse objects in STRING and return structure.
-- 
1.8.3.2


[-- Attachment #3: Type: text/plain, Size: 46 bytes --]


-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Should comments break paragraphs?

[Bastien]

Eric Schulte <schulte.eric@gmail.com> writes:

> Unless I'm mistaken, I think this is exactly what my previous patch does
> (reproduced here for ease of reference).

Yes, this is it.

I also reproduce Nicolas snippet elsewhere in the thread, which might
be useful for other:

  (defun my-comment-removal (backend)
    (delete-matching-lines "^[ \t]*#\\( \\|$\\)"))
  (add-hook 'org-export-before-parsing-hook 'my-comment-removal)


I won't discuss this further, because I feel like the discussion
was over before it even started.

-- 
 Bastien

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Bastien <bzg@gnu.org> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>> And the direction is wrong... Parsing shouldn't modify the buffer being
>> parsed, ever. But you can use a hook for that purpose.
>
> I didn't suggest that parsing should modify the before: I said "where
> comments are deleted before parsing and exporting".
>
> There should be an easy solution for that.

Yes, a hook, as I suggested:

  (defun my-comment-removal (backend)
    (delete-matching-lines "^[ \t]*#\\( \\|$\\)"))
  (add-hook 'org-export-before-parsing-hook 'my-comment-removal)

I maintain it is dangerous to make it a default (or even make it easier
than that) because it will cause a discrepancy between the real buffer
contents and what is really parsed. It can create strange things,
really.

> I didn't suggest to redefine comments.

Of course you did. You said comments should be available anywhere but in
source blocks, didn't you? It isn't possible without redefining them in
Org.

>> You have to change every part of Org that
>> assumes there will be no comment in its way (lists, agenda, babel,
>> parser and probably more I can't think of).
>>
>> If it's an HTML/ODT export issue, it's far easier to patch the export
>> back-ends instead. 10 lines of code in each one, maybe.
>
> This is a general pre-export issue, it does not depend on the
> exporters themselves.

What you want here is to give comments a special meaning just before
export, even though they don't have it otherwise. As an example, you can
remove the comment here just before export:

  text
  # comment
  text

but `org-element-at-point' will still see two paragraphs and a comment
between them nonetheless.

But, IIUC, the problem the OP wants to solve is different: he wants the
example above to be treated as a paragraph, not as two of them. In fact
this is more about paragraphs than comments (i.e. paragraphs definition
in targeted back-ends is different than Org's).

> So again, what prevents us to make it easy for users to treat comments
> as no-line before parsing and exporting?

It is already easy, see hook above. Nothing prevents you from making it
easier. But this is wrong.


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Eric S Fraga]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

[...]

>   (defun my-comment-removal (backend)
>     (delete-matching-lines "^[ \t]*#\\( \\|$\\)"))
>   (add-hook 'org-export-before-parsing-hook 'my-comment-removal)

Nicolas,

quick question as I am trying to understand the various hooks and
filters that make up the (not so new anymore ;-) exporter.

This little snippet works just fine for comment lines.  However, I tried
changing the regexp to one that matches an :ignoreheading: tag on a
headline.  This works when I do a LaTeX export but not an HTML one.  Why
would the behaviour be different in each case?

I have tried using org-export-before-processing-hook as well but I get
the same behaviour.

I am soooo confused!

Thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.3.50.1, Org release_8.0.5-337-g9f3bed

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> This little snippet works just fine for comment lines.  However, I tried
> changing the regexp to one that matches an :ignoreheading: tag on a
> headline.  This works when I do a LaTeX export but not an HTML one.  Why
> would the behaviour be different in each case?

It shouldn't. Would you mind sending your hook?


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Eric S Fraga]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Hello,
>
> Eric S Fraga <e.fraga@ucl.ac.uk> writes:
>
>> This little snippet works just fine for comment lines.  However, I tried
>> changing the regexp to one that matches an :ignoreheading: tag on a
>> headline.  This works when I do a LaTeX export but not an HTML one.  Why
>> would the behaviour be different in each case?
>
> It shouldn't. Would you mind sending your hook?

Actually, it works so sorry about the noise.  I just tried emacs -Q and
realised that one of my other customisations was getting in the way.

For the record, this is what I have which (almost) does what I want:

#+begin_src emacs-lisp
  (defun esf/remove-lines-with-ignore-heading-tag (backend)
    (message "Deleting lines with ignore heading tag")
    (delete-matching-lines "^\\*+.*[ \t]+[a-ZA-Z0-9:]*:ignoreheading:[a-ZA-Z0-9:]*$"))
  (add-hook 'org-export-before-processing-hook 'esf/remove-lines-with-ignore-heading-tag)
#+end_src

I say almost because I actually want to replace matching lines by "\n"
and not simply delete them (sort of going against the whole
thread... ;-).  But I'll figure this bit out -- good emacs lisp practice
for me!

thanks,
eric

-- 
: Eric S Fraga (0xFFFCF67D), Emacs 24.3.50.1, Org release_8.0.5-337-g9f3bed

Re: Should comments break paragraphs?

[Suvayu Ali]

On Tue, Jul 16, 2013 at 07:51:38PM +0200, Nicolas Goaziou wrote:
> Eric Schulte <schulte.eric@gmail.com> writes:
> 
> >> Even if you move it to ox.el, this is not an acceptable solution. Think
> >> about the following example
> >>
> >>  - item 1
> >>
> >>    # with a comment
> >>
> >>  - item 2
> >>
> >> If you simply delete matching lines, you break the list.
> >>
> >
> > Yes, I would argue that this list should be broken, because with or
> > without the comment the items are separated by two blank lines.
> 
> On the other hand, visually, this list looks perfectly correct.

Actually I can think of at least one use case where you would want that
to be a list.  Say when drafting, you have a list of things you are
discussing, but you have yet to fill in the details and the comment has
some notes/reminder for tomorrow.  In this case, adding missing content
should not change the structure of your document, i.e. join two lists
into one.

Cheers,

-- 
Suvayu

Open source is the future. It sets us free.

Re: Should comments break paragraphs?

[Eric Schulte]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>> So are you proposing to remove comments from Org-mode entirely? 
>
> Certainly not.
>

Great, then I think we're much closer together on this than I originally
thought.

>
> I'm on the "good enough" side and I simply suggest to live with it.
> Also, if you badly need inline comments, there's the empty "comment"
> macro:
>
>   #+MACRO: comment 
>
>   Some text {{{comment(This is my comment)}}} and some more.
>

I'm not suggesting adding inline comments, only that comment lines be
treated as "" instead of "\n".

>
>> But if we treat comments as semantically empty, then maybe they should
>> be ignored by the parser?
>
> Yes, we can ignore them at the parser level. But what about the rest of
> Org? There are many places that do not rely on the parser and do not
> ignore comments (lists being an example, agenda being another one, and
> I'm pretty sure comments at some locations can break Babel too).
>
> Something could be done, but again, it would certainly not be
> a one-liner in org-element.el or ox.el.
>

I'm happy to work to fix places where babel is breaking comments.

>
>>> Even if you move it to ox.el, this is not an acceptable solution. Think
>>> about the following example
>>>
>>>  - item 1
>>>
>>>    # with a comment
>>>
>>>  - item 2
>>>
>>> If you simply delete matching lines, you break the list.
>>>
>>
>> Yes, I would argue that this list should be broken, because with or
>> without the comment the items are separated by two blank lines.
>
> On the other hand, visually, this list looks perfectly correct.
>
>> In my opinion comments as semantically empty lines is a simple rule
>> for both users and parsers to understand.
>
> Comments are already almost semantically empty lines. That's why:
>
>   Text
>   # Comment
>   Other Text
>
> generates two paragraphs in Org (and in every back-end excepted `latex',
> but that's a different problem). They basically act as a paragraph
> separator, much like blank lines.
>
> Also, note that if they really are empty lines, they cannot be inlined
> within paragraphs either.
>

Sorry I was unclear, see above, by "semantically empty" I meant
replacing comment lines with "" instead of the current behavior (except
in the latex exporter) which replaces them with "\n".

Cheers,

>
>
> Regards,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Eric Schulte <schulte.eric@gmail.com> writes:

> Sorry I was unclear, see above, by "semantically empty" I meant
> replacing comment lines with "" instead of the current behavior (except
> in the latex exporter) which replaces them with "\n".

They are _not_ replaced by "\n". They are replaced by "". But the parser
sees two paragraphs, and so does the exporter (even if they are not
separated by a blank line).


Regards,

-- 
Nicolas Goaziou

Re: Should comments break paragraphs?

[Andreas Leha]

Eric Schulte <schulte.eric@gmail.com> writes:


[...]

>
> I'm happy to work to fix places where babel is breaking comments.
>

I guess this would be one of these cases:

--8<---------------cut here---------------start------------->8---
* Test

#+name: flowdiagram
#+header: :exports results
# #+header: :results raw
#+begin_src latex
  Hello World
#+end_src

#+results: flowdiagram
Hello World
--8<---------------cut here---------------end--------------->8---


[...]

Regards,
Andreas

Re: Should comments break paragraphs?

[Eric Schulte]

Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:

> Eric Schulte <schulte.eric@gmail.com> writes:
>
>
> [...]
>
>>
>> I'm happy to work to fix places where babel is breaking comments.
>>

I should add the conditional *if* we're going to improve comment support
globally (e.g., in the parser as well).  Else I am emphatically *not* up
for a host of new parsing edge cases. :)

Best,

>>
>
> I guess this would be one of these cases:
>
> --8<---------------cut here---------------start------------->8---
> * Test
>
> #+name: flowdiagram
> #+header: :exports results
> # #+header: :results raw
> #+begin_src latex
>   Hello World
> #+end_src
>
> #+results: flowdiagram
> Hello World
> --8<---------------cut here---------------end--------------->8---
>
>
> [...]
>
> Regards,
> Andreas
>
>

-- 
Eric Schulte
http://cs.unm.edu/~eschulte

Re: Should comments break paragraphs?

[Andreas Leha]

Eric Schulte <schulte.eric@gmail.com> writes:

> Andreas Leha <andreas.leha@med.uni-goettingen.de> writes:
>
>> Eric Schulte <schulte.eric@gmail.com> writes:
>>
>>
>> [...]
>>
>>>
>>> I'm happy to work to fix places where babel is breaking comments.
>>>
>
> I should add the conditional *if* we're going to improve comment support
> globally (e.g., in the parser as well).  Else I am emphatically *not* up
> for a host of new parsing edge cases. :)
>

That is how I read your offer.  Just to give an example that I stumbled
upon while experimenting with latex->tikz evaluation.

[...]

Regards,
Andreas

Re: Should comments break paragraphs?

[Nicolas Goaziou]

Hello,

Eric Schulte <schulte.eric@gmail.com> writes:

> I should add the conditional *if* we're going to improve comment support
> globally (e.g., in the parser as well).  Else I am emphatically *not* up
> for a host of new parsing edge cases. :)

*If* we're going to improve comments support (i.e. make comments real comments
like in programming languages), I can take care of the parser and even,
with some additional time, org-list.

But there's also the agenda to handle:

  * Headline
  # A annoying comment since agenda expects planning to be just below
  # the headline.
  SCHEDULED: <...>

And tables too:

  | a | b |
  # I really want to comment the row below, without breaking formulas.
  | c | d |

I'm not up for any of these two.

BTW, *if* comments are to become real comments, they should be allowed
even in source blocks. They would then become context free syntax,
which is much easier to handle. It would then be possible to rely a bit
more on newcomment.el.


Regards,

-- 
Nicolas Goaziou