Inline comments

Inline comments

[ypuntot]

[-- Attachment #1: Type: text/plain, Size: 212 bytes --]

Hi

Could it be possible to make inline comments (# something like this) ?

I found this, but it doesn't seem to world, or I misused it.
https://gist.github.com/andersjohansson/6baa1e42ad4d7353e125

Best regards

[-- Attachment #2: Type: text/html, Size: 732 bytes --]

Re: Inline comments

[Max Nikulin]

On 22/06/2023 07:04, ypuntot wrote:
> 
> Could it be possible to make inline comments (# something like this) ?
> 
> I found this, but it doesn't seem to world, or I misused it.
> https://gist.github.com/andersjohansson/6baa1e42ad4d7353e125

Unfortunately the only native comments that Org has are ones that 
separate block-level elements. There are some workarounds however: 
@@comment: ...@@ taking advantage of unknown export backend, 
{{{comment(...)}}} macro expanding to nothing. Perhaps a dedicated babel 
language may be defined exporting nothing by default.

Previous discussion:

Max Nikulin to emacs-orgmode… Re: [PATCH] ox: fix comment exported as a 
blank line. Thu, 2 Jun 2022 23:09:04 +0700.
https://list.orgmode.org/e384d381-0a60-f777-b285-e0b8e62b14a9@gmail.com

[PATCH] org-faq.org: Inline comments

[Max Nikulin]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

On 22/06/2023 08:50, Max Nikulin wrote:
> On 22/06/2023 07:04, ypuntot wrote:
>>
>> Could it be possible to make inline comments (# something like this) ?
...
> Previous discussion:
...

I think, it is time to add a FAQ entry, see the attachment.

To my taste FAQ is rather long already. If you have an idea of another 
place on Worg where such tricks may be added then, please, share it.

[-- Attachment #2: 0001-org-faq.org-Inline-comments.patch --]
[-- Type: text/x-patch, Size: 3414 bytes --]

From a8dd1797588bdab7f63628579414c65d7444f387 Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Wed, 28 Jun 2023 22:40:40 +0700
Subject: [PATCH] org-faq.org: Inline comments

* org-faq.org (Export): New heading describing workarounds
for inline comments.
---
 org-faq.org | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/org-faq.org b/org-faq.org
index ab661395..7048ab2d 100644
--- a/org-faq.org
+++ b/org-faq.org
@@ -4492,6 +4492,59 @@ ** How can I suppress the page number in the footer of an exported PDF?
 
 : #+LATEX: \thispagestyle{empty}
 
+** Is it possible to add inline comments?
+:PROPERTIES:
+:CUSTOM_ID: inline-comments
+:END:
+
+Unfortunately [[https://orgmode.org/manual/Comment-Lines.html][comments]]
+in Org are block-level elements, so they break paragraphs.
+However some workarounds are possible.
+
+Choose a non-standard export backend name and use it in
+[[https://orgmode.org/worg/org-syntax.html#Export_Snippets][export snippets]]:
+=@@comment: rewrite it @@=  or even =@@c: be concise@@=.
+
+Define a [[https://orgmode.org/manual/Macro-Replacement.html][macro]]
+that expands to nothing
+: #+MACRO: comment
+an use it as ={{{comment( not ideal )}}}=.
+A downside of this approach is warnings issued by ~org-lint~.
+
+Define a [[https://orgmode.org/manual/Adding-Hyperlink-Types.html][custom link type]]
+#+begin_src elisp :exports code :eval never-export
+  (org-link-set-parameters "rem" :export (lambda (&rest _) ""))
+#+end_src
+A bonus is that you may create a
+=[[rem: hidden part of comment][...]]= try =C-h .= (~display-local-help~)
+or =C-c C-l= (~org-insert-link~) to read first part of the comment.
+Another link syntax is =<rem: comment>=.  Beware of patterns that may
+appear in plain text and be recognized as links even without
+enclosing brackets like =rem:10=.  Use a unique sequence of letters
+as the link type.
+
+Use inline [[https://orgmode.org/manual/Structure-of-Code-Blocks.html][source code blocks]]
+for a programming language that does not appear in your text.
+Just suppress export
+: #+header-args:csh :exports none :eval never-export
+and =src_csh{code}= becomes indistinguishable from comments.
+Perhaps with a custom ~org-babel~ backend that evaluates
+code to the same string you may control whether comments
+should appear in exported file.
+
+It is safer to avoid lines that have no text besides comments.
+They may be treated as paragraph separators.
+
+Unfortunately never approach suggested above works on sites for
+hosting of open source code projects. Literal markup
+(or broken links) will appear in e.g. formatted =README.org= files.
+
+Discussions on the mailing list:
+- [[https://list.orgmode.org/5210ac1c-ed73-4b82-a296-41cf90b9f0a7@gmail.com/][Inline comments. Thu, 22 Jun 2023 02:04:15 +0200]]
+- [[https://list.orgmode.org/mid:87zg8jwp26.fsf@localhost/][Re: [BUG] No space after footnote with org-export-with-footnotes set to nil. Sat, 11 Mar 2023 10:38:25 +0000]]
+- [[https://list.orgmode.org/87618398-cf6d-86b4-ba59-144f70d0f96e@7d.nz/][[PATCH] ox: fix comment exported as a blank line. Wed, 1 Jun 2022 07:30:01 +0200]]
+- [[https://list.orgmode.org/alpine.DEB.2.22.394.2107161156080.74845@shell3.miskatonic.org/][Comments break up a paragraph when writing one-setence-per-line. Fri, 16 Jul 2021 12:06:54 -0400]]
+
 * Backup
 
 #+index: Backup
-- 
2.25.1

Re: [PATCH] org-faq.org: Inline comments

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>>> Could it be possible to make inline comments (# something like this) ?
> ...
>> Previous discussion:
> ...
>
> I think, it is time to add a FAQ entry, see the attachment.

The proposed FAQ entry is overwhelming. It would work fine as a blog,
but not as a quick answer to a question. I recommend leaving only the
@@comment:...@@ part - it is what we use ourselves in WORG sources.

> To my taste FAQ is rather long already. If you have an idea of another 
> place on Worg where such tricks may be added then, please, share it.

FAQ is probably fine. We may add a CSS there that will fold the
answers by default.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] org-faq.org: Inline comments

[Max Nikulin]

On 29/06/2023 17:47, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> I think, it is time to add a FAQ entry, see the attachment.
> 
> The proposed FAQ entry is overwhelming. It would work fine as a blog,

A blog with a couple of posts a year has a little sense. Moreover it is 
closer to a wiki article because it is supposed to be updated. That is 
why I asked for an alternative location on Worg for bonus or extra 
material for the manual.

>> To my taste FAQ is rather long already. If you have an idea of another
>> place on Worg where such tricks may be added then, please, share it.
> 
> FAQ is probably fine. We may add a CSS there that will fold the
> answers by default.

When all entries are opened, it is possible to use search in browser. I 
am against the idea of hiding answers. However the table of contents, 
that is rather long and revealed on hover only, makes it harder to 
search in the TOC at first.

I have tried a random query "org-mode renumber footnotes" that is quite 
close to an entry from the document. Nether Google nor Bing offered FAQ 
on first 2 pages of search results. So FAQ is not friendly to search 
engines for some reasons. I am unsure if the rank may be improved by 
e.g. microdata schema.org attributes.

Re: [PATCH] org-faq.org: Inline comments

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> The proposed FAQ entry is overwhelming. It would work fine as a blog,
>
> A blog with a couple of posts a year has a little sense. Moreover it is 
> closer to a wiki article because it is supposed to be updated. That is 
> why I asked for an alternative location on Worg for bonus or extra 
> material for the manual.

I have no idea about alternative location.
Maybe someone more familiar with WORG. Bastien?

>> FAQ is probably fine. We may add a CSS there that will fold the
>> answers by default.
>
> When all entries are opened, it is possible to use search in browser. I 
> am against the idea of hiding answers. However the table of contents, 
> that is rather long and revealed on hover only, makes it harder to 
> search in the TOC at first.

Maybe we can add a proper TOC at the top? Or convert the on-hover TOC
into proper one. TOC on top + detailed answers is how many FAQ pages are
designed.

> I have tried a random query "org-mode renumber footnotes" that is quite 
> close to an entry from the document. Nether Google nor Bing offered FAQ 
> on first 2 pages of search results. So FAQ is not friendly to search 
> engines for some reasons. I am unsure if the rank may be improved by 
> e.g. microdata schema.org attributes.

I wish we had somebody who is familiar with these things. I personally
have no idea what can be wrong and where to start to solve this problem.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] org-faq.org: Inline comments

[Bastien Guerry]

Ihor Radchenko <yantar92@posteo.net> writes:

> Max Nikulin <manikulin@gmail.com> writes:
>
>>> The proposed FAQ entry is overwhelming. It would work fine as a blog,
>>
>> A blog with a couple of posts a year has a little sense. Moreover it is 
>> closer to a wiki article because it is supposed to be updated. That is 
>> why I asked for an alternative location on Worg for bonus or extra 
>> material for the manual.
>
> I have no idea about alternative location.
> Maybe someone more familiar with WORG. Bastien?

FWIW, for such cases, I think a dedicated Worg page would make sense.

Also, yes, Worg css needs a lotta love.

-- 
 Bastien

Re: [PATCH] org-faq.org: Inline comments

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> FAQ is probably fine. We may add a CSS there that will fold the
>> answers by default.
>
> When all entries are opened, it is possible to use search in browser. I 
> am against the idea of hiding answers. However the table of contents, 
> that is rather long and revealed on hover only, makes it harder to 
> search in the TOC at first.

I am now watching
https://fosdem.org/2023/schedule/event/a11y_eaa_bfsg_wcag_wai_aria_wtf/

They suggest a combination of <details><summary> to keep hidden text
searchable. See https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Org FAQ design (Re: [PATCH] org-faq.org: Inline comments)

[Max Nikulin]

On 01/07/2023 22:24, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>>> FAQ is probably fine. We may add a CSS there that will fold the
>>> answers by default.
>>
>> When all entries are opened, it is possible to use search in browser. I
>> am against the idea of hiding answers. However the table of contents,
>> that is rather long and revealed on hover only, makes it harder to
>> search in the TOC at first.
> 
> I am now watching
> https://fosdem.org/2023/schedule/event/a11y_eaa_bfsg_wcag_wai_aria_wtf/

I have skimmed through the presentation.

> They suggest a combination of <details><summary> to keep hidden text
> searchable. See https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary

Unsupported by Firefox:

https://bugzilla.mozilla.org/show_bug.cgi?id=1724299
Implement auto-expanding <details>

I am unsure if it is possible to implement auto-expanding on link to 
specific question for Firefox.

However <details> is better than accordions in most cases.

I would consider to put TOC into <details open>

P.S. Have you noticed that MDN pages have "skip to content" link hidden 
by default, but revealed on using keyboard (TAB).

[PATCH v2] org-faq.org: Inline comments

[Max Nikulin]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

On 29/06/2023 17:47, Ihor Radchenko wrote:
> Max Nikulin writes:
>> I think, it is time to add a FAQ entry, see the attachment.
> 
> The proposed FAQ entry is overwhelming. It would work fine as a blog,
> but not as a quick answer to a question. I recommend leaving only the
> @@comment:...@@ part

See a shortened variant.

> - it is what we use ourselves in WORG sources.

I have not found it in the worg repository. There are enough @@html:@@ 
with a handful of @@latex:@@.

[-- Attachment #2: v2-0001-org-faq.org-Inline-comments.patch --]
[-- Type: text/x-patch, Size: 1941 bytes --]

From fc587b06c20e865636f16104c13b3f239680e7bf Mon Sep 17 00:00:00 2001
From: Max Nikulin <manikulin@gmail.com>
Date: Wed, 28 Jun 2023 22:40:40 +0700
Subject: [PATCH v2] org-faq.org: Inline comments

* org-faq.org (Export): New heading describing workarounds
for inline comments.
---
 org-faq.org | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/org-faq.org b/org-faq.org
index ab661395..42cd2ef4 100644
--- a/org-faq.org
+++ b/org-faq.org
@@ -4492,6 +4492,38 @@ ** How can I suppress the page number in the footer of an exported PDF?
 
 : #+LATEX: \thispagestyle{empty}
 
+** Is it possible to add inline comments?
+:PROPERTIES:
+:CUSTOM_ID: inline-comments
+:END:
+
+Unfortunately [[https://orgmode.org/manual/Comment-Lines.html][comments]]
+in Org mode are block-level elements, so they break paragraphs.
+
+As a workaround choose a non-standard export backend name and use it in
+[[https://orgmode.org/worg/org-syntax.html#Export_Snippets][export snippets]]:
+=@@comment: rewrite it @@= or even =@@c: be concise@@=.
+It is safer to avoid export snippets having no other content before or after them
+on the same line.
+#+begin_src org
+The following line may become a patagraph separator.
+@@comment: might give unexpected effect @@
+Put some text before @@comment: a better variant
+@@ and after instread.
+#+end_src
+
+Alternatives are:
+a [[https://orgmode.org/manual/Macro-Replacement.html][macro]]
+expanding to nothing,
+a [[https://orgmode.org/manual/Adding-Hyperlink-Types.html][custom link type]]
+exporting to empty string,
+[[https://orgmode.org/manual/Structure-of-Code-Blocks.html][source code blocks]]
+with exporting and evaluation suppressed through default header arguments.
+
+Unfortunately sites for hosting of open source code projects
+do not support such tricks and inline comments may appear in formatted
+=README.org= or other files.
+
 * Backup
 
 #+index: Backup
-- 
2.25.1

Re: [PATCH v2] org-faq.org: Inline comments

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> The proposed FAQ entry is overwhelming. It would work fine as a blog,
>> but not as a quick answer to a question. I recommend leaving only the
>> @@comment:...@@ part
>
> See a shortened variant.

Thanks!

>> - it is what we use ourselves in WORG sources.
>
> I have not found it in the worg repository. There are enough @@html:@@ 
> with a handful of @@latex:@@.

Yeah, not in WORG.
https://orgmode.org/quickstart.html#text-comments

> +** Is it possible to add inline comments?
> +:PROPERTIES:
> +:CUSTOM_ID: inline-comments
> +:END:
> +
> +Unfortunately [[https://orgmode.org/manual/Comment-Lines.html][comments]]
> +in Org mode are block-level elements, so they break paragraphs.

Please, avoid "unfortunately". This reads kinda strange.
Comments do break paragraphs, which is our design decision. Some people
like it, some don't.

> +It is safer to avoid export snippets having no other content before or after them
> +on the same line.
> +#+begin_src org
> +The following line may become a patagraph separator.
> +@@comment: might give unexpected effect @@
> +Put some text before @@comment: a better variant
> +@@ and after instread.
> +#+end_src

May you please elaborate?
If you see inline export block starting a paragraph, it is a bug.

> +Alternatives are:
> +a [[https://orgmode.org/manual/Macro-Replacement.html][macro]]
> +expanding to nothing,
> +a [[https://orgmode.org/manual/Adding-Hyperlink-Types.html][custom link type]]
> +exporting to empty string,
> +[[https://orgmode.org/manual/Structure-of-Code-Blocks.html][source code blocks]]
> +with exporting and evaluation suppressed through default header arguments.

Again, this is not helpful as FAQ entry. We do not need to provide
overwhelming number of possibilities here.

> +Unfortunately sites for hosting of open source code projects
> +do not support such tricks and inline comments may appear in formatted
> +=README.org= or other files.

Why do we need to mention it here?
If you really want to explain this problem, it should be a dedicated FAQ
entry. Rendering Org files on Github and other sites has many more
problems not limited to inline export snippets.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH v2] org-faq.org: Inline comments

[Max Nikulin]

On 08/07/2023 16:31, Ihor Radchenko wrote:
>>> - it is what we use ourselves in WORG sources.

> Max Nikulin writes:
>> I have not found it in the worg repository. There are enough @@html:@@
>> with a handful of @@latex:@@.
> 
> Yeah, not in WORG.
> https://orgmode.org/quickstart.html#text-comments

It makes the FAQ entry having no value if your suggestions are taken 
into account. (I would add a link to annotating that is another sort of 
comments instead of dropping content further.) So I am giving up and 
leaving opportunity to propose changes to somebody else.

The section in quickstart was added in

64922ca 2020-10-04 14:28:00 -0400 Tom Gillespie: quickstart add sections 
for comments and macros at the end

I am surprises that quickstart is more detailed than the manual. However 
I have no idea how to have this section in both documents with proper 
balance in respect to details. Literal copy may be distracting for 
readers familiar with quickstart.

Bing rates quickstart reasonably, but Google avoids it and I have no 
idea why. Some hypotheses:
- quickstart uses generic metadata for the whole site, not specific for 
this page
- <h1> repeating <title> has display:none property
- invalid markup in index.html (a pitfall with affiliated keywords 
breaking paragraphs) for the link to quickstart:

<p>
<a href="quickstart.html">
</p>

<figure id="org490449d">
<img src="resources/img/quickstart.svg" alt="Running stickfigure" 
class="org-svg" title="Start using Org. You'll never stop.">

</figure>

<p>
</a>

I am unsure, but I would expect something like

<a href="quickstart.html" title="Getting started with Org-mode"
    ><img src="resources/img/quickstart.svg" alt="Quickstart"
          class="org-svg"></a>

>> +#+begin_src org
>> +The following line may become a patagraph separator.
>> +@@comment: might give unexpected effect @@
>> +Put some text before @@comment: a better variant
>> +@@ and after instread.
>> +#+end_src
> 
> May you please elaborate?
> If you see inline export block starting a paragraph, it is a bug.

I failed to convince you that it is a general issue, not a LaTeX 
specific one.

Ihor Radchenko to emacs-orgmode… Re: [BUG] No space after footnote with 
org-export-with-footnotes set to nil [9.6.1 ( @ 
/Users/test/.emacs.d/elpa/28.0/develop/org-9.6.1/)] Tue, 14 Mar 2023 
12:19:11 +0000. https://list.orgmode.org/87fsa7o79c.fsf@localhost

try to export to MarkDown. Even for LaTeX the fix is in the development 
branch only. There are enough users of built-in Org that deserve to be 
warned.

Re: [PATCH v2] org-faq.org: Inline comments

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 08/07/2023 16:31, Ihor Radchenko wrote:
>>>> - it is what we use ourselves in WORG sources.
>
>> Max Nikulin writes:
>>> I have not found it in the worg repository. There are enough @@html:@@
>>> with a handful of @@latex:@@.
>> 
>> Yeah, not in WORG.
>> https://orgmode.org/quickstart.html#text-comments
>
> It makes the FAQ entry having no value if your suggestions are taken 
> into account. (I would add a link to annotating that is another sort of 
> comments instead of dropping content further.) So I am giving up and 
> leaving opportunity to propose changes to somebody else.

https://git.sr.ht/~bzg/worg/commit/f598c7ef

> Bing rates quickstart reasonably, but Google avoids it and I have no 
> idea why. Some hypotheses:
> - quickstart uses generic metadata for the whole site, not specific for 
> this page
> - <h1> repeating <title> has display:none property
> - invalid markup in index.html (a pitfall with affiliated keywords 
> breaking paragraphs) for the link to quickstart:
>
> <p>
> <a href="quickstart.html">
> </p>
>
> <figure id="org490449d">
> <img src="resources/img/quickstart.svg" alt="Running stickfigure" 
> class="org-svg" title="Start using Org. You'll never stop.">
>
> </figure>
>
> <p>
> </a>
>
> I am unsure, but I would expect something like
>
> <a href="quickstart.html" title="Getting started with Org-mode"
>     ><img src="resources/img/quickstart.svg" alt="Quickstart"
>           class="org-svg"></a>

I have no idea here. I wish somebody more familiar with search engine
optimization chimed in.

>>> +#+begin_src org
>>> +The following line may become a patagraph separator.
>>> +@@comment: might give unexpected effect @@
>>> +Put some text before @@comment: a better variant
>>> +@@ and after instread.
>>> +#+end_src
>> 
>> May you please elaborate?
>> If you see inline export block starting a paragraph, it is a bug.
>
> I failed to convince you that it is a general issue, not a LaTeX 
> specific one.
>
> Ihor Radchenko to emacs-orgmode… Re: [BUG] No space after footnote with 
> org-export-with-footnotes set to nil [9.6.1 ( @ 
> /Users/test/.emacs.d/elpa/28.0/develop/org-9.6.1/)] Tue, 14 Mar 2023 
> 12:19:11 +0000. https://list.orgmode.org/87fsa7o79c.fsf@localhost
>
> try to export to MarkDown. Even for LaTeX the fix is in the development 
> branch only. There are enough users of built-in Org that deserve to be 
> warned.

The linked message was talking about macros. I still think that for
macros specifically it was not a bug - macros may be expanded into
multiple paragraphs, which is perfectly fine. We should not force macro
expansion to ensure a single paragraph.

As for comments, I tend to agree that it is indeed a bug.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH v2] org-faq.org: Inline comments

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

>>>> +#+begin_src org
>>>> +The following line may become a patagraph separator.
>>>> +@@comment: might give unexpected effect @@
>>>> +Put some text before @@comment: a better variant
>>>> +@@ and after instread.
>>>> +#+end_src
>>> 
>>> May you please elaborate?
>>> If you see inline export block starting a paragraph, it is a bug.
> ...
>
> As for comments, I tend to agree that it is indeed a bug.

I am no longer able to reproduce the problem with @@comment:...@@
splitting paragraph in the above example.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH v2] org-faq.org: Inline comments

[Max Nikulin]

On 15/04/2024 19:17, Ihor Radchenko wrote:
> 
>>>>> +#+begin_src org
>>>>> +The following line may become a patagraph separator.
>>>>> +@@comment: might give unexpected effect @@
>>>>> +Put some text before @@comment: a better variant
>>>>> +@@ and after instread.
>>>>> +#+end_src
> 
> I am no longer able to reproduce the problem with @@comment:...@@
> splitting paragraph in the above example.

I see no changes in respect to ox-md on the main branch.

---- 8< ----
The following line may become a paragraph separator.
@@comment: might give unexpected effect @@
Put some text before @@comment: a better variant
@@ and after instead.
---- >8 ----

is exported to

---- 8< ----
The following line may become a paragraph separator.

Put some text before  and after instead.
---- >8 ----

Trailing whitespace after export snippets without a transcoder (was: [PATCH v2] org-faq.org: Inline comments)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>>>>>> +#+begin_src org
>>>>>> +The following line may become a patagraph separator.
>>>>>> +@@comment: might give unexpected effect @@
>>>>>> +Put some text before @@comment: a better variant
>>>>>> +@@ and after instread.
>>>>>> +#+end_src
>> 
>> I am no longer able to reproduce the problem with @@comment:...@@
>> splitting paragraph in the above example.
>
> I see no changes in respect to ox-md on the main branch.

Right.
Upon debugging further, to my great surprise, I found the following
comment in ox.el:

			;; Export snippets never return a nil value so
			;; that white spaces following them are never
			;; ignored.
			(and (eq type 'export-snippet) "")

and then even a test:

;; Ignored export snippets do not remove any blank.
  (should
   (equal "begin  end\n"
	  (org-test-with-parsed-data "begin @@test:A@@ end"
	    (org-export-data-with-backend
	     tree
	     (org-export-create-backend
	      :transcoders
	      '((paragraph . (lambda (paragraph contents info) contents))
		(section . (lambda (section contents info) contents))))
	     info))))

The test was added in the commit
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a6d9fd82e

    Back-end developers should pay attention to the fact that white spaces
    before and after an ignored export snippet now are accumulated in the
    output.

I have no clue about the rationale of this special behaviour - it dates
back to the days when Org export was merged. It is also not documented
anywhere, AFAIK.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Trailing whitespace after export snippets without a transcoder

[Max Nikulin]

On 20/04/2024 18:14, Ihor Radchenko wrote:
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a6d9fd82e
> 
>      Back-end developers should pay attention to the fact that white spaces
>      before and after an ignored export snippet now are accumulated in the
>      output.
> 
> I have no clue about the rationale of this special behaviour - it dates
> back to the days when Org export was merged. It is also not documented
> anywhere, AFAIK.

I would not expect that the space after the following export snippet is 
ignored in the case of ox-html or ox-latex backend:

     A space@@ascii:*@@ character.

The space may be put inside the export snippet if the intention is to 
omit it for output formats other than plain text. So current behavior is 
perfectly reasonable and flexible enough.

The issue is empty lines that serve as paragraph separators and that may 
appear due to expansion of a macro or due to skipped export snippets. 
Perhaps transcoders of other elements, e.g. links, may return empty 
strings as well.

Re: Trailing whitespace after export snippets without a transcoder

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

>> I have no clue about the rationale of this special behaviour - it dates
>> back to the days when Org export was merged. It is also not documented
>> anywhere, AFAIK.
>
> I would not expect that the space after the following export snippet is 
> ignored in the case of ox-html or ox-latex backend:
>
>      A space@@ascii:*@@ character.
>
> The space may be put inside the export snippet if the intention is to 
> omit it for output formats other than plain text. So current behavior is 
> perfectly reasonable and flexible enough.

Hmm. We actually have a similar scenario in `org-export--prune-tree'
with a slightly different logic - only keep spaces when previous object
does not have any.

I am attaching tentative patch that will duplicate the logic for export
snippets as well. And for any other object where transcoder returns nil.
WDYT?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-export-data-Handle-trailing-spaces-when-transcod.patch --]
[-- Type: text/x-patch, Size: 3472 bytes --]

From 54939c4044fb407b068c0666c258ccd01e59c2af Mon Sep 17 00:00:00 2001
Message-ID: <54939c4044fb407b068c0666c258ccd01e59c2af.1713703523.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sun, 21 Apr 2024 15:37:18 +0300
Subject: [PATCH 1/2] org-export-data: Handle trailing spaces when transcoder
 returns nil

* lisp/ox.el (org-export-data): When transcoder returns nil, handle
trailing spaces after an object the same way `org-export--prune-tree'
does.  Remove special handling of export snippets that unconditionally
keep their trailing spaces.

Link: https://orgmode.org/list/87h6fwmgkm.fsf@localhost
---
 lisp/ox.el | 43 ++++++++++++++++++++++++++++++++-----------
 1 file changed, 32 insertions(+), 11 deletions(-)

diff --git a/lisp/ox.el b/lisp/ox.el
index fc746950d..ccc4c94ce 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -1930,15 +1930,11 @@ (defun org-export-data (data info)
 			   (eq (plist-get info :with-archived-trees) 'headline)
 			   (org-element-property :archivedp data)))
 		  (let ((transcoder (org-export-transcoder data info)))
-		    (or (and (functionp transcoder)
-                             (if (eq type 'link)
-			         (broken-link-handler
-			          (funcall transcoder data nil info))
-                               (funcall transcoder data nil info)))
-			;; Export snippets never return a nil value so
-			;; that white spaces following them are never
-			;; ignored.
-			(and (eq type 'export-snippet) ""))))
+		    (and (functionp transcoder)
+                         (if (eq type 'link)
+			     (broken-link-handler
+			      (funcall transcoder data nil info))
+                           (funcall transcoder data nil info)))))
 		 ;; Element/Object with contents.
 		 (t
 		  (let ((transcoder (org-export-transcoder data info)))
@@ -1979,8 +1975,33 @@ (defun org-export-data (data info)
 	  (puthash
 	   data
 	   (cond
-	    ((not results) "")
-	    ((memq type '(nil org-data plain-text raw)) results)
+	    ((not results)
+             ;; TRANSCODER returned nil.  When DATA is an object,
+             ;; interpret this as if DATA should be ignored (see
+             ;; `org-export--prune-tree').  Keep spaces in place of
+             ;; removed element, if necessary.
+	     ;; Example: "Foo.[10%] Bar" would become
+	     ;; "Foo.Bar" if we do not keep spaces.
+             ;; Another example: "A space@@ascii:*@@ character."
+             ;; should become "A space character" in non-ASCII export.
+             (let ((post-blank (org-element-post-blank data)))
+               (or
+                (unless (or (not post-blank)
+                            (zerop post-blank)
+                            (eq 'element (org-element-class data)))
+                  (let ((previous (org-export-get-previous-element data info)))
+		    (unless (or (not previous)
+			        (pcase (org-element-type previous)
+				  (`plain-text
+				   (string-match-p
+				    (rx  whitespace eos) previous))
+				  (_ (org-element-post-blank previous))))
+                      ;; When previous element does not have
+                      ;; trailing spaces, keep the trailing
+                      ;; spaces from DATA.
+		      (make-string post-blank ?\s))))
+                "")))
+            ((memq type '(nil org-data plain-text raw)) results)
 	    ;; Append the same white space between elements or objects
 	    ;; as in the original buffer, and call appropriate filters.
 	    (t
-- 
2.44.0


[-- Attachment #3: Type: text/plain, Size: 763 bytes --]


> The issue is empty lines that serve as paragraph separators and that may 
> appear due to expansion of a macro or due to skipped export snippets. 
> Perhaps transcoders of other elements, e.g. links, may return empty 
> strings as well.

Right. This is a special case in MD where blank lines separate
paragraphs. Just like in ox-latex, where we fixed exactly same thing
reported in https://orgmode.org/list/tufdb6$11h2$1@ciao.gmane.io

It is also a side effect of the fact that newlines are not considered a
part of the Org markup objects.

I do not think that we need to handle this Org mode-wide (it will be
difficult and will likely cause breaking changes). We can only adjust
the export backends sensitive to blank lines.

See the attached tentative fix.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0001-ox-md-ox-ascii-ox-texinfo-Strip-blank-lines-from-par.patch --]
[-- Type: text/x-patch, Size: 4474 bytes --]

From 08c584b90ca6950e4186093bf5742d7443448254 Mon Sep 17 00:00:00 2001
Message-ID: <08c584b90ca6950e4186093bf5742d7443448254.1713704215.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sun, 21 Apr 2024 15:54:48 +0300
Subject: [PATCH] ox-md, ox-ascii, ox-texinfo: Strip blank lines from
 paragraphs

* lisp/org-macs.el (org-remove-blank-lines): New helper function to
strip blank lines from string.
* lisp/ox-ascii.el (org-ascii-paragraph):
* lisp/ox-latex.el (org-latex-paragraph):
* lisp/ox-md.el (org-md-paragraph):
* lisp/ox-texinfo.el (org-texinfo-paragraph): Strip blank lines from
paragraphs - these exporters are using blank lines as paragraph
separators.

Reported-by: Max Nikulin <manikulin@gmail.com>
Link: https://orgmode.org/list/v00le7$frp$1@ciao.gmane.io
---
 lisp/org-macs.el   | 4 ++++
 lisp/ox-ascii.el   | 6 ++++++
 lisp/ox-latex.el   | 4 +---
 lisp/ox-md.el      | 6 ++++++
 lisp/ox-texinfo.el | 7 ++++++-
 5 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 0046f3493..85bf6e4fa 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -1554,6 +1554,10 @@ (defun org-remove-tabs (s &optional width)
 	     t t s)))
   s)
 
+(defun org-remove-blank-lines (s)
+  "Remove blank lines in S."
+  (replace-regexp-in-string (rx "\n" (1+ (0+ space) "\n")) "\n" s))
+
 (defun org-wrap (string &optional width lines)
   "Wrap string to either a number of lines, or a width in characters.
 If WIDTH is non-nil, the string is wrapped to that width, however many lines
diff --git a/lisp/ox-ascii.el b/lisp/ox-ascii.el
index db4356ec6..e767f66cf 100644
--- a/lisp/ox-ascii.el
+++ b/lisp/ox-ascii.el
@@ -1651,6 +1651,12 @@ (defun org-ascii-paragraph (paragraph contents info)
   "Transcode a PARAGRAPH element from Org to ASCII.
 CONTENTS is the contents of the paragraph, as a string.  INFO is
 the plist used as a communication channel."
+  ;; Ensure that we do not create multiple paragraphs, when a single
+  ;; paragraph is expected.
+  ;; Multiple newlines may appear in CONTENTS, for example, when
+  ;; certain objects are stripped from export, leaving single newlines
+  ;; before and after.
+  (setq contents (org-remove-blank-lines contents))
   (org-ascii--justify-element
    (let ((indented-line-width (plist-get info :ascii-indented-line-width)))
      (if (not (wholenump indented-line-width)) contents
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 8a10f9390..cae7bb3b2 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -3040,9 +3040,7 @@ (defun org-latex-paragraph (_paragraph contents _info)
   ;; Multiple newlines may appear in CONTENTS, for example, when
   ;; certain objects are stripped from export, leaving single newlines
   ;; before and after.
-  (replace-regexp-in-string
-   (rx "\n" (1+ (0+ space) "\n")) "\n"
-   contents))
+  (org-remove-blank-lines contents))
 
 
 ;;;; Plain List
diff --git a/lisp/ox-md.el b/lisp/ox-md.el
index fa2beeb95..28f0a4cf6 100644
--- a/lisp/ox-md.el
+++ b/lisp/ox-md.el
@@ -628,6 +628,12 @@ (defun org-md-paragraph (paragraph contents _info)
   "Transcode PARAGRAPH element into Markdown format.
 CONTENTS is the paragraph contents.  INFO is a plist used as
 a communication channel."
+  ;; Ensure that we do not create multiple paragraphs, when a single
+  ;; paragraph is expected.
+  ;; Multiple newlines may appear in CONTENTS, for example, when
+  ;; certain objects are stripped from export, leaving single newlines
+  ;; before and after.
+  (setq contents (org-remove-blank-lines contents))
   (let ((first-object (car (org-element-contents paragraph))))
     ;; If paragraph starts with a #, protect it.
     (if (and (stringp first-object) (string-prefix-p "#" first-object))
diff --git a/lisp/ox-texinfo.el b/lisp/ox-texinfo.el
index 4aef9c41c..fc9ec9209 100644
--- a/lisp/ox-texinfo.el
+++ b/lisp/ox-texinfo.el
@@ -1517,7 +1517,12 @@ (defun org-texinfo-paragraph (_paragraph contents _info)
   "Transcode a PARAGRAPH element from Org to Texinfo.
 CONTENTS is the contents of the paragraph, as a string.  INFO is
 the plist used as a communication channel."
-  contents)
+  ;; Ensure that we do not create multiple paragraphs, when a single
+  ;; paragraph is expected.
+  ;; Multiple newlines may appear in CONTENTS, for example, when
+  ;; certain objects are stripped from export, leaving single newlines
+  ;; before and after.
+  (org-remove-blank-lines contents))
 
 ;;;; Plain List
 
-- 
2.44.0


[-- Attachment #5: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Trailing whitespace after export snippets without a transcoder

[Max Nikulin]

On 21/04/2024 20:00, Ihor Radchenko wrote:
> Max Nikulin writes:
>>       A space@@ascii:*@@ character.
> 
> Hmm. We actually have a similar scenario in `org-export--prune-tree'
> with a slightly different logic - only keep spaces when previous object
> does not have any.

I like this idea. At first I even believed that backend-specific code 
would not be necessary.

> I do not think that we need to handle this Org mode-wide (it will be
> difficult and will likely cause breaking changes).

I have not figured out why it may become a breaking changes and what 
backends need blank lines inside paragraph. I would make stripping empty 
lines default behavior with some option to disable this feature.

> See the attached tentative fix.

Since zero width spaces are part of Org syntax, they need special treatment.

---- 8< ----
#+macro: empty (eval "")

Some *bold*​@@comment: *@@ text.
@@comment: line@@
More /italic/​{{{empty}}} text.
{{{empty}}}
Last line.
---- >8 ----

LaTeX export:
---- 8< ----
Some \textbf{bold}​text.
More \emph{italic}​ text.

Last line.
---- >8 ----

Notice visible space character disappeared after "bold". I am leaving up 
to you to decide if empty line appeared due to a macro is a bug or a 
feature. If I remember it correctly, your opinion is that a macro 
expanding to multiple paragraphs is a valid one.

Re: Trailing whitespace after export snippets without a transcoder

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 2366 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

>> I do not think that we need to handle this Org mode-wide (it will be
>> difficult and will likely cause breaking changes).
>
> I have not figured out why it may become a breaking changes and what 
> backends need blank lines inside paragraph. I would make stripping empty 
> lines default behavior with some option to disable this feature.

For example, consider an HTML exporter that aligns tags nicely and keeps
blank lines between markup blocks for readability.  If we remove such
blank lines unconditionally, it will be problematic.

>> See the attached tentative fix.
>
> Since zero width spaces are part of Org syntax, they need special treatment.

They are not a part of Org syntax, and we currently do not handle them
specially. They still work as escape-character simply because Org syntax
defines markup boundaries using a closed set of whitespace characters -
(rx (any " \t")). So, any non-tab non-space whitespace will be an
equivalent of zero-width space for all practical purposes.

> ---- 8< ----
> #+macro: empty (eval "")
>
> Some *bold*​@@comment: *@@ text.
> @@comment: line@@
> More /italic/​{{{empty}}} text.
> {{{empty}}}
> Last line.
> ---- >8 ----
>
> LaTeX export:
> ---- 8< ----
> Some \textbf{bold}​text.
> More \emph{italic}​ text.
>
> Last line.
> ---- >8 ----
>
> Notice visible space character disappeared after "bold".

I guess that I can change the condition to not include trailing space
from (rx whitespace eol) to (rx (any " \t|) eol).

See the attached updated version of the patch set.

> ... I am leaving up 
> to you to decide if empty line appeared due to a macro is a bug or a 
> feature. If I remember it correctly, your opinion is that a macro 
> expanding to multiple paragraphs is a valid one.

Yes. I do believe that we should keep macros as dumb as possible, so
that people can use them in the most flexible ways, including breaking
paragraphs, if so desired.

A more annoying one is

First line
@@comment:foo@@
last line.

vs.

First line
@@comment:foo
@@last line.

where we encounter the peculiarity of Org syntax with trailing tabs and
spaces included as part of the object, but not newlines.

But I do not see any good way to address this problem without rewriting
half of Org mode.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v2-0001-org-export-data-Handle-trailing-spaces-when-trans.patch --]
[-- Type: text/x-patch, Size: 5708 bytes --]

From 229a563dc38e1fdfd63be2dfebb1a9e9023e44b2 Mon Sep 17 00:00:00 2001
Message-ID: <229a563dc38e1fdfd63be2dfebb1a9e9023e44b2.1713812419.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sun, 21 Apr 2024 15:37:18 +0300
Subject: [PATCH v2 1/2] org-export-data: Handle trailing spaces when
 transcoder returns nil

* lisp/ox.el (org-export--keep-spaces): New helper function containing
logic about keeping spaces in place of removed object from
`org-export--prune-tree'.  The logic is modified to keep spaces in the
case when previous plain-string object ends with a whitespace, but not
" " or "\t".  This can happen, for example, when there is a trailing
zero-width space.  We do want to keep spaces in such scenario.
(org-export-data): When transcoder returns nil, handle
trailing spaces after an object the same way `org-export--prune-tree'
does.  Remove special handling of export snippets that unconditionally
keep their trailing spaces.
(org-export--prune-tree): Use the helper function.

Link: https://orgmode.org/list/87h6fwmgkm.fsf@localhost
---
 lisp/ox.el | 67 ++++++++++++++++++++++++++++++++++--------------------
 1 file changed, 42 insertions(+), 25 deletions(-)

diff --git a/lisp/ox.el b/lisp/ox.el
index fc746950d..6f6689188 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -1880,6 +1880,38 @@ (defun org-export-transcoder (blob info)
       (let ((transcoder (cdr (assq type (plist-get info :translate-alist)))))
 	(and (functionp transcoder) transcoder)))))
 
+(defun org-export--keep-spaces (data info)
+  "Non-nil, when post-blank spaces after removing DATA should be preserved.
+INFO is the info channel.
+
+This function returns nil, when previous exported element already has
+trailing spaces or when DATA does not have non-zero non-nil
+`:post-blank' property.
+
+When the return value is non-nil, it is a string containing the trailing
+spaces."
+  ;; When DATA is an object, interpret this as if DATA should be
+  ;; ignored (see `org-export--prune-tree').  Keep spaces in place of
+  ;; removed element, if necessary.  Example: "Foo.[10%] Bar" would
+  ;; become "Foo.Bar" if we do not keep spaces.  Another example: "A
+  ;; space@@ascii:*@@ character."  should become "A space character"
+  ;; in non-ASCII export.
+  (let ((post-blank (org-element-post-blank data)))
+    (unless (or (not post-blank)
+                (zerop post-blank)
+                (eq 'element (org-element-class data)))
+      (let ((previous (org-export-get-previous-element data info)))
+	(unless (or (not previous)
+		    (pcase (org-element-type previous)
+		      (`plain-text
+		       (string-match-p
+			(rx (any " \t") eos) previous))
+		      (_ (org-element-post-blank previous))))
+          ;; When previous element does not have
+          ;; trailing spaces, keep the trailing
+          ;; spaces from DATA.
+	  (make-string post-blank ?\s))))))
+
 ;;;###autoload
 (defun org-export-data (data info)
   "Convert DATA into current backend format.
@@ -1930,15 +1962,11 @@ (defun org-export-data (data info)
 			   (eq (plist-get info :with-archived-trees) 'headline)
 			   (org-element-property :archivedp data)))
 		  (let ((transcoder (org-export-transcoder data info)))
-		    (or (and (functionp transcoder)
-                             (if (eq type 'link)
-			         (broken-link-handler
-			          (funcall transcoder data nil info))
-                               (funcall transcoder data nil info)))
-			;; Export snippets never return a nil value so
-			;; that white spaces following them are never
-			;; ignored.
-			(and (eq type 'export-snippet) ""))))
+		    (and (functionp transcoder)
+                         (if (eq type 'link)
+			     (broken-link-handler
+			      (funcall transcoder data nil info))
+                           (funcall transcoder data nil info)))))
 		 ;; Element/Object with contents.
 		 (t
 		  (let ((transcoder (org-export-transcoder data info)))
@@ -1979,8 +2007,8 @@ (defun org-export-data (data info)
 	  (puthash
 	   data
 	   (cond
-	    ((not results) "")
-	    ((memq type '(nil org-data plain-text raw)) results)
+	    ((not results) (or (org-export--keep-spaces data info) ""))
+            ((memq type '(nil org-data plain-text raw)) results)
 	    ;; Append the same white space between elements or objects
 	    ;; as in the original buffer, and call appropriate filters.
 	    (t
@@ -2641,24 +2669,13 @@ (defun org-export--prune-tree (data info)
 		(let ((type (org-element-type data)))
 		  (if (org-export--skip-p data info selected excluded)
 		      (if (memq type '(table-cell table-row)) (push data ignore)
-			(let ((post-blank (org-element-post-blank data)))
-			  (if (or (not post-blank) (zerop post-blank)
-				  (eq 'element (org-element-class data)))
-			      (org-element-extract data)
+                        (if-let ((keep-spaces (org-export--keep-spaces data info)))
 			    ;; Keep spaces in place of removed
 			    ;; element, if necessary.
 			    ;; Example: "Foo.[10%] Bar" would become
 			    ;; "Foo.Bar" if we do not keep spaces.
-			    (let ((previous (org-export-get-previous-element data info)))
-			      (if (or (not previous)
-				      (pcase (org-element-type previous)
-					(`plain-text
-					 (string-match-p
-					  (rx  whitespace eos) previous))
-					(_ (org-element-post-blank previous))))
-				  ;; Previous object ends with whitespace already.
-				  (org-element-extract data)
-				(org-element-set data (make-string post-blank ?\s)))))))
+                            (org-element-set data keep-spaces)
+			  (org-element-extract data)))
 		    (if (and (eq type 'headline)
 			     (eq (plist-get info :with-archived-trees)
 				 'headline)
-- 
2.44.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: v2-0002-ox-md-ox-ascii-ox-texinfo-Strip-blank-lines-from-.patch --]
[-- Type: text/x-patch, Size: 4662 bytes --]

From 3fa3ed068fcfc58470430a5c4bae3a5ffd1ca3ed Mon Sep 17 00:00:00 2001
Message-ID: <3fa3ed068fcfc58470430a5c4bae3a5ffd1ca3ed.1713812419.git.yantar92@posteo.net>
In-Reply-To: <229a563dc38e1fdfd63be2dfebb1a9e9023e44b2.1713812419.git.yantar92@posteo.net>
References: <229a563dc38e1fdfd63be2dfebb1a9e9023e44b2.1713812419.git.yantar92@posteo.net>
From: Ihor Radchenko <yantar92@posteo.net>
Date: Sun, 21 Apr 2024 15:54:48 +0300
Subject: [PATCH v2 2/2] ox-md, ox-ascii, ox-texinfo: Strip blank lines from
 paragraphs

* lisp/org-macs.el (org-remove-blank-lines): New helper function to
strip blank lines from string.
* lisp/ox-ascii.el (org-ascii-paragraph):
* lisp/ox-latex.el (org-latex-paragraph):
* lisp/ox-md.el (org-md-paragraph):
* lisp/ox-texinfo.el (org-texinfo-paragraph): Strip blank lines from
paragraphs - these exporters are using blank lines as paragraph
separators.

Reported-by: Max Nikulin <manikulin@gmail.com>
Link: https://orgmode.org/list/v00le7$frp$1@ciao.gmane.io
---
 lisp/org-macs.el   | 4 ++++
 lisp/ox-ascii.el   | 6 ++++++
 lisp/ox-latex.el   | 4 +---
 lisp/ox-md.el      | 6 ++++++
 lisp/ox-texinfo.el | 7 ++++++-
 5 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 1254ddb54..93803bfe9 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -1244,6 +1244,10 @@ (defun org-remove-tabs (s &optional width)
 	     t t s)))
   s)
 
+(defun org-remove-blank-lines (s)
+  "Remove blank lines in S."
+  (replace-regexp-in-string (rx "\n" (1+ (0+ space) "\n")) "\n" s))
+
 (defun org-wrap (string &optional width lines)
   "Wrap string to either a number of lines, or a width in characters.
 If WIDTH is non-nil, the string is wrapped to that width, however many lines
diff --git a/lisp/ox-ascii.el b/lisp/ox-ascii.el
index db4356ec6..e767f66cf 100644
--- a/lisp/ox-ascii.el
+++ b/lisp/ox-ascii.el
@@ -1651,6 +1651,12 @@ (defun org-ascii-paragraph (paragraph contents info)
   "Transcode a PARAGRAPH element from Org to ASCII.
 CONTENTS is the contents of the paragraph, as a string.  INFO is
 the plist used as a communication channel."
+  ;; Ensure that we do not create multiple paragraphs, when a single
+  ;; paragraph is expected.
+  ;; Multiple newlines may appear in CONTENTS, for example, when
+  ;; certain objects are stripped from export, leaving single newlines
+  ;; before and after.
+  (setq contents (org-remove-blank-lines contents))
   (org-ascii--justify-element
    (let ((indented-line-width (plist-get info :ascii-indented-line-width)))
      (if (not (wholenump indented-line-width)) contents
diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index 5c19e1fe7..2267a604e 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -3039,9 +3039,7 @@ (defun org-latex-paragraph (_paragraph contents _info)
   ;; Multiple newlines may appear in CONTENTS, for example, when
   ;; certain objects are stripped from export, leaving single newlines
   ;; before and after.
-  (replace-regexp-in-string
-   (rx "\n" (1+ (0+ space) "\n")) "\n"
-   contents))
+  (org-remove-blank-lines contents))
 
 
 ;;;; Plain List
diff --git a/lisp/ox-md.el b/lisp/ox-md.el
index fa2beeb95..28f0a4cf6 100644
--- a/lisp/ox-md.el
+++ b/lisp/ox-md.el
@@ -628,6 +628,12 @@ (defun org-md-paragraph (paragraph contents _info)
   "Transcode PARAGRAPH element into Markdown format.
 CONTENTS is the paragraph contents.  INFO is a plist used as
 a communication channel."
+  ;; Ensure that we do not create multiple paragraphs, when a single
+  ;; paragraph is expected.
+  ;; Multiple newlines may appear in CONTENTS, for example, when
+  ;; certain objects are stripped from export, leaving single newlines
+  ;; before and after.
+  (setq contents (org-remove-blank-lines contents))
   (let ((first-object (car (org-element-contents paragraph))))
     ;; If paragraph starts with a #, protect it.
     (if (and (stringp first-object) (string-prefix-p "#" first-object))
diff --git a/lisp/ox-texinfo.el b/lisp/ox-texinfo.el
index 4aef9c41c..fc9ec9209 100644
--- a/lisp/ox-texinfo.el
+++ b/lisp/ox-texinfo.el
@@ -1517,7 +1517,12 @@ (defun org-texinfo-paragraph (_paragraph contents _info)
   "Transcode a PARAGRAPH element from Org to Texinfo.
 CONTENTS is the contents of the paragraph, as a string.  INFO is
 the plist used as a communication channel."
-  contents)
+  ;; Ensure that we do not create multiple paragraphs, when a single
+  ;; paragraph is expected.
+  ;; Multiple newlines may appear in CONTENTS, for example, when
+  ;; certain objects are stripped from export, leaving single newlines
+  ;; before and after.
+  (org-remove-blank-lines contents))
 
 ;;;; Plain List
 
-- 
2.44.0


[-- Attachment #4: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Trailing whitespace after export snippets without a transcoder

[Max Nikulin]

On 23/04/2024 02:01, Ihor Radchenko wrote:
> For example, consider an HTML exporter that aligns tags nicely and
> keeps blank lines between markup blocks for readability.  If we
> remove such blank lines unconditionally, it will be problematic.

I consider that just newlines are enough to make HTML markup human 
readable. I believe blank lines appear in HTML due to conditional 
constructs interpreted by various template engines and almost nobody 
cares concerning actual formatting in such cases.

However I proposed to make this feature an option that is turned on by 
default.

> I guess that I can change the condition to not include trailing space
> from (rx whitespace eol) to (rx (any " \t|) eol).

One more time I forgot that neither \n nor non-breakable space are 
included into post-blank.

I think, more permissive regexp may be used. At least it should accept 
newlines and any space after it

     (rx (any " \t" eol) (zero-or-more whitespace) eos)

Moreover, post-blank of the pruned object may be ignored when the 
following element starts with spaces other than purely zero width ones.

In my opinion, keeping extra spaces (e.g. post-blank ones from pruned 
objects) makes less harm than aggressively stripping them. Anyway some 
backends must normalize spaces (while for others they do not matter).

While newline characters are not affected, this part of change does not 
affect accidental split of paragraphs.

My feeling is that extensive test suite is required. It would be easier 
to review what cases are not handled yet.

Re: Trailing whitespace after export snippets without a transcoder

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> On 23/04/2024 02:01, Ihor Radchenko wrote:
>> For example, consider an HTML exporter that aligns tags nicely and
>> keeps blank lines between markup blocks for readability.  If we
>> remove such blank lines unconditionally, it will be problematic.
>
> I consider that just newlines are enough to make HTML markup human 
> readable. I believe blank lines appear in HTML due to conditional 
> constructs interpreted by various template engines and almost nobody 
> cares concerning actual formatting in such cases.

I looked further, and it turns out that Org export is already overriding
the blank lines produced by the exporters.

In particular, we have

	    ((memq type '(nil org-data plain-text raw)) results)
            ;; Append the same white space between elements or objects
	    ;; as in the original buffer, and call appropriate filters.
	    (t
	     (org-export-filter-apply-functions
	      (plist-get info (intern (format ":filter-%s" type)))
	      (let ((blank (or (org-element-post-blank data) 0)))
		(if (eq (org-element-class data parent) 'object)
		    (concat results (make-string blank ?\s))
		  (concat (org-element-normalize-string results)
			  (make-string blank ?\n))))
	      info))

For now, we only override newlines between elements, not objects.
For objects (but not plain-text), we unconditionally append :post-blank.

I conclude that it is actually OK to go a step further and cleanup
newlines after objects, before appending the post-blank.

Same for plain-text.

>> I guess that I can change the condition to not include trailing space
>> from (rx whitespace eol) to (rx (any " \t|) eol).
>
> One more time I forgot that neither \n nor non-breakable space are 
> included into post-blank.

> I think, more permissive regexp may be used. At least it should accept 
> newlines and any space after it
>
>      (rx (any " \t" eol) (zero-or-more whitespace) eos)

I see how special case for \n is useful. Not should about non-breakable
space.

> Moreover, post-blank of the pruned object may be ignored when the 
> following element starts with spaces other than purely zero width ones.

This can never happen, AFAIK.
All the spaces after any object become a part of its :post-blank
attribute.

> My feeling is that extensive test suite is required. It would be easier 
> to review what cases are not handled yet.

May you summarize various examples that should be considered?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Trailing whitespace after export snippets without a transcoder

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

>> I consider that just newlines are enough to make HTML markup human 
>> readable. I believe blank lines appear in HTML due to conditional 
>> constructs interpreted by various template engines and almost nobody 
>> cares concerning actual formatting in such cases.
>
> I looked further, and it turns out that Org export is already overriding
> the blank lines produced by the exporters.
> ...
> For now, we only override newlines between elements, not objects.
> For objects (but not plain-text), we unconditionally append :post-blank.
>
> I conclude that it is actually OK to go a step further and cleanup
> newlines after objects, before appending the post-blank.
>
> Same for plain-text.

Not so easy, unfortunately. Cleaning up spaces (in various degrees)
always leads to cascades of failed tests.

I did not find a good way to cleanup blank lines between objects without
overriding custom export filters or presenting these filters with
inaccurate information.

So, I went with the approach modifying individual export backends for now.

Applied, onto main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=14848726a
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=6d5fa8198

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>