Seemingly inconsistent whitespace ownership

Seemingly inconsistent whitespace ownership

[Tom Alexander]

The org-mode documentation states: "With the exception of list items, blank lines belong to the preceding element with the narrowest possible scope. For example, if at the end of a section we have a paragraph and a blank line, that blank line is considered part of the paragraph."

https://orgmode.org/worg/org-syntax.html#org9bc833d

So I created a test org-mode document with exactly that scenario described:
```
foo bar.




* Lorem
baz




* Ipsum
alpha




beta
```

and then I ran a M-x eval-expression (message "%s" (pp-to-string (org-element-parse-buffer))) to see the parse tree. I'll paste the output at the end because its a wall of text, but analyzing it:

The zeroth section has :begin at 1 and :end at 14, which is expected.

Its only child is a paragraph which has a :begin at 1 and :end at 10, which only includes the line break at the end of "foo bar." but this is exactly the scenario described in the documentation (a paragraph at the end of a section) so I would expect the paragraph to end at 14, not 10.

The "Lorem" section is repeating the test to prove this isn't behavior unique to the zeroth section. It behaved the same, so there is no need to go over it.

The "Ipsum" section is where I put two separate paragraphs with a bunch of blank lines in between. This time the "alpha" paragraph goes all the way to character 48 which means it includes ALL of the blank lines between "alpha" and "beta".

So the "Ipsum" section proves that ":begin" to ":end" is supposed to contain the trailing blank lines of an element.

I am using org-mode version: Org mode version 9.6.6 (release_9.6.6 @ /usr/share/emacs/29.1/lisp/org/)

And finally, that parse tree I mentioned:

```
(org-data
 (:begin 1 :contents-begin 1 :contents-end 53 :end 53 :robust-begin 3 :robust-end 51 :post-blank 0 :post-affiliated 1 :path nil :mode org-data :CATEGORY nil :granularity nil)
 (section
  (:begin 1 :end 14 :contents-begin 1 :contents-end 10 :robust-begin 1 :robust-end 8 :post-blank 4 :post-affiliated 1 :mode first-section :granularity nil :parent #0)
  (paragraph
   (:begin 1 :end 10 :contents-begin 1 :contents-end 10 :post-blank 0 :post-affiliated 1 :mode top-comment :granularity nil :parent #1)
   #("foo bar.\n" 0 9
     (:parent #2))))
 (headline
  (:raw-value "Lorem" :begin 14 :end 30 :pre-blank 0 :contents-begin 22 :contents-end 26 :robust-begin nil :robust-end nil :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :post-blank 4 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 14 :title
	      (#("Lorem" 0 5
		 (:parent #1)))
	      :mode nil :granularity nil :parent #0)
  (section
   (:begin 22 :end 30 :contents-begin 22 :contents-end 26 :robust-begin 22 :robust-end 24 :post-blank 4 :post-affiliated 22 :mode section :granularity nil :parent #1)
   (paragraph
    (:begin 22 :end 26 :contents-begin 22 :contents-end 26 :post-blank 0 :post-affiliated 22 :mode planning :granularity nil :parent #2)
    #("baz\n" 0 4
      (:parent #3)))))
 (headline
  (:raw-value "Ipsum" :begin 30 :end 53 :pre-blank 0 :contents-begin 38 :contents-end 53 :robust-begin 40 :robust-end 51 :level 1 :priority nil :tags nil :todo-keyword nil :todo-type nil :post-blank 0 :footnote-section-p nil :archivedp nil :commentedp nil :post-affiliated 30 :title
	      (#("Ipsum" 0 5
		 (:parent #1)))
	      :mode nil :granularity nil :parent #0)
  (section
   (:begin 38 :end 53 :contents-begin 38 :contents-end 53 :robust-begin 38 :robust-end 51 :post-blank 0 :post-affiliated 38 :mode section :granularity nil :parent #1)
   (paragraph
    (:begin 38 :end 48 :contents-begin 38 :contents-end 44 :post-blank 4 :post-affiliated 38 :mode planning :granularity nil :parent #2)
    #("alpha\n" 0 6
      (:parent #3)))
   (paragraph
    (:begin 48 :end 53 :contents-begin 48 :contents-end 53 :post-blank 0 :post-affiliated 48 :mode nil :granularity nil :parent #2)
    #("beta\n" 0 5
      (:parent #3))))))
```

--
Tom Alexander

Re: Seemingly inconsistent whitespace ownership

[Ihor Radchenko]

"Tom Alexander" <tom@fizz.buzz> writes:

> The org-mode documentation states: "With the exception of list items, blank lines belong to the preceding element with the narrowest possible scope. For example, if at the end of a section we have a paragraph and a blank line, that blank line is considered part of the paragraph."
>
> https://orgmode.org/worg/org-syntax.html#org9bc833d
>
> So I created a test org-mode document with exactly that scenario described:
> ...
> and then I ran a M-x eval-expression (message "%s" (pp-to-string (org-element-parse-buffer))) to see the parse tree. I'll paste the output at the end because its a wall of text, but analyzing it:
> ...
> Its only child is a paragraph which has a :begin at 1 and :end at 10, which only includes the line break at the end of "foo bar." but this is exactly the scenario described in the documentation (a paragraph at the end of a section) so I would expect the paragraph to end at 14, not 10.

You are right and you revealed an existing inconsistency between our
incremental and recursive parsers (org-element-parse-buffer is the
recursive parser).

Fixed, on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=bfe1179dc

Also, in addition to list items, footnote-definitions do not extend
their contents to the trailing blank lines.
I now clarified it in org-syntax document.
https://git.sr.ht/~bzg/worg/commit/3e91f5be

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Seemingly inconsistent whitespace ownership

[Tom Alexander]

[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]

On August 11, 2023 5:20:08 AM EDT, Ihor Radchenko <yantar92@posteo.net> wrote:
>"Tom Alexander" <tom@fizz.buzz> writes:
>
>> The org-mode documentation states: "With the exception of list items, blank lines belong to the preceding element with the narrowest possible scope. For example, if at the end of a section we have a paragraph and a blank line, that blank line is considered part of the paragraph."
>>
>> https://orgmode.org/worg/org-syntax.html#org9bc833d
>>
>> So I created a test org-mode document with exactly that scenario described:
>> ...
>> and then I ran a M-x eval-expression (message "%s" (pp-to-string (org-element-parse-buffer))) to see the parse tree. I'll paste the output at the end because its a wall of text, but analyzing it:
>> ...
>> Its only child is a paragraph which has a :begin at 1 and :end at 10, which only includes the line break at the end of "foo bar." but this is exactly the scenario described in the documentation (a paragraph at the end of a section) so I would expect the paragraph to end at 14, not 10.
>
>You are right and you revealed an existing inconsistency between our
>incremental and recursive parsers (org-element-parse-buffer is the
>recursive parser).
>
>Fixed, on main.
>https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=bfe1179dc
>
>Also, in addition to list items, footnote-definitions do not extend
>their contents to the trailing blank lines.
>I now clarified it in org-syntax document.
>https://git.sr.ht/~bzg/worg/commit/3e91f5be
>
>-- 
>Ihor Radchenko // yantar92,
>Org mode contributor,
>Learn more about Org mode at <https://orgmode.org/>.
>Support Org development at <https://liberapay.com/org-mode>,
>or support my work at <https://liberapay.com/yantar92>

Oh neat, I didn't know there were two! I'll check that out. Thanks!

[-- Attachment #2: Type: text/html, Size: 2335 bytes --]