Incorrect link is stored (org-elements?)

Incorrect link is stored (org-elements?)

[Max Nikulin]

`org-store-link' under some conditions may pick a link to wrong heading. 
Emacs-26.3, Org main~1.

--- >8 ---
* Right

Save this file, kill the buffer, open it again.
Put cursor to the empty line below this paragraph,
type something, e.g. =a=, undo =C-/=, =M-x org-store-link=.
"#its_wrong" is stored instead of "*Right".

** Wrong
:PROPERTIES:
:CUSTOM_ID: its_wrong
:END:
--- 8< ---

Re: Incorrect link is stored (org-elements?)

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 249 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

> `org-store-link' under some conditions may pick a link to wrong heading. 
> Emacs-26.3, Org main~1.

I can reproduce, though not exactly following your steps.
Can you try the attached patch?

Best,
Ihor


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-org-element-at-point-Fix-breakage-when-called-at-the.patch --]
[-- Type: text/x-diff, Size: 1650 bytes --]

From 6465741dff757af23b10d1f0a412eb41d05f4c75 Mon Sep 17 00:00:00 2001
Message-Id: <6465741dff757af23b10d1f0a412eb41d05f4c75.1636458311.git.yantar92@gmail.com>
From: Ihor Radchenko <yantar92@gmail.com>
Date: Tue, 9 Nov 2021 19:42:14 +0800
Subject: [PATCH] org-element-at-point: Fix breakage when called at the end of
 section

* lisp/org-element.el (org-element-at-point): Do no unconditionally
return first matching cached element when point is within blank lines
after section element and CACHED-ONLY is non-nil.

Fixes https://list.orgmode.org/smbl59$qjm$1@ciao.gmane.io/T/#u
---
 lisp/org-element.el | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index 9ea13db9c..ad5c2bbd0 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -7510,9 +7510,6 @@ (defun org-element-at-point (&optional pom cached-only)
                                (and (org-element-property :contents-begin element)
                                     (>= pom (org-element-property :begin element))
                                     (< pom (org-element-property :contents-begin element)))
-                               (and (org-element-property :contents-end element)
-                                    (< pom (org-element-property :end element))
-                                    (>= pom (org-element-property :contents-end element)))
                                (and (not (org-element-property :contents-end element))
                                     (>= pom (org-element-property :begin element))
                                     (< pom (org-element-property :end element)))))))
-- 
2.32.0

Re: Incorrect link is stored (org-elements?)

[Max Nikulin]

On 09/11/2021 18:46, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> `org-store-link' under some conditions may pick a link to wrong heading.
>> Emacs-26.3, Org main~1.
> 
> I can reproduce, though not exactly following your steps.
> Can you try the attached patch?

With equivalent patch (attached one has additional differences in spaces 
around) my particular case works. I have no idea which additional 
scenarios it may affect. I did not test the patch extensively. I faced 
the issue while I was preparing a patch to fix storing double #CUSTOM_ID 
links.

P.S. I am puzzled that `org-element-at-point' returns paragraph even 
when patch is not applied. After such call `org-store-link' does what I 
expect. However "after undo" erroneous state survives emacs restart, 
thanks to org-persist, I believe.

Re: Incorrect link is stored (org-elements?)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> With equivalent patch (attached one has additional differences in spaces 
> around) my particular case works. I have no idea which additional 
> scenarios it may affect. I did not test the patch extensively. I faced 
> the issue while I was preparing a patch to fix storing double #CUSTOM_ID 
> links.

I pushed the patch upstream.

> P.S. I am puzzled that `org-element-at-point' returns paragraph even 
> when patch is not applied. After such call `org-store-link' does what I 
> expect. However "after undo" erroneous state survives emacs restart, 
> thanks to org-persist, I believe.

The bug was in very specific case. org-element-at-point now has a
special fast mode of operation when CACHED-ONLY argument is non-nil. In
this mode, it does not try to parse anything, but only checks the
nearest cached element and returns it if point is at the element
beginning, within +begin/end line of the element, or inside a
non-greater element. The error happened when the cached element was a
section and point was within blank lines at the end of the section
exactly one symbol before the following headline. In such cases,
org-element-at-point must follow some special rules (see comments in
org-element--parse-to), but it did not in this specific scenario.
On top of that, org-element-parse-to cannot return section and the
current code assumes that we ever attempt to return section when point
is at the beginning of section. So, it returns
(org-element-at-point (1+ pos)) instead. Hence, the next headline
was wrongly returned when (1) point is 1- beginning of next headline
(2) we are within blank lines at the end of section (3) some edit in
buffer just cleared everything in cache inside current section
(4) org-element-at-point is called with non-nil CACHED-ONLY argument.

Just calling org-element-at-point did not trigger error. You would need
to call (org-element-at-point nil 'cached)

Best,
Ihor

Re: Incorrect link is stored (org-elements?)

[Max Nikulin]

On 10/11/2021 20:48, Ihor Radchenko wrote:
> 
> I pushed the patch upstream.
> 
>> P.S. I am puzzled that `org-element-at-point' returns paragraph even
>> when patch is not applied. After such call `org-store-link' does what I
>> expect. However "after undo" erroneous state survives emacs restart,
>> thanks to org-persist, I believe.
> 
> The bug was in very specific case. org-element-at-point now has a
> special fast mode of operation when CACHED-ONLY argument is non-nil. In
> this mode, it does not try to parse anything, but only checks the
> nearest cached element and returns it if point is at the element
> beginning, within +begin/end line of the element, or inside a
> non-greater element. The error happened when the cached element was a
> section and point was within blank lines at the end of the section
> exactly one symbol before the following headline. In such cases,
> org-element-at-point must follow some special rules (see comments in
> org-element--parse-to), but it did not in this specific scenario.
> On top of that, org-element-parse-to cannot return section and the
> current code assumes that we ever attempt to return section when point
> is at the beginning of section. So, it returns
> (org-element-at-point (1+ pos)) instead. Hence, the next headline
> was wrongly returned when (1) point is 1- beginning of next headline
> (2) we are within blank lines at the end of section (3) some edit in
> buffer just cleared everything in cache inside current section
> (4) org-element-at-point is called with non-nil CACHED-ONLY argument.

Thank you for the fix and for the detailed explanation, Ihor. I was 
naively suspecting some weird issue with undo, maybe similar to the one 
found with interference of tables and non-latin input methods.