[BUG] org-get-outline-path misbehave in some scenarios when org-element-use-cache is t Inbox

[BUG] org-get-outline-path misbehave in some scenarios when org-element-use-cache is t Inbox

[Rodrigo Morales]

I've noticed that =org-get-outline-path= report incorrect information
when =org-element-use-cache= is =t=. This mail shows some experiments
that demonstrates this bug.

* Experiment 1

In this experiment, we define a =func= which contains the sexps that
we want to run when =org-element-use-cache= is =t= and when it is
=nil=. In =func=, we use =org-map-entries= to loop through headlines
that have depth 2. In each headline, we move the point two lines below
and insert a new line containing =FOO=.

#+HEADER: :results output
#+begin_src elisp
(let ((func (lambda ()
              (with-temp-buffer
                (org-mode)
                (insert "* 1
,** 1-1
a
,** 1-2
a
,* 2
,** 2-1
b
,** 2-2
b
")
                (princ (format "--- before\n%s\n--- before\n"
                         (buffer-substring-no-properti
es
                          (point-min) (point-max))))
                (org-map-entries
                 (lambda ()
                   ;; We get the outline path before calling
                   ;; (forward-line 2) to ensure that we are referring
                   ;; to the headline in the current iteration
                   (princ (format "org-get-outline-path: %s\n"
(org-get-outline-path t)))
                   ;; Move two lines below the current headline A. After
                   ;; executing (forward-line 2), the point is where
                   ;; the next headline B starts.
                   (forward-line 2)
                   ;; We add a line which moves the headline B, one
                   ;; line below
                   (insert "FOO" "\n"))
                 "LEVEL=2")
                (princ (format "--- after \n%s\n---after\n"
                         (buffer-substring-no-properti
es
                          (point-min) (point-max))))))))
  (princ "Experiment 1.1 (org-element-use-cache nil)\n")
  (let ((org-element-use-cache nil))
    (funcall func))
  (princ "Experiment 1.2 (org-element-use-cache t)\n")
  (let ((org-element-use-cache t))
    (funcall func)))
#+end_src

#+RESULTS:
#+begin_example
Experiment 1.1 (org-element-use-cache nil)
--- before
,* 1
,** 1-1
a
,** 1-2
a
,* 2
,** 2-1
b
,** 2-2
b

--- before
org-get-outline-path: (1 1-1)
org-get-outline-path: (1 1-2)
org-get-outline-path: (2 2-1)
org-get-outline-path: (2 2-2)
--- after
,* 1
,** 1-1
a
FOO
,** 1-2
a
FOO
,* 2
,** 2-1
b
FOO
,** 2-2
b
FOO

---after
Experiment 1.2 (org-element-use-cache t)
--- before
,* 1
,** 1-1
a
,** 1-2
a
,* 2
,** 2-1
b
,** 2-2
b

--- before
org-get-outline-path: (1 1-1)
org-get-outline-path: (1 1-1 1-2)
org-get-outline-path: (1 2 2-1)
org-get-outline-path: (1 2 2-2)
--- after
,* 1
,** 1-1
a
FOO
,** 1-2
a
FOO
,* 2
,** 2-1
b
FOO
,** 2-2
b
FOO

---after
#+end_example

In the results above, we can see that in =Experiment 1.2=,
=org-get-outline-path= returns wrong information (note that the
results are lists of length 3), while in =Experiment 1.1= this doesn't
happen.

* Experiment 2

I noticed that when we pass one paameter (i.e. ="FOO\n"=) instead of
two as we did in the previous experiment (i.e. ="FOO" "\n"=) to
=insert=, the bug doesn't happen. See minimal reproducible example
below.

In this group of experiments we pass ="FOO\n"= instead of ="FOO" "\n"=
to =insert=.

#+HEADER: :results output
#+begin_src elisp
(let ((func (lambda ()
              (with-temp-buffer
                (org-mode)
                (insert "* 1
,** 1-1
a
,** 1-2
a
,* 2
,** 2-1
b
,** 2-2
b
")
                (princ (format "--- before\n%s\n--- before\n"
                         (buffer-substring-no-properties
                          (point-min) (point-max))))
                (org-map-entries
                 (lambda ()
                   (princ (format "org-get-outline-path: %s\n"
(org-get-outline-path t)))
                   (forward-line 2)
                   (insert "FOO\n"))
                 "LEVEL=2")
                (princ (format "--- after \n%s\n---after\n"
                         (buffer-substring-no-properties
                          (point-min) (point-max))))))))
  (princ "Experiment 2.1 (org-element-use-cache nil)\n")
  (let ((org-element-use-cache nil))
    (funcall func))
  (princ "Experiment 2.2 (org-element-use-cache t)\n")
  (let ((org-element-use-cache t))
    (funcall func)))
#+end_src

#+RESULTS:
#+begin_example
Experiment 2.1 (org-element-use-cache nil)
--- before
,* 1
,** 1-1
a
,** 1-2
a
,* 2
,** 2-1
b
,** 2-2
b

--- before
org-get-outline-path: (1 1-1)
org-get-outline-path: (1 1-2)
org-get-outline-path: (2 2-1)
org-get-outline-path: (2 2-2)
--- after
,* 1
,** 1-1
a
FOO
,** 1-2
a
FOO
,* 2
,** 2-1
b
FOO
,** 2-2
b
FOO

---after
Experiment 2.2 (org-element-use-cache t)
--- before
,* 1
,** 1-1
a
,** 1-2
a
,* 2
,** 2-1
b
,** 2-2
b

--- before
org-get-outline-path: (1 1-1)
org-get-outline-path: (1 1-2)
org-get-outline-path: (2 2-1)
org-get-outline-path: (2 2-2)
--- after
,* 1
,** 1-1
a
FOO
,** 1-2
a
FOO
,* 2
,** 2-1
b
FOO
,** 2-2
b
FOO

---after
#+end_example

In the results above, we can see that both Experiment 2.1 and
Experiment 2.2 have the same output.

* Personal thoughts

With regards to experiment 2, we know that passing either ="FOO\n"= or
="FOO" "\n"= insert the same content, so I'm wondering how come
passing different parameters to =insert= which result in the same
outcome can affect the behavior of =org-get-outline-path=.

Re: [BUG] org-get-outline-path misbehave in some scenarios when org-element-use-cache is t Inbox

[Ihor Radchenko]

Rodrigo Morales <moralesrodrigo1100@gmail.com> writes:

> I've noticed that =org-get-outline-path= report incorrect information
> when =org-element-use-cache= is =t=. This mail shows some experiments
> that demonstrates this bug.

Thanks for reporting!
It was a bug in parser cache code.
Fixed, on bugfix.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=27a41d418

> With regards to experiment 2, we know that passing either ="FOO\n"= or
> ="FOO" "\n"= insert the same content, so I'm wondering how come
> passing different parameters to =insert= which result in the same
> outcome can affect the behavior of =org-get-outline-path=.

The difference is that "FOO\n" is processed a single buffer
modification (insert a string), while "FOO" "\n" triggers two
modifications (insert string, insert another string). The latter case
was not processed properly, missing that "\n" created a new heading:
1. Insert "FOO" ->
FOO** 1-2
2. Insert "\n"  ->
FOO
** 1-2

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] org-get-outline-path misbehave in some scenarios when org-element-use-cache is t Inbox

[Rodrigo Morales]

For those interested: here's another minimal working example:

#+HEADER: :results output
#+begin_src elisp
(with-temp-buffer
  (org-mode)
  (insert "* 1
,** 1.1
,* 2
")
  ;; jump to the first headline to print the value of :end
  (goto-char (point-min))
  (princ (format "%s\n" (org-element-property :end (org-element-context))))
  ;; search
  (search-forward "* 2")
  (beginning-of-line)
  (insert "a" "\n")
  ;; jump to the first headline to print the value of :end
  (goto-char (point-min))
  (princ (format "%s\n" (org-element-property :end (org-element-context)))))
#+end_src

Results of evaluation in v9.6.7 (unexpected behavior)

#+RESULTS:
#+begin_example
12
18
#+end_example

Results of evaluation in latest development main (expected behavior)

#+RESULTS:
#+begin_example
12
14
#+end_example

On Tue, 15 Aug 2023 at 19:03, Rodrigo Morales
<moralesrodrigo1100@gmail.com> wrote:
>
> I've noticed that =org-get-outline-path= report incorrect information
> when =org-element-use-cache= is =t=. This mail shows some experiments
> that demonstrates this bug.
>
> * Experiment 1
>
> In this experiment, we define a =func= which contains the sexps that
> we want to run when =org-element-use-cache= is =t= and when it is
> =nil=. In =func=, we use =org-map-entries= to loop through headlines
> that have depth 2. In each headline, we move the point two lines below
> and insert a new line containing =FOO=.
>
> #+HEADER: :results output
> #+begin_src elisp
> (let ((func (lambda ()
>               (with-temp-buffer
>                 (org-mode)
>                 (insert "* 1
> ,** 1-1
> a
> ,** 1-2
> a
> ,* 2
> ,** 2-1
> b
> ,** 2-2
> b
> ")
>                 (princ (format "--- before\n%s\n--- before\n"
>                          (buffer-substring-no-properti
> es
>                           (point-min) (point-max))))
>                 (org-map-entries
>                  (lambda ()
>                    ;; We get the outline path before calling
>                    ;; (forward-line 2) to ensure that we are referring
>                    ;; to the headline in the current iteration
>                    (princ (format "org-get-outline-path: %s\n"
> (org-get-outline-path t)))
>                    ;; Move two lines below the current headline A. After
>                    ;; executing (forward-line 2), the point is where
>                    ;; the next headline B starts.
>                    (forward-line 2)
>                    ;; We add a line which moves the headline B, one
>                    ;; line below
>                    (insert "FOO" "\n"))
>                  "LEVEL=2")
>                 (princ (format "--- after \n%s\n---after\n"
>                          (buffer-substring-no-properti
> es
>                           (point-min) (point-max))))))))
>   (princ "Experiment 1.1 (org-element-use-cache nil)\n")
>   (let ((org-element-use-cache nil))
>     (funcall func))
>   (princ "Experiment 1.2 (org-element-use-cache t)\n")
>   (let ((org-element-use-cache t))
>     (funcall func)))
> #+end_src
>
> #+RESULTS:
> #+begin_example
> Experiment 1.1 (org-element-use-cache nil)
> --- before
> ,* 1
> ,** 1-1
> a
> ,** 1-2
> a
> ,* 2
> ,** 2-1
> b
> ,** 2-2
> b
>
> --- before
> org-get-outline-path: (1 1-1)
> org-get-outline-path: (1 1-2)
> org-get-outline-path: (2 2-1)
> org-get-outline-path: (2 2-2)
> --- after
> ,* 1
> ,** 1-1
> a
> FOO
> ,** 1-2
> a
> FOO
> ,* 2
> ,** 2-1
> b
> FOO
> ,** 2-2
> b
> FOO
>
> ---after
> Experiment 1.2 (org-element-use-cache t)
> --- before
> ,* 1
> ,** 1-1
> a
> ,** 1-2
> a
> ,* 2
> ,** 2-1
> b
> ,** 2-2
> b
>
> --- before
> org-get-outline-path: (1 1-1)
> org-get-outline-path: (1 1-1 1-2)
> org-get-outline-path: (1 2 2-1)
> org-get-outline-path: (1 2 2-2)
> --- after
> ,* 1
> ,** 1-1
> a
> FOO
> ,** 1-2
> a
> FOO
> ,* 2
> ,** 2-1
> b
> FOO
> ,** 2-2
> b
> FOO
>
> ---after
> #+end_example
>
> In the results above, we can see that in =Experiment 1.2=,
> =org-get-outline-path= returns wrong information (note that the
> results are lists of length 3), while in =Experiment 1.1= this doesn't
> happen.
>
> * Experiment 2
>
> I noticed that when we pass one paameter (i.e. ="FOO\n"=) instead of
> two as we did in the previous experiment (i.e. ="FOO" "\n"=) to
> =insert=, the bug doesn't happen. See minimal reproducible example
> below.
>
> In this group of experiments we pass ="FOO\n"= instead of ="FOO" "\n"=
> to =insert=.
>
> #+HEADER: :results output
> #+begin_src elisp
> (let ((func (lambda ()
>               (with-temp-buffer
>                 (org-mode)
>                 (insert "* 1
> ,** 1-1
> a
> ,** 1-2
> a
> ,* 2
> ,** 2-1
> b
> ,** 2-2
> b
> ")
>                 (princ (format "--- before\n%s\n--- before\n"
>                          (buffer-substring-no-properties
>                           (point-min) (point-max))))
>                 (org-map-entries
>                  (lambda ()
>                    (princ (format "org-get-outline-path: %s\n"
> (org-get-outline-path t)))
>                    (forward-line 2)
>                    (insert "FOO\n"))
>                  "LEVEL=2")
>                 (princ (format "--- after \n%s\n---after\n"
>                          (buffer-substring-no-properties
>                           (point-min) (point-max))))))))
>   (princ "Experiment 2.1 (org-element-use-cache nil)\n")
>   (let ((org-element-use-cache nil))
>     (funcall func))
>   (princ "Experiment 2.2 (org-element-use-cache t)\n")
>   (let ((org-element-use-cache t))
>     (funcall func)))
> #+end_src
>
> #+RESULTS:
> #+begin_example
> Experiment 2.1 (org-element-use-cache nil)
> --- before
> ,* 1
> ,** 1-1
> a
> ,** 1-2
> a
> ,* 2
> ,** 2-1
> b
> ,** 2-2
> b
>
> --- before
> org-get-outline-path: (1 1-1)
> org-get-outline-path: (1 1-2)
> org-get-outline-path: (2 2-1)
> org-get-outline-path: (2 2-2)
> --- after
> ,* 1
> ,** 1-1
> a
> FOO
> ,** 1-2
> a
> FOO
> ,* 2
> ,** 2-1
> b
> FOO
> ,** 2-2
> b
> FOO
>
> ---after
> Experiment 2.2 (org-element-use-cache t)
> --- before
> ,* 1
> ,** 1-1
> a
> ,** 1-2
> a
> ,* 2
> ,** 2-1
> b
> ,** 2-2
> b
>
> --- before
> org-get-outline-path: (1 1-1)
> org-get-outline-path: (1 1-2)
> org-get-outline-path: (2 2-1)
> org-get-outline-path: (2 2-2)
> --- after
> ,* 1
> ,** 1-1
> a
> FOO
> ,** 1-2
> a
> FOO
> ,* 2
> ,** 2-1
> b
> FOO
> ,** 2-2
> b
> FOO
>
> ---after
> #+end_example
>
> In the results above, we can see that both Experiment 2.1 and
> Experiment 2.2 have the same output.
>
> * Personal thoughts
>
> With regards to experiment 2, we know that passing either ="FOO\n"= or
> ="FOO" "\n"= insert the same content, so I'm wondering how come
> passing different parameters to =insert= which result in the same
> outcome can affect the behavior of =org-get-outline-path=.