BUG: org cycling regression when using the legacy folding style overlays

BUG: org cycling regression when using the legacy folding style overlays

[Kaushal Modi]

Hello,

I was trying to create a minimal reproducible example for a different
issue and I ended up with this other issue that I didn't expect and so
I am reporting that first.

Issue: Opening an Org file gives (wrong-number-of-arguments #<subr
looking-at> 2) error.

=====
Warning (org-element-cache): org-element--cache: Org parser error in
t.org::88. Resetting.
 The error was: (wrong-number-of-arguments #<subr looking-at> 2)
 Backtrace:
nil
 Please report this to Org mode mailing list (M-x org-submit-bug-report).
=====

Recipe:

1. Clone https://gitlab.com/kaushalmodi/org-mode-mwe to a temp
directory (say, in /tmp)
2. cd to the org repo and check out the main branch. The Org version
used when creating this report was release_9.5.3-520-g4dda0d.
3. emacs -Q -L ./lisp/
/tmp/org-mode-mwe/bug-recipes/collapse-all-post-subtrees/t.org &
4. Evaluate the "code__fold_style" src block in that test file; it
sets the fold style to overlays
5. Evaluate the "code__collapse_all_posts" src block; this code hides
all the subtrees with non-empty values for EXPORT_FILE_NAME or
CUSTOM_ID properties.

If you skip step 4, you will find the collapsing of all subtrees where
EXPORT_FILE_NAME or CUSTOM_ID is set happens as expected in step 5.

But if you evaluate step 4, most (or all?) subtrees remain uncollapsed.
Additionally, once step 4 + 5 are evaluated, the default TAB binding
that should cycle a subtree's visibility also starts behaving
erratically.

For example, move cursor to line 68 (* Image / Figure) in the test
file t.org after completing all of the above steps.

You will find these lines visible:

=====
* Image / Figure                                                      :image:
** Image links
:PROPERTIES:
:EXPORT_FILE_NAME: image-links
:EXPORT_DATE: 2017-07-15T07:49:44-04:00
:END:
=====

Hit TAB once, and now you see this: it hid only the title of the
subtree heading under it, but still shows the property drawer
expanded!

=====
* Image / Figure                                                      :image:
:PROPERTIES:
:EXPORT_FILE_NAME: image-links
:EXPORT_DATE: 2017-07-15T07:49:44-04:00
:END:
=====

Hit TAB once again, you will see:

=====
* Image / Figure                                                      :image:
** Image links...
=====

Now if you move the point to line 69 (* Image links), hitting the TAB
will not do any kind of cycling at all.

--
Kaushal Modi

Re: BUG: org cycling regression when using the legacy folding style overlays

[Ihor Radchenko]

Kaushal Modi <kaushal.modi@gmail.com> writes:

> I was trying to create a minimal reproducible example for a different
> issue and I ended up with this other issue that I didn't expect and so
> I am reporting that first.
>
> Issue: Opening an Org file gives (wrong-number-of-arguments #<subr
> looking-at> 2) error.

Do I understand correctly that your reproducer has nothing to do with
this error?

> 1. Clone https://gitlab.com/kaushalmodi/org-mode-mwe to a temp
> directory (say, in /tmp)
> 2. cd to the org repo and check out the main branch. The Org version
> used when creating this report was release_9.5.3-520-g4dda0d.
> 3. emacs -Q -L ./lisp/
> /tmp/org-mode-mwe/bug-recipes/collapse-all-post-subtrees/t.org &
> 4. Evaluate the "code__fold_style" src block in that test file; it
> sets the fold style to overlays
> 5. Evaluate the "code__collapse_all_posts" src block; this code hides
> all the subtrees with non-empty values for EXPORT_FILE_NAME or
> CUSTOM_ID properties.

Your code__collapse_all_posts block makes use of hide-subtree from
outline.el. Please, do not use it. outline.el is no longer considered
compatible with Org. It was not in the past, and even more so now. This
has been announced in ORG-NEWS.

Best,
Ihor

Re: BUG: org cycling regression when using the legacy folding style overlays

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

On Mon, May 30, 2022, 7:45 PM Ihor Radchenko <yantar92@gmail.com> wrote:

> Kaushal Modi <kaushal.modi@gmail.com> writes:
>
> > I was trying to create a minimal reproducible example for a different
> > issue and I ended up with this other issue that I didn't expect and so
> > I am reporting that first.
> >
> > Issue: Opening an Org file gives (wrong-number-of-arguments #<subr
> > looking-at> 2) error.
>
> Do I understand correctly that your reproducer has nothing to do with
> this error?
>

Oddly, I was able to reproduce that error 2 times in an emacs -Q session,
but after resetting the Org element cache, that error went away. So turns
out that the same cache can be reused between emacs -Q and regular Emacs
sessions?

Your code__collapse_all_posts block makes use of hide-subtree from
> outline.el. Please, do not use it. outline.el is no longer considered
> compatible with Org. It was not in the past, and even more so now. This
> has been announced in ORG-NEWS.
>

Thanks! I definitely missed that. I'll read up on how to do the same
without hide-subtree.

>

[-- Attachment #2: Type: text/html, Size: 1951 bytes --]

Re: BUG: org cycling regression when using the legacy folding style overlays

[Kaushal Modi]

On Mon, May 30, 2022 at 7:45 PM Ihor Radchenko <yantar92@gmail.com> wrote:
> Your code__collapse_all_posts block makes use of hide-subtree from
> outline.el. Please, do not use it. outline.el is no longer considered
> compatible with Org. It was not in the past, and even more so now. This
> has been announced in ORG-NEWS.

I read the ORG-NEWS:

> The new folding backend breaks some of the =outline-*= functions that
> rely on the details of visibility state implementation in
> =outline.el=.  The old Org folding backend was compatible with the
> =outline.el= folding, but it is not the case anymore with the new
> backend.  From now on, using =outline-*= functions is strongly
> discouraged when working with Org files.

It says that the new folding backend won't work with outline.el. But
in this case, I am still using the old backend.
Are the outline.el functions expected to stop working for the old backend too?

I replaced hide-subtree with org-fold-hide-subtree and my subtree
collpasing function seems to work as before even with
org-fold-core-style set to 'overlays. I will update this thread if I
see any issue with that.

Thanks!

Re: BUG: org cycling regression when using the legacy folding style overlays

[Ihor Radchenko]

Kaushal Modi <kaushal.modi@gmail.com> writes:

> I read the ORG-NEWS:
>
>> The new folding backend breaks some of the =outline-*= functions that
>> rely on the details of visibility state implementation in
>> =outline.el=.  The old Org folding backend was compatible with the
>> =outline.el= folding, but it is not the case anymore with the new
>> backend.  From now on, using =outline-*= functions is strongly
>> discouraged when working with Org files.
>
> It says that the new folding backend won't work with outline.el. But
> in this case, I am still using the old backend.
> Are the outline.el functions expected to stop working for the old backend too?

org-fold-core-style 'overlays is not enabling the old backend, but
rather enables legacy support in the new backend.

Strictly speaking, I can simplify the 'overlays option to work exactly
like the old backend, but it will also require changing some other
defaults, which is awkward. Also, it will discourage non-complying
packages to keep using outline.el and instead ask users to change to the
old backend. I would prefer to avoid this.

'overlays option is intended to be a emergency solution for people
experiencing major issues with the new folding mechanism. It will be
eventually removed.

Best,
Ihor

Re: BUG: org cycling regression when using the legacy folding style overlays

[Ihor Radchenko]

Kaushal Modi <kaushal.modi@gmail.com> writes:

> Oddly, I was able to reproduce that error 2 times in an emacs -Q session,
> but after resetting the Org element cache, that error went away. So turns
> out that the same cache can be reused between emacs -Q and regular Emacs
> sessions?

org-persist-directly does have a default value and it may use cache
located in $XDG_CACHE_DIR/org-persist. It is probably a good idea not to
load cache when emacs -Q is called, but I am not sure how to detect it.

Best,
Ihor