Re: org-persist - bug report

Re: org-persist - bug report

[Ihor Radchenko]

Alastair Burt <alastair.burt@gmail.com> writes:

> Hi there,
>
> I couldn't find an issue-tracking system for org-mode as they have for
> projects on github. So I'm emailing you directly.

Thanks! The bug reporting for Org mode is by email. You can send email
to Org mailing list at emacs-orgmode@gnu.org (also, see "1.4 Feedback"
section of Org manual or https://orgmode.org/ The issue tracker is
derived from the email and can be found online in
https://updates.orgmode.org/

I am CCing this message the mailing list.

> I just upgraded my org-mode to the latest version in git. Whenever I
> visited a certain org file, my Emacs became unusable. Any attempt to use
> M-x (execute-extended-command) resulted in max-specpdl-size errors as did
> using C-x C-c to exit Emacs. Some but not all of the errors mentioned
> "Error during file deserialization". After an hour's debugging, I tracked
> this down to org-persist. Deleting "~/.cache/org-persist" made the problem
> go away.

Your error looks bad, but I never saw it on my side. Are you able to
reproduce it with clean Emacs config? Does it happen immediately after
you open the file or did you change the file before the error appeared?

> I never managed to produce a backtrace, even with debug-on-error set to t.
> This was hard to debug!

This indicates some problem with cache code. The cache code is
asynchronous and thus difficult to debug. You may produce a useful
backtrace if you have

(setq org-element--cache-self-verify 'backtrace)
(setq org-element--cache-self-verify-frequency 1)

If the above is not enough, try the following just before actions that
trigger the error:

(setq org-element--cache-diagnostics t)

Best,
Ihor

Re: org-persist - bug report

[Colin Baxter 😺]

>>>>> Ihor Radchenko <yantar92@gmail.com> writes:

    > Alastair Burt <alastair.burt@gmail.com> writes:
    >> Hi there,
    >> 
    >> I couldn't find an issue-tracking system for org-mode as they
    >> have for projects on github. So I'm emailing you directly.

    > Thanks! The bug reporting for Org mode is by email. You can send
    > email to Org mailing list at emacs-orgmode@gnu.org (also, see "1.4
    > Feedback" section of Org manual or https://orgmode.org/ The issue
    > tracker is derived from the email and can be found online in
    > https://updates.orgmode.org/

    > I am CCing this message the mailing list.

    >> I just upgraded my org-mode to the latest version in
    >> git. Whenever I visited a certain org file, my Emacs became
    >> unusable. Any attempt to use M-x (execute-extended-command)
    >> resulted in max-specpdl-size errors as did using C-x C-c to exit
    >> Emacs. Some but not all of the errors mentioned "Error during
    >> file deserialization". After an hour's debugging, I tracked this
    >> down to org-persist. Deleting "~/.cache/org-persist" made the
    >> problem go away.

I just stumbled on this error myself. The culprit was a agenda file - I
mean, an org-mode file called by org-agenda. I'll pursue further when
I've got time (not today!)

Best wishes,

Colin.

Re: org-persist - bug report

[Ihor Radchenko]

Colin Baxter 😺 <m43cap@yandex.com> writes:

>     >> I just upgraded my org-mode to the latest version in
>     >> git. Whenever I visited a certain org file, my Emacs became
>     >> unusable. Any attempt to use M-x (execute-extended-command)
>     >> resulted in max-specpdl-size errors as did using C-x C-c to exit
>     >> Emacs. Some but not all of the errors mentioned "Error during
>     >> file deserialization". After an hour's debugging, I tracked this
>     >> down to org-persist. Deleting "~/.cache/org-persist" made the
>     >> problem go away.
>
> I just stumbled on this error myself. The culprit was a agenda file - I
> mean, an org-mode file called by org-agenda. I'll pursue further when
> I've got time (not today!)

If you see the error again, can you also try with the following?

(setq org-element--cache-self-verify 'backtrace)
(setq org-element--cache-self-verify-frequency 1)

If a warning appears, backtrace could be helpful. If not, it should be
something to do with org-element-cache-map.

Best,
Ihor

Re: org-persist - bug report

[Colin Baxter 😺]

>>>>> Ihor Radchenko <yantar92@gmail.com> writes:

    > Colin Baxter 😺 <m43cap@yandex.com> writes:
    >> >> I just upgraded my org-mode to the latest version in >>
    >> git. Whenever I visited a certain org file, my Emacs became >>
    >> unusable. Any attempt to use M-x (execute-extended-command) >>
    >> resulted in max-specpdl-size errors as did using C-x C-c to exit
    >> >> Emacs. Some but not all of the errors mentioned "Error during
    >> >> file deserialization". After an hour's debugging, I tracked
    >> this >> down to org-persist. Deleting "~/.cache/org-persist" made
    >> the >> problem go away.
    >> 
    >> I just stumbled on this error myself. The culprit was a agenda
    >> file - I mean, an org-mode file called by org-agenda. I'll pursue
    >> further when I've got time (not today!)

    > If you see the error again, can you also try with the following?

    > (setq org-element--cache-self-verify 'backtrace)
    > (setq org-element--cache-self-verify-frequency 1)

I set the above in my ~/.emacs

    > If a warning appears, backtrace could be helpful. If not, it
    > should be something to do with org-element-cache-map.

The max-specpdl-size error has happened again when I tried to add a note
to an agenda item. Emacs hung and had to be killed. The debugger buffer
was empty and the only other information given was the message:

mapc: Lisp nesting exceeds `max-lisp-eval-depth'

I hope this helps.

Best wishes,

Colin.

Re: org-persist - bug report

[Ihor Radchenko]

Colin Baxter 😺 <m43cap@yandex.com> writes:

>     > (setq org-element--cache-self-verify 'backtrace)
>     > (setq org-element--cache-self-verify-frequency 1)
>
> I set the above in my ~/.emacs
>
>     > If a warning appears, backtrace could be helpful. If not, it
>     > should be something to do with org-element-cache-map.
>
> The max-specpdl-size error has happened again when I tried to add a note
> to an agenda item. Emacs hung and had to be killed. The debugger buffer
> was empty and the only other information given was the message:
>
> mapc: Lisp nesting exceeds `max-lisp-eval-depth'

It indicates yet another possible problematic place. But I still need
more information.

I just pushed a change that can help with debugging this.
If you update to current main and set `org-element--cache-interrupt-C-g'
to t, you can press "C-g" 6 times (1 +
org-element--cache-interrupt-C-g-max-count) to force-quit the "hang".

When you encounter the hang again, please try to reproduce it after
force quitting. If you figure out some way to reproduce, set
`org-element--cache-diagnostics' to t just before reproducing
(M-: (setq org-element--cache-diagnostics t) <RET>). Then, post the
contents added to your *Warnings* buffer. It will be very verbose and
can hopefully help me figure out the problem.

Best,
Ihor

Re: org-persist - bug report

[Max Nikulin]

On 21/10/2021 02:04, Colin Baxter 😺 wrote:
> 
> The max-specpdl-size error has happened again when I tried to add a note
> to an agenda item. Emacs hung and had to be killed. The debugger buffer
> was empty and the only other information given was the message:
> 
> mapc: Lisp nesting exceeds `max-lisp-eval-depth'

Ihor, I have seen your instructions for more debug info, but I have not 
tried them yet. In the meanwhile I have tracked my problem down to to 
the following minimal file

--- >8 ---
#+begin_example org
   ,* H
#+end_example
--- 8< ---

M-x org-lint

Debugger entered--Lisp error: (error "Variable binding depth exceeds 
max-specpdl-size")
...
org-filename-concat

Actually I can not update another example block after editing it in a 
special buffer (C-c C-').

commit a149fdd5699b9279846fb11184529f8cf47b36b5
Author: Ihor Radchenko <yantar92@gmail.com>
Date:   Thu Oct 21 16:50:10 2021 +0800

Emacs 1.26.3

Another observation:

C-c * on a line to make it a header causes the following warning:

Warning (emacs): org-element--cache: Unregistered buffer modifications 
detected. Resetting
  The buffer is: example.org
  Current command: nil

Feel free to response to another thread, there are several ones for 
problems with new cache.

org-persist? error "Variable binding depth exceeds max-specpdl-size" (was: org-persist - bug report)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> Ihor, I have seen your instructions for more debug info, but I have not 
> tried them yet. In the meanwhile I have tracked my problem down to to 
> the following minimal file
>
> --- >8 ---
> #+begin_example org
>    ,* H
> #+end_example
> --- 8< ---
>
> M-x org-lint
>
> Debugger entered--Lisp error: (error "Variable binding depth exceeds 
> max-specpdl-size")
> ...
> org-filename-concat

I cannot reproduce with your minimal example, but the error indicates
some issue with persistent cache. Can you:

1. Go to cache directory (~/.cache/org-persist)
2. Open "index" file and find the record for your minimal file and find
   :persist-file property. That property is relative cache file path.
3. Attach the contents of the :persist-file

Then, can you delete your cache folder and try to reproduce again?

By the way, did your Emacs crash recently?

> Another observation:
>
> C-c * on a line to make it a header causes the following warning:
>
> Warning (emacs): org-element--cache: Unregistered buffer modifications 
> detected. Resetting
>   The buffer is: example.org
>   Current command: nil

I cannot reproduce this as well. The fact that "Current command: nil" is
suspicious. Is "C-c *" bound to org-ctrl-c-star for you?

Best,
Ihor

Re: org-persist - bug report

[Colin Baxter 😺]

>>>>> Max Nikulin <manikulin@gmail.com> writes:

    > On 21/10/2021 02:04, Colin Baxter 😺 wrote:
    >> 
    >> The max-specpdl-size error has happened again when I tried to add
    >> a note to an agenda item. Emacs hung and had to be killed. The
    >> debugger buffer was empty and the only other information given
    >> was the message:
    >> 
    >> mapc: Lisp nesting exceeds `max-lisp-eval-depth'

    > Ihor, I have seen your instructions for more debug info, but I
    > have not tried them yet. In the meanwhile I have tracked my
    > problem down to to the following minimal file

    > --- >8 ---

    > #+begin_example org ,* H #+end_example

    > --- 8< ---

    > M-x org-lint

    > Debugger entered--Lisp error: (error "Variable binding depth
    > exceeds max-specpdl-size") ...  org-filename-concat

I can reproduce that. Then emacs just hangs. I have the latest org-mode.

Best wishes,

Colin.

Re: org-persist - bug report

[Ihor Radchenko]

Colin Baxter 😺 <m43cap@yandex.com> writes:

>     > M-x org-lint
>
>     > Debugger entered--Lisp error: (error "Variable binding depth
>     > exceeds max-specpdl-size") ...  org-filename-concat
>
> I can reproduce that. Then emacs just hangs. I have the latest org-mode.

Thanks for another confirmation. I tried harder to reproduce and I
managed to. There is no problem on Emacs 28, but I can see the error
using Emacs 26.3. I used the following command line from inside the Git
folder to run Emacs (debug.org is the provided minimal file):

emacs -Q -L ./lisp -l org  /tmp/debug.org

Interestingly, the following does not produce error:

emacs -Q -L ./lisp -l org -l org-lint /tmp/debug.org

So, there is something strange about loading org-lint library.
I tried to instrument org-lint, but
M-x debug-on-entry <RET> org-lint <RET> M-x org-lint <RET> somehow did
not trigger the debugger.

I am confused here. Can someone knowledgeable about Emacs load
mechanisms provide any clue about what may be happening?

Best,
Ihor

Re: org-persist - bug report

[Ihor Radchenko]

Ihor Radchenko <yantar92@gmail.com> writes:

> ... There is no problem on Emacs 28, but I can see the error
> using Emacs 26.3. I used the following command line from inside the Git
> folder to run Emacs (debug.org is the provided minimal file):
>
> emacs -Q -L ./lisp -l org  /tmp/debug.org
>
> Interestingly, the following does not produce error:
>
> emacs -Q -L ./lisp -l org -l org-lint /tmp/debug.org

Another data point:

If I compile Org before testing, the error disappears:

make
emacs -Q -L ./lisp -l org  /tmp/debug.org

Best,
Ihor

Re: org-persist - bug report

[Ihor Radchenko]

It was malformed add-hook call. Fixed in 5315773e8.

Re: org-persist - bug report

[Colin Baxter 😺]

>>>>> Ihor Radchenko <yantar92@gmail.com> writes:

    > It was malformed add-hook call. Fixed in 5315773e8.

Great! I've pulled org-mode and I can confirm I no longer see my
org-persist errors. Excellent. I'm looking at the vc-diff buffer and
it's amazing that such a small omission has such a major effect.

Best wishes,

Colin.

Re: org-persist - bug report

[Max Nikulin]

On 21/10/2021 23:03, Ihor Radchenko wrote:
> 
> It was malformed add-hook call. Fixed in 5315773e8.

Thank you, Ihor. I do not see problems I have noticed today anymore.

Re: org-persist - bug report

[Alastair Burt]

Thanks. Worked for me