29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Yuchen Pei]

emacs-29 756244c69638bc3562d8cd2c1aa126216e99fb29 build.

Not sure how to reprod with something from scratch. But here are the
steps to reprod with a large "organic" org file with emacs -Q

- (add-hook 'before-save-hook 'time-stamp)
- Open a very large org file with a "Time-stamp:" at the beginning of the
  file
- (setq org-refile-use-cache t)
- (setq org-refile-use-outline-path t)
- (setq org-refile-targets '((org-agenda-files :maxlevel . 5)))
- (setq org-goto-interface 'outline-path-completion)
- M-x org-goto, which triggers `org-refile-get-targets`, wait for it to
  finish
- Make a minimal change to the file, e.g. add a space then delete it.
- M-x save-buffer / C-x C-s - and it takes too long to save

The problem disappears if I remove the hook or do not run org-goto and
trigger generation of the cache, and the problem does not manifest in
28.2 either.

Best,
Yuchen

-- 
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
          <https://ypei.org/assets/ypei-pubkey.txt>

Re: 29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Ihor Radchenko]

Yuchen Pei <id@ypei.org> writes:

> emacs-29 756244c69638bc3562d8cd2c1aa126216e99fb29 build.
>
> Not sure how to reprod with something from scratch. But here are the
> steps to reprod with a large "organic" org file with emacs -Q
>
> - (add-hook 'before-save-hook 'time-stamp)
> - Open a very large org file with a "Time-stamp:" at the beginning of the
>   file
> - (setq org-refile-use-cache t)
> - (setq org-refile-use-outline-path t)
> - (setq org-refile-targets '((org-agenda-files :maxlevel . 5)))
> - (setq org-goto-interface 'outline-path-completion)
> - M-x org-goto, which triggers `org-refile-get-targets`, wait for it to
>   finish

Potentially a duplicate of https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58558
`org-refile-get-targets' creates a bunch of markers in buffer and in the
discussion of bug#58558 we have found that
`parse-sexp-lookup-properties' set to non-nil dramatically slows down
searching in buffer, with slowdown scaling with the number of markers.
There is also a fix.

Yuchen, may you try to set `parse-sexp-lookup-properties' locally to nil
in your large Org buffer? If it helps, that's it.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: 29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Yuchen Pei]

On 4 June 2023 16:23:45 GMT+10:00, Ihor Radchenko <yantar92@posteo.net> wrote:
>Yuchen Pei <id@ypei.org> writes:
>
>> emacs-29 756244c69638bc3562d8cd2c1aa126216e99fb29 build.
>>
>> Not sure how to reprod with something from scratch. But here are the
>> steps to reprod with a large "organic" org file with emacs -Q
>>
>> - (add-hook 'before-save-hook 'time-stamp)
>> - Open a very large org file with a "Time-stamp:" at the beginning of the
>>   file
>> - (setq org-refile-use-cache t)
>> - (setq org-refile-use-outline-path t)
>> - (setq org-refile-targets '((org-agenda-files :maxlevel . 5)))
>> - (setq org-goto-interface 'outline-path-completion)
>> - M-x org-goto, which triggers `org-refile-get-targets`, wait for it to
>>   finish
>
>Potentially a duplicate of https://debbugs.gnu.org/cgi/bugreport.cgi?bug=58558
>`org-refile-get-targets' creates a bunch of markers in buffer and in the
>discussion of bug#58558 we have found that
>`parse-sexp-lookup-properties' set to non-nil dramatically slows down
>searching in buffer, with slowdown scaling with the number of markers.
>There is also a fix.
>
>Yuchen, may you try to set `parse-sexp-lookup-properties' locally to nil
>in your large Org buffer? If it helps, that's it.
>

Thanks. Tried that, and it did not fix the issue.

Re: 29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Ihor Radchenko]

Yuchen Pei <id@ypei.org> writes:

>>Yuchen, may you try to set `parse-sexp-lookup-properties' locally to nil
>>in your large Org buffer? If it helps, that's it.
>>
>
> Thanks. Tried that, and it did not fix the issue.

Ok. I tried to open a large Org file and somehow got:

    > ~/Git/emacs/src/emacs -Q -l /tmp/bug.el 
    double free or corruption (out)
    Fatal error 6: Aborted

But only once. Next attempts did not show any obvious slowdown when
opening a large (22Mb) file, adding # Time-stamp: <>, and C-x C-s

On Emacs master.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: 29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Yuchen Pei]

On 12 June 2023 23:10:00 GMT+10:00, Ihor Radchenko <yantar92@posteo.net> wrote:
>Yuchen Pei <id@ypei.org> writes:
>
>>>Yuchen, may you try to set `parse-sexp-lookup-properties' locally to nil
>>>in your large Org buffer? If it helps, that's it.
>>>
>>
>> Thanks. Tried that, and it did not fix the issue.
>
>Ok. I tried to open a large Org file and somehow got:
>
>    > ~/Git/emacs/src/emacs -Q -l /tmp/bug.el 
>    double free or corruption (out)
>    Fatal error 6: Aborted
>
>But only once. Next attempts did not show any obvious slowdown when
>opening a large (22Mb) file, adding # Time-stamp: <>, and C-x C-s
>
>On Emacs master.
>

Thanks for checking, but that is not the whole reprod process - it misses generating a big refile cache etc, see my original message.

Re: 29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Ihor Radchenko]

Yuchen Pei <id@ypei.org> writes:

>>On Emacs master.
>>
>
> Thanks for checking, but that is not the whole reprod process - it misses generating a big refile cache etc, see my original message.

Ok. I tried to repeat your steps.
I was able to reproduce using Emacs 29 and some older build of Emacs 30.
But not on master.

Profiler does not reveal anything, so it is somewhere in C internals.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: 29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Yuchen Pei]

On Mon 2023-06-12 18:50:36 +0000, Ihor Radchenko wrote:

> Yuchen Pei <id@ypei.org> writes:
>
>>>On Emacs master.
>>>
>>
>> Thanks for checking, but that is not the whole reprod process - it
>> misses generating a big refile cache etc, see my original message.
>
> Ok. I tried to repeat your steps.
> I was able to reproduce using Emacs 29 and some older build of Emacs 30.
> But not on master.
>
> Profiler does not reveal anything, so it is somewhere in C internals.

Thanks. What should be the next course of action to address this issue?
I'd be happy to investigate and get it fixed in Emacs 29. Is it worth
the effot to git-bisect to find the good commit?

Best,
Yuchen

-- 
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
          <https://ypei.org/assets/ypei-pubkey.txt>

Re: 29.0.91 756244c696 Saving large org file can be slow with time-stamp before-save-hook

[Ihor Radchenko]

Yuchen Pei <id@ypei.org> writes:

>> Profiler does not reveal anything, so it is somewhere in C internals.
>
> Thanks. What should be the next course of action to address this issue?
> I'd be happy to investigate and get it fixed in Emacs 29. Is it worth
> the effot to git-bisect to find the good commit?

I think git-bisect will make sense here, if you also cannot reproduce on
master. You already know that 756244c696 is bad.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>