bug#67326: 28.1.90.1; time-stamp slowdown in large org file after org-refile-get-targets, caused by e7b5912b23 (Improvements to long lines handling)

bug#67326: 28.1.90.1; time-stamp slowdown in large org file after org-refile-get-targets, caused by e7b5912b23 (Improvements to long lines handling)

[Yuchen Pei]

Previously reported in emacs-devel at
https://lists.gnu.org/archive/html/emacs-devel/2023-06/msg00113.html

Emacs built with

./configure --with-native-compilation --with-imagemagick --with-json

Possibly related to bug#57207

Set up: foo.org is a 12MB file, longest line 4753, about 21000
headlines. It contains the timestamp thingie "Time-stamp: <....>" as one
of the first few lines that already has a timestamp from before. Say the
following file is called bar.el

#+begin_src emacs-lisp
(add-hook 'before-save-hook 'time-stamp)
(require 'org-refile)
(setq org-refile-use-cache t)
(setq org-refile-use-outline-path t)
(setq org-refile-targets '((nil :maxlevel . 5)))
(setq org-goto-interface 'outline-path-completion)
(setq large-file-warning-threshold 15000000)
(find-file "foo.org")
(org-refile-get-targets)
#+end_src

To reproduce, do

./src/emacs -Q -l bar.el foo.org

followed by eval

(time-stamp)

It will take 10+ seconds to complete, and 10+ seconds if I undo
afterwards.

Like bug#57207,

(setq long-line-threshold nil)

is a workaround, and (time-stamp) completes instantly.

I could not measure the precise time or have the whole process run in
batch mode because:

1. It does not affect batch mode, presumably because of the change has
   to do with x display
2. (time-stamp) has funny pseudo-asynchronous behaviour: if I do
   (progn (time-stamp) (message "Done.")) the message evals immediately,
   long before time-stamp "completes" and unhangs emacs.

BTW what is the commit that fixed bug#57207? I searched for commits in
master and emacs-29 with message containing 57207 but could not find
any.

Best,
Yuchen

--
Dr Yuchen Pei | https://ypei.org | Timezone: UTC+11
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
https://ypei.org/assets/ypei-pubkey.txt

bug#67327: 28.1.90.1; time-stamp slowdown in large org file after org-refile-get-targets, caused by e7b5912b23 (Improvements to long lines handling)

[Yuchen Pei]

Sorry, typo in version number, should be 29.1.90.1
> [... 66 lines elided]

Best,
Yuchen

--
Dr Yuchen Pei | https://ypei.org | Timezone: UTC+11
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
https://ypei.org/assets/ypei-pubkey.txt

bug#67326: 28.1.90.1; time-stamp slowdown in large org file after org-refile-get-targets, caused by e7b5912b23 (Improvements to long lines handling)

[Eli Zaretskii]

> From: Yuchen Pei <id@ypei.org>
> Date: Tue, 21 Nov 2023 23:23:16 +1100
> 
> 
> Previously reported in emacs-devel at
> https://lists.gnu.org/archive/html/emacs-devel/2023-06/msg00113.html
> 
> Emacs built with
> 
> ./configure --with-native-compilation --with-imagemagick --with-json
> 
> Possibly related to bug#57207
> 
> Set up: foo.org is a 12MB file, longest line 4753, about 21000
> headlines. It contains the timestamp thingie "Time-stamp: <....>" as one
> of the first few lines that already has a timestamp from before. Say the
> following file is called bar.el
> 
> #+begin_src emacs-lisp
> (add-hook 'before-save-hook 'time-stamp)
> (require 'org-refile)
> (setq org-refile-use-cache t)
> (setq org-refile-use-outline-path t)
> (setq org-refile-targets '((nil :maxlevel . 5)))
> (setq org-goto-interface 'outline-path-completion)
> (setq large-file-warning-threshold 15000000)
> (find-file "foo.org")
> (org-refile-get-targets)
> #+end_src
> 
> To reproduce, do
> 
> ./src/emacs -Q -l bar.el foo.org
> 
> followed by eval
> 
> (time-stamp)
> 
> It will take 10+ seconds to complete, and 10+ seconds if I undo
> afterwards.
> 
> Like bug#57207,
> 
> (setq long-line-threshold nil)
> 
> is a workaround, and (time-stamp) completes instantly.

Adding Gregory to the discussion.

> I could not measure the precise time or have the whole process run in
> batch mode because:
> 
> 1. It does not affect batch mode, presumably because of the change has
>    to do with x display
> 2. (time-stamp) has funny pseudo-asynchronous behaviour: if I do
>    (progn (time-stamp) (message "Done.")) the message evals immediately,
>    long before time-stamp "completes" and unhangs emacs.

I think you misinterpret what happens: what takes a long time is not
time-stamp, but the redisplay after it.  IOW, it isn't the evaluation
that takes time, it's the redisplay after it.  But maybe I'm missing
something.

> BTW what is the commit that fixed bug#57207? I searched for commits in
> master and emacs-29 with message containing 57207 but could not find
> any.

It's 1c837c42c2, I think.

bug#67326: 28.1.90.1; time-stamp slowdown in large org file after org-refile-get-targets, caused by e7b5912b23 (Improvements to long lines handling)

[Yuchen Pei]

retitle 67326 29.1.90.1; time-stamp slowdown in large org file after org-refile-get-targets, caused by e7b5912b23 (Improvements to long lines handling)
thanks
> [... 66 lines elided]

Best,
Yuchen

--
Dr Yuchen Pei | https://ypei.org | Timezone: UTC+11
PGP Key: 47F9 D050 1E11 8879 9040  4941 2126 7E93 EF86 DFD0
https://ypei.org/assets/ypei-pubkey.txt

bug#67326: 29.1.90.1; time-stamp slowdown in large org file after org-refile-get-targets, caused by e7b5912b23 (Improvements to long lines handling)

[Gregory Heytings]

Hi Yuchen,

Thanks for your bug report, and sorry for the delay.

>
> Set up: foo.org is a 12MB file, longest line 4753, about 21000 
> headlines. It contains the timestamp thingie "Time-stamp: <....>" as one 
> of the first few lines that already has a timestamp from before.
>

I can't reproduce that bug alas, because I don't have that file.  I 
understand that it may contain private information.  Could you perhaps 
create a censored version of that file, and send it to me (privately, a 12 
MB attachments is too large for this list)?

>
> Like bug#57207,
>

This bug is likely not directly related to that one, as Ihor confirmed 
that it was fixed (and no other similar reports were filed since then).

>
> BTW what is the commit that fixed bug#57207? I searched for commits in 
> master and emacs-29 with message containing 57207 but could not find 
> any.
>

The reason for that is that development happened on feature branches, on 
which commits were not tagged as fixing specific bugs.