bug#71644: 30.0.50; Severe slowdown in larger files with markers beginning in emacs 29+

[Mitchell <mitchellahren@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6025 bytes --]

Thank you for the response, Eli.

> Maybe the reason is that Emacs 30 now
> uses Org 9.7.4, whereas you have a local installation of 9.6.30?

Upgrading to Org 9.7.4 and reinstalling counsel did seem to improve the
abbrev expansion speed slightly on my Emacs 30.0.50 build, at least at
first. It’s still not nearly as snappy as 28.2 the longer I edit the
buffer. I will work on a reliable recipe to recreate this slowness.

> Why did you decide the problem was due to markers?  I see a single
> change in the implementation of markers in Emacs 29 as compared to
> Emacs 28, and no changes at all in Emacs 30 (except one that affects
> the Android, I think).

I thought it was markers because (1) overwriting a few functions in
counsel.el to be the versions at
https://gist.github.com/kings2u/5a8acbf0986f0848be66169d2dba7260 so that
counsel-outline cleaned up the markers it created caused the bug to go
away, and (2) the bug also exists when instead of counsel-outline at Step 5
I used org-refile (with `org-refile-use-cache` is set to `t`), and
`org-refile-use-cache` seems to create many markers in the buffer. But
maybe counsel-outline and org-refile are both doing something else that
cause the bug. (FWIW I did also notice that the abbrev expansions were
slightly quicker in Step 7 after using org-refile than after
counsel-outline...)

> I _can_ reproduce this in Emacs 29.3, where indeed
> expanding any of the two abbrevs takes about 0.5 sec to show on the
> screen.

Abbrev expansions on my machine take longer. When I re-create the bug with
counsel-outline, the abbrevs take over a second each to expand. Maybe your
hardware is faster than mine?

> I see 3400 markers created by counsel-outline in this case, which is
> not too many, IMO.

Interesting, maybe it isn’t markers after all. Can I ask how you count
markers? I couldn’t find anything in the docs or online.

On Wed, Jun 19, 2024 at 6:57 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Mitchell <mitchellahren@gmail.com>
> > Date: Tue, 18 Jun 2024 23:25:18 -0600
> >
> > I recently upgraded from emacs 28.2 to 29.3. When I resumed working in a
> large 11MB org-mode file that
> > never gave me problems with speed on 28.2, I noticed a severe slowdown
> when editing the buffer after
> > invoking `counsel-outline`. It took a while for me to isolate the issue,
> but it seems to be caused by the many
> > markers that `counsel-outline` creates in the buffer. After invoking the
> command, every subsequent edit to the
> > buffer renders much slower on screen than it did in 28.2.
> >
> > This problem is not only caused by `counsel-outline`, but other
> functions that create a lot of markers in the
> > buffer, like `org-refile` if `org-refile-use-cache` is set to `t`. I
> tried upgrading my system to emacs 30.0.50,
> > which I've used to generate this bug report, but the issue persists.
>
> I cannot reproduce this on my system, with the current master branch,
> i.e. Emacs 30.  I _can_ reproduce this in Emacs 29.3, where indeed
> expanding any of the two abbrevs takes about 0.5 sec to show on the
> screen.  But in Emacs 30 it's instantaneous, even though I used an
> unoptimized build of Emacs 30.  Maybe the reason is that Emacs 30 now
> uses Org 9.7.4, whereas you have a local installation of 9.6.30?
>
> Why did you decide the problem was due to markers?  I see a single
> change in the implementation of markers in Emacs 29 as compared to
> Emacs 28, and no changes at all in Emacs 30 (except one that affects
> the Android, I think).
>
> > Using emacs 29+, when I use `profiler-start` (CPU) between Steps 4 and
> 5, and then `profiler-stop` after Step
> > 7, I get a very big entry (e.g., 70%) for `redisplay_internal (C
> function)` in the `profiler-report`. Using the
> > `profiler` at the same points using emacs 28.2, I don’t get a big entry
> for `redisplay_internal (C function)`.
>
> This doesn't necessarily mean the problem is due to the markers (but
> doesn't refute that, either).
>
> I see 3400 markers created by counsel-outline in this case, which is
> not too many, IMO.
>
> > Here are the steps to reproduce the issue:
> >
> > 1. Start emacs 29.3 (or 30.0.50) with emacs -q.
> > 2. Paste the minimal config at
> https://gist.github.com/kings2u/30b7a22dfa38fe0d5dc18adaf0e5e9de into the
> > scratch buffer and eval-buffer. (Note on my system I'm using the
> `counsel` version current as of
> > counsel-20240502.743, but any recent version will reproduce the bug).
> > 3. Open the following big file in org-mode:
> > https://gist.github.com/kings2u/c123a30de7e507a153a9500f03f08a9e
> > 4. `goto-line` 35.
> > 5. `M-x counsel-outline` and simply press enter to navigate point to the
> headline at line 34.
> > 6. Go to the end of the line and press `enter` to start typing on a new
> line at line 35.
> > 7. Start typing `n` `space`, or `t` `space` several times very fast and
> observe how long it takes for the abbrev
> > expansions (defined in the minimal config above) to show up on the
> screen.
>
> I didn't want to install all those packages (counsel is just the tip
> of the iceberg, it wants you to install ivy and swiper and whatnot),
> so I simply unpacked their tarballs from ELPA, and did this instead of
> steps 1 and 2 above:
>
>   (add-to-list 'load-path "~/foo/ivy-0.14.2")
>   (add-to-list 'load-path "~/foo/swiper-0.14.2")
>   (setq-default abbrev-mode t)
>   (setq save-abbrevs nil)
>   (add-hook 'minibuffer-setup-hook (lambda () (setq-local abbrev-mode
> nil)))
>   (define-abbrev global-abbrev-table "n" "and")
>   (define-abbrev global-abbrev-table "t" "the")
>   (define-abbrev global-abbrev-table "th" "that")
>
>   M-x load-file RET ~/foo/counsel-0.14.2/counsel.el RET
>   M-x eval-region RET
>
> Then I did all the rest of steps.  As mentioned, I do see the slow
> responses in Emacs 29.3, but not in the latest master branch of the
> Emacs Git repository.
>

[-- Attachment #2: Type: text/html, Size: 7372 bytes --]