bug#71644: 30.0.50; Severe slowdown in larger files with markers beginning in emacs 29+

[Eli Zaretskii <eliz@gnu.org>]

> From: Mitchell <mitchellahren@gmail.com>
> Date: Tue, 18 Jun 2024 23:25:18 -0600
> 
> I recently upgraded from emacs 28.2 to 29.3. When I resumed working in a large 11MB org-mode file that
> never gave me problems with speed on 28.2, I noticed a severe slowdown when editing the buffer after
> invoking `counsel-outline`. It took a while for me to isolate the issue, but it seems to be caused by the many
> markers that `counsel-outline` creates in the buffer. After invoking the command, every subsequent edit to the
> buffer renders much slower on screen than it did in 28.2.
> 
> This problem is not only caused by `counsel-outline`, but other functions that create a lot of markers in the
> buffer, like `org-refile` if `org-refile-use-cache` is set to `t`. I tried upgrading my system to emacs 30.0.50,
> which I've used to generate this bug report, but the issue persists. 

I cannot reproduce this on my system, with the current master branch,
i.e. Emacs 30.  I _can_ reproduce this in Emacs 29.3, where indeed
expanding any of the two abbrevs takes about 0.5 sec to show on the
screen.  But in Emacs 30 it's instantaneous, even though I used an
unoptimized build of Emacs 30.  Maybe the reason is that Emacs 30 now
uses Org 9.7.4, whereas you have a local installation of 9.6.30?

Why did you decide the problem was due to markers?  I see a single
change in the implementation of markers in Emacs 29 as compared to
Emacs 28, and no changes at all in Emacs 30 (except one that affects
the Android, I think).

> Using emacs 29+, when I use `profiler-start` (CPU) between Steps 4 and 5, and then `profiler-stop` after Step
> 7, I get a very big entry (e.g., 70%) for `redisplay_internal (C function)` in the `profiler-report`. Using the
> `profiler` at the same points using emacs 28.2, I don’t get a big entry for `redisplay_internal (C function)`.

This doesn't necessarily mean the problem is due to the markers (but
doesn't refute that, either).

I see 3400 markers created by counsel-outline in this case, which is
not too many, IMO.

> Here are the steps to reproduce the issue:
> 
> 1. Start emacs 29.3 (or 30.0.50) with emacs -q.
> 2. Paste the minimal config at https://gist.github.com/kings2u/30b7a22dfa38fe0d5dc18adaf0e5e9de into the
> scratch buffer and eval-buffer. (Note on my system I'm using the `counsel` version current as of
> counsel-20240502.743, but any recent version will reproduce the bug).
> 3. Open the following big file in org-mode:
> https://gist.github.com/kings2u/c123a30de7e507a153a9500f03f08a9e
> 4. `goto-line` 35.
> 5. `M-x counsel-outline` and simply press enter to navigate point to the headline at line 34.
> 6. Go to the end of the line and press `enter` to start typing on a new line at line 35.
> 7. Start typing `n` `space`, or `t` `space` several times very fast and observe how long it takes for the abbrev
> expansions (defined in the minimal config above) to show up on the screen.

I didn't want to install all those packages (counsel is just the tip
of the iceberg, it wants you to install ivy and swiper and whatnot),
so I simply unpacked their tarballs from ELPA, and did this instead of
steps 1 and 2 above:

  (add-to-list 'load-path "~/foo/ivy-0.14.2")
  (add-to-list 'load-path "~/foo/swiper-0.14.2")
  (setq-default abbrev-mode t)
  (setq save-abbrevs nil)
  (add-hook 'minibuffer-setup-hook (lambda () (setq-local abbrev-mode nil)))
  (define-abbrev global-abbrev-table "n" "and")
  (define-abbrev global-abbrev-table "t" "the")
  (define-abbrev global-abbrev-table "th" "that")

  M-x load-file RET ~/foo/counsel-0.14.2/counsel.el RET
  M-x eval-region RET

Then I did all the rest of steps.  As mentioned, I do see the slow
responses in Emacs 29.3, but not in the latest master branch of the
Emacs Git repository.