bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

[Pierre Neidhardt]

I originally reported this bug on Evil's issue tracker:

	https://github.com/emacs-evil/evil/issues/1006

The recipe with Evil is as follows:

- Start Emacs.

- `M-x eshell'.

- Insert lots of colored text. On Linux, you can call `dmesg -L=always' a few times.

- Go to normal state and press `x' wherever you can delete a character.

It should be possible to reproduce without Evil, I just could not figure
out a slow operation to replace the last step in the recipe.

According to the Evil maintainer, the issue comes from markers left
behind.

In fact, if I run `clone-buffer', the cloned buffer does not suffer from
this issue.  I think `clone-buffer' does not copy markers.

By the way, is there a way to scan for markers other than
`buffer-has-markers-at'?
Is there a way to remove them?


In GNU Emacs 25.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.26)
 of 2017-12-04 built on arojas
Windowing system distributor 'The X.Org Foundation', version 11.0.11905000
System Description:	Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft --with-modules
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 -fno-plt' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
<#secure method=pgpmime mode=sign>

-- 
Pierre Neidhardt

You will gain money by an immoral action.

bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

[Noam Postavsky]

Pierre Neidhardt <ambrevar@gmail.com> writes:

> The recipe with Evil is as follows:
>
> - Start Emacs.
>
> - `M-x eshell'.
>
> - Insert lots of colored text. On Linux, you can call `dmesg -L=always' a few times.
>
> - Go to normal state and press `x' wherever you can delete a character.
>
> It should be possible to reproduce without Evil, I just could not figure
> out a slow operation to replace the last step in the recipe.
>
> According to the Evil maintainer, the issue comes from markers left
> behind.

So does this fix it?

--- i/lisp/ansi-color.el
+++ w/lisp/ansi-color.el
@@ -417,3 +417,7 @@ ansi-color-apply-on-region
 		 start-marker end-marker (ansi-color--find-face codes))
-	(setq ansi-color-context-region (if codes (list codes)))))))
+	(setq ansi-color-context-region (if codes (list codes)))))
+    ;; Clean up our temporary markers.
+    (unless (eq start-marker (cadr ansi-color-context-region))
+      (set-marker start-marker nil))
+    (set-marker end-marker nil)))

Also, I wonder if doing this would help also?

    (setq
     ansi-color-apply-face-function
     (lambda (beg end face)
       (when face
         (put-text-property beg end 'face face))))

That should make Emacs use text properties instead of overlays for the
colored text.

bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1744 bytes --]

The patch does not seem to do it.  Unless load order matters?

The redifinition of `ansi-color-apply-face-function` works however, plus it
comes at the bonus of making Eshell an order of magnitude faster!
I haven't tested estensively yet, but it seems to be a much better default.

On Mon, Jan 1, 2018 at 3:07 AM, Noam Postavsky <
npostavs@users.sourceforge.net> wrote:

> Pierre Neidhardt <ambrevar@gmail.com> writes:
>
> > The recipe with Evil is as follows:
> >
> > - Start Emacs.
> >
> > - `M-x eshell'.
> >
> > - Insert lots of colored text. On Linux, you can call `dmesg -L=always'
> a few times.
> >
> > - Go to normal state and press `x' wherever you can delete a character.
> >
> > It should be possible to reproduce without Evil, I just could not figure
> > out a slow operation to replace the last step in the recipe.
> >
> > According to the Evil maintainer, the issue comes from markers left
> > behind.
>
> So does this fix it?
>
> --- i/lisp/ansi-color.el
> +++ w/lisp/ansi-color.el
> @@ -417,3 +417,7 @@ ansi-color-apply-on-region
>                  start-marker end-marker (ansi-color--find-face codes))
> -       (setq ansi-color-context-region (if codes (list codes)))))))
> +       (setq ansi-color-context-region (if codes (list codes)))))
> +    ;; Clean up our temporary markers.
> +    (unless (eq start-marker (cadr ansi-color-context-region))
> +      (set-marker start-marker nil))
> +    (set-marker end-marker nil)))
>
> Also, I wonder if doing this would help also?
>
>     (setq
>      ansi-color-apply-face-function
>      (lambda (beg end face)
>        (when face
>          (put-text-property beg end 'face face))))
>
> That should make Emacs use text properties instead of overlays for the
> colored text.
>

[-- Attachment #2: Type: text/html, Size: 2362 bytes --]

bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

[Noam Postavsky]

Pierre Neidhardt <ambrevar@gmail.com> writes:

> The patch does not seem to do it.  Unless load order matters?

Load order should not matter.

> The redifinition of `ansi-color-apply-face-function` works however,
> plus it comes at the bonus of making Eshell an order of magnitude
> faster!
> I haven't tested estensively yet, but it seems to be a much better
> default.

Aha, maybe it's not a question of markers, but rather another case of
Bug#26051 "overlays may make emacs very slow".

(anyway, ansi-color-apply-to-region doesn't set markers per colour
boundary; it's just 1 or 2 markers per call (eshell calls it once per
~4096 bytes))

The question is then whether there is some case where using overlays is
necessary, or is it just a pessimization?  For reference, I took the
text property using definition from man.el (so it at least works for
that case):

    (defun Man-fontify-manpage ()
      [...]
      (let ((ansi-color-apply-face-function
             (lambda (beg end face)
               (when face
                 (put-text-property beg end 'face face))))

bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 519 bytes --]


> The question is then whether there is some case where using overlays is
> necessary, or is it just a pessimization?  For reference, I took the
> text property using definition from man.el (so it at least works for
> that case):

Any idea what the original intent was?
I can't seem to see why an overlay would be needed for Eshell's output.

-- 
Pierre Neidhardt

Abstainer, n.:
	A weak person who yields to the temptation of denying himself a
	pleasure.
		-- Ambrose Bierce, "The Devil's Dictionary"

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

[Noam Postavsky]

Pierre Neidhardt <ambrevar@gmail.com> writes:

>> The question is then whether there is some case where using overlays is
>> necessary, or is it just a pessimization?  For reference, I took the
>> text property using definition from man.el (so it at least works for
>> that case):
>
> Any idea what the original intent was?
> I can't seem to see why an overlay would be needed for Eshell's output.

Hard to say.  It might just be a case of going with ansi-color.el's
default.  ansi-color-apply-on-region was changed to use overlays in [1:
0e3c1e3ea6], but it's unclear why.  The commit message just says

    (ansi-color-apply-on-region): Use extents or overlays instead of
    text-properties.

[1: 0e3c1e3ea6]: 2001-01-09 11:38:28 +0000
  (ansi-color-process-output): Use markers instead of positions for start and end of region.[...]
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=0e3c1e3ea6a27e20d0252661336afe9fc84b21f5

bug#29854: 25.3; Eshell buffer editing gets slower as colored output grows

[Noam Postavsky]

tags 29854 fixed
close 29854 27.1
quit

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> Pierre Neidhardt <ambrevar@gmail.com> writes:
>
>>> The question is then whether there is some case where using overlays is
>>> necessary, or is it just a pessimization?  For reference, I took the
>>> text property using definition from man.el (so it at least works for
>>> that case):
>>
>> Any idea what the original intent was?
>> I can't seem to see why an overlay would be needed for Eshell's output.
>
> Hard to say.  It might just be a case of going with ansi-color.el's
> default.  ansi-color-apply-on-region was changed to use overlays in [1:
> 0e3c1e3ea6], but it's unclear why.  The commit message just says
>
>     (ansi-color-apply-on-region): Use extents or overlays instead of
>     text-properties.

I've changed to use text properties (but font-lock-face rather than
face) in master.  We'll see if it causes trouble.

4fd9048e94 2019-05-04T15:33:20-04:00 "Avoid slow overlay ansi coloring in eshell (Bug#29854)"
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=4fd9048e940d38364caf4abe9b209f9288c78544