bug#32523: 27.0.50; Emacs hangs when killing rectangle

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Joseph Mingrone]

[-- Attachment #1: Type: text/plain, Size: 4301 bytes --]

Here is a recipe to make Emacs (nearly) hang indefinitely.

1. emacs -Q

2. Visit the file found https://ftfl.ca/misc/big_file_hangs_emacs.txt

3. M-x toggle-truncate-lines

4. Use rectangle-mark-mode (C-x SPC) to mark the rectangle that starts
   at the top left of the file (point 1), and includes the leading white
   space, the line numbers, and the space after the line numbers (point
   468848).

5. Kill the rectangle with C-x r k.

For me, the Emacs process will continue to use 100% CPU and Emacs is
almost completely unresponsive and has to be killed.  Some actions such
as saving the file may complete, but only after a few minutes.

This is with a recent commit (eb787d7) on the master branch.

In GNU Emacs 27.0.50 (build 1, amd64-portbld-freebsd11.2, GTK+ Version 3.22.29)
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --disable-build-details --localstatedir=/var
 --with-gameuser=games:games --without-libsystemd --without-mini-gmp
 --with-wide-int=no --with-x --enable-acl --without-dbus --without-gconf
 --with-gnutls --without-gsettings --with-json --with-lcms2
 --with-m17n-flt --with-mailutils --with-modules --with-libotf
 --with-toolkit-scroll-bars --with-threads --with-xft --with-xim
 --with-xml2 --without-xwidgets --with-file-notification=kqueue
 --with-sound=oss --with-x-toolkit=gtk3 --without-cairo --with-gif
 --with-jpeg --with-imagemagick --with-png --with-rsvg --with-tiff
 --with-xpm --x-libraries=/usr/local/lib --x-includes=/usr/local/include
 --prefix=/usr/local --mandir=/usr/local/man --disable-silent-rules
 --infodir=/usr/local/share/emacs/info/
 --build=amd64-portbld-freebsd11.2 'CFLAGS=-O2 -pipe -fstack-protector
 -isystem /usr/local/include -fno-strict-aliasing' 'CPPFLAGS=-isystem
 /usr/local/include' 'LDFLAGS= -fstack-protector -L/usr/local/lib''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND NOTIFY ACL GNUTLS LIBXML2
FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 MODULES
THREADS JSON LCMS2 GMP

Important settings:
  value of $LANG: en_CA.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv dired dired-loaddefs format-spec rfc822 mml
easymenu mml-sec password-cache epa derived epg epg-config gnus-util
rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue lcms2
dynamic-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 94849 6162)
 (symbols 48 20212 1)
 (strings 32 28576 1649)
 (string-bytes 1 752582)
 (vectors 16 14291)
 (vector-slots 8 505822 8998)
 (floats 8 47 70)
 (intervals 56 253 0)
 (buffers 992 11))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Eli Zaretskii]

> From: Joseph Mingrone <jrm@ftfl.ca>
> Date: Fri, 24 Aug 2018 21:59:48 -0300
> 
> Here is a recipe to make Emacs (nearly) hang indefinitely.
> 
> 1. emacs -Q
> 
> 2. Visit the file found https://ftfl.ca/misc/big_file_hangs_emacs.txt
> 
> 3. M-x toggle-truncate-lines
> 
> 4. Use rectangle-mark-mode (C-x SPC) to mark the rectangle that starts
>    at the top left of the file (point 1), and includes the leading white
>    space, the line numbers, and the space after the line numbers (point
>    468848).
> 
> 5. Kill the rectangle with C-x r k.
> 
> For me, the Emacs process will continue to use 100% CPU and Emacs is
> almost completely unresponsive and has to be killed.  Some actions such
> as saving the file may complete, but only after a few minutes.

It doesn't hang, it just takes very long to finish that operation (3
min on my machine with an unoptimized build; should be something like
1 to 1.5 min in an optimized build).

This belongs to the "Emacs is very slow with long lines" class of
problems: the file has 2900-character lines.  If this file will never
include any text, I suggest to visit it with
"M-x find-file-literally", then the problem of slowness will go away.

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Joseph Mingrone]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Joseph Mingrone <jrm@ftfl.ca>
>> Date: Fri, 24 Aug 2018 21:59:48 -0300

>> Here is a recipe to make Emacs (nearly) hang indefinitely.

>> 1. emacs -Q

>> 2. Visit the file found https://ftfl.ca/misc/big_file_hangs_emacs.txt

>> 3. M-x toggle-truncate-lines

>> 4. Use rectangle-mark-mode (C-x SPC) to mark the rectangle that starts
>>    at the top left of the file (point 1), and includes the leading white
>>    space, the line numbers, and the space after the line numbers (point
>>    468848).

>> 5. Kill the rectangle with C-x r k.

>> For me, the Emacs process will continue to use 100% CPU and Emacs is
>> almost completely unresponsive and has to be killed.  Some actions such
>> as saving the file may complete, but only after a few minutes.

> It doesn't hang, it just takes very long to finish that operation (3
> min on my machine with an unoptimized build; should be something like
> 1 to 1.5 min in an optimized build).

> This belongs to the "Emacs is very slow with long lines" class of
> problems: the file has 2900-character lines.  If this file will never
> include any text, I suggest to visit it with
> "M-x find-file-literally", then the problem of slowness will go away.

Thanks for the `find-file-literally' tip.

The rectangle does eventually get cut for me as well.  Ignoring speed,
the problem is that Emacs is unusable afterwards.  For example, if I go
away for an hour or so, then return, the Emacs process will still be
using something close to 100% CPU and trying to doing something simple,
like moving the point forward, may take minutes.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Stefan Kangas]

Joseph Mingrone <jrm@ftfl.ca> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Joseph Mingrone <jrm@ftfl.ca>
>>> Date: Fri, 24 Aug 2018 21:59:48 -0300
>
>>> Here is a recipe to make Emacs (nearly) hang indefinitely.
>
>>> 1. emacs -Q
>
>>> 2. Visit the file found https://ftfl.ca/misc/big_file_hangs_emacs.txt
>
>>> 3. M-x toggle-truncate-lines
>
>>> 4. Use rectangle-mark-mode (C-x SPC) to mark the rectangle that starts
>>>    at the top left of the file (point 1), and includes the leading white
>>>    space, the line numbers, and the space after the line numbers (point
>>>    468848).
>
>>> 5. Kill the rectangle with C-x r k.
>
>>> For me, the Emacs process will continue to use 100% CPU and Emacs is
>>> almost completely unresponsive and has to be killed.  Some actions such
>>> as saving the file may complete, but only after a few minutes.
>
>> It doesn't hang, it just takes very long to finish that operation (3
>> min on my machine with an unoptimized build; should be something like
>> 1 to 1.5 min in an optimized build).
>
>> This belongs to the "Emacs is very slow with long lines" class of
>> problems: the file has 2900-character lines.  If this file will never
>> include any text, I suggest to visit it with
>> "M-x find-file-literally", then the problem of slowness will go away.
>
> Thanks for the `find-file-literally' tip.
>
> The rectangle does eventually get cut for me as well.  Ignoring speed,
> the problem is that Emacs is unusable afterwards.  For example, if I go
> away for an hour or so, then return, the Emacs process will still be
> using something close to 100% CPU and trying to doing something simple,
> like moving the point forward, may take minutes.

Are you still seeing this behaviour?

If yes, could you try to run the profiler to see what Emacs is spending
so much time doing?

Best regards,
Stefan Kangas

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Fri, 21 Aug 2020 04:51:37 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 32523@debbugs.gnu.org
> 
> > The rectangle does eventually get cut for me as well.  Ignoring speed,
> > the problem is that Emacs is unusable afterwards.  For example, if I go
> > away for an hour or so, then return, the Emacs process will still be
> > using something close to 100% CPU and trying to doing something simple,
> > like moving the point forward, may take minutes.
> 
> Are you still seeing this behaviour?

I bet he does.  the problem with the slow responses after cutting the
rectangle is that Emacs performs redisplay each type the user types
some character.  The redisplay can be very small and optimized, but it
can also be much more thorough; for example, typing "M-x" typically
triggers a thorough redisplay.  Each time we need to perform a
non-trivial redisplay, the same problem with long lines hits again.

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> > The rectangle does eventually get cut for me as well.  Ignoring speed,
>> > the problem is that Emacs is unusable afterwards.  For example, if I go
>> > away for an hour or so, then return, the Emacs process will still be
>> > using something close to 100% CPU and trying to doing something simple,
>> > like moving the point forward, may take minutes.
>>
>> Are you still seeing this behaviour?
>
> I bet he does.  the problem with the slow responses after cutting the
> rectangle is that Emacs performs redisplay each type the user types
> some character.  The redisplay can be very small and optimized, but it
> can also be much more thorough; for example, typing "M-x" typically
> triggers a thorough redisplay.  Each time we need to perform a
> non-trivial redisplay, the same problem with long lines hits again.

Ah, so you interpret what he writes to mean that he leaves Emacs _in the
same buffer_ and then sees these results?  Yes, that makes sense.  I
somehow assumed he meant that this was persistent even after closing the
problematic buffer, but he didn't say that explicitly.

Asking the same questions here as in another bug report:

Is there anything more we can/should do in this case short of rewriting
the display engine?  Does it make sense to track this limitation in a
bug report?

etc/PROBLEMS says:

*** Editing files with very long lines is slow.

For example, simply moving through a file that contains hundreds of
thousands of characters per line is slow, and consumes a lot of CPU.
This is a known limitation of Emacs with no solution at this time.

Best regards,
Stefan Kangas

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Fri, 21 Aug 2020 07:06:29 -0700
> Cc: jrm@ftfl.ca, 32523@debbugs.gnu.org
> 
> Is there anything more we can/should do in this case short of rewriting
> the display engine?  Does it make sense to track this limitation in a
> bug report?

I answered this in bug#22143.

bug#32523: 27.0.50; Emacs hangs when killing rectangle

[Joseph Mingrone]

On Fri, 2020-08-21 at 07:06, Stefan Kangas <stefan@marxist.se> wrote:

> Eli Zaretskii <eliz@gnu.org> writes:

>>> > The rectangle does eventually get cut for me as well.  Ignoring speed,
>>> > the problem is that Emacs is unusable afterwards.  For example, if I go
>>> > away for an hour or so, then return, the Emacs process will still be
>>> > using something close to 100% CPU and trying to doing something simple,
>>> > like moving the point forward, may take minutes.

>>> Are you still seeing this behaviour?

>> I bet he does.  the problem with the slow responses after cutting the
>> rectangle is that Emacs performs redisplay each type the user types
>> some character.  The redisplay can be very small and optimized, but it
>> can also be much more thorough; for example, typing "M-x" typically
>> triggers a thorough redisplay.  Each time we need to perform a
>> non-trivial redisplay, the same problem with long lines hits again.

> Ah, so you interpret what he writes to mean that he leaves Emacs _in the
> same buffer_ and then sees these results?  Yes, that makes sense.  I
> somehow assumed he meant that this was persistent even after closing the
> problematic buffer, but he didn't say that explicitly.

> Asking the same questions here as in another bug report:

> Is there anything more we can/should do in this case short of rewriting
> the display engine?  Does it make sense to track this limitation in a
> bug report?

> etc/PROBLEMS says:

> *** Editing files with very long lines is slow.

> For example, simply moving through a file that contains hundreds of
> thousands of characters per line is slow, and consumes a lot of CPU.
> This is a known limitation of Emacs with no solution at this time.

> Best regards,
> Stefan Kangas

Hello Stefan,

I just tested the recipe in GNU Emacs 28.0.50 (build 1,
amd64-portbld-freebsd12.1, GTK+ Version 3.24.20, cairo version 1.16.0)
and it's still an issue in that Emacs is difficult to use after
performing the rectangle kill.  However, after the operation completes
*and I kill the buffer visiting the large file*, Emacs become responsive
again.  It's been a long time, but I recall that this wasn't the case in
the past.  I recall having to kill and restart Emacs before it became
usable again.

Joe