bug#44464: 28.0.50; slow shell output

bug#44464: 28.0.50; slow shell output

[Andreas Röhler]

[-- Attachment #1: Type: text/plain, Size: 3301 bytes --]

When running

curl -L -H "Accept: text/html" http://dbpedia.org/resource/Earth

output stops at several times.
It fontifies parts of the output - fontification is gone when finished. 
See attachments for freezes during output

No delay in system bash BTW.

In GNU Emacs 28.0.50 (build 1, i686-pc-linux-gnu, GTK+ Version 3.14.5, 
cairo version 1.14.0)
  of 2020-06-10
Repository revision: ee8b2742d7f6d03daea37f1bac48c2746f7ca789
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description: Debian GNU/Linux 8 (jessie)

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark set [4 times]
Quit
Mark set
5 0

Configured using:
  'configure --with-modules --with-mailutils'

Configured features:
XPM JPEG TIFF GIF PNG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL
GNUTLS LIBXML2 FREETYPE LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE
XIM MODULES THREADS PDUMPER GMP

Important settings:
   value of $LANG: de_DE.UTF-8
   locale-coding-system: utf-8-unix

Major mode: Shell

Minor modes in effect:
   shell-dirtrack-mode: t
   tooltip-mode: t
   global-eldoc-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search seq
byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date subr-x
cl-loaddefs cl-lib shell pcomplete comint ansi-color ring tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 8 173153 8669)
  (symbols 24 30041 1)
  (strings 16 65618 1981)
  (string-bytes 1 1087478)
  (vectors 8 13865)
  (vector-slots 4 381518 22024)
  (floats 8 25 46)
  (intervals 28 2390 2)
  (buffers 564 11)
  (heap 1024 15133 888))


[-- Attachment #2: slow-shell-1.png --]
[-- Type: image/png, Size: 205893 bytes --]

[-- Attachment #3: slow-shell-2.png --]
[-- Type: image/png, Size: 315259 bytes --]

bug#44464: 28.0.50; slow shell output

[Basil L. Contovounesios]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> When running
>
> curl -L -H "Accept: text/html" http://dbpedia.org/resource/Earth
>
> output stops at several times.

Isn't this just because of long lines in the output?
This is a well-known performance pitfall in Emacs.

-- 
Basil

bug#44464: 28.0.50; slow shell output

[Phil Sainty]

On 5/11/20 11:55 pm, Basil L. Contovounesios wrote:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>> When running
>> curl -L -H "Accept: text/html" http://dbpedia.org/resource/Earth
>> output stops at several times.
> 
> Isn't this just because of long lines in the output?
> This is a well-known performance pitfall in Emacs.

I can reproduce the issue (in 27.1), and it seems to me that it's
all about that one paragraph of right-to-left text.

If I disable `bidi-display-reordering', I see no slow-downs.

Eli, I note that setting `bidi-paragraph-direction' in the buffer
(with bidi-display-reordering enabled) does not seem to have any
obvious effect.  Setting it to `left-to-right', I still see
right-to-left movement through that portion of the text, and the
associated slowness.  Is that expected?

I'll add that the line in question is not *especially* long
(2374 characters); however it does seem to be a significant factor,
as enabling longlines-mode eliminates the slowness around that text.


-Phil

bug#44464: 28.0.50; slow shell output

[Eli Zaretskii]

> Cc: 44464@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
> From: Phil Sainty <psainty@orcon.net.nz>
> Date: Fri, 6 Nov 2020 02:54:17 +1300
> 
> I can reproduce the issue (in 27.1), and it seems to me that it's
> all about that one paragraph of right-to-left text.
> 
> If I disable `bidi-display-reordering', I see no slow-downs.

I don't see any particularly bad slowdown at all, but maybe I didn't
make all the steps (I don't have 'curl').  Do you see the same
slowdown if you fetch the URL with, say, WGET, and then visit it in
Emacs?  If not, how to reproduce this without 'curl'?

> Eli, I note that setting `bidi-paragraph-direction' in the buffer
> (with bidi-display-reordering enabled) does not seem to have any
> obvious effect.

It normally doesn't.  Only some very special files can benefit from
setting bidi-paragraph-direction non-nil.

> Setting it to `left-to-right', I still see right-to-left movement
> through that portion of the text, and the associated slowness.  Is
> that expected?

Yes.  Paragraph direction doesn't disable reordering, it just
determines whether the text will start at the left or the right edge
of the window.

> I'll add that the line in question is not *especially* long
> (2374 characters); however it does seem to be a significant factor,
> as enabling longlines-mode eliminates the slowness around that text.

I might be able to say something intelligent about this if I have a
recipe for reproducing the slowdown, and have some quantitative idea
of the slowdown (how much time does it take for Emacs to respond to
some specific command).

Thanks.

bug#44464: 28.0.50; slow shell output

[Basil L. Contovounesios]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 44464@debbugs.gnu.org, Eli Zaretskii <eliz@gnu.org>
>> From: Phil Sainty <psainty@orcon.net.nz>
>> Date: Fri, 6 Nov 2020 02:54:17 +1300
>> 
>> I can reproduce the issue (in 27.1), and it seems to me that it's
>> all about that one paragraph of right-to-left text.
>> 
>> If I disable `bidi-display-reordering', I see no slow-downs.
>
> I don't see any particularly bad slowdown at all, but maybe I didn't
> make all the steps (I don't have 'curl').  Do you see the same
> slowdown if you fetch the URL with, say, WGET, and then visit it in
> Emacs?  If not, how to reproduce this without 'curl'?

0. emacs -Q
1. M-x shell RET
2. C-x 1
3. wget -O- http://dbpedia.org/resource/Earth
4. RET

The printing to *shell* happens at a speed that I'm accustomed to in
Emacs shell emulators, but there's a noticeable pause for a fraction of
a second at a couple of lines, one of which contains Arabic script.

While printing, the last line is also highlighted in blue, i.e. with the
comint-highlight-prompt face.  You can also see this with e.g.

  $ while :; do printf hi; sleep 1; done

-- 
Basil

In GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo version 1.16.0, Xaw3d scroll bars)
 of 2020-11-05 built on thunk
Repository revision: 771046f5d888c703c875d8629fdf227429a36d3d
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Debian GNU/Linux bullseye/sid

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O2 -march=native' --config-cache
 --prefix=/home/blc/.local --with-x-toolkit=lucid
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT
LIBOTF ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS
LIBSYSTEMD JSON PDUMPER LCMS2

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

bug#44464: 28.0.50; slow shell output

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Cc: Phil Sainty <psainty@orcon.net.nz>,  andreas.roehler@easy-emacs.de,
>   44464@debbugs.gnu.org
> Date: Thu, 05 Nov 2020 14:41:07 +0000
> 
> 0. emacs -Q
> 1. M-x shell RET
> 2. C-x 1
> 3. wget -O- http://dbpedia.org/resource/Earth
> 4. RET

Thanks.  Looks like a regression between Emacs 26.3 and 27.1.  And
it's somehow related to auto-composition-mode, because turning that
off makes Emacs fast again.

Bisections welcome.

> While printing, the last line is also highlighted in blue, i.e. with the
> comint-highlight-prompt face.

I think this is part normal, as the last portion of display in shell
mode is always highlighted like that.