bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Gregor Zattler]

Dear emacs developers,

under screen with emacs from repo

emacsclient -t <some-filename>

almost instantaneously shows the scratch buffer (or some other
last used buffer), then for 2 secs nothing happens, then the
window shows the buffer which corresponds to some-file-name and
one is able to work with it.

This happens for some values of TERM.


recipe:

emacs-snapshot -Q --daemon

now do

/usr/bin/time screen -m -c /dev/null  -T $TERMINALNAME  sh -c "emacsclient.emacs-snapshot -n -t /tmp/emacsslow/AA${RANDOM}${RANDOM}BB ; emacsclient.emacs-snapshot --eval '(server-edit)'"

for different values of TERMINALNAME

Here is a sorted list of elapsed times for terminfo definitions
from my debian testing system (the elapsed times in this list are
the median of 11 rounds with each terminfo definiton):

0:00.05elapsed screen2
0:00.05elapsed screen3
0:00.05elapsed screen.Eterm
0:00.05elapsed screen.gnome
0:00.05elapsed screen.konsole
0:00.05elapsed screen.linux
0:00.05elapsed screen.mlterm
0:00.05elapsed screen.mrxvt
0:00.05elapsed screen.rxvt
0:00.05elapsed screen.teraterm
0:00.05elapsed screen.vte
0:00.05elapsed screen.xterm-new
0:00.05elapsed screen.xterm-r6

some of them are *much* slower:
0:02.06elapsed screen-16color
0:02.07elapsed screen-16color-bce
0:02.07elapsed screen-16color-s
0:02.07elapsed screen-bce.Eterm
0:02.07elapsed screen-bce.gnome
0:02.07elapsed screen-bce.konsole
0:02.07elapsed screen-bce.linux
0:02.07elapsed screen-bce.mlterm
0:02.07elapsed screen-bce.mrxvt
0:02.07elapsed screen-bce.rxvt
0:02.08elapsed screen-256color-s
0:02.09elapsed screen-256c-bce-s --> screen-256color-bce-s

Sadly I use the last one, which is a link to the actual terminfo
file since screen (1) does not accept TERM values with a length
of more then 20 characters.

I did the same timing tests with emacs24 as packaged for debian
testing:

0:00.02elapsed screen3
0:00.02elapsed screen.mrxvt
0:00.03elapsed screen-16color
0:00.03elapsed screen-16color-bce
0:00.03elapsed screen-16color-s
0:00.03elapsed screen2
0:00.03elapsed screen-bce.Eterm
0:00.03elapsed screen-bce.gnome
0:00.03elapsed screen-bce.konsole
0:00.03elapsed screen-bce.linux
0:00.03elapsed screen-bce.mlterm
0:00.03elapsed screen-bce.mrxvt
0:00.03elapsed screen-bce.rxvt
0:00.03elapsed screen.gnome
0:00.03elapsed screen.konsole
0:00.03elapsed screen.linux
0:00.03elapsed screen.mlterm
0:00.03elapsed screen.rxvt
0:00.03elapsed screen.teraterm
0:00.03elapsed screen.vte
0:00.03elapsed screen.xterm-new
0:00.04elapsed screen-256c-bce-s
0:00.04elapsed screen-256color-s
0:00.04elapsed screen.xterm-r6
0:00.10elapsed screen.Eterm

There is no such 2 secs delay.

I did a git bisect on the repo and this produced:

6607a79c6e7c7554059557c0db78c26c57314f24 is the first bad commit
commit 6607a79c6e7c7554059557c0db78c26c57314f24
Author: Daniel Colascione <dancol@dancol.org>
Date:   Thu Apr 17 00:54:23 2014 -0700

    2014-04-17  Daniel Colascione  <dancol@dancol.org>

        Add support for bracketed paste mode; add infrastructure for
        managing terminal mode enabling and disabling automatically.

        * xt-mouse.el:
        (xterm-mouse-mode): Simplify.
        (xterm-mouse-tracking-enable-sequence)
        (xterm-mouse-tracking-disable-sequence): New constants.
        (turn-on-xterm-mouse-tracking-on-terminal)
        (turn-off-xterm-mouse-tracking-on-terminal): Use
        tty-mode-set-strings and tty-mode-reset-strings terminal
        parameters instead of random hooks.
        (turn-on-xterm-mouse-tracking)
        (turn-off-xterm-mouse-tracking): Delete.

        * term/xterm.el (xterm-extra-capabilities): Fix bitrotted comment.
        (xterm-paste-ending-sequence): New constant.
        (xterm-paste): New command used for bracketed paste support.

        (xterm-modify-other-keys-terminal-list): Delete obsolete variable.
        (terminal-init-xterm-bracketed-paste-mode): New function.
        (terminal-init-xterm): Call it.
        (terminal-init-xterm-modify-other-keys): Use tty-mode-set-strings
        and tty-mode-reset-strings instead of random hooks.
        (xterm-turn-on-modify-other-keys)
        (xterm-turn-off-modify-other-keys)
        (xterm-remove-modify-other-keys): Delete obsolete functions.

        * term/screen.el: Rewrite to just use the xterm code.  Add
        copyright notice.  Mention tmux.

:040000 040000 35e7957f61e4d66ebd209cd1c4e866a28d4b2530 ab84f10010ef069579268a06a8cb87297a99222f M      doc
:040000 040000 6e356fc3442cded91abe85539d87c010a5b55b74 97ed2cb729e1981da13e3087f31ca1d889d207f5 M      etc
:040000 040000 1f0fa9c7b097af1630c809920c763294996bc9e1 e2cb31100399d2a62f34c29014f588c097033f8d M      lisp
:040000 040000 502e344d50623e95ecd35483334810ab76f9f5da f2e77d45d90ad1964f395e75cbd318ee01e4b7a3 M      src
bisect run success


0 grfz@boo:~/src/emacs$ git bisect log
git bisect start
# bad: [70ae6e9f3ca829e4a22290a8f2434343e2f0e127] * admin/notes/changelogs: Mention `quoting'.
git bisect bad 70ae6e9f3ca829e4a22290a8f2434343e2f0e127
# good: [87cc4be37367f4039a1fb6bda8ba681bb279475f] Bump version to 24.3.91
git bisect good 87cc4be37367f4039a1fb6bda8ba681bb279475f
# bad: [83e103c61b9d775709624a44e7ed5b20ba44847d] Improve Scheme font-locking for (define ((foo ...) ...) ...).
git bisect bad 83e103c61b9d775709624a44e7ed5b20ba44847d
# good: [5910501252ec0067857e36c1d73e7b522d83b861] * emacs-lisp/eldoc.el (eldoc-print-current-symbol-info): Refactor out eldoc-documentation-function-default.
(eldoc-documentation-function-default): New function. (eldoc-documentation-function): Change value.
git bisect good 5910501252ec0067857e36c1d73e7b522d83b861
# good: [f0f301dd19212bf133aacd7fd592e126958309d8] Merge from emacs-24; up to r116940
git bisect good f0f301dd19212bf133aacd7fd592e126958309d8
# bad: [069e400fbf7d0e404438a922c9f5839b1d4902da] Tweak documentation for previous change
git bisect bad 069e400fbf7d0e404438a922c9f5839b1d4902da
# good: [6f100a7a58857ce2838a11eec596073837ba7858] Remove DATA_SEG_BITS.
git bisect good 6f100a7a58857ce2838a11eec596073837ba7858
# good: [085874f070af05df7a80b8e10cb5f88696bd685e] * GNUmakefile: Speed up 'make bootstrap' in fresh checkout.
git bisect good 085874f070af05df7a80b8e10cb5f88696bd685e
# bad: [ed507155bec59e1dd1844b4593d75aec1906b594] Merge from emacs-24; up to r116982
git bisect bad ed507155bec59e1dd1844b4593d75aec1906b594
# bad: [5c1915d10b3716879785fe49f5cfe20beeb37090] * term.c (tty_send_additional_strings): No need to fflush here,
git bisect bad 5c1915d10b3716879785fe49f5cfe20beeb37090
# bad: [6607a79c6e7c7554059557c0db78c26c57314f24] 2014-04-17  Daniel Colascione  <dancol@dancol.org>
git bisect bad 6607a79c6e7c7554059557c0db78c26c57314f24
# first bad commit: [6607a79c6e7c7554059557c0db78c26c57314f24] 2014-04-17  Daniel Colascione  <dancol@dancol.org>




I have no idea if or how this is related to the problem at hand,
but screen.el was changed, therefore I hope this info helps.

I wished, emacsclient would open files be fast again.


I would be happy to answer questions or do some testing
if someone provides me with instruction.

Thanks for your attention, Gregor

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Stefan Monnier]

> almost instantaneously shows the scratch buffer (or some other
> last used buffer), then for 2 secs nothing happens, then the
> window shows the buffer which corresponds to some-file-name and
> one is able to work with it.

> This happens for some values of TERM.

Can you describe as precisely as possible the text-terminal you're
using, and tell us what C-h l says right after this 2s delay?

> I did a git bisect on the repo and this produced:
[...]
>     2014-04-17  Daniel Colascione  <dancol@dancol.org>
>         Add support for bracketed paste mode; add infrastructure for
>         managing terminal mode enabling and disabling automatically.

Dan, it looks like the magic sequence to enable bracketed paste mode
will need to use some version checking code :-(
Can you take a look at it?


        Stefan

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

On 05/31/2014 06:23 PM, Stefan Monnier wrote:
>> almost instantaneously shows the scratch buffer (or some other
>> last used buffer), then for 2 secs nothing happens, then the
>> window shows the buffer which corresponds to some-file-name and
>> one is able to work with it.
> 
>> This happens for some values of TERM.
> 
> Can you describe as precisely as possible the text-terminal you're
> using, and tell us what C-h l says right after this 2s delay?
> 
>> I did a git bisect on the repo and this produced:
> [...]
>>     2014-04-17  Daniel Colascione  <dancol@dancol.org>
>>         Add support for bracketed paste mode; add infrastructure for
>>         managing terminal mode enabling and disabling automatically.
> 
> Dan, it looks like the magic sequence to enable bracketed paste mode
> will need to use some version checking code :-(
> Can you take a look at it?

Sure --- but I'm not sure why the result varies depending on the value
of TERM. Aren't we sending the same sequence regardless? Jim's recent
complaint about another two second delay seems related. I feel like the
terminal-echo recognition must be wrong somehow.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

On 05/31/2014 06:25 PM, Daniel Colascione wrote:
> On 05/31/2014 06:23 PM, Stefan Monnier wrote:
>>> almost instantaneously shows the scratch buffer (or some other
>>> last used buffer), then for 2 secs nothing happens, then the
>>> window shows the buffer which corresponds to some-file-name and
>>> one is able to work with it.
>>
>>> This happens for some values of TERM.
>>
>> Can you describe as precisely as possible the text-terminal you're
>> using, and tell us what C-h l says right after this 2s delay?
>>
>>> I did a git bisect on the repo and this produced:
>> [...]
>>>     2014-04-17  Daniel Colascione  <dancol@dancol.org>
>>>         Add support for bracketed paste mode; add infrastructure for
>>>         managing terminal mode enabling and disabling automatically.
>>
>> Dan, it looks like the magic sequence to enable bracketed paste mode
>> will need to use some version checking code :-(
>> Can you take a look at it?
> 
> Sure --- but I'm not sure why the result varies depending on the value
> of TERM. Aren't we sending the same sequence regardless? Jim's recent
> complaint about another two second delay seems related. I feel like the
> terminal-echo recognition must be wrong somehow.

Also, that change also made term/screen.el use term/xterm.el. It's not
the bracketed paste stuff per se that's causing the problem, but the
terminal echo stuff.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Stefan Monnier]

> Also, that change also made term/screen.el use term/xterm.el.

Ah, maybe that's the problem, indeed.

> It's not the bracketed paste stuff per se that's causing the problem,

I hope so.

> but the terminal echo stuff.

Not sure what you mean by "terminal echo".


        Stefan

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Gregor Zattler]

Hi Stefan, Daniel,
* Stefan Monnier <monnier@iro.umontreal.ca> [31. May. 2014]:
>> almost instantaneously shows the scratch buffer (or some other
>> last used buffer), then for 2 secs nothing happens, then the
>> window shows the buffer which corresponds to some-file-name and
>> one is able to work with it.
> 
>> This happens for some values of TERM.
> 
> Can you describe as precisely as possible the text-terminal you're
> using,

rxvt-unicode (urxvt) v9.20 - released: 2014-04-26
options: perl,xft,styles,combining,blink,iso14755,unicode3,encodings=eu+vn+jp+jp-ext+kr+Zusammenhang+Zusammenhang-ext,fade,transparent,tint,pixbuf,XIM,frills,selectionscrolling,wh
eel,slipwheel,smart-resize,cursorBlink,pointerBlank,scrollbars=plain+rxvt+NeXT+xterm

> and tell us what C-h l says right after this 2s delay?

This is from the instance I'm writing this reply to your message:

l e s , c o m b i n i n g , b l i n k , i s o 1 4 7
5 5 , u n i c o d e 3 , e n c o d i n g s = e u + v
n + j p + j p - e x t + k r + z h + z h - e x t , f
a d e , t r a n s p a r e n t , t i n t , p i x b u
f , X I M , f r i l l s , s e l e c t i o n s c r o
l l i n g , w h RET e e l , s l i p w h e e l , s m
a r t - r e s i z e , c u r s o r B l i n k , p o i
n t e r B l a n k , s c r o l l b a r s = p l a i n
+ r x v t + N e X T + x t e r m ESC O A ESC O A ESC
O A ESC O A ESC O A ESC O A ESC O B ESC O B ESC O B
ESC O B ESC O B ESC O B ESC O B ESC O B ESC O B ESC
O B ESC O B ESC O A ESC O A C-a DEL C-x C-s ESC [ >
8 3 ; 4 0 2 0 0 ; 0 c C-h l

This is after the very first connection to a emacs server started with:
emacs-snapshot -Q --daemon:

ESC [ > 8 3 ; 4 0 2 0 0 ; 0 c C-h l



Thanks, Gregor

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Gregor Zattler]

Hi Daniel, Stefan,
* Daniel Colascione <dancol@dancol.org> [31. May. 2014]:
> On 05/31/2014 06:25 PM, Daniel Colascione wrote:
>> On 05/31/2014 06:23 PM, Stefan Monnier wrote:
>>> Dan, it looks like the magic sequence to enable bracketed paste mode
>>> will need to use some version checking code :-(
>>> Can you take a look at it?
>> 
>> Sure --- but I'm not sure why the result varies depending on the value
>> of TERM. Aren't we sending the same sequence regardless? Jim's recent
>> complaint about another two second delay seems related. I feel like the
>> terminal-echo recognition must be wrong somehow.
> 
> Also, that change also made term/screen.el use term/xterm.el.


I tested emacsclients snappiness under screen with other
(= non screen*) values of TERM like so:

TERM=rxvt-unicode-256color emacsclient.emacs-snapshot -t bugreport

TERM=rxvt-unicode-256color is snappy
TERM=xterm is slow

So there would be a workaround for me but I do not know of the
side effects of forcing the use of a non screen* value for TERM.

And for completeness: TERM=xterm is snappy on a pure xterm
without using screen.

Thanks for looking into this, Gregor

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Stefan Monnier]

> rxvt-unicode (urxvt) v9.20 - released: 2014-04-26
> options:
> perl,xft,styles,combining,blink,iso14755,unicode3,encodings=eu+vn+jp+jp-ext+kr+Zusammenhang+Zusammenhang-ext,fade,transparent,tint,pixbuf,XIM,frills,selectionscrolling,wh
> eel,slipwheel,smart-resize,cursorBlink,pointerBlank,scrollbars=plain+rxvt+NeXT+xterm

Now, I'm confused: are you running Emacs inside rxvt-unicode or inside
screen (at least in theory, if Emacs is running inside screen, it
shouldn't be affected by whether that screen runs inside an xterm, rxvt,
or anything else: that's screen's problem).

> This is after the very first connection to a emacs server started with:
> emacs-snapshot -Q --daemon:

> ESC [ > 8 3 ; 4 0 2 0 0 ; 0 c C-h l

Hmm... I recently installed a patch for terminals using "ESC [ > 83 ..."
as above.  Could you try again (with the emacs-24 code, since it hasn't
been merged into trunk yet) to see if that fixes it, by any chance?

After trying out rxvt-unicode and screen here, I think you're running
Emacs inside screen rather than inside rxvt, and I think Jim was also
running his Emacs inside screen rather than inside an OSX Terminal.app.
Jim, can you confirm?


        Stefan

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Jim Meyering]

On Tue, Jun 17, 2014 at 7:07 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> rxvt-unicode (urxvt) v9.20 - released: 2014-04-26
>> options:
>> perl,xft,styles,combining,blink,iso14755,unicode3,encodings=eu+vn+jp+jp-ext+kr+Zusammenhang+Zusammenhang-ext,fade,transparent,tint,pixbuf,XIM,frills,selectionscrolling,wh
>> eel,slipwheel,smart-resize,cursorBlink,pointerBlank,scrollbars=plain+rxvt+NeXT+xterm
>
> Now, I'm confused: are you running Emacs inside rxvt-unicode or inside
> screen (at least in theory, if Emacs is running inside screen, it
> shouldn't be affected by whether that screen runs inside an xterm, rxvt,
> or anything else: that's screen's problem).
>
>> This is after the very first connection to a emacs server started with:
>> emacs-snapshot -Q --daemon:
>
>> ESC [ > 8 3 ; 4 0 2 0 0 ; 0 c C-h l
>
> Hmm... I recently installed a patch for terminals using "ESC [ > 83 ..."
> as above.  Could you try again (with the emacs-24 code, since it hasn't
> been merged into trunk yet) to see if that fixes it, by any chance?
>
> After trying out rxvt-unicode and screen here, I think you're running
> Emacs inside screen rather than inside rxvt, and I think Jim was also
> running his Emacs inside screen rather than inside an OSX Terminal.app.
> Jim, can you confirm?

Yes.  Sorry for the inaccuracy.  There was indeed a screen
intermediary between the OSX Terminal.app and my emacs process.

bug#17648: 24.4.50; regression: emacsclient under screen (1) slow for some values of TERM

[Stefan Monnier]

forcemerge 17607 17648
thanks

> Yes.  Sorry for the inaccuracy.  There was indeed a screen
> intermediary between the OSX Terminal.app and my emacs process.

Aha!  So the problem was not with Terminal.app but with Screen.
So presumably this bug should also now be fixed in emacs-24.


        Stefan