bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Jorgen Schaefer]

Hello!
Since a few days (I recompiled Emacs the first time on Wed 15th or Thu
16th, not sure when this was introduced before that) I see terminal
glyph garbage on certain deletion operations in emacs -nw.

A reliable way to trigger this for me is M-DEL, though many deletion
operations cause spurious data to be displayed. C-l will remove said
glyphs, and neither scrolling, normal editing nor indeed
single-character deletion will trigger this bug.

This is in a Gnome Terminal with TERM=xterm, sshing to a server, and
Emacs started in a screen with TERM=screen.

The Emacs built from ref 0465c9dd (last Apr 7th commit) does not exhibit
this behavior.

I know from at least one other person that recent builds with emacs -nw
work fine for them, so this is likely going to be tricky to debug. I'm
happy to do any testing needed.

Thanks!
Jorgen


Configured using:
 `configure --without-x'

Configured features:
SOUND DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB

Important settings:
  value of $LC_ALL: 
  value of $LC_COLLATE: de_DE.UTF-8
  value of $LC_CTYPE: de_DE.UTF-8
  value of $LC_MESSAGES: POSIX
  value of $LC_MONETARY: POSIX
  value of $LC_NUMERIC: POSIX
  value of $LC_TIME: POSIX
  value of $LANG: POSIX
  locale-coding-system: utf-8-unix

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Eli Zaretskii]

> From: Jorgen Schaefer <contact@jorgenschaefer.de>
> Date: Fri, 17 Apr 2015 11:33:32 +0200
> 
> Since a few days (I recompiled Emacs the first time on Wed 15th or Thu
> 16th, not sure when this was introduced before that) I see terminal
> glyph garbage on certain deletion operations in emacs -nw.
> 
> A reliable way to trigger this for me is M-DEL, though many deletion
> operations cause spurious data to be displayed. C-l will remove said
> glyphs, and neither scrolling, normal editing nor indeed
> single-character deletion will trigger this bug.
> 
> This is in a Gnome Terminal with TERM=xterm, sshing to a server, and
> Emacs started in a screen with TERM=screen.
> 
> The Emacs built from ref 0465c9dd (last Apr 7th commit) does not exhibit
> this behavior.
> 
> I know from at least one other person that recent builds with emacs -nw
> work fine for them, so this is likely going to be tricky to debug. I'm
> happy to do any testing needed.

Can you "git bisect" to find the offending commit, please?

TIA

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Stefan Monnier]

> A reliable way to trigger this for me is M-DEL, though many deletion
> operations cause spurious data to be displayed. C-l will remove said

This is likely coming from the "OSC 52" SetSelection support
recently added and which your gnome-terminal apparently doesn't support
yet term/xterm.el doesn't realize it.

Could you add some debug `message's to check that
terminal-init-xterm-activate-set-selection is indeed called (it
apparently shouldn't).  The problem is likely in xterm--version-handler
to add also some `message's there to see what `version' number is sent
by gnome-terminal (and why the code doesn't set it back to 200)?


        Stefan

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Jorgen Schäfer]

On Fri, Apr 17, 2015 at 2:26 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>
> Can you "git bisect" to find the offending commit, please?

Took a while, but here we go:

2b2fd3965f8c096b39de7e0418d37433269a1dce is the first bad commit.

I can confirm that redefining xterm--set-selection to a no-op fixes the problem.

(terminal-parameter nil 'terminal-initted) => terminal-init-screen

Regards,
Jorgen

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Eli Zaretskii]

> Date: Fri, 17 Apr 2015 17:32:54 +0200
> From: Jorgen Schäfer <contact@jorgenschaefer.de>
> Cc: 20356@debbugs.gnu.org
> 
> On Fri, Apr 17, 2015 at 2:26 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > Can you "git bisect" to find the offending commit, please?
> 
> Took a while, but here we go:
> 
> 2b2fd3965f8c096b39de7e0418d37433269a1dce is the first bad commit.

Thanks!

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Ivan Shmakov]

>>>>> Jorgen Schäfer <contact@jorgenschaefer.de> writes:

[…]

 > I can confirm that redefining xterm--set-selection to a no-op fixes
 > the problem.

	As there’s no possible way – in my setup – to /reliably/ detect
	OSC 52 support, and as I don’t need the feature anyway, I’ve
	simply disabled it with (setq select-enable-clipboard nil).

	My guess is that a new etc/PROBLEMS entry may turn to be useful
	to others.

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Stefan Monnier]

>>>>> Jorgen Schäfer <contact@jorgenschaefer.de> writes:
>> I can confirm that redefining xterm--set-selection to a no-op fixes
>> the problem.

But could you do the tests I requested so that it gets disabled
automatically by testing the terminal?

>>>>> "Ivan" == Ivan Shmakov <ivan@siamics.net> writes:
> 	As there’s no possible way – in my setup – to /reliably/ detect
> 	OSC 52 support,

Why not?


        Stefan

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Jorgen Schäfer]

On Fri, Apr 17, 2015 at 4:39 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> A reliable way to trigger this for me is M-DEL, though many deletion
>> operations cause spurious data to be displayed. C-l will remove said
>
> This is likely coming from the "OSC 52" SetSelection support
> recently added and which your gnome-terminal apparently doesn't support
> yet term/xterm.el doesn't realize it.
>
> Could you add some debug `message's to check that
> terminal-init-xterm-activate-set-selection is indeed called (it
> apparently shouldn't).

It is indeed being called.

> The problem is likely in xterm--version-handler
> to add also some `message's there to see what `version' number is sent
> by gnome-terminal (and why the code doesn't set it back to 200)?

The value of `str` after the first while loop is "83;40100;0" here. This is what
screen reports, and version is set to 240 in this case. Later, it calls
`terminal-init-xterm-activate-set-selection` if version is greater than 203.
Commenting this out fixes things for me again.

Regards,
Jorgen

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Stefan Monnier]

> The value of `str` after the first while loop is "83;40100;0"
> here.  This is what screen reports, and version is set to 240 in this

Wait, so you're not just running within gnome-terminal, but within
screen within gnome-terminal?
I guess that's the crux of the matter.  The code actually needs support
from the actual terminal (i.e gnome-terminal in this case), but the
version check is only told about screen's version.

Hmm...


        Stefan

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Jorgen Schäfer]

On Fri, Apr 24, 2015 at 4:07 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> The value of `str` after the first while loop is "83;40100;0"
>> here.  This is what screen reports, and version is set to 240 in this
>
> Wait, so you're not just running within gnome-terminal, but within
> screen within gnome-terminal?

Yep. This is a local Gnome terminal, running ssh to a server, where
Emacs runs in screen. :-)

(The same screen can also be attached to an ssh session from a Gnome
terminal on a
completely different GNU/Linux distribution and system, as well as to
a JuiceSSH session
on an Android tablet or a phone. The other gnome terminal exhibits the
same problem.
I have not used JuiceSSH much lately, so can't say if there are any
peculiar problems
there.)

Regards,
Jorgen

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Thierry Volpiatto]

Don't know if it is related to this bug, but recently I had my urxvt
full of stranges characters after compiling elisp files containing
errors with a make file.
The error in these files was a call to set-slot-value with the OBJ
argument forgotten.
If it help I can try to reproduce.

-- 
Thierry
Get my Gnupg key:
gpg --keyserver pgp.mit.edu --recv-keys 59F29997 

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Stefan Monnier]

> (The same screen can also be attached to an ssh session from a Gnome
> terminal on a completely different GNU/Linux distribution and system,
> as well as to a JuiceSSH session on an Android tablet or a phone. The
> other gnome terminal exhibits the same problem.  I have not used
> JuiceSSH much lately, so can't say if there are any peculiar
> problems there.)

Exactly.  I think the only sane solution is to disable this new feature
under `screen'.


        Stefan

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Stefan Monnier]

> This is in a Gnome Terminal with TERM=xterm, sshing to a server, and
> Emacs started in a screen with TERM=screen.

Can you confirm that the patch below fixes the problem you're seeing?


        Stefan


diff --git a/lisp/term/screen.el b/lisp/term/screen.el
index 3587c4f..41fd916 100644
--- a/lisp/term/screen.el
+++ b/lisp/term/screen.el
@@ -1,9 +1,22 @@
 ;;; screen.el --- terminal initialization for screen and tmux  -*- lexical-binding: t -*-
 ;; Copyright (C) 1995, 2001-2015 Free Software Foundation, Inc.
 
+(require 'term/xterm)
+
+(defcustom xterm-screen-extra-capabilities '(modifyOtherKeys)
+  "Extra capabilities supported under \"screen\".
+Some features of screen depend on the terminal emulator in which
+it runs, which can change when the screen session is moved to another tty."
+  :type xterm--extra-capabilities-type
+  :group 'xterm)
+
 (defun terminal-init-screen ()
   "Terminal initialization function for screen."
-  ;; Treat a screen terminal similar to an xterm.
-  (tty-run-terminal-initialization (selected-frame) "xterm"))
+  ;; Treat a screen terminal similar to an xterm, but don't use
+  ;; xterm-extra-capabilities's `check' setting since that doesn't seem
+  ;; to work so well (it depends too much on the surrounding terminal
+  ;; emulator, which can change during the session, bug#20356).
+  (let ((xterm-extra-capabilities xterm-screen-extra-capabilities))
+    (tty-run-terminal-initialization (selected-frame) "xterm")))
 
 ;; screen.el ends here
diff --git a/lisp/term/xterm.el b/lisp/term/xterm.el
index 726ecf9..4311647 100644
--- a/lisp/term/xterm.el
+++ b/lisp/term/xterm.el
@@ -29,6 +29,13 @@
   :version "24.1"
   :group 'terminals)
 
+(defconst xterm--extra-capabilities-type
+  ;; NOTE: If you add entries here, make sure to update
+  ;; `terminal-init-xterm' as well.
+  '(set (const :tag "modifyOtherKeys support" modifyOtherKeys)
+        (const :tag "report background" reportBackground)
+        (const :tag "set X selection" setSelection)))
+
 (defcustom xterm-extra-capabilities 'check
   "Whether Xterm supports some additional, more modern, features.
 If nil, just assume that it does not.
@@ -40,13 +47,8 @@ The relevant features are:
   reportBackground -- if supported, Xterm reports its background color
   setSelection     -- if supported, Xterm saves yanked text to the X selection"
   :version "24.1"
-  :type '(choice (const :tag "No" nil)
-                 (const :tag "Check" check)
-                 ;; NOTE: If you add entries here, make sure to update
-                 ;; `terminal-init-xterm' as well.
-                 (set (const :tag "modifyOtherKeys support" modifyOtherKeys)
-                      (const :tag "report background" reportBackground)
-                      (const :tag "set X selection" setSelection))))
+  :type `(choice (const :tag "Check" check)
+                 ,xterm--extra-capabilities-type))
 
 (defcustom xterm-max-cut-length 100000
   "Maximum number of bytes to cut into xterm using the OSC 52 sequence.
@@ -623,8 +625,8 @@ string bytes that can be copied is 3/4 of this value."
           (setq version 200))
         (when (equal (match-string 1 str) "83")
           ;; `screen' (which returns 83;40003;0) seems to also lack support for
-          ;; some of these (bug#17607).
-          (setq version 240))
+          ;; some of these (bug#17607, bug#20356).
+          (setq version 200))
         ;; If version is 242 or higher, assume the xterm supports
         ;; reporting the background color (TODO: maybe earlier
         ;; versions do too...)

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Jorgen Schäfer]

On Fri, May 1, 2015 at 12:31 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> This is in a Gnome Terminal with TERM=xterm, sshing to a server, and
>> Emacs started in a screen with TERM=screen.
>
> Can you confirm that the patch below fixes the problem you're seeing?

Hello!
Using (require 'xterm "term/xterm") instead of (require 'term/xterm),
this compiles
and I do not see the glyphs anymore. So yes, this fixes the problem.

Thank you!
Jorgen

bug#20356: 25.0.50; Strange terminal glyphs in emacs -nw in recent master

[Stefan Monnier]

> Using (require 'xterm "term/xterm") instead of (require 'term/xterm),
> this compiles and I do not see the glyphs anymore.

Indeed, there was a little problem there, which I think I've now fixed.

> So yes, this fixes the problem.

Great, installed, thank you,


        Stefan