bug#67070: 29.1; broken keybindings with emacs 29

bug#67070: 29.1; broken keybindings with emacs 29

[Enrico Scholz]

Emacs 29 seems to inject always a 'super' key event after 'meta'.
E.g. when I try to press 'M-x', emacs reports

| M-s-x is undefined

Ditto with any other key combination which involves 'meta'.

Things work as expected with emacs 28.3 and before.


My ~/.Xmodmap is

---
keycode 12  = 3 numbersign section
keycode 20  = minus underscore ssharp
keycode 26  = e E EuroSign
keycode 54  = c C cent
keycode 34  = bracketleft braceleft udiaeresis
keycode 47  = semicolon   colon     odiaeresis
keycode 48  = apostrophe  quotedbl  adiaeresis
keycode 28  = t T XF86Time

keycode 66  = Control_L

!keycode 37  = Mode_switch Caps_Lock
keycode 37  = Mode_switch
keycode 133  = Alt_L
keycode 64 = Meta_L

!keycode 108 = Mode_switch
keycode 108 = Meta_R Meta_R
keycode 134 = Meta_R
keycode 135 = Alt_R

clear Mod1
clear Mod4
clear Lock
clear control
add Lock = Caps_Lock
add control = Control_L Control_R
add Mod1 = Alt_L Alt_R
add Mod4 = Meta_L Meta_R

keycode 67 = F1 F1
keycode 68 = F2 F2
keycode 69 = F3 F3
keycode 70 = F4 F4
keycode 71 = F5 F5
keycode 72 = F6 F6
keycode 73 = F7 F7
keycode 74 = F8 F8
keycode 75 = F9 F9
!keycode 76 = F10 F10
keycode 95 = F11 F11
keycode 96 = F12 F12
---


In GNU Emacs 29.1 (build 1, x86_64-redhat-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.17.8) of 2023-09-24 built on
 e1d0f5fdfea948b9bb25e212c9c623e4
Windowing system distributor 'The X.Org Foundation', version 11.0.12014000
System Description: Fedora Linux 39 (Container Image)

Configured using:
 'configure --build=x86_64-redhat-linux-gnu
 --host=x86_64-redhat-linux-gnu --program-prefix=
 --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr
 --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc
 --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64
 --libexecdir=/usr/libexec --localstatedir=/var --runstatedir=/run
 --sharedstatedir=/var/lib --mandir=/usr/share/man
 --infodir=/usr/share/info --with-dbus --with-gif --with-jpeg --with-png
 --with-rsvg --with-tiff --with-xpm --with-x-toolkit=gtk3 --with-gpm=no
 --with-xwidgets --with-modules --with-harfbuzz --with-cairo --with-json
 --with-native-compilation=aot --with-tree-sitter --with-sqlite3
 --with-webp --with-xinput2 build_alias=x86_64-redhat-linux-gnu
 host_alias=x86_64-redhat-linux-gnu CC=gcc 'CFLAGS=-DMAIL_USE_LOCKF -O2
 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches
 -pipe -Wall -Werror=format-security
 -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS
 -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
 -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer '
 LDFLAGS=-Wl,-z,relro
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig CXX=g++
 'CXXFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g
 -grecord-gcc-switches -pipe -Wall -Werror=format-security
 -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS
 -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
 -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer ''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM XWIDGETS
GTK3 ZLIB

Important settings:
  value of $LANG: C
  locale-coding-system: nil

Major mode: ELisp/d

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: (only . t)
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  overwrite-mode: overwrite-mode-textual

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
comp comp-cstr warnings icons subr-x rx cl-seq cl-macs gv cl-extra
help-mode bytecomp byte-compile cl-lib sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads xwidget-internal dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 77729 7271)
 (symbols 48 7149 0)
 (strings 32 19806 2855)
 (string-bytes 1 596322)
 (vectors 16 15672)
 (vector-slots 8 328685 10196)
 (floats 8 27 46)
 (intervals 56 340 0)
 (buffers 984 12))

bug#67070: 29.1; broken keybindings with emacs 29

[Eli Zaretskii]

> From: Enrico Scholz <enrico.scholz@ensc.de>
> Date: Sat, 11 Nov 2023 11:32:26 +0100
> 
> Emacs 29 seems to inject always a 'super' key event after 'meta'.
> E.g. when I try to press 'M-x', emacs reports
> 
> | M-s-x is undefined
> 
> Ditto with any other key combination which involves 'meta'.
> 
> Things work as expected with emacs 28.3 and before.
> 
> 
> My ~/.Xmodmap is
> 
> ---
> keycode 12  = 3 numbersign section
> keycode 20  = minus underscore ssharp
> keycode 26  = e E EuroSign
> keycode 54  = c C cent
> keycode 34  = bracketleft braceleft udiaeresis
> keycode 47  = semicolon   colon     odiaeresis
> keycode 48  = apostrophe  quotedbl  adiaeresis
> keycode 28  = t T XF86Time
> 
> keycode 66  = Control_L
> 
> !keycode 37  = Mode_switch Caps_Lock
> keycode 37  = Mode_switch
> keycode 133  = Alt_L
> keycode 64 = Meta_L
> 
> !keycode 108 = Mode_switch
> keycode 108 = Meta_R Meta_R
> keycode 134 = Meta_R
> keycode 135 = Alt_R
> 
> clear Mod1
> clear Mod4
> clear Lock
> clear control
> add Lock = Caps_Lock
> add control = Control_L Control_R
> add Mod1 = Alt_L Alt_R
> add Mod4 = Meta_L Meta_R
> 
> keycode 67 = F1 F1
> keycode 68 = F2 F2
> keycode 69 = F3 F3
> keycode 70 = F4 F4
> keycode 71 = F5 F5
> keycode 72 = F6 F6
> keycode 73 = F7 F7
> keycode 74 = F8 F8
> keycode 75 = F9 F9
> !keycode 76 = F10 F10
> keycode 95 = F11 F11
> keycode 96 = F12 F12
> ---

Thanks.

Po Lu, any comments or suggestions?

bug#67070: 29.1; broken keybindings with emacs 29

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Emacs 29 seems to inject always a 'super' key event after 'meta'.
>> E.g. when I try to press 'M-x', emacs reports
>>
>> | M-s-x is undefined
>>
>> Ditto with any other key combination which involves 'meta'.

FWIW I had a similar issue a few weeks ago, with Emacs master.  I don't
think it was an Emacs problem though, IIRC I ended up resolving it by
tweaking my ~/.Xmodmap, adding or reordering a `clear` line.

When you say that you press `M-x`, which key exactly are you pressing?
A good way to gain more insight here would be to start `xev` and note
which keycodes it records when you say `M-x`.


Best,

Eshel

bug#67070: 29.1; broken keybindings with emacs 29

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Enrico Scholz <enrico.scholz@ensc.de>
>> Date: Sat, 11 Nov 2023 11:32:26 +0100
>> 
>> Emacs 29 seems to inject always a 'super' key event after 'meta'.
>> E.g. when I try to press 'M-x', emacs reports
>> 
>> | M-s-x is undefined
>> 
>> Ditto with any other key combination which involves 'meta'.
>> 
>> Things work as expected with emacs 28.3 and before.

Please type "xmodmap -query", then post the output here.  Two virtual
modifiers are probably being tied to one keycode.

bug#67070: 29.1; broken keybindings with emacs 29

[Enrico Scholz via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eshel Yaron <me@eshelyaron.com> writes:

>>> Emacs 29 seems to inject always a 'super' key event after 'meta'.
>>> E.g. when I try to press 'M-x', emacs reports
>>>
>>> | M-s-x is undefined
>>>
>>> Ditto with any other key combination which involves 'meta'.
>
> FWIW I had a similar issue a few weeks ago, with Emacs master.  I don't
> think it was an Emacs problem though, IIRC I ended up resolving it by
> tweaking my ~/.Xmodmap, adding or reordering a `clear` line.
>
> When you say that you press `M-x`, which key exactly are you pressing?

The key left of space (labeled "Alt") or right of it ("Alt Gr").


With xkbcomp from Fedora 38, I saw that '<SUPR>' was still assigned to
Mod4 ('xkbcomp :0 - | grep Mod4').  With Fedora 39 this is not the case
anymore (but problem still persists) and I can recover there by
reloading the xkb map

E.g. with xkbcomp-1.4.6:

1. setxkbmap && xmodmap ~/.Xmodmap

   --> emacs sees 'M-s-x'

2. xkbcomp :0 /tmp/bad && xkbcomp /tmp/bad :0

   --> emacs sees 'M-x' as expected


I solved to problem for me by switching to xkb; but this is much more
complicated than Xmodmap and because it seems to be impossible to clear a
'modifier_map' I have to copy & paste large parts of predefined keyboard
layouts.


> A good way to gain more insight here would be to start `xev` and note
> which keycodes it records when you say `M-x`.

KeyPress event, serial 30, synthetic NO, window 0xa600001,
    root 0x777, subw 0x0, time 196163413, (100,38), root:(2022,554),
    state 0x10, keycode 64 (keysym 0xffe7, Meta_L), same_screen YES,
    XLookupString gives 0 bytes: 
    XmbLookupString gives 0 bytes: 
    XFilterEvent returns: False

KeyPress event, serial 33, synthetic NO, window 0xa600001,
    root 0x777, subw 0x0, time 196169285, (100,38), root:(2022,554),
    state 0x50, keycode 53 (keysym 0x78, x), same_screen YES,
    XLookupString gives 1 bytes: (78) "x"
    XmbLookupString gives 1 bytes: (78) "x"
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0xa600001,
    root 0x777, subw 0x0, time 196169413, (100,38), root:(2022,554),
    state 0x50, keycode 53 (keysym 0x78, x), same_screen YES,
    XLookupString gives 1 bytes: (78) "x"
    XFilterEvent returns: False

KeyRelease event, serial 33, synthetic NO, window 0xa600001,
    root 0x777, subw 0x0, time 196172045, (100,38), root:(2022,554),
    state 0x50, keycode 64 (keysym 0xffe7, Meta_L), same_screen YES,
    XLookupString gives 0 bytes: 
    XFilterEvent returns: False

---

$ xmodmap -query
xmodmap:  up to 4 keys per modifier, (keycodes in parentheses):

shift       Shift_L (0x32),  Shift_R (0x3e)
lock      
control     Control_L (0x42),  Control_R (0x69)
mod1        Alt_L (0x85),  Alt_R (0x87),  Alt_L (0xcc)
mod2        Num_Lock (0x4d)
mod3        ISO_Level5_Shift (0xcb)
mod4        Meta_L (0x40),  Meta_R (0x6c),  Meta_R (0x86),  Meta_L (0xcd)
mod5        ISO_Level3_Shift (0x5c)

$ setxkbmap -query
rules:      evdev
model:      pc105
layout:     us



Enrico

bug#67070: 29.1; broken keybindings with emacs 29

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Enrico Scholz <enrico.scholz@sigma-chemnitz.de> writes:

> $ xmodmap -query
> xmodmap:  up to 4 keys per modifier, (keycodes in parentheses):
>
> shift       Shift_L (0x32),  Shift_R (0x3e)
> lock      
> control     Control_L (0x42),  Control_R (0x69)
> mod1        Alt_L (0x85),  Alt_R (0x87),  Alt_L (0xcc)
> mod2        Num_Lock (0x4d)
> mod3        ISO_Level5_Shift (0xcb)
> mod4        Meta_L (0x40),  Meta_R (0x6c),  Meta_R (0x86),  Meta_L (0xcd)
> mod5        ISO_Level3_Shift (0x5c)

What if you assign Super_L to a modifier bit just for testing?
I think this should coax XKB into assigning Super_L to a modifier
bit besides Mod4.

bug#67070: 29.1; broken keybindings with emacs 29

[Enrico Scholz via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Po Lu <luangruo@yahoo.com> writes:

>> $ xmodmap -query
>> xmodmap:  up to 4 keys per modifier, (keycodes in parentheses):
>>
>> shift       Shift_L (0x32),  Shift_R (0x3e)
>> lock      
>> control     Control_L (0x42),  Control_R (0x69)
>> mod1        Alt_L (0x85),  Alt_R (0x87),  Alt_L (0xcc)
>> mod2        Num_Lock (0x4d)
>> mod3        ISO_Level5_Shift (0xcb)
>> mod4        Meta_L (0x40),  Meta_R (0x6c),  Meta_R (0x86),  Meta_L (0xcd)
>> mod5        ISO_Level3_Shift (0x5c)
>
> What if you assign Super_L to a modifier bit just for testing?

After an additional

| add Mod3 = Super_L Super_R

emacs reports 'H-M-x is undefined' on 'Meta_L (0x40) + x'.  Difference
to previous xmodmap is

| -mod3        ISO_Level5_Shift (0xcb)
| +mod3        ISO_Level5_Shift (0xcb),  Super_L (0xce)


As before, I can recover by 'xkbcomp :0 - | xkbcomp - :0'.




Enrico

bug#67070: 29.1; broken keybindings with emacs 29

[Enrico Scholz via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Enrico Scholz <enrico.scholz@sigma-chemnitz.de> writes:

>> What if you assign Super_L to a modifier bit just for testing?
>
> After an additional
>
> | add Mod3 = Super_L Super_R
>
> emacs reports 'H-M-x is undefined' on 'Meta_L (0x40) + x'.

Doing

| add Mod3 = Super_L Super_R Hyper_R Hyper_L

generates the expected 'M-x'



The Super or Hyper keys are probably coming from

$ setxkbmap
$ xkbcomp :0 - | grep Mod4
    modifier_map Mod4 { <LWIN> };
    modifier_map Mod4 { <RWIN> };
    modifier_map Mod4 { <SUPR> };
    modifier_map Mod4 { <HYPR> };

$ xmodmap ~/.Xmodmap
$ xkbcomp :0 - | grep Mod4
    modifier_map Mod4 { <LALT> };
    modifier_map Mod4 { <RALT> };
    modifier_map Mod4 { <RWIN> };
    modifier_map Mod4 { <META> };

  --> bad 'M-s-x' or 'H-M-x'

$ xkbcomp :0 - | xkbcomp - :0
$ xkbcomp :0 - | grep Mod4
    modifier_map Mod4 { <LALT> };
    modifier_map Mod4 { <RALT> };
    modifier_map Mod4 { <RWIN> };
    modifier_map Mod4 { <META> };

  --> expected 'M-x'



Enrico

bug#67070: 29.1; broken keybindings with emacs 29

[Eli Zaretskii]

Ping!  Can we please make some further progress with this issue?

> From: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
> Cc: Eshel Yaron <me@eshelyaron.com>, Eli Zaretskii <eliz@gnu.org>,
>         67070@debbugs.gnu.org
> Date: Thu, 16 Nov 2023 15:33:52 +0100
> 
> Enrico Scholz <enrico.scholz@sigma-chemnitz.de> writes:
> 
> >> What if you assign Super_L to a modifier bit just for testing?
> >
> > After an additional
> >
> > | add Mod3 = Super_L Super_R
> >
> > emacs reports 'H-M-x is undefined' on 'Meta_L (0x40) + x'.
> 
> Doing
> 
> | add Mod3 = Super_L Super_R Hyper_R Hyper_L
> 
> generates the expected 'M-x'
> 
> 
> 
> The Super or Hyper keys are probably coming from
> 
> $ setxkbmap
> $ xkbcomp :0 - | grep Mod4
>     modifier_map Mod4 { <LWIN> };
>     modifier_map Mod4 { <RWIN> };
>     modifier_map Mod4 { <SUPR> };
>     modifier_map Mod4 { <HYPR> };
> 
> $ xmodmap ~/.Xmodmap
> $ xkbcomp :0 - | grep Mod4
>     modifier_map Mod4 { <LALT> };
>     modifier_map Mod4 { <RALT> };
>     modifier_map Mod4 { <RWIN> };
>     modifier_map Mod4 { <META> };
> 
>   --> bad 'M-s-x' or 'H-M-x'
> 
> $ xkbcomp :0 - | xkbcomp - :0
> $ xkbcomp :0 - | grep Mod4
>     modifier_map Mod4 { <LALT> };
>     modifier_map Mod4 { <RALT> };
>     modifier_map Mod4 { <RWIN> };
>     modifier_map Mod4 { <META> };
> 
>   --> expected 'M-x'
> 
> 
> 
> Enrico
> 

bug#67070: 29.1; broken keybindings with emacs 29

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> Ping!  Can we please make some further progress with this issue?

What's the current status here?

>> From: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
>> Cc: Eshel Yaron <me@eshelyaron.com>, Eli Zaretskii <eliz@gnu.org>,
>>         67070@debbugs.gnu.org
>> Date: Thu, 16 Nov 2023 15:33:52 +0100
>>
>> Enrico Scholz <enrico.scholz@sigma-chemnitz.de> writes:
>>
>> >> What if you assign Super_L to a modifier bit just for testing?
>> >
>> > After an additional
>> >
>> > | add Mod3 = Super_L Super_R
>> >
>> > emacs reports 'H-M-x is undefined' on 'Meta_L (0x40) + x'.
>>
>> Doing
>>
>> | add Mod3 = Super_L Super_R Hyper_R Hyper_L
>>
>> generates the expected 'M-x'
>>
>>
>>
>> The Super or Hyper keys are probably coming from
>>
>> $ setxkbmap
>> $ xkbcomp :0 - | grep Mod4
>>     modifier_map Mod4 { <LWIN> };
>>     modifier_map Mod4 { <RWIN> };
>>     modifier_map Mod4 { <SUPR> };
>>     modifier_map Mod4 { <HYPR> };
>>
>> $ xmodmap ~/.Xmodmap
>> $ xkbcomp :0 - | grep Mod4
>>     modifier_map Mod4 { <LALT> };
>>     modifier_map Mod4 { <RALT> };
>>     modifier_map Mod4 { <RWIN> };
>>     modifier_map Mod4 { <META> };
>>
>>   --> bad 'M-s-x' or 'H-M-x'
>>
>> $ xkbcomp :0 - | xkbcomp - :0
>> $ xkbcomp :0 - | grep Mod4
>>     modifier_map Mod4 { <LALT> };
>>     modifier_map Mod4 { <RALT> };
>>     modifier_map Mod4 { <RWIN> };
>>     modifier_map Mod4 { <META> };
>>
>>   --> expected 'M-x'
>>
>>
>>
>> Enrico
>>

bug#67070: 29.1; broken keybindings with emacs 29

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas <stefankangas@gmail.com> writes:

> What's the current status here?

The difficulty in such bugs where a modifier configuration set with
xmodmap does not apply correctly to Emacs 29 arise from XKB's deficient
backwards compatibility with such configurations.  My suggestion is to
exchange the xmodmap configuration for an XKB equivalent, where the
relations between key and virtual modifier and that between virtual
modifier and modifier bit are set down in an unambiguous manner.  In
addition to that, there could also be an option to disable XKB.