bug#38296: Allow Option key to be modifier for non-char key and mouse events

["Mattias Engdegård" <mattiase@acm.org>]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

Mac users, especially those of us with a non-US keyboard, need the Option keys for entering various characters, and thus set 'ns-alternate-modifier' to nil, or use 'ns-right-alternate-modifier' to employ only one of them for this purpose. However, this prevents use of that key as modifier for function keys (arrows, backspace etc) or mouse actions.

This patch adds the variable 'ns-alternate-modifier-char-entry' which, when non-nil, inhibits the Option key translation to Emacs modifiers for character keys but allows it for any other event. For example, setting

(setq ns-alternate-modifier 'super)
(setq ns-alternate-modifier-char-entry t)

makes Option act as Super with function keys and mouse clicks, but retains the standard macOS semantics for character entry (for example, Option + p = π).


[-- Attachment #2: 0001-Add-ns-alternate-modifier-char-entry.patch --]
[-- Type: application/octet-stream, Size: 7468 bytes --]

From 57e5811920211a41cfd97db9da30acebc32024e7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Tue, 19 Nov 2019 20:54:49 +0100
Subject: [PATCH] Add ns-alternate-modifier-char-entry

Allow the Option keys to be used for normal character entry while
having them generate Emacs modifiers for other events
by setting 'ns-alternate-modifier-char-entry' to non-nil.

* doc/emacs/macos.texi (Mac / GNUstep Basics): Document the variable.
* lisp/cus-start.el (standard): Allow customisation.
* src/nsterm.m (EV_MODIFIERS2): Add argument.
(EV_MODIFIERS): Adapt EV_MODIFIERS2 invocation.
(ns_get_shifted_character, [EmacsView keyDown:]): Use the new variable.
(syms_of_nsterm): Define the new variable.
* etc/NEWS: Mention the new variable.
---
 doc/emacs/macos.texi |  6 ++++++
 etc/NEWS             |  6 ++++++
 lisp/cus-start.el    |  1 +
 src/nsterm.m         | 35 +++++++++++++++++++++++++----------
 4 files changed, 38 insertions(+), 10 deletions(-)

diff --git a/doc/emacs/macos.texi b/doc/emacs/macos.texi
index d9920957ad..85ede7bd68 100644
--- a/doc/emacs/macos.texi
+++ b/doc/emacs/macos.texi
@@ -61,6 +61,12 @@ Mac / GNUstep Basics
 ignore them, in which case you get the default behavior of macOS
 accentuation system from the right @key{Option} key.
 
+@vindex ns-alternate-modifier-char-entry
+  To make the @key{Option} keys retain their macOS behaviour for
+entering characters but work as Emacs modifiers with function keys and
+mouse actions, set the @code{ns-alternate-modifier-char-entry}
+variable to a non-nil value.
+
   @kbd{S-mouse-1} adjusts the region to the click position,
 just like @kbd{mouse-3} (@code{mouse-save-then-kill}); it does not pop
 up a menu for changing the default face, as @kbd{S-mouse-1} normally
diff --git a/etc/NEWS b/etc/NEWS
index db00d40bb6..75f54304fe 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -3218,6 +3218,12 @@ Previously it was supported only in the Cygwin-w32 build.
 ** Emacs now handles key combinations involving the macOS "command"
 and "option" modifier keys more correctly.
 
++++
+** New user option 'ns-alternate-modifier-char-entry'.
+Set to non-nil to allow the macOS "option" keys to be used for normal
+character entry while treating them as Emacs modifiers for function keys
+and mouse events.
+
 ** The special handling of 'frame-title-format' on NS where setting it
 to 't' would enable the macOS proxy icon has been replaced with a
 separate variable, 'ns-use-proxy-icon'.  'frame-title-format' will now
diff --git a/lisp/cus-start.el b/lisp/cus-start.el
index e4b6d8f2d6..9d50108204 100644
--- a/lisp/cus-start.el
+++ b/lisp/cus-start.el
@@ -470,6 +470,7 @@ minibuffer-prompt-properties--setter
 		      (const control) (const meta)
 		      (const alt) (const hyper)
 		      (const super)) "23.3")
+	     (ns-alternate-modifier-char-entry ns boolean "27.1")
 	     (ns-function-modifier
 	      ns
 	      (choice (const :tag "No modifier (work as function)" none)
diff --git a/src/nsterm.m b/src/nsterm.m
index e1d745e332..38a404620d 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -380,7 +380,7 @@ - (NSColor *)colorUsingDefaultColorSpace
   return modifiers;
 }
 
-#define EV_MODIFIERS2(flags)                                            \
+#define EV_MODIFIERS2(flags, enable_alternate_mod)                      \
   (((flags & NSEventModifierFlagHelp) ?                                 \
     hyper_modifier : 0)                                                 \
    | ((flags & NSEventModifierFlagShift) ?                              \
@@ -397,13 +397,15 @@ - (NSColor *)colorUsingDefaultColorSpace
                           NSEventModifierFlagCommand,                   \
                           ns_command_modifier,                          \
                           ns_right_command_modifier)                    \
-   | ev_modifiers_helper (flags, NSLeftAlternateKeyMask,                \
-                          NSRightAlternateKeyMask,                      \
-                          NSEventModifierFlagOption,                    \
-                          ns_alternate_modifier,                        \
-                          ns_right_alternate_modifier))
+   | (enable_alternate_mod                                              \
+      ? ev_modifiers_helper (flags, NSLeftAlternateKeyMask,             \
+                             NSRightAlternateKeyMask,                   \
+                             NSEventModifierFlagOption,                 \
+                             ns_alternate_modifier,                     \
+                             ns_right_alternate_modifier)               \
+      : 0))
 
-#define EV_MODIFIERS(e) EV_MODIFIERS2 ([e modifierFlags])
+#define EV_MODIFIERS(e) EV_MODIFIERS2 ([e modifierFlags], true)
 
 #define EV_UDMODIFIERS(e)                                      \
     ((([e type] == NSEventTypeLeftMouseDown) ? down_modifier : 0)       \
@@ -2630,13 +2632,15 @@ so some key presses (TAB) are swallowed by the system.  */
   NSTRACE ("ns_get_shifted_character");
 
   if ((flags & NSRightAlternateKeyMask) == NSRightAlternateKeyMask
-      && (EQ (ns_right_alternate_modifier, Qnone)
+      && (ns_alternate_modifier_char_entry
+          || EQ (ns_right_alternate_modifier, Qnone)
           || (EQ (ns_right_alternate_modifier, Qleft)
               && EQ (ns_alternate_modifier, Qnone))))
     modifiers |= rightOptionKey;
 
   if ((flags & NSLeftAlternateKeyMask) == NSLeftAlternateKeyMask
-      && EQ (ns_alternate_modifier, Qnone))
+      && (ns_alternate_modifier_char_entry
+          || EQ (ns_alternate_modifier, Qnone)))
     modifiers |= optionKey;
 
   if ((flags & NSRightCommandKeyMask) == NSRightCommandKeyMask
@@ -6287,7 +6291,8 @@ In that case we use UCKeyTranslate (ns_get_shifted_character)
          modifier keys, which returns 0 for shift-like modifiers.
          Therefore its return value is the set of control-like
          modifiers.  */
-      emacs_event->modifiers = EV_MODIFIERS2 (flags);
+      bool enable_alt = fnKeysym || !ns_alternate_modifier_char_entry;
+      emacs_event->modifiers = EV_MODIFIERS2 (flags, enable_alt);
 
       /* Function keys (such as the F-keys, arrow keys, etc.) set
          modifiers as though the fn key has been pressed when it
@@ -9414,6 +9419,16 @@ Convert an X font name (XLFD) to an NS font name.
 at all, allowing it to be used at a lower level for accented character entry.");
   ns_right_alternate_modifier = Qleft;
 
+  DEFVAR_BOOL ("ns-alternate-modifier-char-entry",
+               ns_alternate_modifier_char_entry,
+               "Whether the option or alternate keys are used for character entry.\n\
+When non-nil, the alternate / option keys work as Emacs modifiers for\n\
+function keys and mouse actions but not with character-generating keys.\n\
+When nil, the keys are interpreted by Emacs for all events.\n\
+In either case, `ns-alternate-modifier' and `ns-right-alternate-modifier'\n\
+describe what Emacs modifiers are being generated.");
+  ns_alternate_modifier_char_entry = false;
+
   DEFVAR_LISP ("ns-command-modifier", ns_command_modifier,
                "This variable describes the behavior of the command key.\n\
 Set to the symbol control, meta, alt, super, or hyper means it is taken to be\n\
-- 
2.21.0 (Apple Git-122)