bug#42915: 27.1; Setting ns-function-modifier to control breaks control for certain keys

[Alan Third <alan@idiocy.org>]

[-- Attachment #1: Type: text/plain, Size: 974 bytes --]

On Tue, Aug 18, 2020 at 10:22:18AM -0400, Ryan C. Thompson wrote:
> On a mac, run:
> 
> emacs -Q --eval "(setq ns-function-modifier 'control)"
> 
> Then try C-right, C-home, and similar key combinations. Emacs will act as if
> control is not being held down, i.e. C-right will run right-char instead of
> right-word. Indeed, C-h k C-right shows the key binding for right, not
> C-right. However, other key combinations, such as C-f, work as expected.
> 
> As far as I can tell, this only started happening after I updated to Emacs
> 27. I was previously using this config with Emacs 26.3 and never saw this
> issue. So I believe this is a new bug in Emacs 27.

Can you please try the attached patch?

Note that there may be some change to how "function" keys, like return
and the arrow keys, are handled when using the fn key. I can't
remember exactly how they worked with Emacs 26, but I believe the new
behaviour should be more consistent across the board.
-- 
Alan Third

[-- Attachment #2: 0001-Handle-fn-functions-keys-better-on-NS-bug-42915.patch --]
[-- Type: text/plain, Size: 3824 bytes --]

From 6460cec4412a0f8bc5a16b91743423bef6c09e14 Mon Sep 17 00:00:00 2001
From: Alan Third <alan@idiocy.org>
Date: Tue, 18 Aug 2020 23:41:08 +0100
Subject: [PATCH] Handle fn + functions keys better on NS (bug#42915)

* doc/emacs/macos.texi (Mac / GNUstep Customization): Document the
caveat regarding fn and function keys.
* src/nsterm.m (EV_MODIFIERS2): Ignore ns_function_modifier if this is
a function key.
([EmacsView keyDown:]): Get rid of the code that unsets the
ns_function_modifier value in modifiers.
---
 doc/emacs/macos.texi |  4 ++++
 src/nsterm.m         | 23 ++++++++++-------------
 2 files changed, 14 insertions(+), 13 deletions(-)

diff --git a/doc/emacs/macos.texi b/doc/emacs/macos.texi
index 00daa8b35d..3fb625b417 100644
--- a/doc/emacs/macos.texi
+++ b/doc/emacs/macos.texi
@@ -143,6 +143,10 @@ Mac / GNUstep Customization
 may also be set to @code{left}, which means to use the same behavior as
 the corresponding left-hand key.
 
+Emacs is unable to detect when the @key{function} key is being used as
+a modifier with function keys. As a result Emacs will ignore the
+modifier set in @code{ns-function-modifier} with these keys.
+
 @subsection Frame Variables
 
 @table @code
diff --git a/src/nsterm.m b/src/nsterm.m
index 98c5b69d68..1c306c1b8e 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -402,7 +402,7 @@ - (NSColor *)colorUsingDefaultColorSpace
     hyper_modifier : 0)                                                 \
    | ((flags & NSEventModifierFlagShift) ?                              \
       shift_modifier : 0)                                               \
-   | ((flags & NS_FUNCTION_KEY_MASK)                                    \
+   | (((flags & NS_FUNCTION_KEY_MASK) && kind == QCordinary)            \
       ? parse_solitary_modifier (mod_of_kind (ns_function_modifier,     \
                                               kind))                    \
       : 0)                                                              \
@@ -6505,25 +6505,22 @@ untranslated characters (returned by the
          we have both shift-like and control-like modifiers because
          the NSEvent API doesn’t let us ignore only some modifiers.
          In that case we use UCKeyTranslate (ns_get_shifted_character)
-         to look up the correct character.  */
+         to look up the correct character.
 
-      /* EV_MODIFIERS2 uses parse_solitary_modifier on all known
+         EV_MODIFIERS2 uses parse_solitary_modifier on all known
          modifier keys, which returns 0 for shift-like modifiers.
          Therefore its return value is the set of control-like
-         modifiers.  */
-      Lisp_Object kind = fnKeysym ? QCfunction : QCordinary;
-      emacs_event->modifiers = EV_MODIFIERS2 (flags, kind);
+         modifiers.
 
-      /* Function keys (such as the F-keys, arrow keys, etc.) set
+         Function keys (such as the F-keys, arrow keys, etc.) set
          modifiers as though the fn key has been pressed when it
          hasn't.  Also some combinations of fn and a function key
          return a different key than was pressed (e.g. fn-<left> gives
-         <home>).  We need to unset the fn modifier in these cases.
-         FIXME: Can we avoid setting it in the first place?  */
-      if (fnKeysym && (flags & NS_FUNCTION_KEY_MASK))
-        emacs_event->modifiers
-          ^= parse_solitary_modifier (mod_of_kind (ns_function_modifier,
-                                                   QCfunction));
+         <home>).  EV_MODIFIERS2 avoids setting the the fn modifier in
+         these cases.  */
+
+      Lisp_Object kind = fnKeysym ? QCfunction : QCordinary;
+      emacs_event->modifiers = EV_MODIFIERS2 (flags, kind);
 
       if (NS_KEYLOG)
         fprintf (stderr, "keyDown: code =%x\tfnKey =%x\tflags = %x\tmods = %x\n",
-- 
2.26.1