Touchscreen support

[Po Lu via "Emacs development discussions." <emacs-devel@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 369 bytes --]


I implemented the events required for full touchscreen support for
XInput 2.  These events are generated only for DirectTouch devices,
which means real touchscreens only, no touchpads.

Please take a look and see if the API I'm exposing is useful, and also
easy enough to implement on non-X platforms.

If there are no problems, I would like to install this.

Thanks.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-support-for-touchscreen-events-generated-by-XInp.patch --]
[-- Type: text/x-patch, Size: 17841 bytes --]

From f9df927f3f70c348ceee50eb88c1ddad7eff2c4f Mon Sep 17 00:00:00 2001
From: Po Lu <luangruo@yahoo.com>
Date: Thu, 16 Dec 2021 15:15:28 +0800
Subject: [PATCH] Add support for touchscreen events generated by XInput 2

* doc/lispref/commands.texi (Input Events): Add "Touchscreen
Events" to menu.
(Touchscreen Events): New node.
* etc/NEWS: Announce new event types.

* src/keyboard.c (make_lispy_event): Handle touchscreen events.
(syms_of_keyboard): New symbols `touchscreen-begin',
`touchscreen-end' and `touchscreen-update'.
* src/termhooks.h (enum event_kind): New touchscreen events.

* src/xfns.c (setup_xi_event_mask): Set up touch event mask if
appropriate.
(x_window) [USE_GTK]: Call `setup_xi_event_mask'.

* src/xterm.c (x_free_xi_devices): Free touchpoints.
(x_init_master_valuators): Handle all devices and set master and
touch mode flags appropriately.
(xi_reset_scroll_valuators_for_device_id):
(x_get_scroll_valuator_delta): Ignore slave devices.

(xi_link_touch_point):
(xi_unlink_touch_point):
(xi_find_touch_point): New functions.
(handle_one_xevent): Handle XI touch events.

* src/xterm.h (struct xi_touch_point_t): New structure.
(struct xi_device_t): Add touchpoints field.
---
 doc/lispref/commands.texi |  53 ++++++++++
 etc/NEWS                  |   6 ++
 src/keyboard.c            |  43 ++++++++
 src/termhooks.h           |  13 +++
 src/xfns.c                |  21 +++-
 src/xterm.c               | 210 +++++++++++++++++++++++++++++++++++---
 src/xterm.h               |  11 ++
 7 files changed, 344 insertions(+), 13 deletions(-)

diff --git a/doc/lispref/commands.texi b/doc/lispref/commands.texi
index cc1c216d57..77a5385241 100644
--- a/doc/lispref/commands.texi
+++ b/doc/lispref/commands.texi
@@ -1175,6 +1175,7 @@ Input Events
 * Button-Down Events::          A button was pushed and not yet released.
 * Repeat Events::               Double and triple click (or drag, or down).
 * Motion Events::               Just moving the mouse, not pushing a button.
+* Touchscreen Events::          Tapping and moving fingers on a touchscreen.
 * Focus Events::                Moving the mouse between frames.
 * Xwidget Events::              Events generated by xwidgets.
 * Misc Events::                 Other events the system can generate.
@@ -1835,6 +1836,58 @@ Motion Events
 as the mouse cursor remains pointing to the same glyph in the text.
 @end defvar
 
+@node Touchscreen Events
+@subsection Touchscreen Events
+@cindex touchscreen events
+
+Some window systems provide support for input devices that react to
+the user's finger, and translate those finger movements into points at
+an on-screen position.  These input devices are known as touchscreens,
+and Emacs reports the movements they generate as @dfn{touchscreen
+events}.
+
+Most individual events generated by a touchscreen only have meaning as
+part of a larger sequence of other events: for instance, the simple
+operation of tapping the touchscreen involves the user placing and
+releasing a finger on the touchscreen, and swiping the display to
+scroll it involves placing a finger, moving it many times upwards or
+downwards, and then releasing the finger.
+
+While a simplistic model consisting of one finger is adequate for taps
+and scrolling, most applications support more complicated gestures
+that involve multiple fingers, where the position of each finger is
+represented by a @dfn{touch point}.  For instance, a ``pinch to zoom''
+gesture might consist of the user placing two fingers and moving them
+individually in opposite directions, where the distance between the
+positions of their individual points determine the amount by which to
+zoom the display, and the center of an imaginary line between those
+positions determines where to pan the display after zooming.
+
+@cindex touch point representation
+The low-level touchscreen events described below can be used to
+implement all the touch sequences described above.  In those events,
+each point is represented by a cons of an arbitrary number identifying
+the point and a mouse position list (@pxref{Click Events}) specifying
+the position of the finger when the event occurred.
+
+@table @code
+@cindex @code{touchscreen-begin} event
+@item (touchscreen-begin @var{point})
+This event is sent when @var{point} is created by the user pressing a
+finger against the touchscreen.
+
+@cindex @code{touchscreen-update} event
+@item (touchscreen-update @var{points})
+This event is sent when an point on the touchscreen has changed
+position.  @var{points} is a list of points containing the up-to-date
+positions of each point currently on the touchscreen.
+
+@cindex @code{touchscreen-end} event
+@item (touchscreen-end @var{point-id})
+This event is sent when the point identified by the number
+@var{point-id} is no longer present on the display.
+@end table
+
 @node Focus Events
 @subsection Focus Events
 @cindex focus event
diff --git a/etc/NEWS b/etc/NEWS
index 1d78f1f5c3..6203537475 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1156,6 +1156,12 @@ This variable is bound to t during the preparation of a "*Help*" buffer.
 month, day, or time.  For example, (date-to-time "2021-12-04") now
 assumes a time of 00:00 instead of signaling an error.
 
++++
+** New events for taking advantages of touchscreen devices.
+The events 'touchscreen-down', 'touchscreen-update' and
+'touchscreen-end' have been added to take better advantage of
+touch-capable display panels.
+
 \f
 * Changes in Emacs 29.1 on Non-Free Operating Systems
 
diff --git a/src/keyboard.c b/src/keyboard.c
index 899c9109c2..ee4b18b012 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -6010,6 +6010,46 @@ make_lispy_event (struct input_event *event)
 	return list2 (Qtouch_end, position);
       }
 
+    case TOUCHSCREEN_BEGIN_EVENT:
+    case TOUCHSCREEN_END_EVENT:
+      {
+	Lisp_Object x, y, id, position;
+	struct frame *f = XFRAME (event->frame_or_window);
+
+	id = event->arg;
+	x = event->x;
+	y = event->y;
+
+	position = make_lispy_position (f, x, y, event->timestamp);
+
+	return list2 (((event->kind
+			== TOUCHSCREEN_BEGIN_EVENT)
+		       ? Qtouchscreen_begin
+		       : Qtouchscreen_end),
+		      Fcons (id, position));
+      }
+
+    case TOUCHSCREEN_UPDATE_EVENT:
+      {
+	Lisp_Object x, y, id, position, tem, it, evt;
+	struct frame *f = XFRAME (event->frame_or_window);
+	evt = Qnil;
+
+	for (tem = event->arg; CONSP (tem); tem = XCDR (tem))
+	  {
+	    it = XCAR (tem);
+
+	    x = XCAR (it);
+	    y = XCAR (XCDR (it));
+	    id = XCAR (XCDR (XCDR (it)));
+
+	    position = make_lispy_position (f, x, y, event->timestamp);
+	    evt = Fcons (Fcons (id, position), evt);
+	  }
+
+	return list2 (Qtouchscreen_update, evt);
+      }
+
 #ifdef USE_TOOLKIT_SCROLL_BARS
 
       /* We don't have down and up events if using toolkit scroll bars,
@@ -12257,6 +12297,9 @@ syms_of_keyboard (void)
 	       doc: /* Normal hook run when clearing the echo area.  */);
 #endif
   DEFSYM (Qecho_area_clear_hook, "echo-area-clear-hook");
+  DEFSYM (Qtouchscreen_begin, "touchscreen-begin");
+  DEFSYM (Qtouchscreen_end, "touchscreen-end");
+  DEFSYM (Qtouchscreen_update, "touchscreen-update");
   Fset (Qecho_area_clear_hook, Qnil);
 
   DEFVAR_LISP ("lucid-menu-bar-dirty-flag", Vlucid_menu_bar_dirty_flag,
diff --git a/src/termhooks.h b/src/termhooks.h
index f64c19e039..587d1eef91 100644
--- a/src/termhooks.h
+++ b/src/termhooks.h
@@ -274,6 +274,19 @@ #define EMACS_TERMHOOKS_H
      In the future, this may take into account other multi-touch
      events generated from touchscreens and such.  */
   , TOUCH_END_EVENT
+
+  /* In a TOUCHSCREEN_UPDATE_EVENT, ARG is a list of elements of the
+     form (X Y ID), where X and Y are the coordinates of the
+     touchpoint relative to the top-left corner of the frame, and ID
+     is a unique number identifying the touchpoint.
+
+     In TOUCHSCREEN_BEGIN_EVENT and TOUCHSCREEN_END_EVENT, ARG is the
+     unique ID of the touchpoint, and X and Y are the frame-relative
+     positions of the touchpoint.  */
+
+  , TOUCHSCREEN_UPDATE_EVENT
+  , TOUCHSCREEN_BEGIN_EVENT
+  , TOUCHSCREEN_END_EVENT
 };
 
 /* Bit width of an enum event_kind tag at the start of structs and unions.  */
diff --git a/src/xfns.c b/src/xfns.c
index b5694829ae..dc25d7bfca 100644
--- a/src/xfns.c
+++ b/src/xfns.c
@@ -2916,7 +2916,7 @@ initial_set_up_x_back_buffer (struct frame *f)
   unblock_input ();
 }
 
-#if defined HAVE_XINPUT2 && !defined USE_GTK
+#if defined HAVE_XINPUT2
 static void
 setup_xi_event_mask (struct frame *f)
 {
@@ -2927,6 +2927,9 @@ setup_xi_event_mask (struct frame *f)
   mask.mask = m = alloca (l);
   memset (m, 0, l);
   mask.mask_len = l;
+
+  block_input ();
+#ifndef USE_GTK
   mask.deviceid = XIAllMasterDevices;
 
   XISetMask (m, XI_ButtonPress);
@@ -2945,14 +2948,25 @@ setup_xi_event_mask (struct frame *f)
 		  &mask, 1);
 
   memset (m, 0, l);
+#endif /* !USE_GTK */
+
   mask.deviceid = XIAllDevices;
 
   XISetMask (m, XI_PropertyEvent);
   XISetMask (m, XI_HierarchyChanged);
   XISetMask (m, XI_DeviceChanged);
+#ifdef XI_TouchBegin
+  if (FRAME_DISPLAY_INFO (f)->xi2_version >= 2)
+    {
+      XISetMask (m, XI_TouchBegin);
+      XISetMask (m, XI_TouchUpdate);
+      XISetMask (m, XI_TouchEnd);
+    }
+#endif
   XISelectEvents (FRAME_X_DISPLAY (f),
 		  FRAME_X_WINDOW (f),
 		  &mask, 1);
+  unblock_input ();
 }
 #endif
 
@@ -3249,6 +3263,11 @@ x_window (struct frame *f)
     unblock_input ();
   }
 #endif
+
+#ifdef HAVE_XINPUT2
+  if (FRAME_DISPLAY_INFO (f)->supports_xi2)
+    setup_xi_event_mask (f);
+#endif
 }
 
 #else /*! USE_GTK */
diff --git a/src/xterm.c b/src/xterm.c
index 646985bdc2..c654fe0ce0 100644
--- a/src/xterm.c
+++ b/src/xterm.c
@@ -353,6 +353,8 @@ x_extension_initialize (struct x_display_info *dpyinfo)
 static void
 x_free_xi_devices (struct x_display_info *dpyinfo)
 {
+  struct xi_touch_point_t *tem, *last;
+
   block_input ();
 
   if (dpyinfo->num_devices)
@@ -362,6 +364,14 @@ x_free_xi_devices (struct x_display_info *dpyinfo)
 	  XIUngrabDevice (dpyinfo->display, dpyinfo->devices[i].device_id,
 			  CurrentTime);
 	  xfree (dpyinfo->devices[i].valuators);
+
+	  tem = dpyinfo->devices[i].touchpoints;
+	  while (tem)
+	    {
+	      last = tem;
+	      tem = tem->next;
+	      xfree (last);
+	    }
 	}
 
       xfree (dpyinfo->devices);
@@ -407,7 +417,7 @@ x_init_master_valuators (struct x_display_info *dpyinfo)
   block_input ();
   x_free_xi_devices (dpyinfo);
   infos = XIQueryDevice (dpyinfo->display,
-			 XIAllMasterDevices,
+			 XIAllDevices,
 			 &ndevices);
 
   if (!ndevices)
@@ -432,6 +442,10 @@ x_init_master_valuators (struct x_display_info *dpyinfo)
 	  xi_device->grab = 0;
 	  xi_device->valuators =
 	    xmalloc (sizeof *xi_device->valuators * device->num_classes);
+	  xi_device->touchpoints = NULL;
+	  xi_device->master_p = (device->use == XIMasterKeyboard
+				 || device->use == XIMasterPointer);
+	  xi_device->direct_p = false;
 
 	  for (int c = 0; c < device->num_classes; ++c)
 	    {
@@ -442,22 +456,36 @@ x_init_master_valuators (struct x_display_info *dpyinfo)
 		  {
 		    XIScrollClassInfo *info =
 		      (XIScrollClassInfo *) device->classes[c];
-		    struct xi_scroll_valuator_t *valuator =
-		      &xi_device->valuators[actual_valuator_count++];
-
-		    valuator->horizontal
-		      = (info->scroll_type == XIScrollTypeHorizontal);
-		    valuator->invalid_p = true;
-		    valuator->emacs_value = DBL_MIN;
-		    valuator->increment = info->increment;
-		    valuator->number = info->number;
+		    struct xi_scroll_valuator_t *valuator;
+
+		    if (xi_device->master_p)
+		      {
+			valuator = &xi_device->valuators[actual_valuator_count++];
+			valuator->horizontal
+			  = (info->scroll_type == XIScrollTypeHorizontal);
+			valuator->invalid_p = true;
+			valuator->emacs_value = DBL_MIN;
+			valuator->increment = info->increment;
+			valuator->number = info->number;
+		      }
+
 		    break;
 		  }
+#endif
+#ifdef XITouchClass /* XInput 2.2 */
+		case XITouchClass:
+		  {
+		    XITouchClassInfo *info;
+
+		    info = (XITouchClassInfo *) device->classes[c];
+		    xi_device->direct_p = info->mode == XIDirectTouch;
+		  }
 #endif
 		default:
 		  break;
 		}
 	    }
+
 	  xi_device->scroll_valuator_count = actual_valuator_count;
 	}
     }
@@ -484,7 +512,7 @@ x_get_scroll_valuator_delta (struct x_display_info *dpyinfo, int device_id,
     {
       struct xi_device_t *device = &dpyinfo->devices[i];
 
-      if (device->device_id == device_id)
+      if (device->device_id == device_id && device->master_p)
 	{
 	  for (int j = 0; j < device->scroll_valuator_count; ++j)
 	    {
@@ -534,6 +562,63 @@ xi_device_from_id (struct x_display_info *dpyinfo, int deviceid)
   return NULL;
 }
 
+#ifdef XI_TouchBegin
+
+static void
+xi_link_touch_point (struct xi_device_t *device,
+		     int detail, double x, double y)
+{
+  struct xi_touch_point_t *touchpoint;
+
+  touchpoint = xmalloc (sizeof *touchpoint);
+  touchpoint->next = device->touchpoints;
+  touchpoint->x = x;
+  touchpoint->y = y;
+  touchpoint->number = detail;
+
+  device->touchpoints = touchpoint;
+}
+
+static void
+xi_unlink_touch_point (int detail,
+		       struct xi_device_t *device)
+{
+  struct xi_touch_point_t *last, *tem;
+
+  for (last = NULL, tem = device->touchpoints; tem;
+       last = tem, tem = tem->next)
+    {
+      if (tem->number == detail)
+	{
+	  if (!last)
+	    device->touchpoints = tem->next;
+	  else
+	    last->next = tem->next;
+
+	  xfree (tem);
+	  return;
+	}
+    }
+
+  emacs_abort ();
+}
+
+static struct xi_touch_point_t *
+xi_find_touch_point (struct xi_device_t *device, int detail)
+{
+  struct xi_touch_point_t *point;
+
+  for (point = device->touchpoints; point; point = point->next)
+    {
+      if (point->number == detail)
+	return point;
+    }
+
+  return NULL;
+}
+
+#endif /* XI_TouchBegin */
+
 static void
 xi_grab_or_ungrab_device (struct xi_device_t *device,
 			  struct x_display_info *dpyinfo,
@@ -570,7 +655,7 @@ xi_reset_scroll_valuators_for_device_id (struct x_display_info *dpyinfo, int id)
   struct xi_device_t *device = xi_device_from_id (dpyinfo, id);
   struct xi_scroll_valuator_t *valuator;
 
-  if (!device)
+  if (!device || !device->master_p)
     return;
 
   if (!device->scroll_valuator_count)
@@ -10242,6 +10327,9 @@ handle_one_xevent (struct x_display_info *dpyinfo,
 
 	      device = xi_device_from_id (dpyinfo, xev->deviceid);
 
+	      if (!device || !device->master_p)
+		goto XI_OTHER;
+
 	      bv.button = xev->detail;
 	      bv.type = xev->evtype == XI_ButtonPress ? ButtonPress : ButtonRelease;
 	      bv.x = lrint (xev->event_x);
@@ -10765,6 +10853,104 @@ handle_one_xevent (struct x_display_info *dpyinfo,
 	  case XI_DeviceChanged:
 	    x_init_master_valuators (dpyinfo);
 	    goto XI_OTHER;
+#ifdef XI_TouchBegin
+	  case XI_TouchBegin:
+	    {
+	      struct xi_device_t *device;
+	      device = xi_device_from_id (dpyinfo, xev->deviceid);
+
+	      if (!device)
+		goto XI_OTHER;
+
+	      if (xi_find_touch_point (device, xev->detail))
+		emacs_abort ();
+
+	      xi_link_touch_point (device, xev->detail, xev->event_x,
+				   xev->event_y);
+	      f = x_any_window_to_frame (dpyinfo, xev->event);
+
+	      if (f && device->direct_p)
+		{
+		  inev.ie.kind = TOUCHSCREEN_BEGIN_EVENT;
+		  inev.ie.timestamp = xev->time;
+		  XSETFRAME (inev.ie.frame_or_window, f);
+		  XSETINT (inev.ie.x, lrint (xev->event_x));
+		  XSETINT (inev.ie.y, lrint (xev->event_y));
+		  XSETINT (inev.ie.arg, xev->detail);
+
+		  XIAllowTouchEvents (dpyinfo->display, xev->deviceid,
+				      xev->detail, xev->event, XIAcceptTouch);
+		}
+
+	      goto XI_OTHER;
+	    }
+	  case XI_TouchUpdate:
+	    {
+	      struct xi_device_t *device;
+	      struct xi_touch_point_t *touchpoint;
+	      Lisp_Object arg = Qnil;
+
+	      device = xi_device_from_id (dpyinfo, xev->deviceid);
+
+	      if (!device)
+		goto XI_OTHER;
+
+	      touchpoint = xi_find_touch_point (device, xev->detail);
+
+	      if (!touchpoint)
+		emacs_abort ();
+
+	      touchpoint->x = xev->event_x;
+	      touchpoint->y = xev->event_y;
+
+	      f = x_any_window_to_frame (dpyinfo, xev->event);
+
+	      if (f && device->direct_p)
+		{
+		  inev.ie.kind = TOUCHSCREEN_UPDATE_EVENT;
+		  inev.ie.timestamp = xev->time;
+		  XSETFRAME (inev.ie.frame_or_window, f);
+
+		  for (touchpoint = device->touchpoints;
+		       touchpoint; touchpoint = touchpoint->next)
+		    {
+		      arg = Fcons (list3i (lrint (touchpoint->x),
+					   lrint (touchpoint->y),
+					   lrint (touchpoint->number)),
+				   arg);
+		    }
+
+		  inev.ie.arg = arg;
+		}
+
+	      goto XI_OTHER;
+	    }
+	  case XI_TouchEnd:
+	    {
+	      struct xi_device_t *device;
+
+	      device = xi_device_from_id (dpyinfo, xev->deviceid);
+
+	      if (!device)
+		goto XI_OTHER;
+
+	      xi_unlink_touch_point (xev->detail, device);
+
+	      f = x_any_window_to_frame (dpyinfo, xev->event);
+
+	      if (f && device->direct_p)
+		{
+		  inev.ie.kind = TOUCHSCREEN_END_EVENT;
+		  inev.ie.timestamp = xev->time;
+		  XSETFRAME (inev.ie.frame_or_window, f);
+		  XSETINT (inev.ie.x, lrint (xev->event_x));
+		  XSETINT (inev.ie.y, lrint (xev->event_y));
+		  XSETINT (inev.ie.arg, xev->detail);
+		}
+
+	      goto XI_OTHER;
+	    }
+#endif
 	  default:
 	    goto XI_OTHER;
 	  }
diff --git a/src/xterm.h b/src/xterm.h
index 7abe168bc6..d9ace002d5 100644
--- a/src/xterm.h
+++ b/src/xterm.h
@@ -179,13 +179,24 @@ #define MAX_CLIP_RECTS 2
   int horizontal;
 };
 
+struct xi_touch_point_t
+{
+  struct xi_touch_point_t *next;
+
+  int number;
+  double x, y;
+};
+
 struct xi_device_t
 {
   int device_id;
   int scroll_valuator_count;
   int grab;
+  bool master_p;
+  bool direct_p;
 
   struct xi_scroll_valuator_t *valuators;
+  struct xi_touch_point_t *touchpoints;
 };
 #endif
 
-- 
2.33.1