all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [PATCH] Add user content APIs for WebKit Xwidgets
@ 2020-08-28  2:25 Qiantan Hong
  2020-08-28 14:37 ` Lars Ingebrigtsen
  2020-08-29  4:10 ` Richard Stallman
  0 siblings, 2 replies; 30+ messages in thread
From: Qiantan Hong @ 2020-08-28  2:25 UTC (permalink / raw)
  To: emacs-devel@gnu.org


[-- Attachment #1.1: Type: text/plain, Size: 604 bytes --]

Hi,

I implemented some primitives to expose some WebKit user content APIs
(user script and script message handlers) for WebKit Xwidgets, both
for WebKitGTK impl and NS/mac impl.

The user script API makes it possible to reliable and predictable injecting
script into a Webkit Xwidget, which is useful for customizing the WebView
behavior.

The script message handler API makes it possible to trigger event in emacs
from JavaScript, and can be used to implement procedure calling from
js to elisp. Currently only the other way around is possible.

The patch is attached.

Best,
Qiantan

qhong@mit.edu




[-- Attachment #1.2.1: Type: text/html, Size: 1114 bytes --]

[-- Attachment #1.2.2: 0002-Implment-some-user-content-APIs-for-WebKit-Xwidgets.txt --]
[-- Type: text/plain, Size: 18294 bytes --]

From cac4d244ef78e2bd77c758c4f13a501b07b28e33 Mon Sep 17 00:00:00 2001
From: Qiantan Hong <qhong@mit.edu>
Date: Thu, 27 Aug 2020 17:02:18 -0400
Subject: [PATCH 2/2] Implment some user content APIs for WebKit Xwidgets

Implement WebKit user scripts and script message handlers.
* src/xwidget.h (store_xwidget_script_message_event): store script
  message event into event queue
* src/xwidget.c (store_xwidget_script_message_event, make-xwidget,
  webkit_script_message_cb, xwidget-webkit-add-user-script,
  xwidget-webkit-remove-all-user-scripts,
  xwidget-webkit-register-script-message,
  xwidget-webkit-unregister-script-message): Implement user script
  and script message handler primitives.
* src/nsxwidget.c (nsxwidget_webkit_add_user_script,
  nsxwidget_webkit_remove_all_user_scripts,
  nsxwidget_webkit_register_script_message,
  nsxwidget_webkit_unregister_script_message,
  initWithFrame, initialize, userContentController): NS
  implementation. Changed naming of a previous used  script message
  handler to avoid namespace pollution.
* src/nsxwidget.h (nsxwidget_webkit_add_user_script,
  nsxwidget_webkit_remove_all_user_scripts,
  nsxwidget_webkit_register_script_message,
  nsxwidget_webkit_unregister_script_message): NS implementation
* lisp/xwidget.el (xwidget-webkit-callback,
  xwidget-webkit-add-script-message-handler,
  xwidget-webkit-remove-script-message-handler):
  let lisp recognize and dispatch script message events
---
 lisp/xwidget.el |  22 ++++++
 src/nsxwidget.h |   5 ++
 src/nsxwidget.m |  80 +++++++++++++++++++--
 src/xwidget.c   | 181 +++++++++++++++++++++++++++++++++++++++++++++++-
 src/xwidget.h   |   5 ++
 5 files changed, 285 insertions(+), 8 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 074320855c..0c202e5bc5 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -298,8 +298,30 @@ xwidget-webkit-callback
              (let ((proc (nth 3 last-input-event))
                    (arg  (nth 4 last-input-event)))
                (funcall proc arg)))
+            ((eq xwidget-event-type 'script-message)
+             (let ((name (nth 3 last-input-event))
+                   (value (nth 4 last-input-event)))
+               (let ((handler-pair (assq name (xwidget-get xwidget 'script-message-handlers))))
+                 (if handler-pair
+                     (funcall (cdr handler-pair) xwidget value)
+                   (xwidget-log "unhandled script message:%s" name)))))
             (t (xwidget-log "unhandled event:%s" xwidget-event-type))))))
 
+(defun xwidget-webkit-add-script-message-handler (xwidget name handler)
+  "Associate HANDLER with script messages of NAME for Webkit XWIDGET."
+  (xwidget-put xwidget 'script-message-handlers
+               (cons (cons name handler) (xwidget-get xwidget 'script-message-handlers))))
+
+(defun xwidget-webkit-remove-script-message-handler (xwidget name)
+  "Remove a handler associated with NAME for Webkit XWIDGET.
+Returns the removed (NAME . HANDLER) pair, or NIL if such handler is not found."
+  (let* ((old-alist (xwidget-get xwidget 'script-message-handlers))
+         (handler-pair (assq name old-alist)))
+    (when handler-pair
+      (xwidget-put xwidget 'script-message-handlers
+                   (delq handler-pair old-alist))
+      handler-pair)))
+
 (defvar bookmark-make-record-function)
 (when (memq window-system '(mac ns))
   (defvar xwidget-webkit-enable-plugins nil
diff --git a/src/nsxwidget.h b/src/nsxwidget.h
index 3d91594c34..f1dc53019a 100644
--- a/src/nsxwidget.h
+++ b/src/nsxwidget.h
@@ -40,6 +40,11 @@ #define NSXWIDGET_H_INCLUDED
 void nsxwidget_webkit_execute_script (struct xwidget *xw, const char *script,
                                       Lisp_Object fun);
 
+void nsxwidget_webkit_add_user_script (struct xwidget *xw, const char *script,
+                                       int injection_time_start, int main_frame_only);
+void nsxwidget_webkit_remove_all_user_scripts (struct xwidget *xw);
+Lisp_Object nsxwidget_webkit_register_script_message (struct xwidget *xw, const char *name);
+void nsxwidget_webkit_unregister_script_message (struct xwidget *xw, const char *name);
 /* Functions for xwidget model.  */
 
 #ifdef __OBJC__
diff --git a/src/nsxwidget.m b/src/nsxwidget.m
index e81ca7fc0c..6c9fb497e4 100644
--- a/src/nsxwidget.m
+++ b/src/nsxwidget.m
@@ -85,7 +85,7 @@ - (id)initWithFrame:(CGRect)frame
         @"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6)"
         @" AppleWebKit/603.3.8 (KHTML, like Gecko)"
         @" Version/11.0.1 Safari/603.3.8";
-      [scriptor addScriptMessageHandler:self name:@"keyDown"];
+      [scriptor addScriptMessageHandler:self name:@"__xwidget_internal_keyDown"];
       [scriptor addUserScript:[[WKUserScript alloc]
                                 initWithSource:xwScript
                                  injectionTime:
@@ -272,23 +272,34 @@ + (void)initialize
       @"}"
       @"function xwKeyDown(event) {"
       @"  if (event.ctrlKey && event.key == 'g') {"
-      @"    window.webkit.messageHandlers.keyDown.postMessage('C-g');"
+      @"    window.webkit.messageHandlers.__xwidget_internal_keyDown.postMessage('C-g');"
       @"  }"
       @"}"
       @"document.addEventListener('keydown', xwKeyDown);"
       ;
 }
 
+static Lisp_Object js_to_lisp (id value);
+
 /* Confirming to WKScriptMessageHandler, listens concerning keyDown in
    webkit. Currently 'C-g'.  */
 - (void)userContentController:(WKUserContentController *)userContentController
       didReceiveScriptMessage:(WKScriptMessage *)message
 {
-  if ([message.body isEqualToString:@"C-g"])
+  if ([message.name isEqualToString:@"__xwidget_internal_keyDown"])
     {
-      /* Just give up focus, no relay "C-g" to emacs, another "C-g"
-         follows will be handled by emacs.  */
-      [self.window makeFirstResponder:self.xw->xv->emacswindow];
+      if ([message.body isEqualToString:@"C-g"])
+        {
+          /* Just give up focus, no relay "C-g" to emacs, another "C-g"
+             follows will be handled by emacs.  */
+          [self.window makeFirstResponder:self.xw->xv->emacswindow];
+        }
+    }
+  else
+    {
+      store_xwidget_script_message_event (self.xw,
+                                          message.name.UTF8String,
+                                          js_to_lisp (message.body));
     }
 }
 
@@ -445,6 +456,61 @@ - (void)userContentController:(WKUserContentController *)userContentController
     }];
 }
 
+void
+nsxwidget_webkit_add_user_script (struct xwidget *xw, const char *script,
+                                  int injection_time_start, int main_frame_only)
+{
+  XwWebView *xwWebView = (XwWebView *) xw->xwWidget;
+  WKUserContentController *scriptor = xwWebView.configuration.userContentController;
+
+  NSString *javascriptString = [NSString stringWithUTF8String:script];
+  WKUserScriptInjectionTime injectionTime = injection_time_start?
+    WKUserScriptInjectionTimeAtDocumentStart : WKUserScriptInjectionTimeAtDocumentEnd;
+  WKUserScript *userScript = [[WKUserScript alloc]
+                               initWithSource: javascriptString
+                                injectionTime: injectionTime
+                               forMainFrameOnly: main_frame_only];
+  [scriptor addUserScript: userScript];
+}
+
+void
+nsxwidget_webkit_remove_all_user_scripts (struct xwidget *xw)
+{
+  XwWebView *xwWebView = (XwWebView *) xw->xwWidget;
+  WKUserContentController *scriptor = xwWebView.configuration.userContentController;
+
+  [scriptor removeAllUserScripts];
+}
+
+Lisp_Object
+nsxwidget_webkit_register_script_message (struct xwidget *xw, const char *name)
+{
+  XwWebView *xwWebView = (XwWebView *) xw->xwWidget;
+  WKUserContentController *scriptor = xwWebView.configuration.userContentController;
+
+  NSString *messageName = [NSString stringWithUTF8String:name];
+
+  @try
+    {
+      [scriptor addScriptMessageHandler:xw->xwWidget name:messageName];
+    }
+  @catch (NSException *e)
+    {
+      return Qnil;
+    }
+  return Qt;
+}
+
+void
+nsxwidget_webkit_unregister_script_message (struct xwidget *xw, const char *name)
+{
+  XwWebView *xwWebView = (XwWebView *) xw->xwWidget;
+  WKUserContentController *scriptor = xwWebView.configuration.userContentController;
+
+  NSString *messageName = [NSString stringWithUTF8String:name];
+  [scriptor removeScriptMessageHandlerForName:messageName];
+}
+
 /* Window containing an xwidget.  */
 
 @implementation XwWindow
@@ -477,7 +543,7 @@ - (BOOL)isFlipped { return YES; }
       WKUserContentController *scriptor =
         ((XwWebView *) xw->xwWidget).configuration.userContentController;
       [scriptor removeAllUserScripts];
-      [scriptor removeScriptMessageHandlerForName:@"keyDown"];
+      [scriptor removeScriptMessageHandlerForName:@"__xwidget_internal_keyDown"];
       [scriptor release];
       if (xw->xv)
         xw->xv->model = Qnil; /* Make sure related view stale.  */
diff --git a/src/xwidget.c b/src/xwidget.c
index 154b3e9c82..d3524acd68 100644
--- a/src/xwidget.c
+++ b/src/xwidget.c
@@ -70,6 +70,15 @@ webkit_decide_policy_cb (WebKitWebView *,
                          WebKitPolicyDecision *,
                          WebKitPolicyDecisionType,
                          gpointer);
+
+struct webkit_script_message_cb_data
+{
+  struct xwidget *xw;
+  char name[0];
+};
+static void webkit_script_message_cb (WebKitUserContentManager *,
+                                          WebKitJavascriptResult *,
+                                          gpointer);
 #endif
 
 
@@ -120,7 +129,8 @@ DEFUN ("make-xwidget",
 
       if (EQ (xw->type, Qwebkit))
         {
-          xw->widget_osr = webkit_web_view_new ();
+          WebKitUserContentManager *scriptor = webkit_user_content_manager_new ();
+          xw->widget_osr = webkit_web_view_new_with_user_content_manager (scriptor);
         }
 
       gtk_widget_set_size_request (GTK_WIDGET (xw->widget_osr), xw->width,
@@ -293,6 +303,21 @@ store_xwidget_js_callback_event (struct xwidget *xw,
   kbd_buffer_store_event (&event);
 }
 
+void
+store_xwidget_script_message_event (struct xwidget *xw,
+                                         const char *name,
+                                         Lisp_Object body)
+{
+  struct input_event event;
+  Lisp_Object xwl;
+  XSETXWIDGET (xwl, xw);
+  EVENT_INIT (event);
+  event.kind = XWIDGET_EVENT;
+  event.frame_or_window = Qnil;
+  event.arg = list4 (intern ("script-message"), xwl, intern (name), body);
+  kbd_buffer_store_event (&event);
+}
+
 
 #ifdef USE_GTK
 void
@@ -481,6 +506,18 @@ webkit_decide_policy_cb (WebKitWebView *webView,
   }
 }
 
+static void webkit_script_message_cb (WebKitUserContentManager *scriptor,
+                                          WebKitJavascriptResult *js_result,
+                                          gpointer data)
+{
+  JSCValue *value = webkit_javascript_result_get_js_value (js_result);
+  struct webkit_script_message_cb_data *arg = data;
+
+  Lisp_Object lisp_value = webkit_js_to_lisp (value);
+  store_xwidget_script_message_event (arg->xw, arg->name, lisp_value);
+}
+
+
 
 /* For gtk3 offscreen rendered widgets.  */
 static gboolean
@@ -922,6 +959,140 @@ DEFUN ("xwidget-webkit-execute-script",
   return Qnil;
 }
 
+DEFUN ("xwidget-webkit-add-user-script",
+       Fxwidget_webkit_add_user_script, Sxwidget_webkit_add_user_script,
+       4, 4, 0,
+       doc: /* Add user SCRIPT to the Webkit XWIDGET.
+INJECTION-TIME is a symbol which can take one of the following values:
+
+- start: SCRIPT is injected when document starts loading
+- end: SCRIPT is injected when document finishes loading
+
+If MAIN_FRAME_ONLY is nil, SCRIPT is injected to all frames.
+Otherwise, SCRIPT is only injected to top frames.*/)
+  (Lisp_Object xwidget, Lisp_Object script,
+   Lisp_Object injection_time, Lisp_Object main_frame_only)
+{
+  WEBKIT_FN_INIT ();
+  CHECK_STRING (script);
+  CHECK_SYMBOL (injection_time);
+
+  script = ENCODE_SYSTEM(script);
+
+  int injection_time_start, mfo;
+  mfo = !NILP (main_frame_only);
+  if (EQ (injection_time, Qstart))
+    injection_time_start = 1;
+  else if (EQ (injection_time, Qend))
+    injection_time_start = 0;
+  else
+    error ("Unknown Xwidget Webkit user script injection time: %s",
+           SDATA (SYMBOL_NAME (injection_time)));
+
+#ifdef USE_GTK
+  WebKitWebView *wkwv = WEBKIT_WEB_VIEW (xw->widget_osr);
+  WebKitUserContentManager *scriptor = webkit_web_view_get_user_content_manager (wkwv);
+
+  int webkit_injected_frames = mfo?
+    WEBKIT_USER_CONTENT_INJECT_TOP_FRAME : WEBKIT_USER_CONTENT_INJECT_ALL_FRAMES;
+  int webkit_injection_time = injection_time_start?
+    WEBKIT_USER_SCRIPT_INJECT_AT_DOCUMENT_START : WEBKIT_USER_SCRIPT_INJECT_AT_DOCUMENT_END;
+  WebKitUserScript *userScript = webkit_user_script_new (SSDATA (script),
+                                                         webkit_injected_frames,
+                                                         webkit_injection_time,
+                                                         NULL, NULL);
+  webkit_user_content_manager_add_script (scriptor, userScript);
+  webkit_user_script_unref (userScript);
+#elif defined NS_IMPL_COCOA
+  nsxwidget_webkit_add_user_script (xw, SSDATA (script), injection_time_start, mfo);
+#endif
+  return Qnil;
+}
+
+DEFUN ("xwidget-webkit-remove-all-user-scripts",
+       Fxwidget_webkit_remove_all_user_scripts, Sxwidget_webkit_remove_all_user_scripts,
+       1, 1, 0,
+       doc: /* Remove all user scripts from XWIDGET.  */)
+  (Lisp_Object xwidget)
+{
+  WEBKIT_FN_INIT ();
+
+#ifdef USE_GTK
+  WebKitWebView *wkwv = WEBKIT_WEB_VIEW (xw->widget_osr);
+  WebKitUserContentManager *scriptor = webkit_web_view_get_user_content_manager (wkwv);
+
+  webkit_user_content_manager_remove_all_scripts (scriptor);
+#elif defined NS_IMPL_COCOA
+  nsxwidget_webkit_remove_all_user_scripts(xw);
+#endif
+  return Qnil;
+}
+
+DEFUN ("xwidget-webkit-register-script-message",
+       Fxwidget_webkit_register_script_message, Sxwidget_webkit_register_script_message,
+       2, 2, 0,
+       doc: /* Register script message with symbol NAME in Webkit XWIDGET.
+Returns T if the operation is successful, NIL otherwise.
+The cause of failure is usually that NAME has already been registered for XWIDGET.  */)
+  (Lisp_Object xwidget, Lisp_Object name)
+{
+  WEBKIT_FN_INIT ();
+  CHECK_SYMBOL (name);
+  const char *sname = SDATA( SYMBOL_NAME (name));
+
+#ifdef USE_GTK
+  WebKitWebView *wkwv = WEBKIT_WEB_VIEW (xw->widget_osr);
+  WebKitUserContentManager *scriptor = webkit_web_view_get_user_content_manager (wkwv);
+
+  gchar *signal_name = g_strconcat ("script-message-received::", sname, NULL);
+  size_t name_length = strlen (sname) + 1;
+  struct webkit_script_message_cb_data *arg = malloc (sizeof *arg + name_length);
+  arg->xw = xw;
+  g_strlcpy (arg->name, sname, name_length);
+  g_signal_connect_data(scriptor, signal_name, G_CALLBACK (webkit_script_message_cb),
+                        arg, (GClosureNotify)free, 0);
+  g_free (signal_name);
+  if (webkit_user_content_manager_register_script_message_handler (scriptor, sname))
+    {
+      return Qt;
+    }
+  else
+    {
+      g_signal_handlers_disconnect_matched  (scriptor,
+                                             G_SIGNAL_MATCH_DATA,
+                                             0, 0, 0, 0, arg);
+      return Qnil;
+    }
+#elif defined NS_IMPL_COCOA
+  return nsxwidget_webkit_register_script_message(xw, sname);
+#endif
+}
+
+DEFUN ("xwidget-webkit-unregister-script-message",
+       Fxwidget_webkit_unregister_script_message, Sxwidget_webkit_unregister_script_message,
+       2, 2, 0,
+       doc: /* Unregister script message with symbol NAME in Webkit XWIDGET.  */)
+  (Lisp_Object xwidget, Lisp_Object name)
+{
+  WEBKIT_FN_INIT ();
+  CHECK_SYMBOL (name);
+  const char *sname = SSDATA( SYMBOL_NAME (name));
+
+#ifdef USE_GTK
+  WebKitWebView *wkwv = WEBKIT_WEB_VIEW (xw->widget_osr);
+  WebKitUserContentManager *scriptor = webkit_web_view_get_user_content_manager (wkwv);
+
+  webkit_user_content_manager_unregister_script_message_handler (scriptor, sname);
+  g_signal_handlers_disconnect_matched  (scriptor,
+                                         G_SIGNAL_MATCH_FUNC,
+                                         0, g_quark_from_string (sname), 0,
+                                         G_CALLBACK (webkit_script_message_cb), 0);
+#elif defined NS_IMPL_COCOA
+  nsxwidget_webkit_unregister_script_message(xw, sname);
+#endif
+  return Qnil;
+}
+
 DEFUN ("xwidget-resize", Fxwidget_resize, Sxwidget_resize, 3, 3, 0,
        doc: /* Resize XWIDGET to NEW_WIDTH, NEW_HEIGHT.  */ )
   (Lisp_Object xwidget, Lisp_Object new_width, Lisp_Object new_height)
@@ -1189,6 +1360,14 @@ syms_of_xwidget (void)
   defsubr (&Sxwidget_webkit_execute_script);
   DEFSYM (Qwebkit, "webkit");
 
+  defsubr (&Sxwidget_webkit_add_user_script);
+  DEFSYM (Qstart, "start");
+  DEFSYM (Qend, "end");
+  defsubr (&Sxwidget_webkit_remove_all_user_scripts);
+
+  defsubr (&Sxwidget_webkit_register_script_message);
+  defsubr (&Sxwidget_webkit_unregister_script_message);
+
   defsubr (&Sxwidget_size_request);
   defsubr (&Sdelete_xwidget_view);
 
diff --git a/src/xwidget.h b/src/xwidget.h
index 40ad8ae833..cfd0ebced6 100644
--- a/src/xwidget.h
+++ b/src/xwidget.h
@@ -162,6 +162,11 @@ #define XG_XWIDGET_VIEW "emacs_xwidget_view"
 void store_xwidget_js_callback_event (struct xwidget *xw,
                                       Lisp_Object proc,
                                       Lisp_Object argument);
+
+void store_xwidget_script_message_event (struct xwidget *xw,
+                                         const char *name,
+                                         Lisp_Object value);
+
 #else
 INLINE_HEADER_BEGIN
 INLINE void syms_of_xwidget (void) {}
-- 
2.20.1 (Apple Git-117)


[-- Attachment #1.2.3: Type: text/html, Size: 2497 bytes --]

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 1858 bytes --]

^ permalink raw reply related	[flat|nested] 30+ messages in thread
* Re: [PATCH] Add user content APIs for WebKit Xwidgets
@ 2022-10-14  6:34 Qiantan Hong
  2022-10-14  7:01 ` Po Lu
  0 siblings, 1 reply; 30+ messages in thread
From: Qiantan Hong @ 2022-10-14  6:34 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: emacs-devel@gnu.org

Hi,

Is there any chance to get this merged now?

Best,
Qiantan






^ permalink raw reply	[flat|nested] 30+ messages in thread

end of thread, other threads:[~2022-10-24  7:20 UTC | newest]

Thread overview: 30+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-08-28  2:25 [PATCH] Add user content APIs for WebKit Xwidgets Qiantan Hong
2020-08-28 14:37 ` Lars Ingebrigtsen
2020-08-28 15:41   ` Qiantan Hong
2020-08-30 13:43     ` Lars Ingebrigtsen
2020-08-29  4:07   ` Richard Stallman
2020-08-29  4:10 ` Richard Stallman
2020-08-29  4:45   ` Qiantan Hong
  -- strict thread matches above, loose matches on Subject: below --
2022-10-14  6:34 Qiantan Hong
2022-10-14  7:01 ` Po Lu
2022-10-14  7:12   ` Qiantan Hong
2022-10-14  7:35     ` Po Lu
2022-10-14 21:13       ` Qiantan Hong
2022-10-15  1:37         ` Qiantan Hong
2022-10-15  7:53           ` Qiantan Hong
2022-10-15 11:23             ` Po Lu
2022-10-15 18:29               ` Qiantan Hong
2022-10-16  0:26                 ` Po Lu
2022-10-15 23:33               ` Qiantan Hong
2022-10-16  4:32                 ` Po Lu
2022-10-16  6:29                   ` Qiantan Hong
2022-10-16  6:41                     ` Po Lu
2022-10-16  6:45                       ` Po Lu
2022-10-23  9:11                       ` Qiantan Hong
2022-10-23 10:58                         ` Po Lu
2022-10-23 22:16                           ` Qiantan Hong
2022-10-24  0:30                             ` Po Lu
2022-10-24  4:17                               ` Qiantan Hong
2022-10-24  5:38                                 ` Po Lu
2022-10-24  5:44                                   ` Qiantan Hong
2022-10-24  7:20                                     ` Po Lu

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.