[PATCH v2 00/15] xwidget webkit improvements

[PATCH v2 00/15] xwidget webkit improvements

[Ricardo Wurmus]

Hello again, readers of emacs-devel!

I was informed that my copyright assignment for Emacs is now complete,
so I revisited my patches for the xwidget feature.  This replaces my
previous patch set with Message-Id: <20160921115018.29576-1-rekado@elephly.net>.

As before, the first patch in this series adjusts the code such that
the WebKit2 API is used, which allows users to build Emacs with the
latest version of WebKit(2)Gtk.  Due to process separation in
WebKit2Gtk, scrolling the WebKit widget is achieved using JavaScript,
and JavaScript evaluation now happens asynchronously.

The remaining patches add features such as automatic resizing of the
WebKit widget, new keybindings, and zoom.

~~ Ricardo


Ricardo Wurmus (15):
  xwidget: Use WebKit2 API
  xwidget: Pass JavaScript return value to optional callback procedure
  Remove scrolled window container around WebKit widget
  xwidget: Do not use `xwidget-execute-script-rv' to insert string
  xwidget: Get title via asynchronous JavaScript.
  xwidget: Simplify functions to scroll to elements
  xwidget: Add function to find element by CSS selector
  xwidget: Get selection with asynchronous JavaScript
  xwidget: Get URL asynchronously.
  xwidget: Remove title hack.
  Let initial WebKit view fill window
  Dynamically resize WebKit widget.
  Implement zoom for WebKit widget.
  xwidget: Bind "beginning-of-buffer" and "end-of-buffer"
  xwidget: Map "previous-line" and "next-line" to scroll

 configure.ac    |   4 +-
 lisp/xwidget.el | 311 ++++++++++++++++++++++-----------------
 src/xwidget.c   | 445 ++++++++++++++++++++++++++++++++------------------------
 src/xwidget.h   |   3 -
 4 files changed, 434 insertions(+), 329 deletions(-)

-- 
2.10.1

[PATCH 01/15] xwidget: Use WebKit2 API

[Ricardo Wurmus]

* configure.ac: Check for webkit2gtk-4.0.
* src/xwidget.c: Adjust to use WebKit2 API.
* lisp/xwidget.el (xwidget-webkit-callback): Adjust matches for
`xwidget-event-type'.
---
 configure.ac    |   4 +-
 lisp/xwidget.el |   5 +-
 src/xwidget.c   | 186 ++++++++++++++++++++++++--------------------------------
 3 files changed, 84 insertions(+), 111 deletions(-)

diff --git a/configure.ac b/configure.ac
index 3f06eff..e16384b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2630,8 +2630,8 @@ if test "$with_xwidgets" != "no"; then
   test "$USE_GTK_TOOLKIT" = "GTK3" && test "$window_system" != "none" ||
     AC_MSG_ERROR([xwidgets requested but gtk3 not used.])
 
-  WEBKIT_REQUIRED=1.4.0
-  WEBKIT_MODULES="webkitgtk-3.0 >= $WEBKIT_REQUIRED"
+  WEBKIT_REQUIRED=2.12
+  WEBKIT_MODULES="webkit2gtk-4.0 >= $WEBKIT_REQUIRED"
   EMACS_CHECK_MODULES([WEBKIT], [$WEBKIT_MODULES])
   HAVE_XWIDGETS=$HAVE_WEBKIT
   test $HAVE_XWIDGETS = yes ||
diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 7a0ca8b..1bae6bb 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -187,7 +187,7 @@ XWIDGET instance, XWIDGET-EVENT-TYPE depends on the originating xwidget."
        "error: callback called for xwidget with dead buffer")
     (with-current-buffer (xwidget-buffer xwidget)
       (let* ((strarg  (nth 3 last-input-event)))
-        (cond ((eq xwidget-event-type 'document-load-finished)
+        (cond ((eq xwidget-event-type 'load-changed)
                (xwidget-log "webkit finished loading: '%s'"
                             (xwidget-webkit-get-title xwidget))
                ;;TODO - check the native/internal scroll
@@ -196,8 +196,7 @@ XWIDGET instance, XWIDGET-EVENT-TYPE depends on the originating xwidget."
                (rename-buffer (format "*xwidget webkit: %s *"
                                       (xwidget-webkit-get-title xwidget)))
                (pop-to-buffer (current-buffer)))
-              ((eq xwidget-event-type
-                   'navigation-policy-decision-requested)
+              ((eq xwidget-event-type 'decide-policy)
                (if (string-match ".*#\\(.*\\)" strarg)
                    (xwidget-webkit-show-id-or-named-element
                     xwidget
diff --git a/src/xwidget.c b/src/xwidget.c
index f5f4da0..78349a8 100644
--- a/src/xwidget.c
+++ b/src/xwidget.c
@@ -27,10 +27,7 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
 #include "keyboard.h"
 #include "gtkutil.h"
 
-#include <webkit/webkitwebview.h>
-#include <webkit/webkitwebnavigationaction.h>
-#include <webkit/webkitdownload.h>
-#include <webkit/webkitwebpolicydecision.h>
+#include <webkit2/webkit2.h>
 
 static struct xwidget *
 allocate_xwidget (void)
@@ -50,34 +47,16 @@ allocate_xwidget_view (void)
 
 static struct xwidget_view *xwidget_view_lookup (struct xwidget *,
 						 struct window *);
-static void webkit_document_load_finished_cb (WebKitWebView *, WebKitWebFrame *,
-					      gpointer);
-static gboolean webkit_download_cb (WebKitWebView *, WebKitDownload *, gpointer);
+static void webkit_view_load_changed_cb (WebKitWebView *,
+                                         WebKitLoadEvent,
+                                         gpointer);
+static gboolean webkit_download_cb (WebKitWebContext *, WebKitDownload *, gpointer);
 
 static gboolean
-webkit_mime_type_policy_typedecision_requested_cb (WebKitWebView *,
-                                                   WebKitWebFrame *,
-                                                   WebKitNetworkRequest *,
-                                                   gchar *,
-                                                   WebKitWebPolicyDecision *,
-                                                   gpointer);
-
-static gboolean
-webkit_new_window_policy_decision_requested_cb (WebKitWebView *,
-                                                WebKitWebFrame *,
-                                                WebKitNetworkRequest *,
-                                                WebKitWebNavigationAction *,
-                                                WebKitWebPolicyDecision *,
-                                                gpointer);
-
-static gboolean
-webkit_navigation_policy_decision_requested_cb (WebKitWebView *,
-                                                WebKitWebFrame *,
-                                                WebKitNetworkRequest *,
-                                                WebKitWebNavigationAction *,
-                                                WebKitWebPolicyDecision *,
-                                                gpointer);
-
+webkit_decide_policy_cb (WebKitWebView *,
+                         WebKitPolicyDecision *,
+                         WebKitPolicyDecisionType,
+                         gpointer);
 
 
 DEFUN ("make-xwidget",
@@ -168,29 +147,17 @@ Returns the newly constructed xwidget, or nil if construction fails.  */)
       if (EQ (xw->type, Qwebkit))
         {
           g_signal_connect (G_OBJECT (xw->widget_osr),
-                            "document-load-finished",
-                            G_CALLBACK (webkit_document_load_finished_cb), xw);
+                            "load-changed",
+                            G_CALLBACK (webkit_view_load_changed_cb), xw);
 
-          g_signal_connect (G_OBJECT (xw->widget_osr),
-                            "download-requested",
+          g_signal_connect (G_OBJECT (webkit_web_context_get_default ()),
+                            "download-started",
                             G_CALLBACK (webkit_download_cb), xw);
 
           g_signal_connect (G_OBJECT (xw->widget_osr),
-                            "mime-type-policy-decision-requested",
-                            G_CALLBACK
-                            (webkit_mime_type_policy_typedecision_requested_cb),
-                            xw);
-
-          g_signal_connect (G_OBJECT (xw->widget_osr),
-                            "new-window-policy-decision-requested",
-                            G_CALLBACK
-                            (webkit_new_window_policy_decision_requested_cb),
-                            xw);
-
-          g_signal_connect (G_OBJECT (xw->widget_osr),
-                            "navigation-policy-decision-requested",
+                            "decide-policy",
                             G_CALLBACK
-                            (webkit_navigation_policy_decision_requested_cb),
+                            (webkit_decide_policy_cb),
                             xw);
         }
 
@@ -284,81 +251,83 @@ store_xwidget_event_string (struct xwidget *xw, const char *eventname,
   kbd_buffer_store_event (&event);
 }
 
-/* TODO deprecated, use load-status.  */
 void
-webkit_document_load_finished_cb (WebKitWebView *webkitwebview,
-                                  WebKitWebFrame *arg1,
-                                  gpointer data)
+webkit_view_load_changed_cb (WebKitWebView *webkitwebview,
+                             WebKitLoadEvent load_event,
+                             gpointer data)
 {
-  struct xwidget *xw = g_object_get_data (G_OBJECT (webkitwebview),
-                                          XG_XWIDGET);
-
-  store_xwidget_event_string (xw, "document-load-finished", "");
+  switch (load_event) {
+  case WEBKIT_LOAD_FINISHED:
+    {
+      struct xwidget *xw = g_object_get_data (G_OBJECT (webkitwebview),
+                                              XG_XWIDGET);
+      store_xwidget_event_string (xw, "load-changed", "");
+      break;
+    }
+  default:
+    break;
+  }
 }
 
 gboolean
-webkit_download_cb (WebKitWebView *webkitwebview,
+webkit_download_cb (WebKitWebContext *webkitwebcontext,
                     WebKitDownload *arg1,
                     gpointer data)
 {
-  struct xwidget *xw = g_object_get_data (G_OBJECT (webkitwebview),
+  WebKitWebView *view = webkit_download_get_web_view(arg1);
+  WebKitURIRequest *request = webkit_download_get_request(arg1);
+  struct xwidget *xw = g_object_get_data (G_OBJECT (view),
                                           XG_XWIDGET);
-  store_xwidget_event_string (xw, "download-requested",
-                              webkit_download_get_uri (arg1));
+
+  store_xwidget_event_string (xw, "download-started",
+                              webkit_uri_request_get_uri(request));
   return FALSE;
 }
 
 static gboolean
-webkit_mime_type_policy_typedecision_requested_cb (WebKitWebView *webView,
-						   WebKitWebFrame *frame,
-						   WebKitNetworkRequest *request,
-						   gchar *mimetype,
-						   WebKitWebPolicyDecision *policy_decision,
-						   gpointer user_data)
+webkit_decide_policy_cb (WebKitWebView *webView,
+                         WebKitPolicyDecision *decision,
+                         WebKitPolicyDecisionType type,
+                         gpointer user_data)
 {
-  /* This function makes webkit send a download signal for all unknown
-     mime types.  TODO: Defer the decision to Lisp, so that it's
-     possible to make Emacs handle mime text for instance.  */
-  if (!webkit_web_view_can_show_mime_type (webView, mimetype))
+  switch (type) {
+  case WEBKIT_POLICY_DECISION_TYPE_RESPONSE:
+    /* This function makes webkit send a download signal for all unknown
+       mime types.  TODO: Defer the decision to Lisp, so that it's
+       possible to make Emacs handle mime text for instance.  */
     {
-      webkit_web_policy_decision_download (policy_decision);
-      return TRUE;
+      WebKitResponsePolicyDecision *response =
+        WEBKIT_RESPONSE_POLICY_DECISION (decision);
+      if (!webkit_response_policy_decision_is_mime_type_supported (response))
+        {
+          webkit_policy_decision_download (decision);
+          return TRUE;
+        }
+      else
+        return FALSE;
+      break;
     }
-  else
+  case WEBKIT_POLICY_DECISION_TYPE_NEW_WINDOW_ACTION:
+  case WEBKIT_POLICY_DECISION_TYPE_NAVIGATION_ACTION:
+    {
+      WebKitNavigationPolicyDecision *navigation_decision =
+        WEBKIT_NAVIGATION_POLICY_DECISION (decision);
+      WebKitNavigationAction *navigation_action =
+        webkit_navigation_policy_decision_get_navigation_action (navigation_decision);
+      WebKitURIRequest *request =
+        webkit_navigation_action_get_request (navigation_action);
+
+      struct xwidget *xw = g_object_get_data (G_OBJECT (webView), XG_XWIDGET);
+      store_xwidget_event_string (xw, "decide-policy",
+                                  webkit_uri_request_get_uri (request));
+      return FALSE;
+      break;
+    }
+  default:
     return FALSE;
+  }
 }
 
-static gboolean
-webkit_new_window_policy_decision_requested_cb (WebKitWebView *webView,
-						WebKitWebFrame *frame,
-						WebKitNetworkRequest *request,
-						WebKitWebNavigationAction *navigation_action,
-						WebKitWebPolicyDecision *policy_decision,
-						gpointer user_data)
-{
-  struct xwidget *xw = g_object_get_data (G_OBJECT (webView), XG_XWIDGET);
-  webkit_web_navigation_action_get_original_uri (navigation_action);
-
-  store_xwidget_event_string (xw, "new-window-policy-decision-requested",
-                              webkit_web_navigation_action_get_original_uri
-                              (navigation_action));
-  return FALSE;
-}
-
-static gboolean
-webkit_navigation_policy_decision_requested_cb (WebKitWebView *webView,
-						WebKitWebFrame *frame,
-						WebKitNetworkRequest *request,
-						WebKitWebNavigationAction *navigation_action,
-						WebKitWebPolicyDecision *policy_decision,
-						gpointer user_data)
-{
-  struct xwidget *xw = g_object_get_data (G_OBJECT (webView), XG_XWIDGET);
-  store_xwidget_event_string (xw, "navigation-policy-decision-requested",
-                              webkit_web_navigation_action_get_original_uri
-                              (navigation_action));
-  return FALSE;
-}
 
 /* For gtk3 offscreen rendered widgets.  */
 static gboolean
@@ -599,8 +568,13 @@ DEFUN ("xwidget-webkit-execute-script",
 {
   WEBKIT_FN_INIT ();
   CHECK_STRING (script);
-  webkit_web_view_execute_script (WEBKIT_WEB_VIEW (xw->widget_osr),
-                                  SSDATA (script));
+  // TODO: provide callback function to do something with the return
+  // value!  This allows us to get rid of the title hack.
+  webkit_web_view_run_javascript (WEBKIT_WEB_VIEW (xw->widget_osr),
+                                  SSDATA (script),
+                                  NULL, /*cancellable*/
+                                  NULL, /*callback*/
+                                  NULL /*user data*/);
   return Qnil;
 }
 
-- 
2.10.1

[PATCH 02/15] xwidget: Pass JavaScript return value to optional callback procedure

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-execute-script): Accept optional
callback argument.
(xwidget-webkit-callback): Handle "javascript-callback" event type.
* src/xwidget.c (xwidget-webkit-execute-script): Accept optional
argument FUN, a Lisp procedure to execute on the JavaScript return
value.
(store_xwidget_js_callback_event, webkit_javascript_finished_cb,
webkit_js_to_lisp): New procedures.
---
 lisp/xwidget.el |  33 +++++++-----
 src/xwidget.c   | 165 +++++++++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 177 insertions(+), 21 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 1bae6bb..69b1002 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -42,7 +42,8 @@
 (declare-function xwidget-webkit-get-title "xwidget.c" (xwidget))
 (declare-function xwidget-size-request "xwidget.c" (xwidget))
 (declare-function xwidget-resize "xwidget.c" (xwidget new-width new-height))
-(declare-function xwidget-webkit-execute-script "xwidget.c" (xwidget script))
+(declare-function xwidget-webkit-execute-script "xwidget.c"
+                  (xwidget script &optional callback))
 (declare-function xwidget-webkit-goto-uri "xwidget.c" (xwidget uri))
 (declare-function xwidget-plist "xwidget.c" (xwidget))
 (declare-function set-xwidget-plist "xwidget.c" (xwidget plist))
@@ -186,22 +187,26 @@ XWIDGET instance, XWIDGET-EVENT-TYPE depends on the originating xwidget."
       (xwidget-log
        "error: callback called for xwidget with dead buffer")
     (with-current-buffer (xwidget-buffer xwidget)
-      (let* ((strarg  (nth 3 last-input-event)))
-        (cond ((eq xwidget-event-type 'load-changed)
-               (xwidget-log "webkit finished loading: '%s'"
-                            (xwidget-webkit-get-title xwidget))
-               ;;TODO - check the native/internal scroll
-               ;;(xwidget-adjust-size-to-content xwidget)
-               (xwidget-webkit-adjust-size-dispatch) ;;TODO xwidget arg
-               (rename-buffer (format "*xwidget webkit: %s *"
-                                      (xwidget-webkit-get-title xwidget)))
-               (pop-to-buffer (current-buffer)))
-              ((eq xwidget-event-type 'decide-policy)
+      (cond ((eq xwidget-event-type 'load-changed)
+             (xwidget-log "webkit finished loading: '%s'"
+                          (xwidget-webkit-get-title xwidget))
+             ;;TODO - check the native/internal scroll
+             ;;(xwidget-adjust-size-to-content xwidget)
+             (xwidget-webkit-adjust-size-dispatch) ;;TODO xwidget arg
+             (rename-buffer (format "*xwidget webkit: %s *"
+                                    (xwidget-webkit-get-title xwidget)))
+             (pop-to-buffer (current-buffer)))
+            ((eq xwidget-event-type 'decide-policy)
+             (let ((strarg  (nth 3 last-input-event)))
                (if (string-match ".*#\\(.*\\)" strarg)
                    (xwidget-webkit-show-id-or-named-element
                     xwidget
-                    (match-string 1 strarg))))
-              (t (xwidget-log "unhandled event:%s" xwidget-event-type)))))))
+                    (match-string 1 strarg)))))
+            ((eq xwidget-event-type 'javascript-callback)
+             (let ((proc (nth 3 last-input-event))
+                   (arg  (nth 4 last-input-event)))
+               (funcall proc arg)))
+            (t (xwidget-log "unhandled event:%s" xwidget-event-type))))))
 
 (defvar bookmark-make-record-function)
 (define-derived-mode xwidget-webkit-mode
diff --git a/src/xwidget.c b/src/xwidget.c
index 78349a8..4f53b93 100644
--- a/src/xwidget.c
+++ b/src/xwidget.c
@@ -28,6 +28,7 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
 #include "gtkutil.h"
 
 #include <webkit2/webkit2.h>
+#include <JavaScriptCore/JavaScript.h>
 
 static struct xwidget *
 allocate_xwidget (void)
@@ -50,6 +51,9 @@ static struct xwidget_view *xwidget_view_lookup (struct xwidget *,
 static void webkit_view_load_changed_cb (WebKitWebView *,
                                          WebKitLoadEvent,
                                          gpointer);
+static void webkit_javascript_finished_cb (GObject *,
+                                           GAsyncResult *,
+                                           gpointer);
 static gboolean webkit_download_cb (WebKitWebContext *, WebKitDownload *, gpointer);
 
 static gboolean
@@ -251,6 +255,22 @@ store_xwidget_event_string (struct xwidget *xw, const char *eventname,
   kbd_buffer_store_event (&event);
 }
 
+static void
+store_xwidget_js_callback_event (struct xwidget *xw,
+                                 Lisp_Object proc,
+                                 Lisp_Object argument)
+{
+  struct input_event event;
+  Lisp_Object xwl;
+  XSETXWIDGET (xwl, xw);
+  EVENT_INIT (event);
+  event.kind = XWIDGET_EVENT;
+  event.frame_or_window = Qnil;
+  event.arg = list4 (intern ("javascript-callback"), xwl, proc, argument);
+  kbd_buffer_store_event (&event);
+}
+
+
 void
 webkit_view_load_changed_cb (WebKitWebView *webkitwebview,
                              WebKitLoadEvent load_event,
@@ -269,6 +289,128 @@ webkit_view_load_changed_cb (WebKitWebView *webkitwebview,
   }
 }
 
+/* Recursively convert a JavaScript value to a Lisp value. */
+Lisp_Object
+webkit_js_to_lisp (JSContextRef context, JSValueRef value)
+{
+  switch (JSValueGetType (context, value))
+    {
+    case kJSTypeString:
+      {
+        JSStringRef js_str_value;
+        gchar *str_value;
+        gsize str_length;
+
+        js_str_value = JSValueToStringCopy (context, value, NULL);
+        str_length = JSStringGetMaximumUTF8CStringSize (js_str_value);
+        str_value = (gchar *)g_malloc (str_length);
+        JSStringGetUTF8CString (js_str_value, str_value, str_length);
+        JSStringRelease (js_str_value);
+        return build_string (str_value);
+      }
+    case kJSTypeBoolean:
+      return (JSValueToBoolean (context, value)) ? Qt : Qnil;
+    case kJSTypeNumber:
+      return make_number (JSValueToNumber (context, value, NULL));
+    case kJSTypeObject:
+      {
+        if (JSValueIsArray (context, value))
+          {
+            JSStringRef pname = JSStringCreateWithUTF8CString("length");
+            JSValueRef len = JSObjectGetProperty (context, (JSObjectRef) value, pname, NULL);
+            int n = JSValueToNumber (context, len, NULL);
+            JSStringRelease(pname);
+
+            Lisp_Object obj;
+            struct Lisp_Vector *p = allocate_vector (n);
+
+            for (int i = 0; i < n; ++i)
+              {
+                p->contents[i] =
+                  webkit_js_to_lisp (context,
+                                     JSObjectGetPropertyAtIndex (context,
+                                                                 (JSObjectRef) value,
+                                                                 i, NULL));
+              }
+            XSETVECTOR (obj, p);
+            return obj;
+          }
+        else
+          {
+            JSPropertyNameArrayRef properties =
+              JSObjectCopyPropertyNames (context, (JSObjectRef) value);
+
+            int n = JSPropertyNameArrayGetCount (properties);
+            Lisp_Object obj;
+
+            // TODO: can we use a regular list here?
+            struct Lisp_Vector *p = allocate_vector (n);
+
+            for (int i = 0; i < n; ++i)
+              {
+                JSStringRef name = JSPropertyNameArrayGetNameAtIndex (properties, i);
+                JSValueRef property = JSObjectGetProperty (context,
+                                                           (JSObjectRef) value,
+                                                           name, NULL);
+                gchar *str_name;
+                gsize str_length;
+                str_length = JSStringGetMaximumUTF8CStringSize (name);
+                str_name = (gchar *)g_malloc (str_length);
+                JSStringGetUTF8CString (name, str_name, str_length);
+                JSStringRelease (name);
+
+                p->contents[i] =
+                  Fcons (build_string (str_name),
+                         webkit_js_to_lisp (context, property));
+              }
+
+            JSPropertyNameArrayRelease (properties);
+            XSETVECTOR (obj, p);
+            return obj;
+          }
+      }
+    case kJSTypeUndefined:
+    case kJSTypeNull:
+    default:
+      return Qnil;
+    }
+}
+
+static void
+webkit_javascript_finished_cb (GObject      *webview,
+                               GAsyncResult *result,
+                               gpointer      lisp_callback)
+{
+    WebKitJavascriptResult *js_result;
+    JSValueRef value;
+    JSGlobalContextRef context;
+    GError *error = NULL;
+    struct xwidget *xw = g_object_get_data (G_OBJECT (webview),
+                                            XG_XWIDGET);
+
+    js_result = webkit_web_view_run_javascript_finish
+      (WEBKIT_WEB_VIEW (webview), result, &error);
+
+    if (!js_result)
+      {
+        g_warning ("Error running javascript: %s", error->message);
+        g_error_free (error);
+        return;
+      }
+
+    context = webkit_javascript_result_get_global_context (js_result);
+    value = webkit_javascript_result_get_value (js_result);
+    Lisp_Object lisp_value = webkit_js_to_lisp (context, value);
+    webkit_javascript_result_unref (js_result);
+
+    // Register an xwidget event here, which then runs the callback.
+    // This ensures that the callback runs in sync with the Emacs
+    // event loop.
+    store_xwidget_js_callback_event (xw, (Lisp_Object)lisp_callback,
+                                     lisp_value);
+}
+
+
 gboolean
 webkit_download_cb (WebKitWebContext *webkitwebcontext,
                     WebKitDownload *arg1,
@@ -562,19 +704,28 @@ DEFUN ("xwidget-webkit-goto-uri",
 
 DEFUN ("xwidget-webkit-execute-script",
        Fxwidget_webkit_execute_script, Sxwidget_webkit_execute_script,
-       2, 2, 0,
-       doc: /* Make the Webkit XWIDGET execute JavaScript SCRIPT.  */)
-  (Lisp_Object xwidget, Lisp_Object script)
+       2, 3, 0,
+       doc: /* Make the Webkit XWIDGET execute JavaScript SCRIPT.  If
+FUN is provided, feed the JavaScript return value to the single
+argument procedure FUN.*/)
+  (Lisp_Object xwidget, Lisp_Object script, Lisp_Object fun)
 {
   WEBKIT_FN_INIT ();
   CHECK_STRING (script);
-  // TODO: provide callback function to do something with the return
-  // value!  This allows us to get rid of the title hack.
+  if (!NILP (fun) && (!FUNCTIONP (fun)))
+    wrong_type_argument (Qinvalid_function, fun);
+
+  void *callback = (FUNCTIONP (fun)) ?
+    &webkit_javascript_finished_cb : NULL;
+
+  // JavaScript execution happens asynchronously.  If an elisp
+  // callback function is provided we pass it to the C callback
+  // procedure that retrieves the return value.
   webkit_web_view_run_javascript (WEBKIT_WEB_VIEW (xw->widget_osr),
                                   SSDATA (script),
                                   NULL, /*cancellable*/
-                                  NULL, /*callback*/
-                                  NULL /*user data*/);
+                                  callback,
+                                  (gpointer) fun);
   return Qnil;
 }
 
-- 
2.10.1

[PATCH 03/15] Remove scrolled window container around WebKit widget

[Ricardo Wurmus]

The WebKit widget can scroll on its own and does not need to wrapped
with a scrolled window container.

* src/xwidget.h: Remove struct member widgetscrolledwindow_osr.
* src/xwidget.c: Remove widgetscrolledwindow_osr.
(xwidget-set-adjustment): Remove.
(xwidget-resize): Resize Webkit widget last.
* lisp/xwidget.el (xwidget-set-adjustment): Remove.
(xwidget-webkit-scroll-up, xwidget-webkit-scroll-down,
xwidget-webkit-scroll-forward, xwidget-webkit-scroll-backward):
Implement scrolling via JavaScript.
---
 lisp/xwidget.el | 18 ++++++++++------
 src/xwidget.c   | 64 ++++-----------------------------------------------------
 src/xwidget.h   |  3 ---
 3 files changed, 16 insertions(+), 69 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 69b1002..d2b9a09 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -36,8 +36,6 @@
 
 (declare-function make-xwidget "xwidget.c"
                   (type title width height arguments &optional buffer))
-(declare-function xwidget-set-adjustment "xwidget.c"
-                  (xwidget axis relative value))
 (declare-function xwidget-buffer "xwidget.c" (xwidget))
 (declare-function xwidget-webkit-get-title "xwidget.c" (xwidget))
 (declare-function xwidget-size-request "xwidget.c" (xwidget))
@@ -137,22 +135,30 @@ Interactively, URL defaults to the string looking like a url around point."
 (defun xwidget-webkit-scroll-up ()
   "Scroll webkit up."
   (interactive)
-  (xwidget-set-adjustment (xwidget-webkit-last-session) 'vertical t 50))
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "window.scrollBy(0, 50);"))
 
 (defun xwidget-webkit-scroll-down ()
   "Scroll webkit down."
   (interactive)
-  (xwidget-set-adjustment (xwidget-webkit-last-session) 'vertical t -50))
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "window.scrollBy(0, -50);"))
 
 (defun xwidget-webkit-scroll-forward ()
   "Scroll webkit forwards."
   (interactive)
-  (xwidget-set-adjustment (xwidget-webkit-last-session) 'horizontal t 50))
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "window.scrollBy(50, 0);"))
 
 (defun xwidget-webkit-scroll-backward ()
   "Scroll webkit backwards."
   (interactive)
-  (xwidget-set-adjustment (xwidget-webkit-last-session) 'horizontal t -50))
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "window.scrollBy(-50, 0);"))
 
 
 ;; The xwidget event needs to go into a higher level handler
diff --git a/src/xwidget.c b/src/xwidget.c
index 4f53b93..8552810 100644
--- a/src/xwidget.c
+++ b/src/xwidget.c
@@ -103,25 +103,9 @@ Returns the newly constructed xwidget, or nil if construction fails.  */)
       gtk_window_resize (GTK_WINDOW (xw->widgetwindow_osr), xw->width,
                          xw->height);
 
-      /* WebKit OSR is the only scrolled component at the moment.  */
-      xw->widgetscrolledwindow_osr = NULL;
-
       if (EQ (xw->type, Qwebkit))
         {
-          xw->widgetscrolledwindow_osr = gtk_scrolled_window_new (NULL, NULL);
-          gtk_scrolled_window_set_min_content_height
-	    (GTK_SCROLLED_WINDOW (xw->widgetscrolledwindow_osr),
-	     xw->height);
-          gtk_scrolled_window_set_min_content_width
-	    (GTK_SCROLLED_WINDOW (xw->widgetscrolledwindow_osr),
-	     xw->width);
-          gtk_scrolled_window_set_policy
-	    (GTK_SCROLLED_WINDOW (xw->widgetscrolledwindow_osr),
-	     GTK_POLICY_ALWAYS, GTK_POLICY_ALWAYS);
-
           xw->widget_osr = webkit_web_view_new ();
-          gtk_container_add (GTK_CONTAINER (xw->widgetscrolledwindow_osr),
-                             GTK_WIDGET (WEBKIT_WEB_VIEW (xw->widget_osr)));
         }
 
       gtk_widget_set_size_request (GTK_WIDGET (xw->widget_osr), xw->width,
@@ -130,7 +114,7 @@ Returns the newly constructed xwidget, or nil if construction fails.  */)
       if (EQ (xw->type, Qwebkit))
         {
           gtk_container_add (GTK_CONTAINER (xw->widgetwindow_osr),
-                             xw->widgetscrolledwindow_osr);
+                             GTK_WIDGET (WEBKIT_WEB_VIEW (xw->widget_osr)));
         }
       else
         {
@@ -140,7 +124,6 @@ Returns the newly constructed xwidget, or nil if construction fails.  */)
 
       gtk_widget_show (xw->widget_osr);
       gtk_widget_show (xw->widgetwindow_osr);
-      gtk_widget_show (xw->widgetscrolledwindow_osr);
 
       /* Store some xwidget data in the gtk widgets for convenient
          retrieval in the event handlers.  */
@@ -482,10 +465,7 @@ xwidget_osr_draw_cb (GtkWidget *widget, cairo_t *cr, gpointer data)
   cairo_rectangle (cr, 0, 0, xv->clip_right, xv->clip_bottom);
   cairo_clip (cr);
 
-  if (xw->widgetscrolledwindow_osr != NULL)
-    gtk_widget_draw (xw->widgetscrolledwindow_osr, cr);
-  else
-    gtk_widget_draw (xw->widget_osr, cr);
+  gtk_widget_draw (xw->widget_osr, cr);
   return FALSE;
 }
 
@@ -767,21 +747,11 @@ DEFUN ("xwidget-resize", Fxwidget_resize, Sxwidget_resize, 3, 3, 0,
   /* If there is an offscreen widget resize it first.  */
   if (xw->widget_osr)
     {
-      /* Use minimum size.  */
-      gtk_widget_set_size_request (GTK_WIDGET (xw->widget_osr),
-                                   xw->width, xw->height);
-
       gtk_window_resize (GTK_WINDOW (xw->widgetwindow_osr), xw->width,
                          xw->height);
-      gtk_scrolled_window_set_min_content_height
-	(GTK_SCROLLED_WINDOW (xw->widgetscrolledwindow_osr),
-	 xw->height);
-      gtk_scrolled_window_set_min_content_width
-	(GTK_SCROLLED_WINDOW (xw->widgetscrolledwindow_osr),
-	 xw->width);
-
       gtk_container_resize_children (GTK_CONTAINER (xw->widgetwindow_osr));
-
+      gtk_widget_set_size_request (GTK_WIDGET (xw->widget_osr), xw->width,
+                                   xw->height);
     }
 
   for (Lisp_Object tail = Vxwidget_view_list; CONSP (tail); tail = XCDR (tail))
@@ -800,30 +770,6 @@ DEFUN ("xwidget-resize", Fxwidget_resize, Sxwidget_resize, 3, 3, 0,
 
 
 
-DEFUN ("xwidget-set-adjustment",
-       Fxwidget_set_adjustment, Sxwidget_set_adjustment, 4, 4, 0,
-       doc: /* Set native scrolling for XWIDGET.
-AXIS can be `vertical' or `horizontal'.
-If RELATIVE is t, scroll relative, otherwise absolutely.
-VALUE is the amount to scroll, either relatively or absolutely.  */)
-  (Lisp_Object xwidget, Lisp_Object axis, Lisp_Object relative,
-   Lisp_Object value)
-{
-  CHECK_XWIDGET (xwidget);
-  CHECK_NUMBER (value);
-  struct xwidget *xw = XXWIDGET (xwidget);
-  GtkAdjustment *adjustment
-    = ((EQ (Qhorizontal, axis)
-	? gtk_scrolled_window_get_hadjustment
-	: gtk_scrolled_window_get_vadjustment)
-       (GTK_SCROLLED_WINDOW (xw->widgetscrolledwindow_osr)));
-  double final_value = XINT (value);
-  if (EQ (Qt, relative))
-    final_value += gtk_adjustment_get_value (adjustment);
-  gtk_adjustment_set_value (adjustment, final_value);
-  return Qnil;
-}
-
 
 DEFUN ("xwidget-size-request",
        Fxwidget_size_request, Sxwidget_size_request,
@@ -1039,8 +985,6 @@ syms_of_xwidget (void)
   defsubr (&Sxwidget_buffer);
   defsubr (&Sset_xwidget_plist);
 
-  defsubr (&Sxwidget_set_adjustment);
-
   DEFSYM (Qxwidget, "xwidget");
 
   DEFSYM (QCxwidget, ":xwidget");
diff --git a/src/xwidget.h b/src/xwidget.h
index 8fc3821..4447abb 100644
--- a/src/xwidget.h
+++ b/src/xwidget.h
@@ -56,9 +56,6 @@ struct xwidget
   GtkWidget *widget_osr;
   GtkWidget *widgetwindow_osr;
 
-  /* Used if the widget (webkit) is to be wrapped in a scrolled window.  */
-  GtkWidget *widgetscrolledwindow_osr;
-
   /* Kill silently if Emacs is exited.  */
   bool_bf kill_without_query : 1;
 };
-- 
2.10.1

[PATCH 04/15] xwidget: Do not use `xwidget-execute-script-rv' to insert string

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-insert-string): Obtain JavaScript
return value via callback instead of using
`xwidget-webkit-execute-script-rv'.
---
 lisp/xwidget.el | 47 +++++++++++++++++++++++------------------------
 1 file changed, 23 insertions(+), 24 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index d2b9a09..a252fd7 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -286,31 +286,30 @@ function findactiveelement(doc){
   ;;TODO the activeelement type needs to be examined, for iframe, etc.
   )
 
-(defun xwidget-webkit-insert-string (xw str)
-  "Insert string STR in the active field in the webkit XW."
+(defun xwidget-webkit-insert-string ()
+  "Prompt for a string and insert it in the active field in the
+current webkit widget."
   ;; Read out the string in the field first and provide for edit.
-  (interactive
-   (let* ((xww (xwidget-webkit-current-session))
-
-          (field-value
-           (progn
-             (xwidget-webkit-execute-script xww xwidget-webkit-activeelement-js)
-             (xwidget-webkit-execute-script-rv
-              xww
-              "findactiveelement(document).value;")))
-          (field-type (xwidget-webkit-execute-script-rv
-                       xww
-                       "findactiveelement(document).type;")))
-     (list xww
-           (cond ((equal "text" field-type)
-                  (read-string "Text: " field-value))
-                 ((equal "password" field-type)
-                  (read-passwd "Password: " nil field-value))
-                 ((equal "textarea" field-type)
-                  (xwidget-webkit-begin-edit-textarea xww field-value))))))
-  (xwidget-webkit-execute-script
-   xw
-   (format "findactiveelement(document).value='%s'" str)))
+  (interactive)
+  (let ((xww (xwidget-webkit-current-session)))
+    (xwidget-webkit-execute-script
+     xww
+     (concat xwidget-webkit-activeelement-js "
+(function () {
+  var res = findactiveelement(document);
+  return [res.value, res.type];
+})();")
+     (lambda (field)
+       (let ((str (pcase field
+                    (`[,val "text"]
+                     (read-string "Text: " val))
+                    (`[,val "password"]
+                     (read-passwd "Password: " nil val))
+                    (`[,val "textarea"]
+                     (xwidget-webkit-begin-edit-textarea xww val)))))
+         (xwidget-webkit-execute-script
+          xww
+          (format "findactiveelement(document).value='%s'" str)))))))
 
 (defvar xwidget-xwbl)
 (defun xwidget-webkit-begin-edit-textarea (xw text)
-- 
2.10.1

[PATCH 05/15] xwidget: Get title via asynchronous JavaScript.

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-callback): Get document title
asynchronously.
---
 lisp/xwidget.el | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index a252fd7..1333365 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -194,13 +194,14 @@ XWIDGET instance, XWIDGET-EVENT-TYPE depends on the originating xwidget."
        "error: callback called for xwidget with dead buffer")
     (with-current-buffer (xwidget-buffer xwidget)
       (cond ((eq xwidget-event-type 'load-changed)
-             (xwidget-log "webkit finished loading: '%s'"
-                          (xwidget-webkit-get-title xwidget))
-             ;;TODO - check the native/internal scroll
-             ;;(xwidget-adjust-size-to-content xwidget)
-             (xwidget-webkit-adjust-size-dispatch) ;;TODO xwidget arg
-             (rename-buffer (format "*xwidget webkit: %s *"
-                                    (xwidget-webkit-get-title xwidget)))
+             (xwidget-webkit-execute-script
+              xwidget "document.title"
+              (lambda (title)
+                (xwidget-log "webkit finished loading: '%s'" title)
+                ;;TODO - check the native/internal scroll
+                ;;(xwidget-adjust-size-to-content xwidget)
+                (xwidget-webkit-adjust-size-dispatch) ;;TODO xwidget arg
+                (rename-buffer (format "*xwidget webkit: %s *" title))))
              (pop-to-buffer (current-buffer)))
             ((eq xwidget-event-type 'decide-policy)
              (let ((strarg  (nth 3 last-input-event)))
-- 
2.10.1

[PATCH 06/15] xwidget: Simplify functions to scroll to elements

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-show-named-element,
xwidget-webkit-show-id-element,
xwidget-webkit-show-id-or-named-element): Simplify functions by
scrolling exclusively with JavaScript.
---
 lisp/xwidget.el | 79 +++++++++++++++++++++++++--------------------------------
 1 file changed, 35 insertions(+), 44 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 1333365..e54d1f8 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -339,62 +339,53 @@ XW is the xwidget identifier, TEXT is retrieved from the webkit."
 For example, use this to display an anchor."
   (interactive (list (xwidget-webkit-current-session)
                      (read-string "Element name: ")))
-  ;;TODO since an xwidget is an Emacs object, it is not trivial to do
-  ;; some things that are taken for granted in a normal browser.
-  ;; scrolling an anchor/named-element into view is one such thing.
-  ;; This function implements a proof-of-concept for this.  Problems
-  ;; remaining: - The selected window is scrolled but this is not
-  ;; always correct - This needs to be interfaced into browse-url
-  ;; somehow.  The tricky part is that we need to do this in two steps:
-  ;; A: load the base url, wait for load signal to arrive B: navigate
-  ;; to the anchor when the base url is finished rendering
-
-  ;; This part figures out the Y coordinate of the element
-  (let ((y (string-to-number
-            (xwidget-webkit-execute-script-rv
-             xw
-             (format
-              "document.getElementsByName('%s')[0].getBoundingClientRect().top"
-              element-name)
-             0))))
-    ;; Now we need to tell Emacs to scroll the element into view.
-    (xwidget-log "scroll: %d" y)
-    (set-window-vscroll (selected-window) y t)))
+  ;; TODO: This needs to be interfaced into browse-url somehow.  The
+  ;; tricky part is that we need to do this in two steps: A: load the
+  ;; base url, wait for load signal to arrive B: navigate to the
+  ;; anchor when the base url is finished rendering
+  (xwidget-webkit-execute-script
+   xw
+   (format "
+(function (query) {
+  var el = document.getElementsByName(query)[0];
+  if (el !== undefined) {
+    window.scrollTo(0, el.offsetTop);
+  }
+})('%s');"
+    element-name)))
 
 (defun xwidget-webkit-show-id-element (xw element-id)
   "Make webkit xwidget XW show an id-element ELEMENT-ID.
 For example, use this to display an anchor."
   (interactive (list (xwidget-webkit-current-session)
                      (read-string "Element id: ")))
-  (let ((y (string-to-number
-            (xwidget-webkit-execute-script-rv
-             xw
-             (format "document.getElementById('%s').getBoundingClientRect().top"
-                     element-id)
-             0))))
-    ;; Now we need to tell Emacs to scroll the element into view.
-    (xwidget-log "scroll: %d" y)
-    (set-window-vscroll (selected-window) y t)))
+  (xwidget-webkit-execute-script
+   xw
+   (format "
+(function (query) {
+  var el = document.getElementById(query);
+  if (el !== null) {
+    window.scrollTo(0, el.offsetTop);
+  }
+})('%s');"
+    element-id)))
 
 (defun xwidget-webkit-show-id-or-named-element (xw element-id)
    "Make webkit xwidget XW show a name or element id ELEMENT-ID.
 For example, use this to display an anchor."
   (interactive (list (xwidget-webkit-current-session)
                      (read-string "Name or element id: ")))
-  (let* ((y1 (string-to-number
-              (xwidget-webkit-execute-script-rv
-               xw
-               (format "document.getElementsByName('%s')[0].getBoundingClientRect().top" element-id)
-               "0")))
-         (y2 (string-to-number
-              (xwidget-webkit-execute-script-rv
-               xw
-               (format "document.getElementById('%s').getBoundingClientRect().top" element-id)
-               "0")))
-         (y3 (max y1 y2)))
-    ;; Now we need to tell Emacs to scroll the element into view.
-    (xwidget-log "scroll: %d" y3)
-    (set-window-vscroll (selected-window) y3 t)))
+  (xwidget-webkit-execute-script
+   xw
+   (format "
+(function (query) {
+  var el = document.getElementById(query) ||
+           document.getElementsByName(query)[0];
+  if (el !== undefined) {
+    window.scrollTo(0, el.offsetTop);
+  }
+})('%s');"
+    element-id)))
 
 (defun xwidget-webkit-adjust-size-to-content ()
   "Adjust webkit to content size."
-- 
2.10.1

[PATCH 07/15] xwidget: Add function to find element by CSS selector

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-show-element): New procedure.
---
 lisp/xwidget.el | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index e54d1f8..d7ef44d 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -334,6 +334,23 @@ XW is the xwidget identifier, TEXT is retrieved from the webkit."
   ;;TODO convert linefeed to \n
   )
 
+(defun xwidget-webkit-show-element (xw element-selector)
+  "Make webkit xwidget XW show a named element ELEMENT-SELECTOR.
+The ELEMENT-SELECTOR must be a valid CSS selector.  For example,
+use this to display an anchor."
+  (interactive (list (xwidget-webkit-current-session)
+                     (read-string "Element selector: ")))
+  (xwidget-webkit-execute-script
+   xw
+   (format "
+(function (query) {
+  var el = document.querySelector(query);
+  if (el !== null) {
+    window.scrollTo(0, el.offsetTop);
+  }
+})('%s');"
+    element-selector)))
+
 (defun xwidget-webkit-show-named-element (xw element-name)
   "Make webkit xwidget XW show a named element ELEMENT-NAME.
 For example, use this to display an anchor."
-- 
2.10.1

[PATCH 08/15] xwidget: Get selection with asynchronous JavaScript

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-get-selection): Add PROC argument to
process selection.
(xwidget-webkit-copy-selection-as-kill): Kill selection in callback.
---
 lisp/xwidget.el | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index d7ef44d..dc31b85 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -504,15 +504,17 @@ DEFAULT is the default return value."
     title))
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-(defun xwidget-webkit-get-selection ()
-  "Get the webkit selection."
-  (xwidget-webkit-execute-script-rv (xwidget-webkit-current-session)
-                                    "window.getSelection().toString();"))
+(defun xwidget-webkit-get-selection (proc)
+  "Get the webkit selection and pass it to PROC."
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "window.getSelection().toString();"
+   proc))
 
 (defun xwidget-webkit-copy-selection-as-kill ()
   "Get the webkit selection and put it on the kill-ring."
   (interactive)
-  (kill-new (xwidget-webkit-get-selection)))
+  (xwidget-webkit-get-selection (lambda (selection) (kill-new selection))))
 
 
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
-- 
2.10.1

[PATCH 09/15] xwidget: Get URL asynchronously.

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-current-url): Kill URL in callback.
---
 lisp/xwidget.el | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index dc31b85..37edd52 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -474,11 +474,11 @@ For example, use this to display an anchor."
 (defun xwidget-webkit-current-url ()
   "Get the webkit url and place it on the kill-ring."
   (interactive)
-  (let* ((rv (xwidget-webkit-execute-script-rv (xwidget-webkit-current-session)
-                                               "document.URL"))
-         (url (kill-new (or rv ""))))
-    (message "url: %s" url)
-    url))
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "document.URL" (lambda (rv)
+                    (let ((url (kill-new (or rv ""))))
+                      (message "url: %s" url)))))
 
 (defun xwidget-webkit-execute-script-rv (xw script &optional default)
   "Same as `xwidget-webkit-execute-script' but with return value.
-- 
2.10.1

[PATCH 10/15] xwidget: Remove title hack.

[Ricardo Wurmus]

* src/xwidget.c (xwidget-webkit-get-title): Remove procedure.
* lisp/xwidget.el (xwidget-webkit-get-title,
xwidget-webkit-execute-script-rv): Remove procedures.
---
 lisp/xwidget.el | 24 ------------------------
 src/xwidget.c   | 22 ----------------------
 2 files changed, 46 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 37edd52..a1b9b50 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -37,7 +37,6 @@
 (declare-function make-xwidget "xwidget.c"
                   (type title width height arguments &optional buffer))
 (declare-function xwidget-buffer "xwidget.c" (xwidget))
-(declare-function xwidget-webkit-get-title "xwidget.c" (xwidget))
 (declare-function xwidget-size-request "xwidget.c" (xwidget))
 (declare-function xwidget-resize "xwidget.c" (xwidget new-width new-height))
 (declare-function xwidget-webkit-execute-script "xwidget.c"
@@ -480,29 +479,6 @@ For example, use this to display an anchor."
                     (let ((url (kill-new (or rv ""))))
                       (message "url: %s" url)))))
 
-(defun xwidget-webkit-execute-script-rv (xw script &optional default)
-  "Same as `xwidget-webkit-execute-script' but with return value.
-XW is the webkit instance.  SCRIPT is the script to execute.
-DEFAULT is the default return value."
-  ;; Notice the ugly "title" hack.  It is needed because the Webkit
-  ;; API at the time of writing didn't support returning values.  This
-  ;; is a wrapper for the title hack so it's easy to remove should
-  ;; Webkit someday support JS return values or we find some other way
-  ;; to access the DOM.
-
-  ;; Reset webkit title.  Not very nice.
-  (let* ((emptytag "titlecantbewhitespaceohthehorror")
-         title)
-    (xwidget-webkit-execute-script xw (format "document.title=\"%s\";"
-                                              (or default emptytag)))
-    (xwidget-webkit-execute-script xw (format "document.title=%s;" script))
-    (setq title (xwidget-webkit-get-title xw))
-    (if (equal emptytag title)
-        (setq title ""))
-    (unless title
-      (setq title default))
-    title))
-
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
 (defun xwidget-webkit-get-selection (proc)
   "Get the webkit selection and pass it to PROC."
diff --git a/src/xwidget.c b/src/xwidget.c
index 8552810..dbd8fc1 100644
--- a/src/xwidget.c
+++ b/src/xwidget.c
@@ -709,27 +709,6 @@ argument procedure FUN.*/)
   return Qnil;
 }
 
-DEFUN ("xwidget-webkit-get-title",
-       Fxwidget_webkit_get_title, Sxwidget_webkit_get_title,
-       1, 1, 0,
-       doc: /* Return the title from the Webkit instance in XWIDGET.
-This can be used to work around the lack of a return value from the
-exec method.  */ )
-  (Lisp_Object xwidget)
-{
-  /* TODO support multibyte strings.  */
-  WEBKIT_FN_INIT ();
-  const gchar *str =
-    webkit_web_view_get_title (WEBKIT_WEB_VIEW (xw->widget_osr));
-  if (str == 0)
-    {
-      /* TODO maybe return Qnil instead.  I suppose webkit returns
-	 null pointer when doc is not properly loaded or something.  */
-      return build_string ("");
-    }
-  return build_string (str);
-}
-
 DEFUN ("xwidget-resize", Fxwidget_resize, Sxwidget_resize, 3, 3, 0,
        doc: /* Resize XWIDGET.  NEW_WIDTH, NEW_HEIGHT define the new size.  */ )
   (Lisp_Object xwidget, Lisp_Object new_width, Lisp_Object new_height)
@@ -975,7 +954,6 @@ syms_of_xwidget (void)
 
   defsubr (&Sxwidget_webkit_goto_uri);
   defsubr (&Sxwidget_webkit_execute_script);
-  defsubr (&Sxwidget_webkit_get_title);
   DEFSYM (Qwebkit, "webkit");
 
   defsubr (&Sxwidget_size_request);
-- 
2.10.1

[PATCH 11/15] Let initial WebKit view fill window

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-new-session): Change default size of
WebKit widget to window size.
---
 lisp/xwidget.el | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index a1b9b50..e5b51ce 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -445,7 +445,9 @@ For example, use this to display an anchor."
     (setq xwidget-webkit-last-session-buffer (switch-to-buffer
                                               (get-buffer-create bufname)))
     (insert " 'a' adjusts the xwidget size.")
-    (setq xw (xwidget-insert 1 'webkit  bufname 1000 1000))
+    (setq xw (xwidget-insert 1 'webkit bufname
+                             (window-pixel-width)
+                             (window-pixel-height)))
     (xwidget-put xw 'callback 'xwidget-webkit-callback)
     (xwidget-webkit-mode)
     (xwidget-webkit-goto-uri (xwidget-webkit-last-session) url)))
-- 
2.10.1

[PATCH 12/15] Dynamically resize WebKit widget.

[Ricardo Wurmus]

* lisp/xwidget.el (xwidget-webkit-auto-adjust-size,
xwidget-webkit-adjust-size-in-frame): New procedures.
(xwidget-webkit-new-session): Remove hint to resize widget with `a'.
(xwidget-webkit-adjust-size-dispatch): Resize current webkit widget.
(xwidget-webkit-adjust-size-to-window): Make non-interactive, add widget and window as arguments.
(xwidget-webkit-callback): Use xwidget-webkit-adjust-size-to-window.
---
 lisp/xwidget.el | 33 +++++++++++++++++++++++++--------
 1 file changed, 25 insertions(+), 8 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index e5b51ce..13e3fdf2 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -199,7 +199,7 @@ XWIDGET instance, XWIDGET-EVENT-TYPE depends on the originating xwidget."
                 (xwidget-log "webkit finished loading: '%s'" title)
                 ;;TODO - check the native/internal scroll
                 ;;(xwidget-adjust-size-to-content xwidget)
-                (xwidget-webkit-adjust-size-dispatch) ;;TODO xwidget arg
+                (xwidget-webkit-adjust-size-to-window xwidget)
                 (rename-buffer (format "*xwidget webkit: %s *" title))))
              (pop-to-buffer (current-buffer)))
             ((eq xwidget-event-type 'decide-policy)
@@ -411,18 +411,18 @@ For example, use this to display an anchor."
 (defun xwidget-webkit-adjust-size-dispatch ()
   "Adjust size according to mode."
   (interactive)
-  (xwidget-webkit-adjust-size-to-window)
+  (xwidget-webkit-adjust-size-to-window (xwidget-webkit-current-session))
   ;; The recenter is intended to correct a visual glitch.
   ;; It errors out if the buffer isn't visible, but then we don't get
   ;; the glitch, so silence errors.
   (ignore-errors
     (recenter-top-bottom)))
 
-(defun xwidget-webkit-adjust-size-to-window ()
-  "Adjust webkit to window."
-  (interactive)
-  (xwidget-resize (xwidget-webkit-current-session) (window-pixel-width)
-                  (window-pixel-height)))
+(defun xwidget-webkit-adjust-size-to-window (xwidget &optional window)
+  "Adjust the size of the webkit XWIDGET to fit the WINDOW."
+  (xwidget-resize xwidget
+                  (window-pixel-width window)
+                  (window-pixel-height window)))
 
 (defun xwidget-webkit-adjust-size (w h)
   "Manually set webkit size to width W, height H."
@@ -437,6 +437,21 @@ For example, use this to display an anchor."
                                  (car (window-inside-pixel-edges)))
                               1000))
 
+(defun xwidget-webkit-auto-adjust-size (window)
+  "Adjust the size of the webkit widget in the given WINDOW."
+  (with-current-buffer (window-buffer window)
+    (when (eq major-mode 'xwidget-webkit-mode)
+      (let ((xwidget (xwidget-webkit-current-session)))
+        (xwidget-webkit-adjust-size-to-window xwidget window)))))
+
+(defun xwidget-webkit-adjust-size-in-frame (frame)
+  "Dynamically adjust webkit widget for all windows of the FRAME."
+  (walk-windows 'xwidget-webkit-auto-adjust-size 'no-minibuf frame))
+
+(eval-after-load 'xwidget-webkit-mode
+  (add-to-list 'window-size-change-functions
+               'xwidget-webkit-adjust-size-in-frame))
+
 (defun xwidget-webkit-new-session (url)
   "Create a new webkit session buffer with URL."
   (let*
@@ -444,7 +459,9 @@ For example, use this to display an anchor."
        xw)
     (setq xwidget-webkit-last-session-buffer (switch-to-buffer
                                               (get-buffer-create bufname)))
-    (insert " 'a' adjusts the xwidget size.")
+    ;; The xwidget id is stored in a text property, so we need to have
+    ;; at least character in this buffer.
+    (insert " ")
     (setq xw (xwidget-insert 1 'webkit bufname
                              (window-pixel-width)
                              (window-pixel-height)))
-- 
2.10.1

[PATCH 13/15] Implement zoom for WebKit widget.

[Ricardo Wurmus]

* src/xwidget.c (xwidget-webkit-zoom): New procedure.
* lisp/xwidget.el: Bind "+" and "-" to zoom in and out, respectively.
(xwidget-webkit-zoom): Declare procedure.
(xwidget-webkit-zoom-in, xwidget-webkit-zoom-out): New procedures.
---
 lisp/xwidget.el | 13 +++++++++++++
 src/xwidget.c   | 20 ++++++++++++++++++++
 2 files changed, 33 insertions(+)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 13e3fdf2..871b729 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -42,6 +42,7 @@
 (declare-function xwidget-webkit-execute-script "xwidget.c"
                   (xwidget script &optional callback))
 (declare-function xwidget-webkit-goto-uri "xwidget.c" (xwidget uri))
+(declare-function xwidget-webkit-zoom "xwidget.c" (xwidget factor))
 (declare-function xwidget-plist "xwidget.c" (xwidget))
 (declare-function set-xwidget-plist "xwidget.c" (xwidget plist))
 (declare-function xwidget-view-window "xwidget.c" (xwidget-view))
@@ -106,6 +107,8 @@ Interactively, URL defaults to the string looking like a url around point."
     (define-key map "t" (lambda () (interactive) (message "o"))) ;FIXME: ?!?
     (define-key map "\C-m" 'xwidget-webkit-insert-string)
     (define-key map "w" 'xwidget-webkit-current-url)
+    (define-key map "+" 'xwidget-webkit-zoom-in)
+    (define-key map "-" 'xwidget-webkit-zoom-out)
 
     ;;similar to image mode bindings
     (define-key map (kbd "SPC")                 'xwidget-webkit-scroll-up)
@@ -131,6 +134,16 @@ Interactively, URL defaults to the string looking like a url around point."
     map)
   "Keymap for `xwidget-webkit-mode'.")
 
+(defun xwidget-webkit-zoom-in ()
+  "Increase webkit view zoom factor."
+  (interactive)
+  (xwidget-webkit-zoom (xwidget-webkit-current-session) 0.1))
+
+(defun xwidget-webkit-zoom-out ()
+  "Decrease webkit view zoom factor."
+  (interactive)
+  (xwidget-webkit-zoom (xwidget-webkit-current-session) -0.1))
+
 (defun xwidget-webkit-scroll-up ()
   "Scroll webkit up."
   (interactive)
diff --git a/src/xwidget.c b/src/xwidget.c
index dbd8fc1..8105b3d 100644
--- a/src/xwidget.c
+++ b/src/xwidget.c
@@ -681,6 +681,25 @@ DEFUN ("xwidget-webkit-goto-uri",
   return Qnil;
 }
 
+DEFUN ("xwidget-webkit-zoom",
+       Fxwidget_webkit_zoom, Sxwidget_webkit_zoom,
+       2, 2, 0,
+       doc: /* Change the zoom factor of the xwidget webkit instance
+referenced by XWIDGET.  */)
+  (Lisp_Object xwidget, Lisp_Object factor)
+{
+  WEBKIT_FN_INIT ();
+  if (FLOATP (factor))
+    {
+      double zoom_change = XFLOAT_DATA (factor);
+      webkit_web_view_set_zoom_level
+        (WEBKIT_WEB_VIEW (xw->widget_osr),
+         webkit_web_view_get_zoom_level
+         (WEBKIT_WEB_VIEW (xw->widget_osr)) + zoom_change);
+    }
+  return Qnil;
+}
+
 
 DEFUN ("xwidget-webkit-execute-script",
        Fxwidget_webkit_execute_script, Sxwidget_webkit_execute_script,
@@ -953,6 +972,7 @@ syms_of_xwidget (void)
   defsubr (&Sset_xwidget_query_on_exit_flag);
 
   defsubr (&Sxwidget_webkit_goto_uri);
+  defsubr (&Sxwidget_webkit_zoom);
   defsubr (&Sxwidget_webkit_execute_script);
   DEFSYM (Qwebkit, "webkit");
 
-- 
2.10.1

[PATCH 14/15] xwidget: Bind "beginning-of-buffer" and "end-of-buffer"

[Ricardo Wurmus]

* lisp/xwidget.el: Rebind "beginning-of-buffer" and "end-of-buffer" to
"xwidget-webkit-scroll-top" and "xwidget-webkit-scroll-bottom",
respectively.
(xwidget-webkit-scroll-top,
xwidget-webkit-scroll-bottom): New procedures.
---
 lisp/xwidget.el | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 871b729..8aa0584 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -129,8 +129,8 @@ Interactively, URL defaults to the string looking like a url around point."
 
     ;; (define-key map [remap move-beginning-of-line] 'image-bol)
     ;; (define-key map [remap move-end-of-line]       'image-eol)
-    ;; (define-key map [remap beginning-of-buffer]    'image-bob)
-    ;; (define-key map [remap end-of-buffer]          'image-eob)
+    (define-key map [remap beginning-of-buffer] 'xwidget-webkit-scroll-top)
+    (define-key map [remap end-of-buffer]       'xwidget-webkit-scroll-bottom)
     map)
   "Keymap for `xwidget-webkit-mode'.")
 
@@ -172,6 +172,19 @@ Interactively, URL defaults to the string looking like a url around point."
    (xwidget-webkit-current-session)
    "window.scrollBy(-50, 0);"))
 
+(defun xwidget-webkit-scroll-top ()
+  "Scroll webkit to the very top."
+  (interactive)
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "window.scrollTo(pageXOffset, 0);"))
+
+(defun xwidget-webkit-scroll-bottom ()
+  "Scroll webkit to the very bottom."
+  (interactive)
+  (xwidget-webkit-execute-script
+   (xwidget-webkit-current-session)
+   "window.scrollTo(pageXOffset, window.document.body.clientHeight);"))
 
 ;; The xwidget event needs to go into a higher level handler
 ;; since the xwidget can generate an event even if it's offscreen.
-- 
2.10.1

[PATCH 15/15] xwidget: Map "previous-line" and "next-line" to scroll

[Ricardo Wurmus]

* lisp/xwidget.el: Map "previous-line" and "next-line" to scrolling
procedures.
---
 lisp/xwidget.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index 8aa0584..6443954 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -124,8 +124,8 @@ Interactively, URL defaults to the string looking like a url around point."
     (define-key map [remap backward-char]       'xwidget-webkit-scroll-backward)
     (define-key map [remap right-char]          'xwidget-webkit-scroll-forward)
     (define-key map [remap left-char]           'xwidget-webkit-scroll-backward)
-    ;; (define-key map [remap previous-line]          'image-previous-line)
-    ;; (define-key map [remap next-line]              'image-next-line)
+    (define-key map [remap previous-line]       'xwidget-webkit-scroll-down)
+    (define-key map [remap next-line]           'xwidget-webkit-scroll-up)
 
     ;; (define-key map [remap move-beginning-of-line] 'image-bol)
     ;; (define-key map [remap move-end-of-line]       'image-eol)
-- 
2.10.1

Re: [PATCH v2 00/15] xwidget webkit improvements

[Paul Eggert]

Thanks. I tried that on Fedora 24 x86-64, and got the following 
diagnostic when using M-x xwidget-webkit-browse-url to visit 
http://cs.ucla.edu.

(emacs:22840): Gtk-WARNING **: Allocating size to GtkOffscreenWindow 
0xd5b6d0 without calling gtk_widget_get_preferred_width/height(). How 
does the code know the size to allocate?

Is this something I should worry about?

A couple of minor things. First, in patch "Dynamically resize WebKit 
widget", this commit-message line exceeds 78 chars:

(xwidget-webkit-adjust-size-to-window): Make non-interactive, add widget and window as arguments.

so "git am" rejected it here. To help prevent this sort of problem in 
the future, run "./autogen.sh git". (72 chars or less is nice, so that 
ChangeLog lines fit in 80 columns.)

Second, in the patch "xwidget: Pass JavaScript return value to optional 
callback procedure" the new function webkit_js_to_lisp should be static. 
To catch this sort of thing in the future, use the --enable-gcc-warnings 
option to 'configure'.

Re: [PATCH v2 00/15] xwidget webkit improvements

[Ricardo Wurmus]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Thanks. I tried that on Fedora 24 x86-64, and got the following 
> diagnostic when using M-x xwidget-webkit-browse-url to visit 
> http://cs.ucla.edu.
>
> (emacs:22840): Gtk-WARNING **: Allocating size to GtkOffscreenWindow 
> 0xd5b6d0 without calling gtk_widget_get_preferred_width/height(). How 
> does the code know the size to allocate?
>
> Is this something I should worry about?

This warning is also produced by the existing xwidget code in Emacs
25.1.  It is harmless, but I did try to remove it and haven’t been
successful yet, as I’m not very familiar with GTK.

> A couple of minor things. First, in patch "Dynamically resize WebKit 
> widget", this commit-message line exceeds 78 chars:
>
> (xwidget-webkit-adjust-size-to-window): Make non-interactive, add widget and window as arguments.
>
> so "git am" rejected it here. To help prevent this sort of problem in 
> the future, run "./autogen.sh git". (72 chars or less is nice, so that 
> ChangeLog lines fit in 80 columns.)
>
> Second, in the patch "xwidget: Pass JavaScript return value to optional 
> callback procedure" the new function webkit_js_to_lisp should be static. 
> To catch this sort of thing in the future, use the --enable-gcc-warnings 
> option to 'configure'.

Thanks for the hints!  I’ll wait for additional feedback before sending
updated patches.

~~ Ricardo

Re: [PATCH v2 00/15] xwidget webkit improvements

[Paul Eggert]

Ricardo Wurmus wrote:

> This warning is also produced by the existing xwidget code in Emacs
> 25.1.  It is harmless, but I did try to remove it and haven’t been
> successful yet, as I’m not very familiar with GTK.

OK, thanks for explaining.

> I’ll wait for additional feedback before sending updated patches.

The patches look OK for a feature that is experimental anyway, so I took the 
liberty of installing them into master, with fixes for the two minor things. 
Thanks again.

Re: [PATCH v2 00/15] xwidget webkit improvements

[Live System User]

Ricardo Wurmus <rekado@elephly.net> writes:

> Paul Eggert <eggert@cs.ucla.edu> writes:
>
>> Thanks. I tried that on Fedora 24 x86-64, and got the following 
>> diagnostic when using M-x xwidget-webkit-browse-url to visit 
>> http://cs.ucla.edu.
>>
>> (emacs:22840): Gtk-WARNING **: Allocating size to GtkOffscreenWindow 
>> 0xd5b6d0 without calling gtk_widget_get_preferred_width/height(). How 
>> does the code know the size to allocate?
>>
>> Is this something I should worry about?
>
> This warning is also produced by the existing xwidget code in Emacs
> 25.1.  It is harmless, but I did try to remove it and haven’t been
> successful yet, as I’m not very familiar with GTK.

  Yes, this error message was reported with a very simple recipe as
  Bug#24343.

  Perhaps someone like Martin could figure out a fix?

  Thanks.

[...]

Re: [PATCH v2 00/15] xwidget webkit improvements

[Live System User]

Ricardo Wurmus <rekado@elephly.net> writes:

> Paul Eggert <eggert@cs.ucla.edu> writes:
>
>> Thanks. I tried that on Fedora 24 x86-64, and got the following 
>> diagnostic when using M-x xwidget-webkit-browse-url to visit 
>> http://cs.ucla.edu.
>>
>> (emacs:22840): Gtk-WARNING **: Allocating size to GtkOffscreenWindow 
>> 0xd5b6d0 without calling gtk_widget_get_preferred_width/height(). How 
>> does the code know the size to allocate?
>>
>> Is this something I should worry about?
>
> This warning is also produced by the existing xwidget code in Emacs
> 25.1.  It is harmless, but I did try to remove it and haven’t been
> successful yet, as I’m not very familiar with GTK.

  Yes, this error message was reported with a very simple recipe as
  Bug#24343.

  Perhaps someone like Martin could figure out a fix?

  Thanks.

[...]

Re: [PATCH v2 00/15] xwidget webkit improvements

[joakim]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Ricardo Wurmus wrote:
>
>> This warning is also produced by the existing xwidget code in Emacs
>> 25.1.  It is harmless, but I did try to remove it and haven’t been
>> successful yet, as I’m not very familiar with GTK.
>
> OK, thanks for explaining.

I think the reason for the warning is that the xwidget code works a
little bit differently layout-wise than the expected use case for gtk
widgets. 

>
>> I’ll wait for additional feedback before sending updated patches.
>
> The patches look OK for a feature that is experimental anyway, so I
> took the liberty of installing them into master, with fixes for the
> two minor things. Thanks again.
>
-- 
Joakim Verona
joakim@verona.se

Re: [PATCH v2 00/15] xwidget webkit improvements

[joakim]

Paul Eggert <eggert@cs.ucla.edu> writes:

> Ricardo Wurmus wrote:
>
>> This warning is also produced by the existing xwidget code in Emacs
>> 25.1.  It is harmless, but I did try to remove it and haven’t been
>> successful yet, as I’m not very familiar with GTK.
>
> OK, thanks for explaining.
>
>> I’ll wait for additional feedback before sending updated patches.
>
> The patches look OK for a feature that is experimental anyway, so I
> took the liberty of installing them into master, with fixes for the
> two minor things. Thanks again.
>

Thanks for working on these patches, I'm looking forward to trying them out!
-- 
Joakim Verona
joakim@verona.se