fine grained control of webkit browsing

fine grained control of webkit browsing

[Perry E. Metzger]

I'm playing a bit with xwidgets webkit browsing for the first time
(because I have Veshboo's patches working for myself on MacOS under
the master branch) and I'm wondering a few things about the
architecture:

1) Say one wants to browse the contents of a buffer, is there a way
to do that?
2) Say one wants to do things like viewing something with javascript
off and without loading remote elements (say because one wants to
safely view some HTML email). What does one need to do to get this
to work?

The documentation on this stuff is very sparse, or if there is more
documentation, I haven't been able to find it.

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: fine grained control of webkit browsing

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 938 bytes --]


Perry E. Metzger <perry@piermont.com> writes:

> 2) Say one wants to do things like viewing something with javascript
> off and without loading remote elements (say because one wants to
> safely view some HTML email). What does one need to do to get this
> to work?

The widget is not fully integrated into Emacs.  The extent to which one
can conveniently communicate with the widget is by sending a string
containing a JavaScript program to the widget and have it return a
value.  Everything else requires exposing Webkit API features to Elisp.

Attached is an old patch that I never submitted that allows one to set
webkit settings via Elisp.  Unfortunately, disabling JavaScript for the
widget also disables JavaScript evaluation of strings that are sent from
Emacs – we use JavaScript for telling the widget to scroll, for example.
(See “xwidget-webkit-scroll-bottom” or “xwidget-webkit-show-element”.)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: settings.patch --]
[-- Type: text/x-patch, Size: 7418 bytes --]

commit b1d2a1ad4a489838027e2f897f65190509a3c34f
Author: Ricardo Wurmus <rekado@elephly.net>
Date:   Sat Oct 29 19:08:54 2016 +0200

    webkit set settings

diff --git a/lisp/xwidget.el b/lisp/xwidget.el
index a4214f054a..11dfc061f2 100644
--- a/lisp/xwidget.el
+++ b/lisp/xwidget.el
@@ -39,6 +39,8 @@
 (declare-function xwidget-buffer "xwidget.c" (xwidget))
 (declare-function xwidget-size-request "xwidget.c" (xwidget))
 (declare-function xwidget-resize "xwidget.c" (xwidget new-width new-height))
+(declare-function xwidget-webkit-set-settings "xwidget.c"
+                  (xwidget settings))
 (declare-function xwidget-webkit-execute-script "xwidget.c"
                   (xwidget script &optional callback))
 (declare-function xwidget-webkit-goto-uri "xwidget.c" (xwidget uri))
diff --git a/src/xwidget.c b/src/xwidget.c
index 4fa906a9f0..fd0ca3edf1 100644
--- a/src/xwidget.c
+++ b/src/xwidget.c
@@ -751,6 +751,84 @@ argument procedure FUN.*/)
   return Qnil;
 }
 
+DEFUN ("xwidget-webkit-set-settings",
+       Fxwidget_webkit_set_settings, Sxwidget_webkit_set_settings,
+       2, 2, 0,
+       doc: /* Set the settings of the Webkit XWIDGET view to the
+values specified in ALIST.*/)
+  (Lisp_Object xwidget, Lisp_Object alist)
+{
+  WEBKIT_FN_INIT ();
+  CHECK_LIST (alist);
+
+  WebKitSettings *settings = webkit_web_view_get_settings (WEBKIT_WEB_VIEW (xw->widget_osr));
+
+  for (int i = 0; CONSP (alist); alist = XCDR (alist))
+    {
+      Lisp_Object elt, key, val;
+
+      elt = XCAR (alist);
+      key = Fcar (elt);
+      CHECK_SYMBOL (key);
+      val = Fcdr (elt);
+
+      // TODO: use a macro instead?
+      if (EQ (key, Qwebkit_allow_file_access_from_file_urls))
+        webkit_settings_set_allow_file_access_from_file_urls (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_allow_modal_dialogs))
+        webkit_settings_set_allow_modal_dialogs (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_allow_file_access_from_file_urls))
+        webkit_settings_set_allow_file_access_from_file_urls (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_allow_modal_dialogs))
+        webkit_settings_set_allow_modal_dialogs (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_allow_universal_access_from_file_urls))
+        webkit_settings_set_allow_universal_access_from_file_urls (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_auto_load_images))
+        webkit_settings_set_auto_load_images (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_draw_compositing_indicators))
+        webkit_settings_set_draw_compositing_indicators (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_accelerated_2d_canvas))
+        webkit_settings_set_enable_accelerated_2d_canvas (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_caret_browsing))
+        webkit_settings_set_enable_caret_browsing (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_developer_extras))
+        webkit_settings_set_enable_developer_extras (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_dns_prefetching))
+        webkit_settings_set_enable_dns_prefetching (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_frame_flattening))
+        webkit_settings_set_enable_frame_flattening (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_javascript))
+        webkit_settings_set_enable_javascript (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_private_browsing))
+        webkit_settings_set_enable_private_browsing (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_webaudio))
+        webkit_settings_set_enable_webaudio (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_webgl))
+        webkit_settings_set_enable_webgl (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_write_console_messages_to_stdout))
+        webkit_settings_set_enable_write_console_messages_to_stdout (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_enable_xss_auditor))
+        webkit_settings_set_enable_xss_auditor (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_javascript_can_access_clipboard))
+        webkit_settings_set_javascript_can_access_clipboard (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_javascript_can_open_windows_automatically))
+        webkit_settings_set_javascript_can_open_windows_automatically (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_load_icons_ignoring_image_load_setting))
+        webkit_settings_set_load_icons_ignoring_image_load_setting (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_media_playback_requires_user_gesture))
+        webkit_settings_set_media_playback_requires_user_gesture (settings, !NILP (val));
+      else if (EQ (key, Qwebkit_zoom_text_only))
+        webkit_settings_set_zoom_text_only (settings, !NILP (val));
+      else
+        signal_error ("Invalid webkit setting", key);
+      i++;
+    }
+
+  webkit_web_view_set_settings (WEBKIT_WEB_VIEW (xw->widget_osr), settings);
+  return Qnil;
+}
+
+
 DEFUN ("xwidget-resize", Fxwidget_resize, Sxwidget_resize, 3, 3, 0,
        doc: /* Resize XWIDGET.  NEW_WIDTH, NEW_HEIGHT define the new size.  */ )
   (Lisp_Object xwidget, Lisp_Object new_width, Lisp_Object new_height)
@@ -999,8 +1077,32 @@ syms_of_xwidget (void)
   defsubr (&Sxwidget_webkit_get_uri);
   defsubr (&Sxwidget_webkit_zoom);
   defsubr (&Sxwidget_webkit_execute_script);
+  defsubr (&Sxwidget_webkit_set_settings);
   DEFSYM (Qwebkit, "webkit");
 
+  /* Symbols used for Webkit view settings */
+  DEFSYM (Qwebkit_allow_file_access_from_file_urls, "webkit-allow-file-access-from-file-urls");
+  DEFSYM (Qwebkit_allow_modal_dialogs, "webkit-allow-modal-dialogs");
+  DEFSYM (Qwebkit_allow_universal_access_from_file_urls, "webkit-allow-universal-access-from-file-urls");
+  DEFSYM (Qwebkit_auto_load_images, "webkit-auto-load-images");
+  DEFSYM (Qwebkit_draw_compositing_indicators, "webkit-draw-compositing-indicators");
+  DEFSYM (Qwebkit_enable_accelerated_2d_canvas, "webkit-enable-accelerated-2d-canvas");
+  DEFSYM (Qwebkit_enable_caret_browsing, "webkit-enable-caret-browsing");
+  DEFSYM (Qwebkit_enable_developer_extras, "webkit-enable-developer-extras");
+  DEFSYM (Qwebkit_enable_dns_prefetching, "webkit-enable-dns-prefetching");
+  DEFSYM (Qwebkit_enable_frame_flattening, "webkit-enable-frame-flattening");
+  DEFSYM (Qwebkit_enable_javascript, "webkit-enable-javascript");
+  DEFSYM (Qwebkit_enable_private_browsing, "webkit-enable-private-browsing");
+  DEFSYM (Qwebkit_enable_webaudio, "webkit-enable-webaudio");
+  DEFSYM (Qwebkit_enable_webgl, "webkit-enable-webgl");
+  DEFSYM (Qwebkit_enable_write_console_messages_to_stdout, "webkit-enable-write-console-messages-to-stdout");
+  DEFSYM (Qwebkit_enable_xss_auditor, "webkit-enable-xss-auditor");
+  DEFSYM (Qwebkit_javascript_can_access_clipboard, "webkit-javascript-can-access-clipboard");
+  DEFSYM (Qwebkit_javascript_can_open_windows_automatically, "webkit-javascript-can-open-windows-automatically");
+  DEFSYM (Qwebkit_load_icons_ignoring_image_load_setting, "webkit-load-icons-ignoring-image-load-setting");
+  DEFSYM (Qwebkit_media_playback_requires_user_gesture, "webkit-media-playback-requires-user-gesture");
+  DEFSYM (Qwebkit_zoom_text_only, "webkit-zoom-text-only");
+
   defsubr (&Sxwidget_size_request);
   defsubr (&Sdelete_xwidget_view);
 

[-- Attachment #3: Type: text/plain, Size: 12 bytes --]


--
Ricardo

Re: fine grained control of webkit browsing

[Perry E. Metzger]

On Tue, 19 Jun 2018 18:19:07 +0200 Ricardo Wurmus
<rekado@elephly.net> wrote:
> Perry E. Metzger <perry@piermont.com> writes:
>
> > 2) Say one wants to do things like viewing something with
> > javascript off and without loading remote elements (say because
> > one wants to safely view some HTML email). What does one need to
> > do to get this to work?
>
> The widget is not fully integrated into Emacs.

It would be neat if it were. One would like, for example,
to be able to move a cursor around in it with keyboard commands,
copy text, and move back to another buffer and paste it without
touching the mouse.

> The extent to which one can conveniently communicate with the widget
> is by sending a string containing a JavaScript program to the widget
> and have it return a value.  Everything else requires exposing
> Webkit API features to Elisp.

Ah, so at the moment, this isn't yet at the point where (for example)
one could safely use it to view HTML email. (To do that, among other
things, one needs to shut off JavaScript and to allow programmatic
control over which images are loaded.)

> Attached is an old patch that I never submitted that allows one to
> set webkit settings via Elisp.  Unfortunately, disabling JavaScript
> for the widget also disables JavaScript evaluation of strings that
> are sent from Emacs -- we use JavaScript for telling the widget to
> scroll, for example. (See ___xwidget-webkit-scroll-bottom___ or
> ___xwidget-webkit-show-element___.)

So it would be necessary to tighten the integration a lot more in
order to make this work as cleanly as one might like?

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: fine grained control of webkit browsing

[joakim]

"Perry E. Metzger" <perry@piermont.com> writes:

> On Tue, 19 Jun 2018 18:19:07 +0200 Ricardo Wurmus
> <rekado@elephly.net> wrote:
>> Perry E. Metzger <perry@piermont.com> writes:
>>
>> > 2) Say one wants to do things like viewing something with
>> > javascript off and without loading remote elements (say because
>> > one wants to safely view some HTML email). What does one need to
>> > do to get this to work?
>>
>> The widget is not fully integrated into Emacs.
>
> It would be neat if it were. One would like, for example,
> to be able to move a cursor around in it with keyboard commands,
> copy text, and move back to another buffer and paste it without
> touching the mouse.
>
>> The extent to which one can conveniently communicate with the widget
>> is by sending a string containing a JavaScript program to the widget
>> and have it return a value.  Everything else requires exposing
>> Webkit API features to Elisp.
>
> Ah, so at the moment, this isn't yet at the point where (for example)
> one could safely use it to view HTML email. (To do that, among other
> things, one needs to shut off JavaScript and to allow programmatic
> control over which images are loaded.)
>
>> Attached is an old patch that I never submitted that allows one to
>> set webkit settings via Elisp.  Unfortunately, disabling JavaScript
>> for the widget also disables JavaScript evaluation of strings that
>> are sent from Emacs -- we use JavaScript for telling the widget to
>> scroll, for example. (See ___xwidget-webkit-scroll-bottom___ or
>> ___xwidget-webkit-show-element___.)
>
> So it would be necessary to tighten the integration a lot more in
> order to make this work as cleanly as one might like?

The original idea was to expose the entire webkit api to emacs using
gobject introspection. There is an initial attempt to do this in one of
the xwidget branches. That work is stalled. I think it would be a
nice project for some enterprising person though.

>
> Perry
-- 
Joakim Verona
joakim@verona.se
+46705459454

Re: fine grained control of webkit browsing

[Perry E. Metzger]

On Tue, 19 Jun 2018 20:58:09 +0200 joakim@verona.se wrote:
> The original idea was to expose the entire webkit api to emacs using
> gobject introspection. There is an initial attempt to do this in
> one of the xwidget branches. That work is stalled. I think it would
> be a nice project for some enterprising person though.

Where is the branch located?

-- 
Perry E. Metzger		perry@piermont.com

Re: fine grained control of webkit browsing

[joakim]

"Perry E. Metzger" <perry@piermont.com> writes:

> On Tue, 19 Jun 2018 20:58:09 +0200 joakim@verona.se wrote:
>> The original idea was to expose the entire webkit api to emacs using
>> gobject introspection. There is an initial attempt to do this in
>> one of the xwidget branches. That work is stalled. I think it would
>> be a nice project for some enterprising person though.
>
> Where is the branch located?

http://git.savannah.gnu.org/cgit/emacs.git/log/?h=xwidget

Xwidget was the original branch that was later stripped down to
xwidget_mvp. xwidget_mvp was the branch that was merged into master.

Xwidget had some other widgets besides the webkit one, and also
embryonic gobject introspection support.

-- 
Joakim Verona
joakim@verona.se

Re: fine grained control of webkit browsing

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The widget is not fully integrated into Emacs.  The extent to which one
  > can conveniently communicate with the widget is by sending a string
  > containing a JavaScript program to the widget and have it return a
  > value.

In what computer does that JavaScript program get executed?
Why use JavaScript here?

-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: fine grained control of webkit browsing

[T.V Raman]

Another possible way of communicating with the browser   might be via
the debugger API e.g. port 9222 on Chrome. Package indium already lets
you do a lot of this and using that  as the integration mechanism might
feel fairly seamless.
-- 

Re: fine grained control of webkit browsing

[Perry E. Metzger]

On Tue, 19 Jun 2018 18:57:08 -0400 Richard Stallman <rms@gnu.org>
wrote:
>   > The widget is not fully integrated into Emacs.  The extent to
>   > which one can conveniently communicate with the widget is by
>   > sending a string containing a JavaScript program to the widget
>   > and have it return a value.  
> 
> In what computer does that JavaScript program get executed?

Inside the same Emacs process on the same machine. WebKit is linked
in to Emacs and controls a window. JavaScript is being injected by
elisp into the WebKit instance to control it.

> Why use JavaScript here?

In the absence of having the full WebKit API exposed to the rest of
Emacs, it's the only obvious way to tell WebKit to do things like
scrolling the window. Yes, it's not really what you want, which is
what we are discussing. Ideally, the entirety of the WebKit API would
be exposed to the elisp layer and one could much more tightly
integrate the browser into the rest of the editor.

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: fine grained control of webkit browsing

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > In what computer does that JavaScript program get executed?

  > Inside the same Emacs process on the same machine. WebKit is linked
  > in to Emacs and controls a window. JavaScript is being injected by
  > elisp into the WebKit instance to control it.

  > > Why use JavaScript here?

Now that I see the architecture, I conclude that using JavaScript this way
is not an ethical problem.  It's free code in a program the user installs, so
ethically it's as good as any other free code in a program the user installs.

We could perhaps expose the interface to Lisp by defining Lisp functions
that work by sending JavaScript code.  Somewhat inelegant in implementation,
but that might not matter to Lisp programs that use these functions.


-- 
Dr Richard Stallman
President, Free Software Foundation (https://gnu.org, https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: fine grained control of webkit browsing

[Perry E. Metzger]

On Wed, 20 Jun 2018 19:26:40 -0400 Richard Stallman <rms@gnu.org>
wrote:
> Now that I see the architecture, I conclude that using JavaScript
> this way is not an ethical problem.  It's free code in a program
> the user installs, so ethically it's as good as any other free code
> in a program the user installs.
> 
> We could perhaps expose the interface to Lisp by defining Lisp
> functions that work by sending JavaScript code.  Somewhat inelegant
> in implementation, but that might not matter to Lisp programs that
> use these functions.

The superior method is to expose the C/C++ API to WebKit to elisp.
However, no one has yet done the work to make this happen.

We do not want to use the JavaScript method you suggest long term
because system safety requires that JavaScript be turned off when
browsing certain content, such as email. If we depend on JavaScript
under such circumstances, we will cease to be able to control the
window and view untrusted content safely at the same time.

For the moment, however, it works okayish.

Perry
-- 
Perry E. Metzger		perry@piermont.com