Browsers and Gstreamer plugins

Browsers and Gstreamer plugins

[Jack Hill]

[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]

Hi Guix,

While we have made progress on #52375 [0], the way forward remains 
unclear. In summary, WebKitGTK expects certain GStreamer plugins to be 
available. Depending on which plugins are missing and the web page 
content, the process corresponding to a browser tab may even crash. 
Currently, we expect folks to manually install the necessary plugins into 
their profile/environment.

Complicating matters, it is not clear to me which plugins WebKitGTK needs 
or optionally makes use of. At least some of the needed plugins are from 
gst-plugins-bad, which upstream considers to be of lesser (code) quality 
[1]. gst-plugins-bad is also a large dependency. Adding it would increase 
the closure size of browsers by almost 1GiB!

[0] https://issues.guix.gnu.org/52375
[1] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/blob/ca8068c6d793d7aaa6f2e2cc6324fdedfe2f33fa/README#L80

I believe that what I have written above is more or less factual. Now for 
my opinion: I think that we should make browsers work out of the box on 
commonly-encountered web content. To that end, I propose that we wrap 
WebKitGTK-powered browsers so that they can find the necessary plugins. I 
have attached a proof-of-concept patch that does just that for Vimb. I 
used the gst-plugins/selection procedure [2] to to add just one plugin 
from gst-plugins-bad that fixed the crash I was seeing in #52374.

Size comparisons:

Existing Vimb: 1397.5 MiB
Vimb with my patch: 1409.9 MiB
Vimb with all of gst-plugins-bad: 2298.6 MiB

Of course this is just the bare-minimum set of plugins. We may want to 
work with WebKitGTK upstream to determine any additional ones that we 
should be including.

[2] The patch depends on the fix for gst-plugins/selection that I 
submitted in https://issues.guix.gnu.org/52730

A note on the approach of wrapping browsers rather than somehow including 
the plugins in WebKitGTK. It is much more obvious and straight forward (to 
me at least) to wrap the browser executable. Also WebKitGTK could in 
theory be used to render content that comes from a controlled environment, 
not the web, and therefore doesn't need the "web plugins". A downside to 
doing it this way is that each browser would need to be wrapped in the 
same set of plugins. Perhaps we can factor that out so that the plugin 
list only has to be maintained in one place.

Questions and comments? How shall we move forward? Is it ok to wrap 
browsers in this way?

Best,
Jack

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-diff; name=0001-gnu-vimb-Wrap-with-required-gstreamer-plugins.patch, Size: 2951 bytes --]

From 44971b5c5c2d355ed6b270810ee7fcb16b106edb Mon Sep 17 00:00:00 2001
From: Jack Hill <jackhill@jackhill.us>
Date: Wed, 22 Dec 2021 11:07:36 -0500
Subject: [PATCH] gnu: vimb: Wrap with required gstreamer plugins.

* gnu/package/web-browsers.scm (vimb)[make-flags]: Swith to using gexps.
[phases]: Add 'wrap-gst-plugins.
[inputs]: Remove input labels and add debugutils from gst-plugins-bad.
---
 gnu/packages/web-browsers.scm | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/gnu/packages/web-browsers.scm b/gnu/packages/web-browsers.scm
index 1a56d9a933..26151fb56f 100644
--- a/gnu/packages/web-browsers.scm
+++ b/gnu/packages/web-browsers.scm
@@ -44,6 +44,7 @@ (define-module (gnu packages web-browsers)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix download)
+  #:use-module (guix gexp)
   #:use-module (guix git-download)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix packages)
@@ -68,6 +69,7 @@ (define-module (gnu packages web-browsers)
   #:use-module (gnu packages gnome)
   #:use-module (gnu packages gnome-xyz)
   #:use-module (gnu packages gnupg)
+  #:use-module (gnu packages gstreamer)
   #:use-module (gnu packages gtk)
   #:use-module (gnu packages image)
   #:use-module (gnu packages libevent)
@@ -551,17 +553,24 @@ (define-public vimb
        (file-name (git-file-name name version))))
     (build-system glib-or-gtk-build-system)
     (arguments
-     '(#:tests? #f                      ; no tests
-       #:make-flags (list "CC=gcc"
-                          "DESTDIR="
-                          (string-append "PREFIX=" %output))
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+     (list #:tests? #f                      ; no tests
+           #:make-flags #~(list "CC=gcc"
+                                "DESTDIR="
+                                (string-append "PREFIX=" #$output))
+           #:phases
+           #~(modify-phases %standard-phases
+               (delete 'configure)
+               (add-after 'glib-or-gtk-wrap 'wrap-gst-plugins
+                 (lambda _
+                   (wrap-program (string-append #$output "/bin/vimb")
+                     `("GST_PLUGIN_SYSTEM_PATH" suffix
+                       (,(getenv "GST_PLUGIN_SYSTEM_PATH")))))))))
     (inputs
-     `(("glib-networking" ,glib-networking)
-       ("gsettings-desktop-schemas" ,gsettings-desktop-schemas)
-       ("webkitgtk" ,webkitgtk-with-libsoup2)))
+     (list glib-networking
+           gsettings-desktop-schemas
+           (gst-plugins/selection gst-plugins-bad
+                                  #:plugins '("debugutils"))
+           webkitgtk-with-libsoup2))
     (native-inputs
      (list pkg-config))
     (home-page "https://fanglingsu.github.io/vimb/")
-- 
2.34.0

Re: Browsers and Gstreamer plugins

[Maxim Cournoyer]

Hello Jack,

Thank you for posting this well written piece here.

Jack Hill <jackhill@jackhill.us> writes:

> Hi Guix,
>
> While we have made progress on #52375 [0], the way forward remains
> unclear. In summary, WebKitGTK expects certain GStreamer plugins to be 
> available. Depending on which plugins are missing and the web page
> content, the process corresponding to a browser tab may even crash. 
> Currently, we expect folks to manually install the necessary plugins
> into their profile/environment.

That sucks; but it really is webkitgtk/browsers' fault; they should have
better reporting so that users know what is missing instead of obtusely
crashing a tab.

> Complicating matters, it is not clear to me which plugins WebKitGTK
> needs or optionally makes use of. At least some of the needed plugins
> are from gst-plugins-bad, which upstream considers to be of lesser
> (code) quality [1]. gst-plugins-bad is also a large dependency. Adding
> it would increase the closure size of browsers by almost 1GiB!
>
> [0] https://issues.guix.gnu.org/52375
> [1]
> https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/blob/ca8068c6d793d7aaa6f2e2cc6324fdedfe2f33fa/README#L80

1 GiB for code deemed of even worst quality than even the 'ugly' plugins
(they're literally at the bottom of the chart, quality-wise) doesn't
sound too appealing :-) [0].

[0]  https://raw.githubusercontent.com/GStreamer/gstreamer/master/README

> I believe that what I have written above is more or less factual. Now
> for my opinion: I think that we should make browsers work out of the
> box on commonly-encountered web content.

How would you define "commonly-encountered" web content?  How will we
handle bug reports of "tab crashed on site X" because a plugin we
thought obscure is used by someone?  It's a bit of a slippery slope
where we may end up wrapping all/too many plugins, because of the issue
I raised in my preceding paragraph (bad reporting from webkitgtk
itself).

Plugins exist for the very reason to allow end users to extend
functionalities the way they see fit, so it seems a bit backward to me
to "wrap plugins in", which makes them non-optional onto users.  On
other systems, they are typically "recommended" but not wrapped in a way
that makes it difficult for users to opt out of them.

> To that end, I propose that
> we wrap WebKitGTK-powered browsers so that they can find the necessary
> plugins. I have attached a proof-of-concept patch that does just that
> for Vimb. I used the gst-plugins/selection procedure [2] to to add
> just one plugin from gst-plugins-bad that fixed the crash I was seeing
> in #52374.
>
> Size comparisons:
>
> Existing Vimb: 1397.5 MiB
> Vimb with my patch: 1409.9 MiB
> Vimb with all of gst-plugins-bad: 2298.6 MiB

This looks reasonable, space wise.

> Of course this is just the bare-minimum set of plugins. We may want to
> work with WebKitGTK upstream to determine any additional ones that we 
> should be including.
>
> [2] The patch depends on the fix for gst-plugins/selection that I
> submitted in https://issues.guix.gnu.org/52730
>
> A note on the approach of wrapping browsers rather than somehow
> including the plugins in WebKitGTK. It is much more obvious and
> straight forward (to me at least) to wrap the browser executable. Also
> WebKitGTK could in theory be used to render content that comes from a
> controlled environment, not the web, and therefore doesn't need the
> "web plugins". A downside to doing it this way is that each browser
> would need to be wrapped in the same set of plugins. Perhaps we can
> factor that out so that the plugin list only has to be maintained in
> one place.
>
>
> Questions and comments? How shall we move forward? Is it ok to wrap
> browsers in this way?

I sympathize with the goal of improving our users experience (by
shipping browsers that don't crash mysteriously left and right), but I'm
not satisfied with the solution of wrapping plugins.  Could we instead
try to fast-forward work that has happened in webkitgtk to produce
better diagnostics about missing plugins?  Such as [1], which has a
patch (with comments on).  It's not a panacea, but having errors about
missing plugins/functionality appear on stderr would be an improvement.
We should also create tickets upstream in webkitgtk-based browsers
requesting that they report missing plugins gracefully in their user
interfaces.

[1]  https://bugs.webkit.org/show_bug.cgi?id=233949

If all this is done and documented and in the waiting queue, then we
could proceed with wrap browsers with a minimal set of plugins as a
stopgap/temporary measure until the upstream issues get resolved.

What do you think?

Thanks,

Maxim