From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marius Bakke Subject: Re: qtwenengine anybody? Date: Mon, 06 Jan 2020 20:23:52 +0100 Message-ID: <87d0bw2xtj.fsf@devup.no> References: <87a77sqtdq.fsf@ambrevar.xyz> <87immgqryj.fsf@gmail.com> <871rt4qnp9.fsf@gmail.com> <87immgp582.fsf@gmail.com> <5955729d-ef57-ffb8-feb8-36a9f53658a2@crazy-compilers.com> <87d0clpbt6.fsf@gmail.com> <87y2v7e0g4.fsf@ambrevar.xyz> <87v9qb56gg.fsf@gmail.com> <87sglfotvj.fsf@ambrevar.xyz> <87o8w354m5.fsf@gmail.com> <87blrod2so.fsf@ambrevar.xyz> <878smsd2pe.fsf@ambrevar.xyz> <87mub7krlq.fsf@gmail.com> <87tv5e9acb.fsf@ambrevar.xyz> <87o8vmn6an.fsf@elephly.net> <87zhf2c5jy.fsf@gmail.com> <87lfql7k9w.fsf@elephly.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:43583) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ioXyr-0004QZ-M2 for guix-devel@gnu.org; Mon, 06 Jan 2020 14:24:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ioXyn-0006bs-B5 for guix-devel@gnu.org; Mon, 06 Jan 2020 14:24:01 -0500 In-Reply-To: <87lfql7k9w.fsf@elephly.net> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus , mike.rosset@gmail.com Cc: guix-devel@gnu.org, GNU Guix maintainers --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ricardo Wurmus writes: > Hi Mike, > > thank you for your patient reply. > >> Ricardo Wurmus writes: >> >>> Does the code include DRM support, for example? Does it phone home? >>> Can we see the patches that upstream applied to the Chromium sources? >>> >> >> QtWebengine does not handle DRM at all. That is handled by a 3rd party >> plugin called wildevine and would require guix to bundle that plugin. >> Something I'm confident is not going to happen. As an added measure my >> proposed qtwebengine declaration has pepper plugins disabled making it >> impossible to load 3rd party plugins either way. >> >> see https://doc.qt.io/qt-5/qtwebengine-features.html#html5-drm >> for reference > > I=E2=80=99m aware of this. My apologies for not expressing myself more > clearly. We probably want to remove the support for the DRM plugin, > following the discussions about ungoogled-chromium at > > https://issues.guix.gnu.org/issue/34565 I have attached a patch that disables support for loading the Widevine plugin, and also removes unused bundled software from the source based on a whitelist, similar to ungoogled-chromium's %PRESERVED-THIRD-PARTY-FILES. It removes some 300+ MiB from the uncompressed sources. The snippet will require some adjustments for every new version of QtWebEngine however, I can help out with that when required. Patch 1/2 is a slightly "cleaned up" version of Mike's latest patch, fixing some cosmetic issues, and makes it respect the daemons requested number of build cores. With these changes, I think the package is ready to go. \o/ --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-gnu-Add-qtwebengine.patch Content-Transfer-Encoding: quoted-printable From=203e373d54e08c95db16d53110051e008e6d08d55b Mon Sep 17 00:00:00 2001 From: Mike Rosset Date: Wed, 1 Jan 2020 08:03:52 -0800 Subject: [PATCH 1/2] gnu: Add qtwebengine. * gnu/packages/qt.scm (qtwebengine): New variable. Signed-off-by: Marius Bakke =2D-- gnu/packages/qt.scm | 144 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm index 795b5e9d2b..d6ceb5b90f 100644 =2D-- a/gnu/packages/qt.scm +++ b/gnu/packages/qt.scm @@ -13,6 +13,7 @@ ;;; Copyright =C2=A9 2018 Eric Bavier ;;; Copyright =C2=A9 2019 Marius Bakke ;;; Copyright =C2=A9 2018 John Soo +;;; Copyright =C2=A9 2020 Mike Rosset ;;; ;;; This file is part of GNU Guix. ;;; @@ -52,30 +53,40 @@ #:use-module (gnu packages flex) #:use-module (gnu packages freedesktop) #:use-module (gnu packages gcc) + #:use-module (gnu packages ghostscript) #:use-module (gnu packages gl) #:use-module (gnu packages glib) + #:use-module (gnu packages gnupg) #:use-module (gnu packages gperf) #:use-module (gnu packages gstreamer) #:use-module (gnu packages gtk) #:use-module (gnu packages icu4c) #:use-module (gnu packages image) + #:use-module (gnu packages libevent) #:use-module (gnu packages linux) #:use-module (gnu packages llvm) #:use-module (gnu packages maths) + #:use-module (gnu packages ninja) #:use-module (gnu packages nss) #:use-module (gnu packages pciutils) #:use-module (gnu packages pcre) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages pulseaudio) + #:use-module (gnu packages protobuf) #:use-module (gnu packages python) #:use-module (gnu packages python-xyz) + #:use-module (gnu packages regex) #:use-module (gnu packages ruby) #:use-module (gnu packages sdl) + #:use-module (gnu packages serialization) #:use-module (gnu packages sqlite) + #:use-module (gnu packages telephony) #:use-module (gnu packages tls) + #:use-module (gnu packages video) #:use-module (gnu packages vulkan) #:use-module (gnu packages xdisorg) + #:use-module (gnu packages xiph) #:use-module (gnu packages xorg) #:use-module (gnu packages xml) #:use-module (srfi srfi-1)) @@ -2020,6 +2031,139 @@ time Web content can be enhanced with native contro= ls.") =20 (license license:lgpl2.1+))) =20 +(define-public qtwebengine + (package + (inherit qtsvg) + (name "qtwebengine") + (version (package-version qtbase)) + (source (origin + (method url-fetch) + (uri (string-append "https://download.qt.io/official_release= s/qt/" + (version-major+minor version) "/" version + "/submodules/" name "-everywhere-src-" + version ".tar.xz")) + (sha256 + (base32 + "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa")))) + (build-system gnu-build-system) + (native-inputs + `(("bison" ,bison) + ("flex" ,flex) + ("gperf" ,gperf) + ("ninja" ,ninja) + ("perl" ,perl) + ("pkg-config" ,pkg-config) + ("python-2" ,python-2) + ("ruby" ,ruby))) + (inputs + `(("alsa-lib" ,alsa-lib) + ("atk" ,atk) + ("cups-minimal" ,cups-minimal) + ("dbus" ,dbus) + ("ffmpeg" ,ffmpeg) + ("fontconfig" ,fontconfig) + ("harbuzz" ,harfbuzz) + ("icu4c" ,icu4c) + ("jsoncpp" ,jsoncpp) + ("lcms" ,lcms) + ("libcap" ,libcap) + ("libevent" ,libevent) + ("libgcrypt" ,libgcrypt) + ("libjpeg" ,libjpeg-turbo) + ("libvpx" ,libvpx) + ;; FIXME: configure does not find system lcms + ;; ("lcms" ,lcms) + ("libwebp" ,libwebp) + ("libx11" ,libx11) + ("libxcb" ,libxcb) + ("libxcomposite" ,libxcomposite) + ("libxcursor" ,libxcursor) + ("libxi" ,libxi) + ("libxkbcommon" ,libxkbcommon) + ;; FIXME: libxml2 needs to built with icu support though it links to + ;; libxml2 configure summary still states. Checking for compatible + ;; system libxml2... no + ("libxml2" ,libxml2) + ("libxrandr" ,libxrandr) + ("libxrender" ,libxrender) + ("libxslt" ,libxslt) + ("libxtst" ,libxtst) + ("mesa" ,mesa) + ("minizip" ,minizip) + ("nss" ,nss) + ("opus" ,opus) + ("pciutils" ,pciutils) + ("protobuf" ,protobuf) + ("pulseaudio" ,pulseaudio) + ("qtbase" ,qtbase) + ("qtdeclarative" ,qtdeclarative) + ("qtmultimedia" ,qtmultimedia) + ("qtwebchannel" ,qtwebchannel) + ("re2" ,re2) + ("snappy" ,snappy) + ("udev" ,eudev) + ("xcb-util" ,xcb-util))) + (arguments + (substitute-keyword-arguments (package-arguments qtsvg) + ((#:phases phases) + `(modify-phases ,phases + (add-before 'configure 'substitute-source + (lambda* (#:key inputs outputs #:allow-other-keys) + (let ((out (assoc-ref outputs "out")) + (nss (assoc-ref inputs "nss")) + (udev (assoc-ref inputs "udev"))) + ;; Qtwebengine is not installed into the same prefix as + ;; qtbase. Some qtbase QTLibraryInfo constants will not + ;; work. Replace with the full path to the qtwebengine + ;; translations and locales in the store. + (substitute* "src/core/web_engine_library_info.cpp" + (("QLibraryInfo::location\\(QLibraryInfo::TranslationsP= ath\\)") + (string-append "QLatin1String(\"" out "/share/qt5/tran= slations\")")) + (("QLibraryInfo::location\\(QLibraryInfo::DataPath\\)") + (string-append "QLatin1String(\"" out "/share/qt5\")")= )) + ;; Substitute full dynamic library path for nss + (substitute* "src/3rdparty/chromium/crypto/nss_util.cc" + (("libnssckbi.so") + (string-append nss "/lib/nss/libnssckbi.so"))) + ;; Substitute full dynamic library path for udev + (substitute* "src/3rdparty/chromium/device/udev_linux/ude= v1_loader.cc" + (("libudev.so.1") + (string-append udev "/lib/libudev.so.1"))) + #t))) + (add-before 'configure 'set-env + (lambda _ + ;; Avoids potential race conditions + (setenv "PYTHONDONTWRITEBYTECODE" "1") + (setenv "NINJAFLAGS" + (string-append "-k1" ;less verbose build output + ;; Respect the '--cores' option of '= guix build'. + " -j" (number->string (parallel-job-= count)))) + #t)) + (replace 'configure + (lambda _ + ;; Valid QT_BUILD_PARTS variables are: + ;; libs tools tests examples demos docs translations + (invoke "qmake" "QT_BUILD_PARTS =3D libs tools" "--" + "--webengine-printing-and-pdf=3Dno" + "--webengine-ffmpeg=3Dsystem" + "--webengine-icu=3Dsystem" + "--webengine-pepper-plugins=3Dno"))))) + ;; Tests are disabled due to Could not find QtWebEngineProcess error + ;; It's possible this can be fixed by setting QTWEBENGINEPROCESS_PA= TH + ;; before running tests It's also possible some tests need a network + ((#:tests? _ #f) #f))) + (native-search-paths + (list (search-path-specification + (file-type 'regular) + (separator #f) + (variable "QTWEBENGINEPROCESS_PATH") + (files '("lib/qt5/libexec/QtWebEngineProcess"))))) + (home-page "https://www.qt.io") + (synopsis "Qt WebEngine module") + (description "Qt5WebEngine provides support for web +applications using the Chromium browser project.") + (license license:lgpl2.1+))) + (define-public dotherside (package (name "dotherside") =2D-=20 2.24.1 --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0002-gnu-qtwebengine-Purge-unused-bundled-software-from-t.patch Content-Transfer-Encoding: quoted-printable From=20487ec682c64fea85ff48ab44d6d918c388f43565 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Fri, 3 Jan 2020 19:32:52 +0100 Subject: [PATCH 2/2] gnu: qtwebengine: Purge unused bundled software from t= he source. * gnu/packages/qt.scm (qtwebengine)[source](modules, snippet): New fields. [inputs]: Add CURL and VALGRIND. =2D-- gnu/packages/qt.scm | 200 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 190 insertions(+), 10 deletions(-) diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm index d6ceb5b90f..38a620c90c 100644 =2D-- a/gnu/packages/qt.scm +++ b/gnu/packages/qt.scm @@ -11,7 +11,7 @@ ;;; Copyright =C2=A9 2018 Nicolas Goaziou ;;; Copyright =C2=A9 2018 Hartmut Goebel ;;; Copyright =C2=A9 2018 Eric Bavier =2D;;; Copyright =C2=A9 2019 Marius Bakke +;;; Copyright =C2=A9 2019, 2020 Marius Bakke ;;; Copyright =C2=A9 2018 John Soo ;;; Copyright =C2=A9 2020 Mike Rosset ;;; @@ -47,6 +47,7 @@ #:use-module (gnu packages cmake) #:use-module (gnu packages compression) #:use-module (gnu packages cups) + #:use-module (gnu packages curl) #:use-module (gnu packages databases) #:use-module (gnu packages documentation) #:use-module (gnu packages fontutils) @@ -83,6 +84,7 @@ #:use-module (gnu packages sqlite) #:use-module (gnu packages telephony) #:use-module (gnu packages tls) + #:use-module (gnu packages valgrind) #:use-module (gnu packages video) #:use-module (gnu packages vulkan) #:use-module (gnu packages xdisorg) @@ -2036,15 +2038,191 @@ time Web content can be enhanced with native contr= ols.") (inherit qtsvg) (name "qtwebengine") (version (package-version qtbase)) =2D (source (origin =2D (method url-fetch) =2D (uri (string-append "https://download.qt.io/official_relea= ses/qt/" =2D (version-major+minor version) "/" vers= ion =2D "/submodules/" name "-everywhere-src-" =2D version ".tar.xz")) =2D (sha256 =2D (base32 =2D "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))= )) + (source + (origin + (method url-fetch) + (uri (string-append "https://download.qt.io/official_releases/qt/" + (version-major+minor version) "/" version + "/submodules/" name "-everywhere-src-" + version ".tar.xz")) + (sha256 + (base32 + "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa")) + (modules '((ice-9 ftw) + (ice-9 match) + (srfi srfi-1) + (srfi srfi-26) + (guix build utils))) + (snippet + '(begin + (let ((preserved-third-party-files + '("base/third_party/dmg_fp" + "base/third_party/dynamic_annotations" + "base/third_party/icu" + "base/third_party/libevent" + "base/third_party/nspr" + "base/third_party/superfasthash" + "base/third_party/symbolize" + "base/third_party/xdg_mime" + "base/third_party/xdg_user_dirs" + "net/third_party/http2" + "net/third_party/mozilla_security_manager" + "net/third_party/nss" + "net/third_party/spdy" + "net/third_party/quic" + "third_party/abseil-cpp" + "third_party/angle" + "third_party/angle/src/common/third_party/base" + "third_party/angle/src/common/third_party/smhasher" + "third_party/angle/src/third_party/compiler" + "third_party/blink" + "third_party/boringssl" + "third_party/boringssl/src/third_party/fiat" + "third_party/breakpad" + "third_party/brotli" + "third_party/ced" + "third_party/crc32c" + "third_party/ffmpeg" + "third_party/googletest" + "third_party/hunspell" + "third_party/iccjpeg" + "third_party/icu" + "third_party/inspector_protocol" + "third_party/jinja2" + "third_party/jstemplate" + "third_party/khronos" + "third_party/leveldatabase" + "third_party/libaddressinput" + "third_party/libjingle_xmpp" + "third_party/libpng" + "third_party/libsrtp" + "third_party/libsync" + "third_party/libudev" + "third_party/libvpx" + "third_party/libwebm" + "third_party/libwebp" + "third_party/libxml" + "third_party/libxslt" + "third_party/libyuv" + "third_party/lss" + "third_party/markupsafe" + "third_party/mesa" + "third_party/metrics_proto" + "third_party/modp_b64" + "third_party/opus" + "third_party/ots" + "third_party/ply" + "third_party/polymer" + "third_party/protobuf" + "third_party/pyjson5" + "third_party/re2" + "third_party/rnnoise" + "third_party/skia" + "third_party/skia/third_party/gif" + "third_party/skia/third_party/skcms" + "third_party/smhasher" + "third_party/snappy" + "third_party/sqlite" + "third_party/usrsctp" + "third_party/web-animations-js" + "third_party/webrtc" + "third_party/webrtc_overrides" + "third_party/widevine/cdm/widevine_cdm_common.h" + "third_party/widevine/cdm/widevine_cdm_version.h" + "third_party/woff2" + "third_party/yasm" + "third_party/zlib" + "url/third_party/mozilla" + "v8/src/third_party/utf8-decoder" + "v8/src/third_party/valgrind" + "v8/third_party/antlr4" + "v8/third_party/inspector_protocol")) + (protected (make-regexp "\\.(gn|gyp)i?$"))) + (define preserved-club + (map (lambda (member) + (string-append "./" member)) + preserved-third-party-files)) + (define (empty? dir) + (equal? (scandir dir) '("." ".."))) + (define (third-party? file) + (if (string-contains file "third_party/") + #t + #f)) + (define (useless? file) + (any (cute string-suffix? <> file) + '(".zip" ".so" ".dll" ".exe" ".jar"))) + (define (parents child) + ;; Return all parent directories of CHILD up to and includi= ng + ;; the closest "third_party". + (let* ((dirs (match (string-split child #\/) + ((dirs ... last) dirs))) + (closest (list-index (lambda (dir) + (string=3D? "third_party" dir= )) + (reverse dirs))) + (delim (- (length dirs) closest))) + (fold (lambda (dir prev) + (cons (string-append (car prev) "/" dir) + prev)) + (list (string-join (list-head dirs delim) "/")) + (list-tail dirs delim)))) + (define (remove-loudly file) + (format #t "deleting ~a...~%" file) + (force-output) + (delete-file file)) + (define (delete-unwanted-files child stat flag base level) + (match flag + ((or 'regular 'symlink 'stale-symlink) + (when (third-party? child) + (unless (or (member child preserved-club) + (any (cute member <> preserved-club) + (parents child)) + (regexp-exec protected child)) + (remove-loudly child))) + (when (and (useless? child) (file-exists? child)) + (remove-loudly child)) + #t) + ('directory-processed + (when (empty? child) + (rmdir child)) + #t) + (_ #t))) + + (with-directory-excursion "src/3rdparty" + ;; TODO: Try removing "gn" too for future versions of qtweb= engine. + (delete-file-recursively "ninja") + + (with-directory-excursion "chromium" + ;; Delete bundled software and binaries that were not exp= licitly + ;; preserved above. + (nftw "." delete-unwanted-files 'depth 'physical) + + ;; Assert that each preserved item is present to catch re= movals. + (for-each (lambda (third-party) + (unless (file-exists? third-party) + (error (format #f "~s does not exist!" thir= d-party)))) + preserved-club) + + ;; Use relative header locations instead of hard coded on= es. + (substitute* + "base/third_party/dynamic_annotations/dynamic_annotat= ions.c" + (("base/third_party/valgrind") "valgrind")) + (substitute* + "third_party/breakpad/breakpad/src/common/linux/libcu= rl_wrapper.h" + (("third_party/curl") "curl")) + + ;; Replace Google Analytics bundle with an empty file and= hope + ;; no one notices. + (mkdir-p "third_party/analytics") + (call-with-output-file + "third_party/analytics/google-analytics-bundle.js" + (lambda (port) + (const #t))))) + + ;; Do not build support for the Widevine DRM. + (substitute* "src/core/config/common.pri" + (("enable_widevine=3Dtrue") + "enable_widevine=3Dfalse")) + #t))))) (build-system gnu-build-system) (native-inputs `(("bison" ,bison) @@ -2059,6 +2237,7 @@ time Web content can be enhanced with native controls= .") `(("alsa-lib" ,alsa-lib) ("atk" ,atk) ("cups-minimal" ,cups-minimal) + ("curl" ,curl) ("dbus" ,dbus) ("ffmpeg" ,ffmpeg) ("fontconfig" ,fontconfig) @@ -2102,6 +2281,7 @@ time Web content can be enhanced with native controls= .") ("re2" ,re2) ("snappy" ,snappy) ("udev" ,eudev) + ("valgrind" ,valgrind) ("xcb-util" ,xcb-util))) (arguments (substitute-keyword-arguments (package-arguments qtsvg) =2D-=20 2.24.1 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAl4TiUgACgkQoqBt8qM6 VPouVAgAwd66VF6KFnT4Ejy0si6UiFFMFKIvsq2Pqy9hLTVkcKdsdwWgEn8x9Dsd VCgd1/5NVwl7LBUNdsmwJ4dbdRGneuRVyJ/ZE0nWPxO0ZZiK9p2BRf+j7H52EltG revblDipqrLARkDEUCVaoSMLUyCN7OSdXp+j4yGnMDMkyOZz8Vy0ZzMztQbEs99E E8va6bwoJ5uoe1+mrb52GbdjTgqUsbryeWQOLhSsOKSOmlzm9VIfpOrlWN1iXw0a Cuc+Lv6DiJu0g0Eg0l9lu0jBWuqEV//02PTQtsWPH6vL0bbv8GHFifAXQcWpc3v1 mM1TXS++yNGb+GhlS0Dv3tx4EbVVug== =zlnL -----END PGP SIGNATURE----- --==-=-=--