qtwenengine anybody?

qtwenengine anybody?

[Hartmut Goebel]

Hi Guix!

Looks like we need to package qtwebengine, as more and more packages
require it.
qtwebengine includes a copy of chromium.

Any volunteers?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

Mike Rosset had it working in a patch that's still waiting on the tracker.
It has at least one issue as far as I get it.  Mike also made an
all-in-one "Qt" package that does not suffer from this issue, a bit like
our texlive mega-package.

I'm all for adding it to master.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Mike Rosset had it working in a patch that's still waiting on the tracker.
> It has at least one issue as far as I get it.  Mike also made an
> all-in-one "Qt" package that does not suffer from this issue, a bit like
> our texlive mega-package.
>
> I'm all for adding it to master.

Thanks for forwarding this to me Pierre. The new monolithic QT packages
with qtwebengine is derived from the now deprecated qt package, which
has since been removed from guix, which is understandable since the
module packages are more ideal.

This new monolithic package stills suffers from the same issue as my
modular qtwebengine. In that there are many in tree third party
dependencies. It's not trivial to simply switch to guix inputs for
these.

This package is an improvement on my module qtwebengine package. Since
it does not suffer from the same locales issues.

Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 212 bytes --]

Mike, can you give some details (maybe one example) on how to replace the
third-party dependencies?  What is holding it back?

I could give it a go if need be.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Mike, can you give some details (maybe one example) on how to replace the
> third-party dependencies?  What is holding it back?
>
> I could give it a go if need be.

I have not tackled this problem yet. I've mainly been focused on get the
package to build. I've fixed the most blaring issue. For example the
package now builds in parallel which improves build times significantly.

In terms of resolving the in tree dependencies. Some can be resolved
using QT configure flags. I don't think I've ported these over from my
module qtwebengine but I will check this when I have time.

The other third party dependencies might need header and gn
substitutions. A good place to start is the ugoogled-chromium package as
to how this is done. Though it might take more work for qtwebengine
because of QT configure and build changes. That's about as far as I got
with looking at this.

Mike

Re: qtwenengine anybody?

[Hartmut Goebel]

Hi,

I'm glad to see there already has been some effort on this. I'll comment
on the issue.

Am 16.12.19 um 15:13 schrieb mike.rosset@gmail.com:
> This new monolithic package stills suffers from the same issue as my
> modular qtwebengine. In that there are many in tree third party
> dependencies. It's not trivial to simply switch to guix inputs for
> these.

IMO we should focus on a stand-alone qtwebengine package to avoid having
packages depending on a *huge* monolith.


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[mike.rosset]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Hi,
>
> I'm glad to see there already has been some effort on this. I'll comment
> on the issue.
>
> Am 16.12.19 um 15:13 schrieb mike.rosset@gmail.com:
>> This new monolithic package stills suffers from the same issue as my
>> modular qtwebengine. In that there are many in tree third party
>> dependencies. It's not trivial to simply switch to guix inputs for
>> these.
>
> IMO we should focus on a stand-alone qtwebengine package to avoid having
> packages depending on a *huge* monolith.

This is the most ideal situation. However the modular qtwebengine makes
some assumptions which are not compatible with guix. For examples it
assumes it will be installed in the same prefix as qtbase. Which is not
possible with guix. The problem is also not readily obvious until you
try to use qtwebengine in a library or a program. Which will causes
locale and translation path issues.

For the nomad browser I had had some hacks to get around this. But those
hacks won't work in all cases. Because it's dependent on arg0
location. Which is not consistent when using an interpreter like guile
or python or if you are using qtwebengine from a dynamic library.
As of yet I have not found a proper way to resolve this.

I believe the monolithic qt package at this time is the shortest route
to having a sane working qtwebengine, until the locale issues can be
resolved.

Mike

Re: qtwenengine anybody?

[Hartmut Goebel]

Am 16.12.19 um 16:45 schrieb mike.rosset@gmail.com:
> The problem is also not readily obvious until you
> try to use qtwebengine in a library or a program. Which will causes
> locale and translation path issues.

If the only issue with qtwebengine are locale and translation, I suggest
to accept these for now. IMHO we need to attract more users (and
developers) for guix, fir which we need more packages.  And missing
qtwebengine blocks some KDE applications form getting build for guix.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[mike.rosset]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 16.12.19 um 16:45 schrieb mike.rosset@gmail.com:
>> The problem is also not readily obvious until you
>> try to use qtwebengine in a library or a program. Which will causes
>> locale and translation path issues.
>
> If the only issue with qtwebengine are locale and translation, I suggest
> to accept these for now. IMHO we need to attract more users (and
> developers) for guix, fir which we need more packages.  And missing
> qtwebengine blocks some KDE applications form getting build for guix.

IMHO the locale, translation would be a blocker still. Because it will
cause programs to fail. It's not just a locale/translation warning.

I'm currently brushing up my latest qtwebengine build. Will resubmit
this patch. I will also see if I can inherit qtsvg again to put this
back on par with other module packages. From there I'll put a simple
test example together demonstrating the locale and translation problem.

Mike

Re: qtwenengine anybody?

[Hartmut Goebel]

Am 16.12.19 um 18:09 schrieb mike.rosset@gmail.com:
> I'm currently brushing up my latest qtwebengine build. Will resubmit
> this patch. I will also see if I can inherit qtsvg again to put this
> back on par with other module packages. From there I'll put a simple
> test example together demonstrating the locale and translation problem.

Sounds good

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[mike.rosset]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 16.12.19 um 18:09 schrieb mike.rosset@gmail.com:
>> I'm currently brushing up my latest qtwebengine build. Will resubmit
>> this patch. I will also see if I can inherit qtsvg again to put this
>> back on par with other module packages. From there I'll put a simple
>> test example together demonstrating the locale and translation problem.
>
> Sounds good

I've resubmitted the patch it can be found here
https://lists.gnu.org/archive/html/guix-patches/2019-12/msg00530.html. My
ML patch workflow is probably not all that correct so I apologize if
there is any issues.

To summarize the package now inherits from qtsvg. The locales issues
have been resolved by substituting qtwebengine's output instead of using
qtbase.

Some dynamically loaded libraries like udev and nss have been
substituted using corresponding inputs.

Some known issues, when building with pulseaudio support. There is some
issues finding pulseaudio socket. I've disabled support for pulseaudio
until I can resolve this.

Additionally on my system when using hardware accelerated OpenGL
neuveau drivers, qtwebengine crashes this does not happen when using the
monolithic qt package. This can avoided by setting the
QT_XCB_FORCE_SOFTWARE_OPENGL to 1.

I have a simple CPP for testing it can be found
here. https://github.com/mrosset/testqt.

for testing I use this enviroment.

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix environment  --ad-hoc qtbase qtwebengine qtdeclarative qtwebchannel coreutils gcc-toolchain nss-certs
--8<---------------cut here---------------end--------------->8---

and then I test with

--8<---------------cut here---------------start------------->8---
cd testqt
qmake
make
./main
--8<---------------cut here---------------end--------------->8---

or there are hardware acceleration problems

--8<---------------cut here---------------start------------->8---
QT_XCB_FORCE_SOFTWARE_OPENGL=1 ./main
--8<---------------cut here---------------end--------------->8---

I can also a substitute if anyone needs them for testing. Since this
build can take some time.


Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 314 bytes --]

Fantastic!

The pulseaudio and OpenGL issues are not blockers in my opinion.  We can
open new issues about them and resolve them later.

> I can also a substitute if anyone needs them for testing. Since this
> build can take some time.

Please do!  Thanks!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Fantastic!
>
> The pulseaudio and OpenGL issues are not blockers in my opinion.  We can
> open new issues about them and resolve them later.
>
>> I can also a substitute if anyone needs them for testing. Since this
>> build can take some time.
>
> Please do!  Thanks!


A substitute is available on https://gx.bufio.org and the public key can
be found
here. http://git.savannah.nongnu.org/cgit/nomad.git/plain/contrib/guix/gx.bufio.org.pub

Mike

Re: qtwenengine anybody?

[Ricardo Wurmus]

mike.rosset@gmail.com writes:

> Hartmut Goebel <h.goebel@crazy-compilers.com> writes:
>
>> Am 16.12.19 um 18:09 schrieb mike.rosset@gmail.com:
>>> I'm currently brushing up my latest qtwebengine build. Will resubmit
>>> this patch. I will also see if I can inherit qtsvg again to put this
>>> back on par with other module packages. From there I'll put a simple
>>> test example together demonstrating the locale and translation problem.
>>
>> Sounds good
>
> I've resubmitted the patch it can be found here
> https://lists.gnu.org/archive/html/guix-patches/2019-12/msg00530.html. My
> ML patch workflow is probably not all that correct so I apologize if
> there is any issues.

For those following along at home: here it is in the bug tracker:

    https://issues.guix.gnu.org/issue/35866

-- 
Ricardo

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 99 bytes --]

The substitution does not apply for me on master.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> The substitution does not apply for me on master.


This should be resolved now. I had to upgrade the vps to avoid OOM
issues. But as I've now it should serve this substitute.

you can test with.

--8<---------------cut here---------------start------------->8---
guix archive --export -r qtwebengine | ssh server guix archive --import
--8<---------------cut here---------------end--------------->8---


If this passes make sure you have
#AF73C7321F90EA89AD29BBA14069EB8EF8F6410325E479F93DE4612F26478726#

in
--8<---------------cut here---------------start------------->8---
/etc/guix/acl
--8<---------------cut here---------------end--------------->8---


Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 8022 bytes --]

mike.rosset@gmail.com writes:

> This should be resolved now. I had to upgrade the vps to avoid OOM
> issues. But as I've now it should serve this substitute.

Sorry, still no substitute for me for the patch you've sent yesterday.

> you can test with.
>
> --8<---------------cut here---------------start------------->8---
> guix archive --export -r qtwebengine | ssh server guix archive --import
> --8<---------------cut here---------------end--------------->8---

Err... What do you mean with the above?  Did you paste the right
command? 

> If this passes make sure you have
> #AF73C7321F90EA89AD29BBA14069EB8EF8F6410325E479F93DE4612F26478726#
>
> in
> --8<---------------cut here---------------start------------->8---
> /etc/guix/acl
> --8<---------------cut here---------------end--------------->8---

I do!

I tried building it myself anyways, and it fails with

--8<---------------cut here---------------start------------->8---
[11690/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o
FAILED: obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o
/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/bin/g++ -MMD -MF obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o.d -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DOFFICIAL_BUILD -DCHROMIUM_BUILD -DFIELDTRIAL_TESTING_ENABLED -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DBLINK_CORE_IMPLEMENTATION=1 -DGL_GLEXT_PROTOTYPES -DUSE_GLX -DUSE_EGL -DBLINK_IMPLEMENTATION=1 -DINSIDE_BLINK -DUSING_SYSTEM_ICU=1 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DHAVE_PTHREAD -DSK_IGNORE_LINEONLY_AA_CONVEX_PATH_OPTS -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_HAS_JPEG_LIBRARY -DSK_SUPPORT_GPU=1 -DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\" -DWTF_USE_WEBAUDIO_FFMPEG=1 -DSUPPORT_WEBGL2_COMPUTE_CONTEXT=1 -DWTF_USE_DEFAULT_RENDER_THEME=1 -DUSE_SYSTEM_LIBJPEG -DUSING_SYSTEM_ICU=1 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DGTEST_RELATIVE_PATH -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_POSIX -DWEBRTC_LINUX -DABSL_ALLOCATOR_NOTHROW=1 -DNO_MAIN_THREAD_WRAPPING -DLIBXSLT_STATIC -DUSE_SYSTEM_ZLIB=1 -I. -Igen -I../../3rdparty/chromium -Igen -Igen -Igen -Igen -Igen -Igen -I../../3rdparty/chromium/third_party/khronos -I../../3rdparty/chromium/gpu -I../../3rdparty/chromium/third_party/libyuv/include -Igen -Igen -Igen -Igen -Igen -Igen -Igen -I../../3rdparty/chromium/third_party/ced/src -I../../3rdparty/chromium/third_party/protobuf/src -I../../3rdparty/chromium/third_party/protobuf/src -Igen/protoc_out -I../../3rdparty/chromium/third_party/boringssl/src/include -I../../3rdparty/chromium/skia/config -I../../3rdparty/chromium/skia/ext -I../../3rdparty/chromium/third_party/skia/include/c -I../../3rdparty/chromium/third_party/skia/include/config -I../../3rdparty/chromium/third_party/skia/include/core -I../../3rdparty/chromium/third_party/skia/include/effects -I../../3rdparty/chromium/third_party/skia/include/encode -I../../3rdparty/chromium/third_party/skia/include/gpu -I../../3rdparty/chromium/third_party/skia/include/images -I../../3rdparty/chromium/third_party/skia/include/lazy -I../../3rdparty/chromium/third_party/skia/include/pathops -I../../3rdparty/chromium/third_party/skia/include/pdf -I../../3rdparty/chromium/third_party/skia/include/pipe -I../../3rdparty/chromium/third_party/skia/include/ports -I../../3rdparty/chromium/third_party/skia/include/utils -I../../3rdparty/chromium/third_party/skia/src/gpu -I../../3rdparty/chromium/third_party/skia/src/sksl -I../../3rdparty/chromium/third_party/angle/include -I../../3rdparty/chromium/third_party/angle/src/common/third_party/base -Igen/angle -I../../3rdparty/chromium/v8/include -Igen/v8/include -I../../3rdparty/chromium/third_party/webrtc_overrides -I../../3rdparty/chromium/third_party/webrtc -I../../3rdparty/chromium/third_party/iccjpeg -I../../3rdparty/chromium/third_party/ots/include -I../../3rdparty/chromium/v8/include -Igen/v8/include -I../../3rdparty/chromium/third_party/libxml/src/include -I../../3rdparty/chromium/third_party/libxml/linux/include -I../../3rdparty/chromium/third_party/libxslt/src -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -funwind-tables -fPIC -pipe -pthread -m64 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-dangling-else -Wno-packed-not-aligned -Wno-missing-field-initializers -Wno-unused-parameter -fno-omit-frame-pointer -fvisibility=hidden -O2 -fno-ident -fdata-sections -ffunction-sections -g0 -isystem../../../../../../gnu/store/8n26nbmw5d9wiiah96bj75hcpgcqkhsg-nss-3.46.1/include/nss -isystem../../../../../../gnu/store/z1g3hlalia7fb0kyf3fv9g1kqk29fbvr-nspr-4.22/include/nspr -isystem ../../3rdparty/chromium/third_party/abseil-cpp -isystem../../../../../../gnu/store/3snpwk7jl8i125bhiilvk9scqc4mnsx7-libpng-1.6.37/include/libpng16 -DLIBXML_STATIC= -std=gnu++14 -Wno-narrowing -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -c gen/third_party/blink/renderer/core/animation/animation_jumbo_2.cc -o obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o
{standard input}: Assembler messages:
{standard input}:3778: Warning: end of file not at end of a line; newline inserted
{standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
[11691/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_7.o
[11692/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_10.o
[11693/14276] CXX obj/media/blink/blink/watch_time_component.o
[11694/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_8.o
[11695/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_3.o
[11696/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_1.o
[11697/14276] CXX obj/third_party/blink/renderer/modules/xr/xr/xr_jumbo_3.o
[11698/14276] CXX obj/third_party/blink/renderer/core/svg/svg/svg_jumbo_10.o
[11699/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_4.o
[11700/14276] CXX obj/third_party/blink/renderer/core/paint/paint/paint_jumbo_5.o
[11701/14276] CXX obj/third_party/blink/renderer/core/paint/paint/paint_jumbo_6.o
[11702/14276] CXX obj/third_party/blink/renderer/core/page/page/page_jumbo_1.o
[11703/14276] CXX obj/third_party/blink/renderer/core/paint/paint/paint_jumbo_4.o
ninja: build stopped: subcommand failed.
make[3]: *** [Makefile.gn_run:332: run_ninja] Error 1
make[3]: Leaving directory '/tmp/guix-build-qtwebengine-5.12.6.drv-0/qtwebengine-everywhere-src-5.12.6/src/core'
make[2]: *** [Makefile:82: sub-gn_run-pro-make_first] Error 2
make[2]: Leaving directory '/tmp/guix-build-qtwebengine-5.12.6.drv-0/qtwebengine-everywhere-src-5.12.6/src/core'
make[1]: *** [Makefile:81: sub-core-make_first] Error 2
make[1]: Leaving directory '/tmp/guix-build-qtwebengine-5.12.6.drv-0/qtwebengine-everywhere-src-5.12.6/src'
make: *** [Makefile:49: sub-src-make_first] Error 2
command "make" "-j" "12" failed with status 2
--8<---------------cut here---------------end--------------->8---

It could be an out-of-memory error.  Any clue how to make sure this was
the case?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Sorry, still no substitute for me for the patch you've sent yesterday.

I tested with a couple of machines and it does server substitutes. maybe
now the substitute cache will have invalidated. or you can manual purge
/var/guix/substitutes.  Hopefully this resolves it self. If not I'm
going to have a new build soon that fixs pulseaudio and the other issues
you mentioned on the bug tracker.

>> --8<---------------cut here---------------start------------->8---
>> guix archive --export -r qtwebengine | ssh server guix archive --import
>> --8<---------------cut here---------------end--------------->8---
>
> Err... What do you mean with the above?  Did you paste the right
> command? 

I was not quite clear here. For me it's easier to build on
workstation. And then I'd like to export to my publish server. But
simply exporting and importing does cause the publish to server the
build as a substitute. Maybe this is not feasible or I misunderstand how
to do this.

>> If this passes make sure you have
>> #AF73C7321F90EA89AD29BBA14069EB8EF8F6410325E479F93DE4612F26478726#
>>
>> in
>> --8<---------------cut here---------------start------------->8---
>> /etc/guix/acl
>> --8<---------------cut here---------------end--------------->8---
>
> I do!
>
> I tried building it myself anyways, and it fails with
>
> [11690/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o
> FAILED: obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o
> /gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/bin/g++ -MMD -MF obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o.d -DV8_DEPRECATION_WARNINGS -DUSE_UDEV -DUSE_AURA=1 -DUSE_NSS_CERTS=1 -DUSE_OZONE=1 -DNO_TCMALLOC -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DOFFICIAL_BUILD -DCHROMIUM_BUILD -DFIELDTRIAL_TESTING_ENABLED -DTOOLKIT_QT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -DBLINK_CORE_IMPLEMENTATION=1 -DGL_GLEXT_PROTOTYPES -DUSE_GLX -DUSE_EGL -DBLINK_IMPLEMENTATION=1 -DINSIDE_BLINK -DUSING_SYSTEM_ICU=1 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DHAVE_PTHREAD -DSK_IGNORE_LINEONLY_AA_CONVEX_PATH_OPTS -DSK_HAS_PNG_LIBRARY -DSK_HAS_WEBP_LIBRARY -DSK_HAS_JPEG_LIBRARY -DSK_SUPPORT_GPU=1 -DSK_GPU_WORKAROUNDS_HEADER=\"gpu/config/gpu_driver_bug_workaround_autogen.h\" -DWTF_USE_WEBAUDIO_FFMPEG=1 -DSUPPORT_WEBGL2_COMPUTE_CONTEXT=1 -DWTF_USE_DEFAULT_RENDER_THEME=1 -DUSE_SYSTEM_LIBJPEG -DUSING_SYSTEM_ICU=1 -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC -DWEBRTC_NON_STATIC_TRACE_EVENT_HANDLERS=0 -DGTEST_RELATIVE_PATH -DWEBRTC_CHROMIUM_BUILD -DWEBRTC_POSIX -DWEBRTC_LINUX -DABSL_ALLOCATOR_NOTHROW=1 -DNO_MAIN_THREAD_WRAPPING -DLIBXSLT_STATIC -DUSE_SYSTEM_ZLIB=1 -I. -Igen -I../../3rdparty/chromium -Igen -Igen -Igen -Igen -Igen -Igen -I../../3rdparty/chromium/third_party/khronos -I../../3rdparty/chromium/gpu -I../../3rdparty/chromium/third_party/libyuv/include -Igen -Igen -Igen -Igen -Igen -Igen -Igen -I../../3rdparty/chromium/third_party/ced/src -I../../3rdparty/chromium/third_party/protobuf/src -I../../3rdparty/chromium/third_party/protobuf/src -Igen/protoc_out -I../../3rdparty/chromium/third_party/boringssl/src/include -I../../3rdparty/chromium/skia/config -I../../3rdparty/chromium/skia/ext -I../../3rdparty/chromium/third_party/skia/include/c -I../../3rdparty/chromium/third_party/skia/include/config -I../../3rdparty/chromium/third_party/skia/include/core -I../../3rdparty/chromium/third_party/skia/include/effects -I../../3rdparty/chromium/third_party/skia/include/encode -I../../3rdparty/chromium/third_party/skia/include/gpu -I../../3rdparty/chromium/third_party/skia/include/images -I../../3rdparty/chromium/third_party/skia/include/lazy -I../../3rdparty/chromium/third_party/skia/include/pathops -I../../3rdparty/chromium/third_party/skia/include/pdf -I../../3rdparty/chromium/third_party/skia/include/pipe -I../../3rdparty/chromium/third_party/skia/include/ports -I../../3rdparty/chromium/third_party/skia/include/utils -I../../3rdparty/chromium/third_party/skia/src/gpu -I../../3rdparty/chromium/third_party/skia/src/sksl -I../../3rdparty/chromium/third_party/angle/include -I../../3rdparty/chromium/third_party/angle/src/common/third_party/base -Igen/angle -I../../3rdparty/chromium/v8/include -Igen/v8/include -I../../3rdparty/chromium/third_party/webrtc_overrides -I../../3rdparty/chromium/third_party/webrtc -I../../3rdparty/chromium/third_party/iccjpeg -I../../3rdparty/chromium/third_party/ots/include -I../../3rdparty/chromium/v8/include -Igen/v8/include -I../../3rdparty/chromium/third_party/libxml/src/include -I../../3rdparty/chromium/third_party/libxml/linux/include -I../../3rdparty/chromium/third_party/libxslt/src -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-protector -funwind-tables -fPIC -pipe -pthread -m64 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wno-unused-local-typedefs -Wno-maybe-uninitialized -Wno-deprecated-declarations -fno-delete-null-pointer-checks -Wno-comments -Wno-dangling-else -Wno-packed-not-aligned -Wno-missing-field-initializers -Wno-unused-parameter -fno-omit-frame-pointer -fvisibility=hidden -O2 -fno-ident -fdata-sections -ffunction-sections -g0 -isystem../../../../../../gnu/store/8n26nbmw5d9wiiah96bj75hcpgcqkhsg-nss-3.46.1/include/nss -isystem../../../../../../gnu/store/z1g3hlalia7fb0kyf3fv9g1kqk29fbvr-nspr-4.22/include/nspr -isystem ../../3rdparty/chromium/third_party/abseil-cpp -isystem../../../../../../gnu/store/3snpwk7jl8i125bhiilvk9scqc4mnsx7-libpng-1.6.37/include/libpng16 -DLIBXML_STATIC= -std=gnu++14 -Wno-narrowing -Wno-attributes -Wno-class-memaccess -Wno-subobject-linkage -fno-exceptions -fno-rtti -fvisibility-inlines-hidden -c gen/third_party/blink/renderer/core/animation/animation_jumbo_2.cc -o obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_2.o
> {standard input}: Assembler messages:
> {standard input}:3778: Warning: end of file not at end of a line; newline inserted
> {standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See <https://gcc.gnu.org/bugs/> for instructions.
> [11691/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_7.o
> [11692/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_10.o
> [11693/14276] CXX obj/media/blink/blink/watch_time_component.o
> [11694/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_8.o
> [11695/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_3.o
> [11696/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_1.o
> [11697/14276] CXX obj/third_party/blink/renderer/modules/xr/xr/xr_jumbo_3.o
> [11698/14276] CXX obj/third_party/blink/renderer/core/svg/svg/svg_jumbo_10.o
> [11699/14276] CXX obj/third_party/blink/renderer/core/animation/animation/animation_jumbo_4.o
> [11700/14276] CXX obj/third_party/blink/renderer/core/paint/paint/paint_jumbo_5.o
> [11701/14276] CXX obj/third_party/blink/renderer/core/paint/paint/paint_jumbo_6.o
> [11702/14276] CXX obj/third_party/blink/renderer/core/page/page/page_jumbo_1.o
> [11703/14276] CXX obj/third_party/blink/renderer/core/paint/paint/paint_jumbo_4.o
> ninja: build stopped: subcommand failed.
> make[3]: *** [Makefile.gn_run:332: run_ninja] Error 1
> make[3]: Leaving directory '/tmp/guix-build-qtwebengine-5.12.6.drv-0/qtwebengine-everywhere-src-5.12.6/src/core'
> make[2]: *** [Makefile:82: sub-gn_run-pro-make_first] Error 2
> make[2]: Leaving directory '/tmp/guix-build-qtwebengine-5.12.6.drv-0/qtwebengine-everywhere-src-5.12.6/src/core'
> make[1]: *** [Makefile:81: sub-core-make_first] Error 2
> make[1]: Leaving directory '/tmp/guix-build-qtwebengine-5.12.6.drv-0/qtwebengine-everywhere-src-5.12.6/src'
> make: *** [Makefile:49: sub-src-make_first] Error 2
> command "make" "-j" "12" failed with status 2
>
> It could be an out-of-memory error.  Any clue how to make sure this was
> the case?

 g++: internal compiler error: Killed (program cc1plus) . does look like
 a oom error. I noticed with this build if you have many cores it needs
 quite alot of ram. I mainly build on a 6 core with 6 threads AMD Ryzen
 5 3600 Processor and 16 gigs of ram with a 1TB nvme drive. Build time
 is roughly 40min.

Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 110 bytes --]

I just had to wait a bit longer, the substitution works now!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 989 bytes --]

mike.rosset@gmail.com writes:

> I have a simple CPP for testing it can be found
> here. https://github.com/mrosset/testqt.
>
> for testing I use this enviroment.
>
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix environment  --ad-hoc qtbase qtwebengine qtdeclarative qtwebchannel coreutils gcc-toolchain nss-certs
> --8<---------------cut here---------------end--------------->8---
>
> and then I test with
>
> --8<---------------cut here---------------start------------->8---
> cd testqt
> qmake
> make
> ./main
> --8<---------------cut here---------------end--------------->8---

This worked for me.

> or there are hardware acceleration problems
>
> --8<---------------cut here---------------start------------->8---
> QT_XCB_FORCE_SOFTWARE_OPENGL=1 ./main
> --8<---------------cut here---------------end--------------->8---

No problem on my end with an AMD RX 580.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 254 bytes --]

I remember an issue was raised regarding the embedded third-party
libraries in the package (I think src/3rdparty).

Any plan regarding this?  Is there anything that cannot be replaced by a
Guix input?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> No problem on my end with an AMD RX 580.

This is surprising but good news none the less. Also I've mailed a
second in the series patch that fixes pulseaudio and address your
comments in the bug tracker. A substitute should be provided for this as
well.

Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 380 bytes --]

mike.rosset@gmail.com writes:

> Pierre Neidhardt <mail@ambrevar.xyz> writes:
>
>> No problem on my end with an AMD RX 580.
>
> This is surprising but good news none the less.

Hmm... On second though, I just navigated GNU.org, which might not
require GL acceleration.  I think I should try with a proper GL test website.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Hmm... On second though, I just navigated GNU.org, which might not
> require GL acceleration.  I think I should try with a proper GL test website.


For me the GL issues will cause issues regardless of the website. For
example you can not scroll the page. But the symptoms may changed
depending on video drivers. Let know how this turns out.

Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

I tested again on an Intel HD Graphics 620.
Everything works for me so far (navigating, smooth scrolling, rendering).

I tested the following:

- https://gnu.org
- https://ambrevar.xyz (has some CSS that may required hardware acceleration)
- https://github.com
- https://youtube.com: playing videos works flawlessly.
- https://get.webgl.org/: the webgl cube is spinning properly :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 294 bytes --]

Mike, have you tried building the package against ungoogled-chromium?

Either way, I vote for including this package into Guix, it seems mature enough.
We can open an issue to replace the bundled chromium with
ungoogled-chromium if need be.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> I tested again on an Intel HD Graphics 620.
> Everything works for me so far (navigating, smooth scrolling, rendering).
>
> I tested the following:
>
> - https://gnu.org
> - https://ambrevar.xyz (has some CSS that may required hardware acceleration)
> - https://github.com
> - https://youtube.com: playing videos works flawlessly.
> - https://get.webgl.org/: the webgl cube is spinning properly :)

Thanks for testing this. one site that suffers with webkit I noticed is
reddit I would add that to your test sites.

I think I found the issues nouaveau in that the those drivers currently
do not support multi threaded OpenGL which qtwebengine uses. Oddly
though I do not need to set QT_XCB_FORCE_SOFTWARE_OPENGL in my
monolithic QT package with qtwebengine. But that package is using the
latest QT 5.14.0 and it's possible it's doing some auto detection and
switching software acceleration on nouveau anyways. I'll have to
investigate this further down the road.

Mike

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Mike, have you tried building the package against ungoogled-chromium?
>
> Either way, I vote for including this package into Guix, it seems mature enough.
> We can open an issue to replace the bundled chromium with
> ungoogled-chromium if need be.

Sorry for the long delay on responding to this topic. I've been pretty
busy. I have done a cursory look at upstreamn ungoogled-chrome and then
started to reason what might need to be done to integrate this into
qtwebengine.

My conclusion is this is going to be way more work then the
benefits. And once this was achieved it still needs to be
maintained. Keep in mind QT already has a patch re base process with
changes that might conflict with ungoogled-chrome. I think the better
approach is to try and fix the in tree dependencies upstream. and let it
trick down to the guix declaration.

In a more ideal world I think QT and the ungogogled project should be
working together since there goals are much on par.

Though maybe given enough time and resources I could find an more ideal
way to solve this. Right now though I don't think I have the time or
resources.

Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

mike.rosset@gmail.com writes:

> My conclusion is this is going to be way more work then the
> benefits. And once this was achieved it still needs to be
> maintained. Keep in mind QT already has a patch re base process with
> changes that might conflict with ungoogled-chrome. I think the better
> approach is to try and fix the in tree dependencies upstream. and let it
> trick down to the guix declaration.
>
> In a more ideal world I think QT and the ungogogled project should be
> working together since there goals are much on par.

Totally, I think you are making a lot of sense here.
Since you are the most knowledgeable on the topic, would you mind
reporting this to upstream and include the links in your patch?  Thanks!

Let me know if there are more changes you'd like to include to your
patch.
If no one objects, I'll merge it within a week.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Let me know if there are more changes you'd like to include to your
> patch.
> If no one objects, I'll merge it within a week.
>
> Cheers!

I think the package is looking pretty good now. I sent some changes
requested by Hartmut today so we might need to wait on any more changes
he might have.

Moving forward I'll start a discussion upstream about how to go about
improving the in tree dependency issue.

I'm going to take a further look at the nouveau issue both on guix system
and I'd like to do more testing of foreign distro.

Mike

Re: qtwenengine anybody?

[Ricardo Wurmus]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> mike.rosset@gmail.com writes:
>
>> My conclusion is this is going to be way more work then the
>> benefits. And once this was achieved it still needs to be
>> maintained. Keep in mind QT already has a patch re base process with
>> changes that might conflict with ungoogled-chrome. I think the better
>> approach is to try and fix the in tree dependencies upstream. and let it
>> trick down to the guix declaration.
>>
>> In a more ideal world I think QT and the ungogogled project should be
>> working together since there goals are much on par.
>
> Totally, I think you are making a lot of sense here.
> Since you are the most knowledgeable on the topic, would you mind
> reporting this to upstream and include the links in your patch?  Thanks!
>
> Let me know if there are more changes you'd like to include to your
> patch.
> If no one objects, I'll merge it within a week.

I’m sorry for making this harder, but I would very much like to first
see evidence that the problematic features of Chromium have in fact been
addressed by QT upstream.

Does the code include DRM support, for example?  Does it phone home?
Can we see the patches that upstream applied to the Chromium sources?

--
Ricardo

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

I just had a quick glimpse at

  git://code.qt.io/qt/qtwebengine-chromium.git

(Branch v5.14.0.)

It has a lot of patches, but I don't understand the sources enough to
make an informed decision.

Maybe Marius or Mike can tell us more.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Hartmut Goebel]

Am 01.01.20 um 23:30 schrieb mike.rosset@gmail.com:
> I think the package is looking pretty good now. I sent some changes
> requested by Hartmut today so we might need to wait on any more changes
> he might have.

I tried applying the patches to see the complete/compiled diff, but `git
am` failed. Can you please send an updated patch or provide a link to
some online-repo to view the complete diff. Thanks

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[mike.rosset]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 01.01.20 um 23:30 schrieb mike.rosset@gmail.com:
>> I think the package is looking pretty good now. I sent some changes
>> requested by Hartmut today so we might need to wait on any more changes
>> he might have.
>
> I tried applying the patches to see the complete/compiled diff, but `git
> am` failed. Can you please send an updated patch or provide a link to
> some online-repo to view the complete diff. Thanks

Apologies this is probably a failure in my ML workflow. And just my luck
I'm in the process of doing some working related moving todoy. Here is a
complete diff
https://gitlab.com/mike.rosset/guix/compare/master...qtwebengine hope
this helps.

Mike

Re: qtwenengine anybody?

[Hartmut Goebel]

Am 02.01.20 um 21:24 schrieb mike.rosset@gmail.com:
> Apologies this is probably a failure in my ML workflow. And just my luck
> I'm in the process of doing some working related moving todoy. Here is a
> complete diff
> https://gitlab.com/mike.rosset/guix/compare/master...qtwebengine hope
> this helps.

Many thanks, this is quite helpful.

For me it looks okay except of two minor points:

- The description should contain at least a brief note about Qt patched
the embedded chromium to (partially) un-google it. If you know details,
this would be great. This of course could be enhanced later, since
changing the description does not force a rebuild of he package.

- Please still honor Ricardo's remark about DRM and phoning home.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[mike.rosset]

Ricardo Wurmus <rekado@elephly.net> writes:

> Does the code include DRM support, for example?  Does it phone home?
> Can we see the patches that upstream applied to the Chromium sources?
>

QtWebengine does not handle DRM at all. That is handled by a 3rd party
plugin called wildevine and would require guix to bundle that plugin.
Something I'm confident is not going to happen. As an added measure my
proposed qtwebengine declaration has pepper plugins disabled making it
impossible to load 3rd party plugins either way.

see https://doc.qt.io/qt-5/qtwebengine-features.html#html5-drm
for reference

In regards to QtWebEngine phoning home https://wiki.qt.io/QtWebEngine
states all google services have been removed. Nothing I've seen
indicates this is not the case.

In terms of patches. You can get a complete diff of chromium sources
by cloning qtwebengine recursively like so.

--8<---------------cut here---------------start------------->8---
git clone http://code.qt.io/qt/qtwebengine.git --recursive
--8<---------------cut here---------------end--------------->8---

Version 5.12.6 of qtwebengine is based on chromium sources 69.0.3497.128
to get a complete diff you can do this from the src/3rdpath directory.
you can then get a complete patch diff for version 5.12.6 like so.

--8<---------------cut here---------------start------------->8---
git diff origin/upstream-master origin/69-based
--8<---------------cut here---------------end--------------->8---

see https://wiki.qt.io/QtWebEngine/Rebase_on_New_Chromium for reference

Mike

Re: qtwenengine anybody?

[mike.rosset]

[-- Attachment #1: Type: text/plain, Size: 1366 bytes --]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 02.01.20 um 21:24 schrieb mike.rosset@gmail.com:
>> Apologies this is probably a failure in my ML workflow. And just my luck
>> I'm in the process of doing some working related moving todoy. Here is a
>> complete diff
>> https://gitlab.com/mike.rosset/guix/compare/master...qtwebengine hope
>> this helps.
>
> Many thanks, this is quite helpful.
>
> For me it looks okay except of two minor points:
>
> - The description should contain at least a brief note about Qt patched
> the embedded chromium to (partially) un-google it. If you know details,
> this would be great. This of course could be enhanced later, since
> changing the description does not force a rebuild of he package.
>
> - Please still honor Ricardo's remark about DRM and phoning home.

Hello Hartmut,

I've upgraded the description field as request see attached. And CC to
35866@debbugs.gnu.org

I've also addressed in depth some Ricardo's concerns.

More then likely you will have problems applying this with git am
again. This final declaration though based on my original
submission. Was moved manually out and back in again. So there is
probably inconsequential git history missing. I could create a new bug
report maybe? Or you can still use the complete diff from
https://gitlab.com/mike.rosset/guix/compare/master...qtwebengine


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0005-gnu-qtwebengine-description-field-now-notes-source-r.patch --]
[-- Type: text/x-patch, Size: 1199 bytes --]

From e67e9e7a6a6b346c9a99759ae75040735e5336d1 Mon Sep 17 00:00:00 2001
From: Mike Rosset <mike.rosset@gmail.com>
Date: Sun, 5 Jan 2020 07:34:47 -0800
Subject: [PATCH 5/5] gnu: qtwebengine: description field now notes source re
 base summary
To: 35866@debbugs.gnu.org

* gnu/packages/qt.scm (qtwebengine)[description]: Note Qt Chromium source re
  base details.
---
 gnu/packages/qt.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index a4a039dc80..e199c67c4f 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -2154,8 +2154,9 @@ time Web content can be enhanced with native controls.")
             (files '("lib/qt5/libexec/QtWebEngineProcess")))))
     (home-page "https://www.qt.io")
     (synopsis "Qt WebEngine module")
-    (description "Qt5WebEngine provides support for web
-applications using the Chromium browser project.")
+    (description "Qt5WebEngine provides support for web applications using the
+Chromium browser project. The Chromium source base has Google services and
+binaries removed along with modular support for system libraries.")
     (license
      (package-license qt))))
 
-- 
2.24.1


[-- Attachment #3: Type: text/plain, Size: 6 bytes --]


Mike

Re: qtwenengine anybody?

[Hartmut Goebel]

Hi Mike,
> I've upgraded the description field as request see attached. And CC to
> 35866@debbugs.gnu.org
>
> I've also addressed in depth some Ricardo's concerns.


For me this would be okay. Ricardo needs to state whether his concerns
are covered.

> https://gitlab.com/mike.rosset/guix/compare/master...qtwebengine

This was perfectly fine for reviewing. Thanks.


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[Ricardo Wurmus]

Hi Mike,

thank you for your patient reply.

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> Does the code include DRM support, for example?  Does it phone home?
>> Can we see the patches that upstream applied to the Chromium sources?
>>
>
> QtWebengine does not handle DRM at all. That is handled by a 3rd party
> plugin called wildevine and would require guix to bundle that plugin.
> Something I'm confident is not going to happen. As an added measure my
> proposed qtwebengine declaration has pepper plugins disabled making it
> impossible to load 3rd party plugins either way.
>
> see https://doc.qt.io/qt-5/qtwebengine-features.html#html5-drm
> for reference

I’m aware of this.  My apologies for not expressing myself more
clearly.  We probably want to remove the support for the DRM plugin,
following the discussions about ungoogled-chromium at

    https://issues.guix.gnu.org/issue/34565

> In regards to QtWebEngine phoning home https://wiki.qt.io/QtWebEngine
> states all google services have been removed. Nothing I've seen
> indicates this is not the case.

Excellent.  Thank you for checking.

-- 
Ricardo

Re: qtwenengine anybody?

[mike.rosset]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Mike,
>
> thank you for your patient reply.
>
>> Ricardo Wurmus <rekado@elephly.net> writes:
>>
>>> Does the code include DRM support, for example?  Does it phone home?
>>> Can we see the patches that upstream applied to the Chromium sources?
>>>
>>
>> QtWebengine does not handle DRM at all. That is handled by a 3rd party
>> plugin called wildevine and would require guix to bundle that plugin.
>> Something I'm confident is not going to happen. As an added measure my
>> proposed qtwebengine declaration has pepper plugins disabled making it
>> impossible to load 3rd party plugins either way.
>>
>> see https://doc.qt.io/qt-5/qtwebengine-features.html#html5-drm
>> for reference
>
> I’m aware of this.  My apologies for not expressing myself more
> clearly.  We probably want to remove the support for the DRM plugin,
> following the discussions about ungoogled-chromium at
>
>     https://issues.guix.gnu.org/issue/34565

Marius Bakke does a good job of summing up DRM when closing this
issue. Widevine and DRM are not part of chromium and requires not only
the widevine binary to be installed by the user. But also in the case of
qtwebengine requires the users to rebuild the package to include pepper
plugin support. The proposed declaration uses this flag
-webengine-pepper-plugins=no which disables flash and widevine support
altogether.

see
https://code.qt.io/cgit/qt/qtwebengine.git/tree/config_help.txt?h=5.12.6#n16
for reference.

Mike

Re: qtwenengine anybody?

[Marius Bakke]

[-- Attachment #1.1: Type: text/plain, Size: 1655 bytes --]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Mike,
>
> thank you for your patient reply.
>
>> Ricardo Wurmus <rekado@elephly.net> writes:
>>
>>> Does the code include DRM support, for example?  Does it phone home?
>>> Can we see the patches that upstream applied to the Chromium sources?
>>>
>>
>> QtWebengine does not handle DRM at all. That is handled by a 3rd party
>> plugin called wildevine and would require guix to bundle that plugin.
>> Something I'm confident is not going to happen. As an added measure my
>> proposed qtwebengine declaration has pepper plugins disabled making it
>> impossible to load 3rd party plugins either way.
>>
>> see https://doc.qt.io/qt-5/qtwebengine-features.html#html5-drm
>> for reference
>
> I’m aware of this.  My apologies for not expressing myself more
> clearly.  We probably want to remove the support for the DRM plugin,
> following the discussions about ungoogled-chromium at
>
>     https://issues.guix.gnu.org/issue/34565

I have attached a patch that disables support for loading the Widevine
plugin, and also removes unused bundled software from the source based
on a whitelist, similar to ungoogled-chromium's
%PRESERVED-THIRD-PARTY-FILES.

It removes some 300+ MiB from the uncompressed sources.  The snippet
will require some adjustments for every new version of QtWebEngine
however, I can help out with that when required.

Patch 1/2 is a slightly "cleaned up" version of Mike's latest patch,
fixing some cosmetic issues, and makes it respect the daemons requested
number of build cores.

With these changes, I think the package is ready to go.  \o/


[-- Attachment #1.2: 0001-gnu-Add-qtwebengine.patch --]
[-- Type: text/x-patch, Size: 8464 bytes --]

From 3e373d54e08c95db16d53110051e008e6d08d55b Mon Sep 17 00:00:00 2001
From: Mike Rosset <mike.rosset@gmail.com>
Date: Wed, 1 Jan 2020 08:03:52 -0800
Subject: [PATCH 1/2] gnu: Add qtwebengine.

* gnu/packages/qt.scm (qtwebengine): New variable.

Signed-off-by: Marius Bakke <mbakke@fastmail.com>
---
 gnu/packages/qt.scm | 144 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 144 insertions(+)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 795b5e9d2b..d6ceb5b90f 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -13,6 +13,7 @@
 ;;; Copyright © 2018 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2019 Marius Bakke <mbakke@fastmail.com>
 ;;; Copyright © 2018 John Soo <jsoo1@asu.edu>
+;;; Copyright © 2020 Mike Rosset <mike.rosset@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -52,30 +53,40 @@
   #:use-module (gnu packages flex)
   #:use-module (gnu packages freedesktop)
   #:use-module (gnu packages gcc)
+  #:use-module (gnu packages ghostscript)
   #:use-module (gnu packages gl)
   #:use-module (gnu packages glib)
+  #:use-module (gnu packages gnupg)
   #:use-module (gnu packages gperf)
   #:use-module (gnu packages gstreamer)
   #:use-module (gnu packages gtk)
   #:use-module (gnu packages icu4c)
   #:use-module (gnu packages image)
+  #:use-module (gnu packages libevent)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages llvm)
   #:use-module (gnu packages maths)
+  #:use-module (gnu packages ninja)
   #:use-module (gnu packages nss)
   #:use-module (gnu packages pciutils)
   #:use-module (gnu packages pcre)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages pulseaudio)
+  #:use-module (gnu packages protobuf)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-xyz)
+  #:use-module (gnu packages regex)
   #:use-module (gnu packages ruby)
   #:use-module (gnu packages sdl)
+  #:use-module (gnu packages serialization)
   #:use-module (gnu packages sqlite)
+  #:use-module (gnu packages telephony)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages video)
   #:use-module (gnu packages vulkan)
   #:use-module (gnu packages xdisorg)
+  #:use-module (gnu packages xiph)
   #:use-module (gnu packages xorg)
   #:use-module (gnu packages xml)
   #:use-module (srfi srfi-1))
@@ -2020,6 +2031,139 @@ time Web content can be enhanced with native controls.")
 
     (license license:lgpl2.1+)))
 
+(define-public qtwebengine
+  (package
+    (inherit qtsvg)
+    (name "qtwebengine")
+    (version (package-version qtbase))
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://download.qt.io/official_releases/qt/"
+                                  (version-major+minor version) "/" version
+                                  "/submodules/" name "-everywhere-src-"
+                                  version ".tar.xz"))
+              (sha256
+               (base32
+                "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("bison" ,bison)
+       ("flex" ,flex)
+       ("gperf" ,gperf)
+       ("ninja" ,ninja)
+       ("perl" ,perl)
+       ("pkg-config" ,pkg-config)
+       ("python-2" ,python-2)
+       ("ruby" ,ruby)))
+    (inputs
+     `(("alsa-lib" ,alsa-lib)
+       ("atk" ,atk)
+       ("cups-minimal" ,cups-minimal)
+       ("dbus" ,dbus)
+       ("ffmpeg" ,ffmpeg)
+       ("fontconfig" ,fontconfig)
+       ("harbuzz" ,harfbuzz)
+       ("icu4c" ,icu4c)
+       ("jsoncpp" ,jsoncpp)
+       ("lcms" ,lcms)
+       ("libcap" ,libcap)
+       ("libevent" ,libevent)
+       ("libgcrypt" ,libgcrypt)
+       ("libjpeg" ,libjpeg-turbo)
+       ("libvpx" ,libvpx)
+       ;; FIXME: configure does not find system lcms
+       ;; ("lcms" ,lcms)
+       ("libwebp" ,libwebp)
+       ("libx11" ,libx11)
+       ("libxcb" ,libxcb)
+       ("libxcomposite" ,libxcomposite)
+       ("libxcursor" ,libxcursor)
+       ("libxi" ,libxi)
+       ("libxkbcommon" ,libxkbcommon)
+       ;; FIXME: libxml2 needs to built with icu support though it links to
+       ;; libxml2 configure summary still states. Checking for compatible
+       ;; system libxml2... no
+       ("libxml2" ,libxml2)
+       ("libxrandr" ,libxrandr)
+       ("libxrender" ,libxrender)
+       ("libxslt" ,libxslt)
+       ("libxtst" ,libxtst)
+       ("mesa" ,mesa)
+       ("minizip" ,minizip)
+       ("nss" ,nss)
+       ("opus" ,opus)
+       ("pciutils" ,pciutils)
+       ("protobuf" ,protobuf)
+       ("pulseaudio" ,pulseaudio)
+       ("qtbase" ,qtbase)
+       ("qtdeclarative" ,qtdeclarative)
+       ("qtmultimedia" ,qtmultimedia)
+       ("qtwebchannel" ,qtwebchannel)
+       ("re2" ,re2)
+       ("snappy" ,snappy)
+       ("udev" ,eudev)
+       ("xcb-util" ,xcb-util)))
+    (arguments
+     (substitute-keyword-arguments (package-arguments qtsvg)
+       ((#:phases phases)
+        `(modify-phases ,phases
+           (add-before 'configure 'substitute-source
+             (lambda* (#:key inputs outputs #:allow-other-keys)
+               (let ((out (assoc-ref outputs "out"))
+                     (nss (assoc-ref inputs "nss"))
+                     (udev (assoc-ref inputs "udev")))
+                 ;; Qtwebengine is not installed into the same prefix as
+                 ;; qtbase. Some qtbase QTLibraryInfo constants will not
+                 ;; work. Replace with the full path to the qtwebengine
+                 ;; translations and locales in the store.
+                 (substitute* "src/core/web_engine_library_info.cpp"
+                   (("QLibraryInfo::location\\(QLibraryInfo::TranslationsPath\\)")
+                    (string-append "QLatin1String(\"" out "/share/qt5/translations\")"))
+                   (("QLibraryInfo::location\\(QLibraryInfo::DataPath\\)")
+                    (string-append "QLatin1String(\"" out "/share/qt5\")")))
+                 ;; Substitute full dynamic library path for nss
+                 (substitute* "src/3rdparty/chromium/crypto/nss_util.cc"
+                   (("libnssckbi.so")
+                    (string-append nss "/lib/nss/libnssckbi.so")))
+                 ;; Substitute full dynamic library path for udev
+                 (substitute* "src/3rdparty/chromium/device/udev_linux/udev1_loader.cc"
+                   (("libudev.so.1")
+                    (string-append udev "/lib/libudev.so.1")))
+                 #t)))
+           (add-before 'configure 'set-env
+             (lambda _
+               ;; Avoids potential race conditions
+               (setenv "PYTHONDONTWRITEBYTECODE" "1")
+               (setenv "NINJAFLAGS"
+                       (string-append "-k1" ;less verbose build output
+                                      ;; Respect the '--cores' option of 'guix build'.
+                                      " -j" (number->string (parallel-job-count))))
+               #t))
+           (replace 'configure
+             (lambda _
+               ;; Valid QT_BUILD_PARTS variables are:
+               ;; libs tools tests examples demos docs translations
+               (invoke "qmake" "QT_BUILD_PARTS = libs tools" "--"
+                       "--webengine-printing-and-pdf=no"
+                       "--webengine-ffmpeg=system"
+                       "--webengine-icu=system"
+                       "--webengine-pepper-plugins=no")))))
+       ;; Tests are disabled due to Could not find QtWebEngineProcess error
+       ;; It's possible this can be fixed by setting QTWEBENGINEPROCESS_PATH
+       ;; before running tests It's also possible some tests need a network
+       ((#:tests? _ #f) #f)))
+    (native-search-paths
+     (list (search-path-specification
+            (file-type 'regular)
+            (separator #f)
+            (variable "QTWEBENGINEPROCESS_PATH")
+            (files '("lib/qt5/libexec/QtWebEngineProcess")))))
+    (home-page "https://www.qt.io")
+    (synopsis "Qt WebEngine module")
+    (description "Qt5WebEngine provides support for web
+applications using the Chromium browser project.")
+    (license license:lgpl2.1+)))
+
 (define-public dotherside
   (package
     (name "dotherside")
-- 
2.24.1


[-- Attachment #1.3: 0002-gnu-qtwebengine-Purge-unused-bundled-software-from-t.patch --]
[-- Type: text/x-patch, Size: 11866 bytes --]

From 487ec682c64fea85ff48ab44d6d918c388f43565 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Fri, 3 Jan 2020 19:32:52 +0100
Subject: [PATCH 2/2] gnu: qtwebengine: Purge unused bundled software from the
 source.

* gnu/packages/qt.scm (qtwebengine)[source](modules, snippet): New fields.
[inputs]: Add CURL and VALGRIND.
---
 gnu/packages/qt.scm | 200 +++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 190 insertions(+), 10 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index d6ceb5b90f..38a620c90c 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -11,7 +11,7 @@
 ;;; Copyright © 2018 Nicolas Goaziou <mail@nicolasgoaziou.fr>
 ;;; Copyright © 2018 Hartmut Goebel <h.goebel@crazy-compilers.com>
 ;;; Copyright © 2018 Eric Bavier <bavier@member.fsf.org>
-;;; Copyright © 2019 Marius Bakke <mbakke@fastmail.com>
+;;; Copyright © 2019, 2020 Marius Bakke <mbakke@fastmail.com>
 ;;; Copyright © 2018 John Soo <jsoo1@asu.edu>
 ;;; Copyright © 2020 Mike Rosset <mike.rosset@gmail.com>
 ;;;
@@ -47,6 +47,7 @@
   #:use-module (gnu packages cmake)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages cups)
+  #:use-module (gnu packages curl)
   #:use-module (gnu packages databases)
   #:use-module (gnu packages documentation)
   #:use-module (gnu packages fontutils)
@@ -83,6 +84,7 @@
   #:use-module (gnu packages sqlite)
   #:use-module (gnu packages telephony)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages valgrind)
   #:use-module (gnu packages video)
   #:use-module (gnu packages vulkan)
   #:use-module (gnu packages xdisorg)
@@ -2036,15 +2038,191 @@ time Web content can be enhanced with native controls.")
     (inherit qtsvg)
     (name "qtwebengine")
     (version (package-version qtbase))
-    (source (origin
-              (method url-fetch)
-              (uri (string-append "https://download.qt.io/official_releases/qt/"
-                                  (version-major+minor version) "/" version
-                                  "/submodules/" name "-everywhere-src-"
-                                  version ".tar.xz"))
-              (sha256
-               (base32
-                "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))))
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://download.qt.io/official_releases/qt/"
+                           (version-major+minor version) "/" version
+                           "/submodules/" name "-everywhere-src-"
+                           version ".tar.xz"))
+       (sha256
+        (base32
+         "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))
+       (modules '((ice-9 ftw)
+                  (ice-9 match)
+                  (srfi srfi-1)
+                  (srfi srfi-26)
+                  (guix build utils)))
+       (snippet
+        '(begin
+           (let ((preserved-third-party-files
+                  '("base/third_party/dmg_fp"
+                    "base/third_party/dynamic_annotations"
+                    "base/third_party/icu"
+                    "base/third_party/libevent"
+                    "base/third_party/nspr"
+                    "base/third_party/superfasthash"
+                    "base/third_party/symbolize"
+                    "base/third_party/xdg_mime"
+                    "base/third_party/xdg_user_dirs"
+                    "net/third_party/http2"
+                    "net/third_party/mozilla_security_manager"
+                    "net/third_party/nss"
+                    "net/third_party/spdy"
+                    "net/third_party/quic"
+                    "third_party/abseil-cpp"
+                    "third_party/angle"
+                    "third_party/angle/src/common/third_party/base"
+                    "third_party/angle/src/common/third_party/smhasher"
+                    "third_party/angle/src/third_party/compiler"
+                    "third_party/blink"
+                    "third_party/boringssl"
+                    "third_party/boringssl/src/third_party/fiat"
+                    "third_party/breakpad"
+                    "third_party/brotli"
+                    "third_party/ced"
+                    "third_party/crc32c"
+                    "third_party/ffmpeg"
+                    "third_party/googletest"
+                    "third_party/hunspell"
+                    "third_party/iccjpeg"
+                    "third_party/icu"
+                    "third_party/inspector_protocol"
+                    "third_party/jinja2"
+                    "third_party/jstemplate"
+                    "third_party/khronos"
+                    "third_party/leveldatabase"
+                    "third_party/libaddressinput"
+                    "third_party/libjingle_xmpp"
+                    "third_party/libpng"
+                    "third_party/libsrtp"
+                    "third_party/libsync"
+                    "third_party/libudev"
+                    "third_party/libvpx"
+                    "third_party/libwebm"
+                    "third_party/libwebp"
+                    "third_party/libxml"
+                    "third_party/libxslt"
+                    "third_party/libyuv"
+                    "third_party/lss"
+                    "third_party/markupsafe"
+                    "third_party/mesa"
+                    "third_party/metrics_proto"
+                    "third_party/modp_b64"
+                    "third_party/opus"
+                    "third_party/ots"
+                    "third_party/ply"
+                    "third_party/polymer"
+                    "third_party/protobuf"
+                    "third_party/pyjson5"
+                    "third_party/re2"
+                    "third_party/rnnoise"
+                    "third_party/skia"
+                    "third_party/skia/third_party/gif"
+                    "third_party/skia/third_party/skcms"
+                    "third_party/smhasher"
+                    "third_party/snappy"
+                    "third_party/sqlite"
+                    "third_party/usrsctp"
+                    "third_party/web-animations-js"
+                    "third_party/webrtc"
+                    "third_party/webrtc_overrides"
+                    "third_party/widevine/cdm/widevine_cdm_common.h"
+                    "third_party/widevine/cdm/widevine_cdm_version.h"
+                    "third_party/woff2"
+                    "third_party/yasm"
+                    "third_party/zlib"
+                    "url/third_party/mozilla"
+                    "v8/src/third_party/utf8-decoder"
+                    "v8/src/third_party/valgrind"
+                    "v8/third_party/antlr4"
+                    "v8/third_party/inspector_protocol"))
+                 (protected (make-regexp "\\.(gn|gyp)i?$")))
+             (define preserved-club
+               (map (lambda (member)
+                      (string-append "./" member))
+                    preserved-third-party-files))
+             (define (empty? dir)
+               (equal? (scandir dir) '("." "..")))
+             (define (third-party? file)
+               (if (string-contains file "third_party/")
+                   #t
+                   #f))
+             (define (useless? file)
+               (any (cute string-suffix? <> file)
+                    '(".zip" ".so" ".dll" ".exe" ".jar")))
+             (define (parents child)
+               ;; Return all parent directories of CHILD up to and including
+               ;; the closest "third_party".
+               (let* ((dirs (match (string-split child #\/)
+                              ((dirs ... last) dirs)))
+                      (closest (list-index (lambda (dir)
+                                             (string=? "third_party" dir))
+                                           (reverse dirs)))
+                      (delim (- (length dirs) closest)))
+                 (fold (lambda (dir prev)
+                         (cons (string-append (car prev) "/" dir)
+                               prev))
+                       (list (string-join (list-head dirs delim) "/"))
+                       (list-tail dirs delim))))
+             (define (remove-loudly file)
+                (format #t "deleting ~a...~%" file)
+                (force-output)
+                (delete-file file))
+             (define (delete-unwanted-files child stat flag base level)
+               (match flag
+                 ((or 'regular 'symlink 'stale-symlink)
+                  (when (third-party? child)
+                    (unless (or (member child preserved-club)
+                                (any (cute member <> preserved-club)
+                                     (parents child))
+                                (regexp-exec protected child))
+                      (remove-loudly child)))
+                  (when (and (useless? child) (file-exists? child))
+                    (remove-loudly child))
+                  #t)
+                 ('directory-processed
+                  (when (empty? child)
+                    (rmdir child))
+                  #t)
+                 (_ #t)))
+
+             (with-directory-excursion "src/3rdparty"
+               ;; TODO: Try removing "gn" too for future versions of qtwebengine.
+               (delete-file-recursively "ninja")
+
+               (with-directory-excursion "chromium"
+                 ;; Delete bundled software and binaries that were not explicitly
+                 ;; preserved above.
+                 (nftw "." delete-unwanted-files 'depth 'physical)
+
+                 ;; Assert that each preserved item is present to catch removals.
+                 (for-each (lambda (third-party)
+                             (unless (file-exists? third-party)
+                               (error (format #f "~s does not exist!" third-party))))
+                           preserved-club)
+
+                 ;; Use relative header locations instead of hard coded ones.
+                 (substitute*
+                     "base/third_party/dynamic_annotations/dynamic_annotations.c"
+                   (("base/third_party/valgrind") "valgrind"))
+                 (substitute*
+                     "third_party/breakpad/breakpad/src/common/linux/libcurl_wrapper.h"
+                   (("third_party/curl") "curl"))
+
+                 ;; Replace Google Analytics bundle with an empty file and hope
+                 ;; no one notices.
+                 (mkdir-p "third_party/analytics")
+                 (call-with-output-file
+                     "third_party/analytics/google-analytics-bundle.js"
+                   (lambda (port)
+                     (const #t)))))
+
+             ;; Do not build support for the Widevine DRM.
+             (substitute* "src/core/config/common.pri"
+               (("enable_widevine=true")
+                "enable_widevine=false"))
+             #t)))))
     (build-system gnu-build-system)
     (native-inputs
      `(("bison" ,bison)
@@ -2059,6 +2237,7 @@ time Web content can be enhanced with native controls.")
      `(("alsa-lib" ,alsa-lib)
        ("atk" ,atk)
        ("cups-minimal" ,cups-minimal)
+       ("curl" ,curl)
        ("dbus" ,dbus)
        ("ffmpeg" ,ffmpeg)
        ("fontconfig" ,fontconfig)
@@ -2102,6 +2281,7 @@ time Web content can be enhanced with native controls.")
        ("re2" ,re2)
        ("snappy" ,snappy)
        ("udev" ,eudev)
+       ("valgrind" ,valgrind)
        ("xcb-util" ,xcb-util)))
     (arguments
      (substitute-keyword-arguments (package-arguments qtsvg)
-- 
2.24.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[mike.rosset]

Marius Bakke <mbakke@fastmail.com> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> Hi Mike,
>>
>> thank you for your patient reply.
>>
>>> Ricardo Wurmus <rekado@elephly.net> writes:
>>>
>>>> Does the code include DRM support, for example?  Does it phone home?
>>>> Can we see the patches that upstream applied to the Chromium sources?
>>>>
>>>
>>> QtWebengine does not handle DRM at all. That is handled by a 3rd party
>>> plugin called wildevine and would require guix to bundle that plugin.
>>> Something I'm confident is not going to happen. As an added measure my
>>> proposed qtwebengine declaration has pepper plugins disabled making it
>>> impossible to load 3rd party plugins either way.
>>>
>>> see https://doc.qt.io/qt-5/qtwebengine-features.html#html5-drm
>>> for reference
>>
>> I’m aware of this.  My apologies for not expressing myself more
>> clearly.  We probably want to remove the support for the DRM plugin,
>> following the discussions about ungoogled-chromium at
>>
>>     https://issues.guix.gnu.org/issue/34565
>
> I have attached a patch that disables support for loading the Widevine
> plugin, and also removes unused bundled software from the source based
> on a whitelist, similar to ungoogled-chromium's
> %PRESERVED-THIRD-PARTY-FILES.
>
> It removes some 300+ MiB from the uncompressed sources.  The snippet
> will require some adjustments for every new version of QtWebEngine
> however, I can help out with that when required.
>
> Patch 1/2 is a slightly "cleaned up" version of Mike's latest patch,
> fixing some cosmetic issues, and makes it respect the daemons requested
> number of build cores.
>
> With these changes, I think the package is ready to go.  \o/

This is really great work Marius I appreciate this addition. I was
actually just looking at this approach yesterday. 

I've merged your two patches with one minor description addition that Hartmut
requested.

Hartmut the merged complete diff can be found here.

https://gitlab.com/mike.rosset/guix/compare/master...merge-qtwebengine

 
Mike 

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1572 bytes --]

Fantastic, Marius!  Some nits below:

>  ;;; Copyright © 2019 Marius Bakke <mbakke@fastmail.com>

2020 for Marius?

> +       ;; FIXME: libxml2 needs to built with icu support though it links to
> +       ;; libxml2 configure summary still states. Checking for compatible
> +       ;; system libxml2... no

Punctuation:

--8<---------------cut here---------------start------------->8---
... still states "Checking for compatible system libxml2... no".
--8<---------------cut here---------------end--------------->8---

> +       ;; Tests are disabled due to Could not find QtWebEngineProcess error
> +       ;; It's possible this can be fixed by setting QTWEBENGINEPROCESS_PATH
> +       ;; before running tests It's also possible some tests need a network

Punctuation:

--8<---------------cut here---------------start------------->8---
Test are disabled due to "Could not find QtWebEngineProcess" error.
It's possible this can be fixed by setting QTWEBENGINEPROCESS_PATH
before running tests.  It's also possible some tests need a network.
--8<---------------cut here---------------end--------------->8---

> +             (define (third-party? file)
> +               (if (string-contains file "third_party/")
> +                   #t
> +                   #f))

Maybe just

--8<---------------cut here---------------start------------->8---
(define (third-party? file)
  (string-contains file "third_party/"))
--8<---------------cut here---------------end--------------->8---

?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 276 bytes --]

One more thing:

> +                    "third_party/widevine/cdm/widevine_cdm_common.h"
> +                    "third_party/widevine/cdm/widevine_cdm_version.h"

Why do we preserve those files if we don't build widevine?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Hartmut Goebel]

Am 07.01.20 um 01:27 schrieb mike.rosset@gmail.com:
>
>> With these changes, I think the package is ready to go.  \o/
> This is really great work Marius I appreciate this addition. I was
> actually just looking at this approach yesterday. 
>
> I've merged your two patches with one minor description addition that Hartmut
> requested.


Great work you've done! I'm looking forward to get this merged to
master, so I can continue working on some more KDE packages.


> Hartmut the merged complete diff can be found here.
> https://gitlab.com/mike.rosset/guix/compare/master...merge-qtwebengine

Fine for me.

Just some super-nitpicky nitpicks:


> +               (base32
> +               
> "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))))

Maybe move has into `base32` line


> +             (define (third-party? file)
> +               (if (string-contains file "third_party/")
> +                   #t
> +                   #f))
The "if" could be omitted, I assume


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: qtwenengine anybody?

[mike.rosset]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Fantastic, Marius!  Some nits below:
>
>>  ;;; Copyright © 2019 Marius Bakke <mbakke@fastmail.com>
>
> 2020 for Marius?
>
>> +       ;; FIXME: libxml2 needs to built with icu support though it links to
>> +       ;; libxml2 configure summary still states. Checking for compatible
>> +       ;; system libxml2... no
>
> Punctuation:
>
> ... still states "Checking for compatible system libxml2... no".
>
>
>> +       ;; Tests are disabled due to Could not find QtWebEngineProcess error
>> +       ;; It's possible this can be fixed by setting QTWEBENGINEPROCESS_PATH
>> +       ;; before running tests It's also possible some tests need a network
>
> Punctuation:
>
> Test are disabled due to "Could not find QtWebEngineProcess" error.
> It's possible this can be fixed by setting QTWEBENGINEPROCESS_PATH
> before running tests.  It's also possible some tests need a network.

Will fix these ASAP, thanks for pointing theses out.

>> +             (define (third-party? file)
>> +               (if (string-contains file "third_party/")
>> +                   #t
>> +                   #f))
>
> Maybe just
>
> (define (third-party? file)
>   (string-contains file "third_party/"))
>
> ?

I have a patch to fix this will include it with punctuation.

Re: qtwenengine anybody?

[mike.rosset]

[-- Attachment #1: Type: text/plain, Size: 1598 bytes --]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 07.01.20 um 01:27 schrieb mike.rosset@gmail.com:
>>
>>> With these changes, I think the package is ready to go.  \o/
>> This is really great work Marius I appreciate this addition. I was
>> actually just looking at this approach yesterday. 
>>
>> I've merged your two patches with one minor description addition that Hartmut
>> requested.
>
>
> Great work you've done! I'm looking forward to get this merged to
> master, so I can continue working on some more KDE packages.
>
>
>> Hartmut the merged complete diff can be found here.
>> https://gitlab.com/mike.rosset/guix/compare/master...merge-qtwebengine
>
> Fine for me.
>
> Just some super-nitpicky nitpicks:
>
>
>> +               (base32
>> +               
>> "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))))
>
> Maybe move has into `base32` line
>
>
>> +             (define (third-party? file)
>> +               (if (string-contains file "third_party/")
>> +                   #t
>> +                   #f))
> The "if" could be omitted, I assume


Hello Hartmut,

I've attached a patch that resolves these suggestions. Along with
patches to fix punctuation suggested by Pierre.

Complete diff can still be found
at. https://gitlab.com/mike.rosset/guix/compare/master...merge-qtwebengine

Marius I've forgot to attach from your series in my last email. I've
attached the rest of the series for you. encase you want to git am.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0003-gnu-qtwebengine-description-field-now-notes-source-r.patch --]
[-- Type: text/x-patch, Size: 1190 bytes --]

From 531edfc848c7253b6768ab5404773a5db54da6f2 Mon Sep 17 00:00:00 2001
From: Mike Rosset <mike.rosset@gmail.com>
Date: Mon, 6 Jan 2020 16:16:18 -0800
Subject: [PATCH 3/5] gnu: qtwebengine: description field now notes source re
 base summary

* gnu/packages/qt.scm (qtwebengine)[description]: Note Qt Chromium source re
  base details.
---
 gnu/packages/qt.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 38a620c90c..87f3470773 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -2340,8 +2340,9 @@ time Web content can be enhanced with native controls.")
             (files '("lib/qt5/libexec/QtWebEngineProcess")))))
     (home-page "https://www.qt.io")
     (synopsis "Qt WebEngine module")
-    (description "Qt5WebEngine provides support for web
-applications using the Chromium browser project.")
+    (description "Qt5WebEngine provides support for web applications using the
+Chromium browser project. The Chromium source base has Google services and
+binaries removed along with modular support for system libraries.")
     (license license:lgpl2.1+)))
 
 (define-public dotherside
-- 
2.24.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0004-gnu-qtwebengine-simplify-snippet-field-third-party-f.patch --]
[-- Type: text/x-patch, Size: 1651 bytes --]

From 1dfcc7e834cf2d6c28d297e4ffa801664f0532a1 Mon Sep 17 00:00:00 2001
From: Mike Rosset <mike.rosset@gmail.com>
Date: Tue, 7 Jan 2020 10:02:27 -0800
Subject: [PATCH 4/5] gnu: qtwebengine: simplify snippet field third-party?
 function

* gnu/packages/qt.scm (qtwebengine)[snippet]: string-contains? returns a
  boolean so 'if condition is not required.

Small syntax improvement. base32 field argument is now on the same line
---
 gnu/packages/qt.scm | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 87f3470773..78486c3622 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -2046,8 +2046,7 @@ time Web content can be enhanced with native controls.")
                            "/submodules/" name "-everywhere-src-"
                            version ".tar.xz"))
        (sha256
-        (base32
-         "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))
+        (base32 "08c60nh95m98mcqk444axs76xi6m9x0wvdxrzk9c2cxwqdbz59fa"))
        (modules '((ice-9 ftw)
                   (ice-9 match)
                   (srfi srfi-1)
@@ -2145,9 +2144,7 @@ time Web content can be enhanced with native controls.")
              (define (empty? dir)
                (equal? (scandir dir) '("." "..")))
              (define (third-party? file)
-               (if (string-contains file "third_party/")
-                   #t
-                   #f))
+               (string-contains file "third_party/"))
              (define (useless? file)
                (any (cute string-suffix? <> file)
                     '(".zip" ".so" ".dll" ".exe" ".jar")))
-- 
2.24.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0005-gnu-qtwebengine-improve-punctuation-within-comments.patch --]
[-- Type: text/x-patch, Size: 1898 bytes --]

From 6916f2f253e127d354be6e2a0bc2ac8af52ad159 Mon Sep 17 00:00:00 2001
From: Mike Rosset <mike.rosset@gmail.com>
Date: Tue, 7 Jan 2020 11:17:41 -0800
Subject: [PATCH 5/5] gnu: qtwebengine: improve punctuation within comments.

* gnu/packages/qt.scm (qtwebengine)[comments]: Add appropriate punctuation to
  input and test comments. This helps differentiate context of some warnings
  and errors.
---
 gnu/packages/qt.scm | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/qt.scm b/gnu/packages/qt.scm
index 78486c3622..0d060de0c3 100644
--- a/gnu/packages/qt.scm
+++ b/gnu/packages/qt.scm
@@ -2257,8 +2257,8 @@ time Web content can be enhanced with native controls.")
        ("libxi" ,libxi)
        ("libxkbcommon" ,libxkbcommon)
        ;; FIXME: libxml2 needs to built with icu support though it links to
-       ;; libxml2 configure summary still states. Checking for compatible
-       ;; system libxml2... no
+       ;; libxml2 configure summary still states "Checking for compatible
+       ;; system libxml2... no"
        ("libxml2" ,libxml2)
        ("libxrandr" ,libxrandr)
        ("libxrender" ,libxrender)
@@ -2325,9 +2325,9 @@ time Web content can be enhanced with native controls.")
                        "--webengine-ffmpeg=system"
                        "--webengine-icu=system"
                        "--webengine-pepper-plugins=no")))))
-       ;; Tests are disabled due to Could not find QtWebEngineProcess error
+       ;; Tests are disabled due to "Could not find QtWebEngineProcess error"
        ;; It's possible this can be fixed by setting QTWEBENGINEPROCESS_PATH
-       ;; before running tests It's also possible some tests need a network
+       ;; before running tests.  It's also possible some tests need a network
        ((#:tests? _ #f) #f)))
     (native-search-paths
      (list (search-path-specification
-- 
2.24.1


[-- Attachment #5: Type: text/plain, Size: 6 bytes --]


Mike

Re: qtwenengine anybody?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> One more thing:
>
>> +                    "third_party/widevine/cdm/widevine_cdm_common.h"
>> +                    "third_party/widevine/cdm/widevine_cdm_version.h"
>
> Why do we preserve those files if we don't build widevine?

They are #include'd by some files unconditionally.  It should be
possible to #ifdef out at least widevine_cdm_common.h; I'll look into it
for the next ungoogled-chromium update.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

mike.rosset@gmail.com writes:

> I've attached a patch that resolves these suggestions. Along with
> patches to fix punctuation suggested by Pierre.
>
> Complete diff can still be found
> at. https://gitlab.com/mike.rosset/guix/compare/master...merge-qtwebengine
>
> Marius I've forgot to attach from your series in my last email. I've
> attached the rest of the series for you. encase you want to git am.

Great!  I've squashed them appropriately and finally merged qtwebengine!

I also moved the package definition alongside the other first party Qt
modules.

Now we just need some packages using it!  :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

Hooray!

Thank you Marius, and thank you Mike for the tremendous effort!

> Now we just need some packages using it!  :-)

We can get started with Qutebrowser.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Mike Rosset]

Marius Bakke writes:

> mike.rosset@gmail.com writes:
>
>> I've attached a patch that resolves these suggestions. Along with
>> patches to fix punctuation suggested by Pierre.
>>
>> Complete diff can still be found
>> at. https://gitlab.com/mike.rosset/guix/compare/master...merge-qtwebengine
>>
>> Marius I've forgot to attach from your series in my last email. I've
>> attached the rest of the series for you. encase you want to git am.
>
> Great!  I've squashed them appropriately and finally merged qtwebengine!
>
> I also moved the package definition alongside the other first party Qt
> modules.
>
> Now we just need some packages using it!  :-)

Thank you Marius, I appreciate this being merged and all the suggestions
everyone provided.

Mike

Re: qtwenengine anybody?

[Mike Rosset]

Pierre Neidhardt writes:

>> Now we just need some packages using it!  :-)
>
> We can get started with Qutebrowser.

This was my first thought as well. Though next is also a good candidate
too? :)

If anyone is working on updating quitebrowser let me know. If not I'll
starting working on this soon.

Side note Pierre, I'll switch GIQT over to this in the next couple of
days as well.

Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

Mike Rosset <mike.rosset@gmail.com> writes:

> This was my first thought as well. Though next is also a good candidate
> too? :)

QtWebengine support is still somewhat experimental in Next.  I haven't
worked on it directly, but I could try and see if it works on Guix.

> If anyone is working on updating quitebrowser let me know. If not I'll
> starting working on this soon.

I think updating qutebrowser to use QtWebengine should be trivial.
If you have time during the next couple of days, please go ahead :)

> Side note Pierre, I'll switch GIQT over to this in the next couple of
> days as well.

Fantastic, thanks!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Mike Rosset]

Pierre Neidhardt writes:

> Hooray!
>
> Thank you Marius, and thank you Mike for the tremendous effort!
>
>> Now we just need some packages using it!  :-)
>
> We can get started with Qutebrowser.

Hello Pierre and Marius,

I've have sent some patches to an open bug #38148 in regards to
qutebrowser being outdated. I'm hoping this will help test out
qtwebengine and close that bug. Two birds with one stone :)

The patch includes python-pyqtwebengine and upgrade of qutebrowser to
1.9.0 and it now uses qtwebengine.

If anyone can help review and make suggestions that would be great.

On second thought maybe I should have emailed them to
guix-patches@gnu.org. Let me know if that's better for reviewing.

Mike

Re: qtwenengine anybody?

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

Hi Mike,

> I've have sent some patches to an open bug #38148 in regards to
> qutebrowser being outdated. I'm hoping this will help test out
> qtwebengine and close that bug. Two birds with one stone :)

Qutebrowser 1.9.0 works now, great job!
I've reviewed those patches; fix the few nits and I'll merge.

> On second thought maybe I should have emailed them to
> guix-patches@gnu.org. Let me know if that's better for reviewing.

If you think the bug title is relevant, then it's the right thing to do.
That said, it seems that you've sent the first patch twice.  To clarify,
this is how you should proceed to send patches to debbugs:

1. Send the first patch only.  If you send with "git send-email", you
should see the message ID in the output.

2. Send all other patches by using the "--in-reply-to=$ABOVE_MESSAGE_ID"
option.  Make sure the patches are in the right order on the command
line.  Example:

  git send-email --to=38148@debbugs.gnu.org \
  --in-reply-to='<20200113103304.9093-1-mike.rosset@gmail.com>' \
  0002-patch-foo.diff 0003-patch-bar.diff
  
Hope that helps!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: qtwenengine anybody?

[Mike Rosset]

> 2. Send all other patches by using the "--in-reply-to=$ABOVE_MESSAGE_ID"
> option.  Make sure the patches are in the right order on the command
> line.  Example:
>
>   git send-email --to=38148@debbugs.gnu.org \
>   --in-reply-to='<20200113103304.9093-1-mike.rosset@gmail.com>' \
>   0002-patch-foo.diff 0003-patch-bar.diff
>
> Hope that helps!

Ah I appreciate this feedback. Hopefully I can improve my ML patch
workflow.

Mike