Qt-5 in core-updates

Qt-5 in core-updates

[Andreas Enge]

Hello,

qt-5 fails to build on both x86_64 and armhf in core-updates with the
following error:
Traceback (most recent call last):
  File "/tmp/nix-build-qt-5.5.1.drv-0/qt-everywhere-opensource-src-5.5.1/qtwebkit/Source/WebKit2/Scripts/generate-messages-header.py", line 28, in <module>
    import webkit2.messages
EOFError: EOF read where object expected
Makefile.WebKit2.DerivedSources:881: recipe for target 'generated/PluginProxyMessages.h' failed
make[3]: *** [generated/PluginProxyMessages.h] Error 1

Surprisingly, it succeeds on i686 (and its build is disabled on mips).

宋文武, you appear to be the most knowledgeable among us concerning Qt;
could you maybe have look?

Thank you,

Andreas

Re: Qt-5 in core-updates

[Andreas Enge]

In fact, the same problem manifests itself in webkitgtk itself:
   http://hydra.gnu.org/build/954055/nixlog/1/tail-reload
after only two minutes of building.

Andreas

Re: Qt-5 in core-updates

[宋文武]

Andreas Enge <andreas@enge.fr> writes:

> Hello,
>
> qt-5 fails to build on both x86_64 and armhf in core-updates with the
> following error:
> Traceback (most recent call last):
>   File "/tmp/nix-build-qt-5.5.1.drv-0/qt-everywhere-opensource-src-5.5.1/qtwebkit/Source/WebKit2/Scripts/generate-messages-header.py", line 28, in <module>
>     import webkit2.messages
> EOFError: EOF read where object expected
> Makefile.WebKit2.DerivedSources:881: recipe for target 'generated/PluginProxyMessages.h' failed
> make[3]: *** [generated/PluginProxyMessages.h] Error 1
>
> Surprisingly, it succeeds on i686 (and its build is disabled on mips).
>
This seems like a parallel build issue after I find that,
Writing a pyc file is not atomic in python:

  <https://bugs.python.org/issue13146>

It was fixed in Python 3.3+, but not in Python 2.7.x (the one for
building WebKit).

I think we can try:
 - Patch our python-2.7.x like Ubuntu does.
   <https://launchpad.net/ubuntu/+archive/primary/+files/python2.7_2.7.10-4ubuntu1.diff.gz>

 - Patch the build phases of qt and webkitgtk to pre-generate *.pyc
   files before the actually building.
   Or disable parallel build completely.

 - Restart the builds on hydra, hope it will pass.

Re: Qt-5 in core-updates

[Ludovic Courtès]

iyzsong@member.fsf.org (宋文武) skribis:

> Andreas Enge <andreas@enge.fr> writes:
>
>> Hello,
>>
>> qt-5 fails to build on both x86_64 and armhf in core-updates with the
>> following error:
>> Traceback (most recent call last):
>>   File "/tmp/nix-build-qt-5.5.1.drv-0/qt-everywhere-opensource-src-5.5.1/qtwebkit/Source/WebKit2/Scripts/generate-messages-header.py", line 28, in <module>
>>     import webkit2.messages
>> EOFError: EOF read where object expected
>> Makefile.WebKit2.DerivedSources:881: recipe for target 'generated/PluginProxyMessages.h' failed
>> make[3]: *** [generated/PluginProxyMessages.h] Error 1
>>
>> Surprisingly, it succeeds on i686 (and its build is disabled on mips).
>>
> This seems like a parallel build issue after I find that,
> Writing a pyc file is not atomic in python:
>
>   <https://bugs.python.org/issue13146>

Ouch.

> It was fixed in Python 3.3+, but not in Python 2.7.x (the one for
> building WebKit).
>
> I think we can try:
>  - Patch our python-2.7.x like Ubuntu does.
>    <https://launchpad.net/ubuntu/+archive/primary/+files/python2.7_2.7.10-4ubuntu1.diff.gz>

Best if we can avoid it for now since that would further delay the merge
of ‘core-updates’.

>  - Patch the build phases of qt and webkitgtk to pre-generate *.pyc
>    files before the actually building.

If that’s simple enough, we could add a phase that run “make
whatever-target-it-is”, indeed.

>    Or disable parallel build completely.

Sounds like the easiest thing to do.  Not sure how long it would take to
build Qt, though.

>  - Restart the builds on hydra, hope it will pass.

I think we should avoid that.

Andreas, are you on it?

Ludo’.

Re: Qt-5 in core-updates

[Andreas Enge]

Thanks, 宋文武, for this thorough research!

On Mon, Jan 25, 2016 at 09:57:13AM +0800, 宋文武 wrote:
>  - Patch our python-2.7.x like Ubuntu does.
>    <https://launchpad.net/ubuntu/+archive/primary/+files/python2.7_2.7.10-4ubuntu1.diff.gz>

To me, this sounds like the cleanest solution, but probably not in this core-
updates run, which has taken quite some time already.

>  - Patch the build phases of qt and webkitgtk to pre-generate *.pyc
>    files before the actually building.

This I would not know how to do, and we would need to do it for webkitgtk
also (of which we have three variants! so an independent question: can this
maybe be reduced? it seems to depend on gnucash).

>  - Restart the builds on hydra, hope it will pass.

This one I already tried, and I also built qt-5 once on my machine.
Áll three builds failed consistently. So something has apparently changed
on core-updates compared to master that triggers this failure.

>    Or disable parallel build completely.

That looks like the best short-term solution. I will try it locally and
push it to core-updates if it works, with "FIXME" remarks for after this
core-updates cycle.

Andreas

Re: Qt-5 in core-updates

[宋文武]

Andreas Enge <andreas@enge.fr> writes:

> Thanks, 宋文武, for this thorough research!
>
> On Mon, Jan 25, 2016 at 09:57:13AM +0800, 宋文武 wrote:
>>  - Patch our python-2.7.x like Ubuntu does.
>>    <https://launchpad.net/ubuntu/+archive/primary/+files/python2.7_2.7.10-4ubuntu1.diff.gz>
>
> To me, this sounds like the cleanest solution, but probably not in this core-
> updates run, which has taken quite some time already.
OK, and in fact I don't know whether the patch will fix this issue or
not.  Waiting help from python folks :-)
>
>>  - Patch the build phases of qt and webkitgtk to pre-generate *.pyc
>>    files before the actually building.
>
> This I would not know how to do, and we would need to do it for webkitgtk
> also (of which we have three variants! so an independent question: can this
> maybe be reduced? it seems to depend on gnucash).
Well, WebKitGTK+ 2.4.9 seem unmaintained and have some security issues:
  <https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/S3VHBCPMPVZ3NBKR7FQZQE6HBUHVEZ3D/>
But currently, we have gnucash, guitarix, and some gnome applications
using it.
>
>>  - Restart the builds on hydra, hope it will pass.
>
> This one I already tried, and I also built qt-5 once on my machine.
> Áll three builds failed consistently. So something has apparently changed
> on core-updates compared to master that triggers this failure.
>
>>    Or disable parallel build completely.
>
> That looks like the best short-term solution. I will try it locally and
> push it to core-updates if it works, with "FIXME" remarks for after this
> core-updates cycle.
Yes, thanks for handling this.

Re: Qt-5 in core-updates

[Andreas Enge]

On Tue, Jan 26, 2016 at 10:23:10AM +0800, 宋文武 wrote:
> Well, WebKitGTK+ 2.4.9 seem unmaintained and have some security issues:
>   <https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/S3VHBCPMPVZ3NBKR7FQZQE6HBUHVEZ3D/>
> But currently, we have gnucash, guitarix, and some gnome applications
> using it.

So even more than I thought. Maybe this would be something to look into
after the core-updates merge, to see whether updating these dependent
packages will help.

> Yes, thanks for handling this.

Thanks for doing the research!

Andreas

Re: Qt-5 in core-updates

[Andreas Enge]

Hello,

now that core-updates is merged, could you maybe investigate about this matter:

On Mon, Jan 25, 2016 at 09:57:13AM +0800, 宋文武 wrote:
> This seems like a parallel build issue after I find that,
> Writing a pyc file is not atomic in python:
>   <https://bugs.python.org/issue13146>
> It was fixed in Python 3.3+, but not in Python 2.7.x (the one for
> building WebKit).
> I think we can try:
>  - Patch our python-2.7.x like Ubuntu does.
>    <https://launchpad.net/ubuntu/+archive/primary/+files/python2.7_2.7.10-4ubuntu1.diff.gz>

It would be good to revert to parallel builds of qt.

Actually, there is a version 2.7.11 of python. Should we update first?
Will this solve our problems?

Andreas

Re: Qt-5 in core-updates

[Andreas Enge]

On Tue, Feb 02, 2016 at 05:47:54PM +0100, Andreas Enge wrote:
> On Mon, Jan 25, 2016 at 09:57:13AM +0800, 宋文武 wrote:
> > This seems like a parallel build issue after I find that,
> > Writing a pyc file is not atomic in python:
> >   <https://bugs.python.org/issue13146>
> > It was fixed in Python 3.3+, but not in Python 2.7.x (the one for
> > building WebKit).
> > I think we can try:
> >  - Patch our python-2.7.x like Ubuntu does.
> >    <https://launchpad.net/ubuntu/+archive/primary/+files/python2.7_2.7.10-4ubuntu1.diff.gz>
> It would be good to revert to parallel builds of qt.
> Actually, there is a version 2.7.11 of python. Should we update first?
> Will this solve our problems?

In fact, there is a very recent release of Qt 5.6, which removes webkit;
so we could try an update. On the other hand, without webkit and the bundled
chromium, there will be no webbrowsers left over... 5.6 is supposed to be
a long term support version, with fixes promised over the next three years.
So would it make sense to reenable chromium (=qtwebengine)?

Andreas