Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Mark H Weaver]

mbakke@fastmail.com (Marius Bakke) writes:

> mbakke pushed a commit to branch master
> in repository guix.
>
> commit 2a087882b1cb2e40e7b63580c675b58a5cfa1b96
> Author: Marius Bakke <mbakke@fastmail.com>
> Date:   Thu Oct 12 18:57:30 2017 +0200
>
>     gnu: mesa: Disable imx driver for armhf-linux.
>     
>     * gnu/packages/gl.scm (mesa)<#:configure-flags>: Don't build imx driver.

What is the rationale for this change?  Although "guix refresh -l mesa"
is failing to work for me at the moment, I'm fairly sure that this
change will entail a massive rebuild on armhf-linux.  In general, this
kind of change belongs on another branch.

      Mark

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1593 bytes --]

Mark H Weaver <mhw@netris.org> writes:

> mbakke@fastmail.com (Marius Bakke) writes:
>
>> mbakke pushed a commit to branch master
>> in repository guix.
>>
>> commit 2a087882b1cb2e40e7b63580c675b58a5cfa1b96
>> Author: Marius Bakke <mbakke@fastmail.com>
>> Date:   Thu Oct 12 18:57:30 2017 +0200
>>
>>     gnu: mesa: Disable imx driver for armhf-linux.
>>     
>>     * gnu/packages/gl.scm (mesa)<#:configure-flags>: Don't build imx driver.
>
> What is the rationale for this change?  Although "guix refresh -l mesa"
> is failing to work for me at the moment, I'm fairly sure that this
> change will entail a massive rebuild on armhf-linux.  In general, this
> kind of change belongs on another branch.

Hello!

Sorry, I should have made the commit comment clearer.  The 'imx' driver
depends on 'etnaviv', which failed to build due to symbols mismatch in
'libdrm'.  From the build log in <https://hydra.gnu.org/build/2318253>:

configure: error: Building with imx requires etnaviv

Note that mesa and libdrm did not build any drivers at all on armhf
until recent commits on 'staging'.  I tried cross-compiling libdrm
to update etnaviv symbols instead, but failed some packages before it.

So currently it's a trial-and-error process to find flags to make mesa
build on armhf.  This means armhf users are currently unable to build
*any* graphical packages, actually.  Given how expensive evaluations
are, I figured we might as well deal with it on 'master'.

Hopefully users affected by this can give a hand in finding a working
libdrm/mesa configuration.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Mark H Weaver]

Hi Marius,

Marius Bakke <mbakke@fastmail.com> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> mbakke@fastmail.com (Marius Bakke) writes:
>>
>>> mbakke pushed a commit to branch master
>>> in repository guix.
>>>
>>> commit 2a087882b1cb2e40e7b63580c675b58a5cfa1b96
>>> Author: Marius Bakke <mbakke@fastmail.com>
>>> Date:   Thu Oct 12 18:57:30 2017 +0200
>>>
>>>     gnu: mesa: Disable imx driver for armhf-linux.
>>>     
>>>     * gnu/packages/gl.scm (mesa)<#:configure-flags>: Don't build imx driver.
>>
>> What is the rationale for this change?  Although "guix refresh -l mesa"
>> is failing to work for me at the moment, I'm fairly sure that this
>> change will entail a massive rebuild on armhf-linux.  In general, this
>> kind of change belongs on another branch.
>
> Hello!
>
> Sorry, I should have made the commit comment clearer.  The 'imx' driver
> depends on 'etnaviv', which failed to build due to symbols mismatch in
> 'libdrm'.  From the build log in <https://hydra.gnu.org/build/2318253>:
>
> configure: error: Building with imx requires etnaviv
>
> Note that mesa and libdrm did not build any drivers at all on armhf
> until recent commits on 'staging'.  I tried cross-compiling libdrm
> to update etnaviv symbols instead, but failed some packages before it.
>
> So currently it's a trial-and-error process to find flags to make mesa
> build on armhf.  This means armhf users are currently unable to build
> *any* graphical packages, actually.  Given how expensive evaluations
> are, I figured we might as well deal with it on 'master'.

Thanks for explaining.  I think this should have been dealt with on the
'staging' branch before merging into 'master'.  This is a pretty bad
situation now for anyone using Guix on armhf.

        Mark

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> Note that mesa and libdrm did not build any drivers at all on armhf
>> until recent commits on 'staging'.  I tried cross-compiling libdrm
>> to update etnaviv symbols instead, but failed some packages before it.
>>
>> So currently it's a trial-and-error process to find flags to make mesa
>> build on armhf.  This means armhf users are currently unable to build
>> *any* graphical packages, actually.  Given how expensive evaluations
>> are, I figured we might as well deal with it on 'master'.
>
> Thanks for explaining.  I think this should have been dealt with on the
> 'staging' branch before merging into 'master'.  This is a pretty bad
> situation now for anyone using Guix on armhf.

Didn't we already talk about this when you merged an earlier 'staging'
branch into 'master' that contained a major GNOME upgrade that was
untested, and broke GNOME desktops for all platforms?

Do you think it should be acceptable to merge a major branch into
'master' where *all* graphical packages are broken on armhf?

Do the Guix maintainers think that this is acceptable behavior?
If so, where does that leave armhf users?

      Mark

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1778 bytes --]

Mark H Weaver <mhw@netris.org> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> Marius Bakke <mbakke@fastmail.com> writes:
>>
>>> Note that mesa and libdrm did not build any drivers at all on armhf
>>> until recent commits on 'staging'.  I tried cross-compiling libdrm
>>> to update etnaviv symbols instead, but failed some packages before it.
>>>
>>> So currently it's a trial-and-error process to find flags to make mesa
>>> build on armhf.  This means armhf users are currently unable to build
>>> *any* graphical packages, actually.  Given how expensive evaluations
>>> are, I figured we might as well deal with it on 'master'.
>>
>> Thanks for explaining.  I think this should have been dealt with on the
>> 'staging' branch before merging into 'master'.  This is a pretty bad
>> situation now for anyone using Guix on armhf.
>
> Didn't we already talk about this when you merged an earlier 'staging'
> branch into 'master' that contained a major GNOME upgrade that was
> untested, and broke GNOME desktops for all platforms?
>
> Do you think it should be acceptable to merge a major branch into
> 'master' where *all* graphical packages are broken on armhf?

I naively assumed that disabling the etnaviv driver was enough, but
concede that it was short-sighted.  Unfortunately I did not notice the
armhf failures until late in the cycle due to manually restarting all
'gobject-introspection' dependents on i686 and x86_64.  And learned that
"new job" failures are not listed in the "newly failing" tab.

I feel terrible for gambling with armhf users' convenience and security
and can only offer a sincere apology.

Hopefully the most recent 'mesa' commit solves this issue, and rest
assured I won't merge a broken package with ~800 dependents again.

Humbly,
Marius

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

On Fri, Oct 13, 2017 at 08:38:13PM -0400, Mark H Weaver wrote:
> Do you think it should be acceptable to merge a major branch into
> 'master' where *all* graphical packages are broken on armhf?

I think we all agree that we should not do this.

As somebody who also works on merging these branches, I'd like to point
out that the continuous integration process for armhf is difficult to
use effectively.

The core of the problem is that the feedback loop for armhf is
unreliable and slow.

Similar problems exist with Hydra for x86_64 and i686, but they are
mitigated because most of us have access to our own x86_64 machines
where we can test changes quickly before asking Hydra to verify a
positive result.

It would be great to have access to an armhf machine where we could do
the same thing, testing changes directly without going through Hydra.

Maybe if one has a fast enough x86_64 computer, they could virtualize an
armhf test machine.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Ludovic Courtès]

Hello,

Marius Bakke <mbakke@fastmail.com> skribis:

> Mark H Weaver <mhw@netris.org> writes:
>
>> Mark H Weaver <mhw@netris.org> writes:
>>
>>> Marius Bakke <mbakke@fastmail.com> writes:
>>>
>>>> Note that mesa and libdrm did not build any drivers at all on armhf
>>>> until recent commits on 'staging'.  I tried cross-compiling libdrm
>>>> to update etnaviv symbols instead, but failed some packages before it.
>>>>
>>>> So currently it's a trial-and-error process to find flags to make mesa
>>>> build on armhf.  This means armhf users are currently unable to build
>>>> *any* graphical packages, actually.  Given how expensive evaluations
>>>> are, I figured we might as well deal with it on 'master'.
>>>
>>> Thanks for explaining.  I think this should have been dealt with on the
>>> 'staging' branch before merging into 'master'.  This is a pretty bad
>>> situation now for anyone using Guix on armhf.
>>
>> Didn't we already talk about this when you merged an earlier 'staging'
>> branch into 'master' that contained a major GNOME upgrade that was
>> untested, and broke GNOME desktops for all platforms?
>>
>> Do you think it should be acceptable to merge a major branch into
>> 'master' where *all* graphical packages are broken on armhf?
>
> I naively assumed that disabling the etnaviv driver was enough, but
> concede that it was short-sighted.  Unfortunately I did not notice the
> armhf failures until late in the cycle due to manually restarting all
> 'gobject-introspection' dependents on i686 and x86_64.  And learned that
> "new job" failures are not listed in the "newly failing" tab.
>
> I feel terrible for gambling with armhf users' convenience and security
> and can only offer a sincere apology.

Thank you, Marius.

As much as I sympathize with Mark’s frustration, and that of anyone
using Guix on ARM, I also understand how hard it is to do the work that
you’ve been doing Marius.

As Leo pointed out, the feedback loop is really slow for ARM if you
don’t have the hardware yourself (we’re doing discontinuous
integration!), which makes things even harder.  It would be ideal if, in
addition to Hydra, ARM users would chime in and report problems before
merges.

From the maintainer’s viewpoint, it just stresses that we must keep
going with the efforts to improve our build infrastructure.  Currently
that has only benefited x86, but perhaps now is a good time to plug our
ARM boxes behind berlin.guixsd.org, for a start.

Thoughts?

Ludo’.

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

On Mon, Oct 16, 2017 at 02:56:38PM +0200, Ludovic Courtès wrote:
> From the maintainer’s viewpoint, it just stresses that we must keep
> going with the efforts to improve our build infrastructure.  Currently
> that has only benefited x86, but perhaps now is a good time to plug our
> ARM boxes behind berlin.guixsd.org, for a start.
> 
> Thoughts?

What sort of interface do we have for querying berlin.guixsd.org's
evalutions (to borrow the Hydra term)?

Is it just `guix weather`, or is there some sort of historical
continuous integration view as with the Hydra interface?

For all the frustrations with Hydra, it's still invaluable to be able to
compare evaluations and look at things like "newly failing jobs", and to
share hyperlinks with each other.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Cuirass HTTP API

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Mon, Oct 16, 2017 at 02:56:38PM +0200, Ludovic Courtès wrote:
>> From the maintainer’s viewpoint, it just stresses that we must keep
>> going with the efforts to improve our build infrastructure.  Currently
>> that has only benefited x86, but perhaps now is a good time to plug our
>> ARM boxes behind berlin.guixsd.org, for a start.
>> 
>> Thoughts?
>
> What sort of interface do we have for querying berlin.guixsd.org's
> evalutions (to borrow the Hydra term)?
>
> Is it just `guix weather`, or is there some sort of historical
> continuous integration view as with the Hydra interface?

Thanks to Mathieu Othacehe, Cuirass now supports a subset of the HTTP
API of Hydra.  Currently the only interface to that is Emacs-Guix, but
we could/should write JavaScript code that does that; any takers?  :-)

So currently we can do:

  M-x guix-hydra-latest-builds

… and from there browse builds, view their status, etc.

We have yet to add support for viewing build logs (!) to ‘guix publish’
and/or Cuirass.

> For all the frustrations with Hydra, it's still invaluable to be able to
> compare evaluations and look at things like "newly failing jobs", and to
> share hyperlinks with each other.

Agreed.  We’re not there yet, but that’s not insurmountable.

BTW, there’s the possibility of a master’s or internship project at the
MDC (Berlin, DE) with Ricardo around Cuirass:

  http://bioinformatics.mdc-berlin.de/mastersprojects.html

I may also be able to recruit a master student or an intern at Inria
(Bordeaux, FR), possibly to work in this area.

If anyone is interested, please let us know!

Ludo’.

Re: Cuirass HTTP API

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> BTW, there’s the possibility of a master’s or internship project at the
> MDC (Berlin, DE) with Ricardo around Cuirass:
>
>   http://bioinformatics.mdc-berlin.de/mastersprojects.html

I should add here that according to HR there cannot be any financial
compensation for the internship project.  Financial compensation is only
available for master students who take on the work as their master’s
project.  Please take note that applicants should be enroled in a
masters’ programme at a university.

People who are not currently studying could of course apply for the
internship, but note that this is not a programming gig, so the
institute won’t pay for the time spent on the internship.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: 01/04: gnu: mesa: Disable imx driver for armhf-linux.

[Ricardo Wurmus]

Leo Famulari <leo@famulari.name> writes:

> Maybe if one has a fast enough x86_64 computer, they could virtualize an
> armhf test machine.

Last time I tried building Java for armhf I tried virtualising on my
workstation in the office, but it was terribly slow.  It took about 4
days to compile the icedtea package (that was before the bootstrapping
work, which increased the number of packages that would have to be
built).

AFAIU we can only use the qemu arm emulator, which does not benefit from
KVM and has a pretty low upper memory limit.

Tricky.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net