core-updates, let’s go!

core-updates, let’s go!

[Ludovic Courtès]

Hello Guix!

‘core-updates’ is doing OK!

  https://hydra.gnu.org/jobset/gnu/core-updates

The remaining issues are GCC 4.8 and 4.9 failing to build, which are
nothing serious and left as an exercise to the reader.

What about starting a new evaluation of all the packages now so we can
merge it early 2018?

Efraim, can you confirm that everything’s alright on aarch64?

We’re lacking a couple of glibc commits from the ‘release/2.26/master’
branch (see below), but Florian Weimer marked them as non-critical
provided we have the previous fixes.  Should we include them anyway and
rebuild the whole thing?  (I’m inclined to answer “no”.)

Thanks,
Ludo’.


commit 633e2f7f3d88df6427aa3a7a984d3a6b796d9611 (HEAD -> release/2.26/master, origin/release/2.26/master)
Author: Florian Weimer <fweimer@redhat.com>
Date:   Thu Dec 14 15:05:57 2017 +0100

    elf: Count components of the expanded path in _dl_init_path [BZ #22607]
    
    (cherry picked from commit 3ff3dfa5af313a6ea33f3393916f30eece4f0171)

commit 43b3cb59b2288953efc26e70fe7c6eb437513b1f
Author: Florian Weimer <fweimer@redhat.com>
Date:   Thu Dec 14 15:18:38 2017 +0100

    elf: Compute correct array size in _dl_init_paths [BZ #22606]
    
    (cherry picked from commit 8a0b17e48b83e933960dfeb8fa08b259f03f310e)

Re: core-updates, let’s go!

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

On Sun, Dec 31, 2017 at 11:27:37AM +0100, Ludovic Courtès wrote:
> Hello Guix!
> 
> ‘core-updates’ is doing OK!
> 
>   https://hydra.gnu.org/jobset/gnu/core-updates
> 
> The remaining issues are GCC 4.8 and 4.9 failing to build, which are
> nothing serious and left as an exercise to the reader.
> 
> What about starting a new evaluation of all the packages now so we can
> merge it early 2018?
> 

I've merged master into core-updates and I'm running 'make' against it
as we speak to make sure I didn't make any obvious errors, and then I'm
about to push it.

> Efraim, can you confirm that everything’s alright on aarch64?
> 

I've been keeping mesa in good shape on aarch64, I'm pretty sure gtk+,
gtk+@2 and qtbase build alright.

> We’re lacking a couple of glibc commits from the ‘release/2.26/master’
> branch (see below), but Florian Weimer marked them as non-critical
> provided we have the previous fixes.  Should we include them anyway and
> rebuild the whole thing?  (I’m inclined to answer “no”.)
> 
> Thanks,
> Ludo’.
> 
> 
> commit 633e2f7f3d88df6427aa3a7a984d3a6b796d9611 (HEAD -> release/2.26/master, origin/release/2.26/master)
> Author: Florian Weimer <fweimer@redhat.com>
> Date:   Thu Dec 14 15:05:57 2017 +0100
> 
>     elf: Count components of the expanded path in _dl_init_path [BZ #22607]
>     
>     (cherry picked from commit 3ff3dfa5af313a6ea33f3393916f30eece4f0171)
> 
> commit 43b3cb59b2288953efc26e70fe7c6eb437513b1f
> Author: Florian Weimer <fweimer@redhat.com>
> Date:   Thu Dec 14 15:18:38 2017 +0100
> 
>     elf: Compute correct array size in _dl_init_paths [BZ #22606]
>     
>     (cherry picked from commit 8a0b17e48b83e933960dfeb8fa08b259f03f310e)

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates, let’s go!

[Danny Milosavljevic]

I'd like to get the patch

    [bug#29856] [PATCH core-updates] guix: python-build-system:  Modify ".py" files in-place.

into this core-updates cycle.  What do you all think?

If we don't do it then a lot of Python programs (the ones who use "console_scripts" in setup.py as they should) will not work (because we emit a shell script wrapper and Python tries to import it - that's going to end badly).  On the other hand, I don't know the effect on other programs (read: I didn't build the world with it - but I did build quite a few Python programs).  You can try it with the guix package ledger-agent and try to invoke "ledger-agent".

The patch only affects files whose names end in ".py" since Python's importer will only find those in any case.  So its possible ill effect is limited.

Hartmut, what do you think?

Re: core-updates, let’s go!

[Danny Milosavljevic]

On Sun, 31 Dec 2017 12:14:16 +0100
Danny Milosavljevic <dannym@scratchpost.org> wrote:

> I'd like to get the patch
> 
>     [bug#29856] [PATCH core-updates] guix: python-build-system:  Modify ".py" files in-place.
> 
> into this core-updates cycle.  What do you all think?
> 
> If we don't do it then a lot of Python programs (the ones who use "console_scripts" in setup.py as they should) will not work (because we emit a shell script wrapper and Python tries to import it - that's going to end badly).  On the other hand, I don't know the effect on other programs (read: I didn't build the world with it - but I did build quite a few Python programs).  You can try it with the guix package ledger-agent and try to invoke "ledger-agent".
> 
> The patch only affects files whose names end in ".py" since Python's importer will only find those in any case.  So its possible ill effect is limited.

^^^ Correction: whose names end in ".py" and that end up in "bin" or "sbin".  Otherwise it would be common.

> 
> Hartmut, what do you think?



-- 
W: https://www.friendly-machines.at/

Re: core-updates, let’s go!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 422 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> I'd like to get the patch
>
>     [bug#29856] [PATCH core-updates] guix: python-build-system:  Modify ".py" files in-place.
>
> into this core-updates cycle.  What do you all think?

We already have a substantial amount of Python changes on core-updates,
so we could save ourselves some trouble by adding this when there's less
"congestion" in case of any problems.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates, let’s go!

[Ricardo Wurmus]

Marius Bakke <mbakke@fastmail.com> writes:

> Danny Milosavljevic <dannym@scratchpost.org> writes:
>
>> I'd like to get the patch
>>
>>     [bug#29856] [PATCH core-updates] guix: python-build-system:  Modify ".py" files in-place.
>>
>> into this core-updates cycle.  What do you all think?
>
> We already have a substantial amount of Python changes on core-updates,
> so we could save ourselves some trouble by adding this when there's less
> "congestion" in case of any problems.

I agree with Marius.

There’s also a change to the wrap-program procedure that does without
the shell wrappers and instead prepends a short Guile program, which is
read as a comment in the target language.  It’s better to play with this
in the next core-updates cycle.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

change to the wrap-program procedure (was: core-updates, let’s go!)

[Hartmut Goebel]

Am 31.12.2017 um 18:27 schrieb Ricardo Wurmus:
> There’s also a change to the wrap-program procedure that does without
> the shell wrappers and instead prepends a short Guile program, which is
> read as a comment in the target language.  It’s better to play with this
> in the next core-updates cycle.

I assume, this is what we discussed in
<https://lists.gnu.org/archive/html/guix-devel/2017-11/msg00041.html>.

I just wonder whether this will keep the encoding-lines in place, like
these:

#!/usr/bin/python
# -*- coding: latin-1 -*-
from __future__ import print_function

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: change to the wrap-program procedure (was: core-updates, let’s go!)

[Ricardo Wurmus]

Hi Hartmut,

> Am 31.12.2017 um 18:27 schrieb Ricardo Wurmus:
>> There’s also a change to the wrap-program procedure that does without
>> the shell wrappers and instead prepends a short Guile program, which is
>> read as a comment in the target language.  It’s better to play with this
>> in the next core-updates cycle.
>
> I assume, this is what we discussed in
> <https://lists.gnu.org/archive/html/guix-devel/2017-11/msg00041.html>.
>
> I just wonder whether this will keep the encoding-lines in place, like
> these:
>
> #!/usr/bin/python
> # -*- coding: latin-1 -*-
> from __future__ import print_function

In the current implementation that I have here it would not.

According to the documentation[1], the regular expression
“coding[=:]\s*([-\w.]+)” must match for a comment on the second line of
a script, so I’d think that the following would still be valid:

--8<---------------cut here---------------start------------->8---
#!/gnu/store/…/bin/guile --no-auto-compile
#!#; -*- coding: latin-1 -*-
#\-(setenv …)
#\-(apply execl …)
#!/gnu/store/…/bin/python3
# -*- coding: latin-1 -*-
…
--8<---------------cut here---------------end--------------->8---

All we need to do is read the second line of the script and see if it is
a coding comment; in that case we’d place the content on the second
line.  I’ll add a special case for this and a test case.  Thanks for
bringing it up!

[1]: https://docs.python.org/3/reference/lexical_analysis.html#encoding-declarations

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: change to the wrap-program procedure

[Hartmut Goebel]

Am 02.01.2018 um 20:01 schrieb Ricardo Wurmus:
> I’ll add a special case for this and a test case.

Great! Thanks.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: change to the wrap-program procedure

[Ricardo Wurmus]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 02.01.2018 um 20:01 schrieb Ricardo Wurmus:
>> I’ll add a special case for this and a test case.
>
> Great! Thanks.

FYI: I sent an initial implementation to guix-patches and got #29951.
We can further discuss things there.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: core-updates, let’s go!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 813 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello Guix!
>
> ‘core-updates’ is doing OK!
>
>   https://hydra.gnu.org/jobset/gnu/core-updates
>
> The remaining issues are GCC 4.8 and 4.9 failing to build, which are
> nothing serious and left as an exercise to the reader.
>
> What about starting a new evaluation of all the packages now so we can
> merge it early 2018?
>
> Efraim, can you confirm that everything’s alright on aarch64?
>
> We’re lacking a couple of glibc commits from the ‘release/2.26/master’
> branch (see below), but Florian Weimer marked them as non-critical
> provided we have the previous fixes.  Should we include them anyway and
> rebuild the whole thing?  (I’m inclined to answer “no”.)

I agree, let's not hold up this branch any longer.  Godspeed!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates, let’s go!

[Ludovic Courtès]

Hello Guix!

I’ve started a new evaluation, let’s see what happens!

We’ll have to stay focused in the coming days to fix everything as
quickly as we can.

Thanks, and happy new year where applicable!  :-)

Ludo’.

Re: core-updates, let’s go!

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

On Mon, Jan 01, 2018 at 06:12:49PM +0100, Ludovic Courtès wrote:
> Hello Guix!
> 
> I’ve started a new evaluation, let’s see what happens!
> 
> We’ll have to stay focused in the coming days to fix everything as
> quickly as we can.
> 
> Thanks, and happy new year where applicable!  :-)
> 
> Ludo’.

I've tracked down the gcc@4.[89] build failures, I'll try to fix gcc@4.9
without affecting libstdc++-boot0

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates, let’s go!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

On Mon, Jan 01, 2018 at 06:12:49PM +0100, Ludovic Courtès wrote:
> We’ll have to stay focused in the coming days to fix everything as
> quickly as we can.

Libtiff failed the first build attempt when the upstream FTP server
timed out. It has succeeded since then, but this will have caused a lot
of dependent packages to fail.

Should I "Restart all dependency failed builds" now?

I changed the libtiff source URL to use HTTP. Hopefully it will be more
reliable.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates, let’s go!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 430 bytes --]

On Wed, Jan 03, 2018 at 01:35:31PM -0500, Leo Famulari wrote:
> Libtiff failed the first build attempt when the upstream FTP server
> timed out. It has succeeded since then, but this will have caused a lot
> of dependent packages to fail.
> 
> Should I "Restart all dependency failed builds" now?

I went ahead and did this because it was too hard to failed packages
that did not fail due to the earlier libtiff problem.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates, let’s go!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 279 bytes --]

On Mon, Jan 01, 2018 at 06:12:49PM +0100, Ludovic Courtès wrote:
> We’ll have to stay focused in the coming days to fix everything as
> quickly as we can.

FYI everyone, I merged master -> core-updates and updated Fossil because
the latest Fossil requires a new SQLite.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates, let’s go!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]

On Mon, Jan 01, 2018 at 06:12:49PM +0100, Ludovic Courtès wrote:
> We’ll have to stay focused in the coming days to fix everything as
> quickly as we can.

The latest core-updates evaluation is coming together [0], but there are
still a few notable issues:

* Pandas is failing to build:

https://hydra.gnu.org/job/gnu/core-updates/python2-pandas-0.19.2.x86_64-linux

* CLISP is failing to build, which breaks SBCL:

https://hydra.gnu.org/job/gnu/core-updates/clisp-2.49-60.x86_64-linux

* Several packages are failing due to changes in ICU related to type
definitions. For example, evolution-data-server is failing to build,
which breaks GNOME:

https://hydra.gnu.org/job/gnu/core-updates/evolution-data-server-3.24.3.x86_64-linux

* ant-bootstrap failed to build, breaking most (all?) of the Java
packages. This is being discussed in <https://bugs.gnu.org/30107>. I
can't retry the ant-bootstrap build because the package is not public,
so there is no corresponding job on Hydra. An example:

https://hydra.gnu.org/build/2443807

* And more!

Why not check if the packages you rely on are working on core-updates?
These are the packages that fail compared to the master branch:

[0] 
https://hydra.gnu.org/eval/109888?compare=master&full=1#tabs-now-fail

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

core-updates: SELinux

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

On Wed, Jan 17, 2018 at 07:50:28PM -0800, Leo Famulari wrote:
> The latest core-updates evaluation is coming together [0], but there are
> still a few notable issues:

Some of the SELinux packages are also failing to build:

https://hydra.gnu.org/build/2414231

I tried updating libsepol to 2.7, and that fixes some things, but it
breaks other packages...

If you use SELinux, please help :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: core-updates: SELinux

[Ricardo Wurmus]

Leo Famulari <leo@famulari.name> writes:

> On Wed, Jan 17, 2018 at 07:50:28PM -0800, Leo Famulari wrote:
>> The latest core-updates evaluation is coming together [0], but there are
>> still a few notable issues:
>
> Some of the SELinux packages are also failing to build:
>
> https://hydra.gnu.org/build/2414231
>
> I tried updating libsepol to 2.7, and that fixes some things, but it
> breaks other packages...
>
> If you use SELinux, please help :)

I updated to 2.7 in master and core-updates.  It built fine on x86_64,
but according to Mark it broke on i686 and armhf.  I’m going to take a
look at this today.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: core-updates, let’s go!

[Kei Kebreau]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

Leo Famulari <leo@famulari.name> writes:

> On Mon, Jan 01, 2018 at 06:12:49PM +0100, Ludovic Courtès wrote:
>> We’ll have to stay focused in the coming days to fix everything as
>> quickly as we can.
>
> The latest core-updates evaluation is coming together [0], but there are
> still a few notable issues:
>
> * Pandas is failing to build:
>
> https://hydra.gnu.org/job/gnu/core-updates/python2-pandas-0.19.2.x86_64-linux
>
> * CLISP is failing to build, which breaks SBCL:
>
> https://hydra.gnu.org/job/gnu/core-updates/clisp-2.49-60.x86_64-linux
>
> * Several packages are failing due to changes in ICU related to type
> definitions. For example, evolution-data-server is failing to build,
> which breaks GNOME:
>
> https://hydra.gnu.org/job/gnu/core-updates/evolution-data-server-3.24.3.x86_64-linux
>
> * ant-bootstrap failed to build, breaking most (all?) of the Java
> packages. This is being discussed in <https://bugs.gnu.org/30107>. I
> can't retry the ant-bootstrap build because the package is not public,
> so there is no corresponding job on Hydra. An example:
>
> https://hydra.gnu.org/build/2443807
>
> * And more!
>
> Why not check if the packages you rely on are working on core-updates?
> These are the packages that fail compared to the master branch:
>
> [0] 
> https://hydra.gnu.org/eval/109888?compare=master&full=1#tabs-now-fail

I have a patch for CLISP that addresses the source of the build
failure. I'm currently testing to see if the resulting binary can build
SBCL properly. Expect to see the patch on the guix-patches list soon!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]