core-updates merged!

core-updates merged!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 420 bytes --]

Guix,

The mythical 'core-updates' branch was just merged to 'master'!

You will notice newer versions of many "core" packages such as glibc,
findutils, make, etc; see 'guix pull --news' for the scoop.  The mailing
list has a (non-exhaustive) overview of the big changes this round:

  https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00412.html

Please report any problems you find to bug-guix@gnu.org.

Enjoy!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates merged!

[Jack Hill]

On Fri, 8 May 2020, Marius Bakke wrote:

> Guix,
>
> The mythical 'core-updates' branch was just merged to 'master'!

Yay! Thanks Marius for shepherding this through.

Best,
Jack

'core-updates' status

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

Hello!

'core-updates' has finished building on x86_64 and i686.  There are a
few failing packages still, but nothing too scary.  Please try to fix
some if you can!

https://hydra.gnu.org/eval/109753?full=1&compare=master#tabs-now-fail

Most of the "dependency failed" armhf packages are because Hydra
believes 'bluez' is failing, even though it has built successfully
twice:

https://hydra.gnu.org/job/gnu/core-updates/bluez-5.45.armhf-linux

What do you think, are we ready to merge this branch?  There are ~2k
armhf packages left in the queue, but many of them have already changed
in 'master' and thus will need to be rebuilt anyway.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 'core-updates' status

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> What do you think, are we ready to merge this branch?  There are ~2k
> armhf packages left in the queue, but many of them have already changed
> in 'master' and thus will need to be rebuilt anyway.

I started a (hopefully final) evaluation while waiting for a consensus.

Please try to `guix pull --branch=core-updates` and check that things
are working fine on your end.  Let's merge this as soon as x86_64 and
i686 is built out in lieu of any new regressions.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 'core-updates' status

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> Marius Bakke <mbakke@fastmail.com> writes:
>
>> What do you think, are we ready to merge this branch?  There are ~2k
>> armhf packages left in the queue, but many of them have already changed
>> in 'master' and thus will need to be rebuilt anyway.
>
> I started a (hopefully final) evaluation while waiting for a consensus.
>
> Please try to `guix pull --branch=core-updates` and check that things
> are working fine on your end.  Let's merge this as soon as x86_64 and
> i686 is built out in lieu of any new regressions.

Hello!

'core-updates' has finished building on x86_64 on i686, and the grafting
failures should now be fixed.  Are we ready to merge this branch? :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: 'core-updates' status

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 342 bytes --]

On Fri, Aug 25, 2017 at 08:34:21PM +0200, Marius Bakke wrote:
> 'core-updates' has finished building on x86_64 on i686, and the grafting
> failures should now be fixed.  Are we ready to merge this branch? :-)

I think it's ready. There are a handful of failing packages left, but I
assume they will be fixed by their users after the merge :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: 'core-updates' status

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Fri, Aug 25, 2017 at 08:34:21PM +0200, Marius Bakke wrote:
>> 'core-updates' has finished building on x86_64 on i686, and the grafting
>> failures should now be fixed.  Are we ready to merge this branch? :-)
>
> I think it's ready. There are a handful of failing packages left, but I
> assume they will be fixed by their users after the merge :)

Seconded!  I upgraded my whole profile a couple of days ago, and I’m
still alive.  :-)

I’ll look at the cross-compilation issue, but it’s too late to realize
there’s a problem in this area I’m afraid.

Ludo’.

core-updates merged!

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Leo Famulari <leo@famulari.name> skribis:
>
>> On Fri, Aug 25, 2017 at 08:34:21PM +0200, Marius Bakke wrote:
>>> 'core-updates' has finished building on x86_64 on i686, and the grafting
>>> failures should now be fixed.  Are we ready to merge this branch? :-)
>>
>> I think it's ready. There are a handful of failing packages left, but I
>> assume they will be fixed by their users after the merge :)
>
> Seconded!  I upgraded my whole profile a couple of days ago, and I’m
> still alive.  :-)

I went ahead and merged it.  Hope you have good bandwidth at the GHM!

\o/

(Also, see you guys there next year ;-))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: core-updates merged!

[Ludovic Courtès]

Howdy!

Marius Bakke <mbakke@fastmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Leo Famulari <leo@famulari.name> skribis:
>>
>>> On Fri, Aug 25, 2017 at 08:34:21PM +0200, Marius Bakke wrote:
>>>> 'core-updates' has finished building on x86_64 on i686, and the grafting
>>>> failures should now be fixed.  Are we ready to merge this branch? :-)
>>>
>>> I think it's ready. There are a handful of failing packages left, but I
>>> assume they will be fixed by their users after the merge :)
>>
>> Seconded!  I upgraded my whole profile a couple of days ago, and I’m
>> still alive.  :-)
>
> I went ahead and merged it.  Hope you have good bandwidth at the GHM!

Actually bandwidth was not good… but the meeting was great.  :-)

Super happy we merged before summer was over, so kudos to you and Leo
and everyone who cajoled the branch!

Who wants to volunteer as a time keeper for the next cycle?  The task of
the time keeper is to make sure we merge on time, which means saying
“no” when someone wants to push one of those infamous last-minute
rebuild-the-world changes, and pinging people regularly when we’re in
the phase where we’re fixing packages.  Any takers?

Ludo’.

PS: Cross-compilation isn’t entirely broken AFAICS:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix build coreutils --target=arm-linux-gnueabihf -n
107.4 MB would be downloaded:
   /gnu/store/qz9v234ibskh09mcrmxjdy6x9bv4zyd0-binutils-cross-arm-linux-gnueabihf-2.28
   /gnu/store/lxs1nmi1a5kp7lz3hk8y2152ipwwpnva-gmp-6.1.2
   /gnu/store/j2nrcggkcxfbggd56bd4nzpaljr87mwv-gcc-cross-arm-linux-gnueabihf-5.4.0
   /gnu/store/z74c0dfgy7vpi0lg9axaz7x40sv9yk9r-glibc-cross-arm-linux-gnueabihf-2.25
   /gnu/store/dbzs50qvfnrddshr0nlxddk0cxd3rlh0-acl-2.2.52
   /gnu/store/wb4kd860x4ql181n073hzswcqb7l8qha-gcc-cross-sans-libc-arm-linux-gnueabihf-5.4.0
   /gnu/store/2dk2zk17allsh1j0hawxwpp47j1db5c5-attr-2.4.47
--8<---------------cut here---------------end--------------->8---

Core-updates

[Andreas Enge]

Hello,

one more thing that would be nice to have is gtk-doc:
   http://hydra.gnu.org:3000/build/1345571
I already tried to upgrade to 1.25, but that did not change the error.
And maybe node for those who are interested in it:
   http://hydra.gnu.org:3000/build/1326189

And there are a few failures of python packages.

Two more cases of non-determinism: The updated versions of dealii and zsh
built without problem on my x86_64 laptop, but fail on hydra (on x86_64
and i686).

Andreas

core-updates merged!

[Ludovic Courtès]

Hello!

Andreas Enge <andreas@enge.fr> skribis:

> one more thing that would be nice to have is gtk-doc:
>    http://hydra.gnu.org:3000/build/1345571
> I already tried to upgrade to 1.25, but that did not change the error.
> And maybe node for those who are interested in it:
>    http://hydra.gnu.org:3000/build/1326189
>
> And there are a few failures of python packages.

I’m afraid these problems are still there, but overall we’re doing
pretty good, so I merged ‘core-updates’ into ‘master’!

Noteworthy changes include the upgrade to libc 2.23, GNU/Hurd and
cross-compilation improvements, closure size reductions, and lots of
things we’ve long forgotten.  :-)

On GuixSD, to ease transition to libc 2.23, you’ll want to install libc
2.22 locale data globally (info "(guix) Locales"):

--8<---------------cut here---------------start------------->8---
(use-modules (gnu system locale))

(operating-system
  ;; …
  (locale-libcs (cons glibc-2.22 %default-locale-libcs)))
--8<---------------cut here---------------end--------------->8---

Enjoy!

Ludo’.

Re: core-updates merged!

[ng0]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Andreas Enge <andreas@enge.fr> skribis:
>
>> one more thing that would be nice to have is gtk-doc:
>>    http://hydra.gnu.org:3000/build/1345571
>> I already tried to upgrade to 1.25, but that did not change the error.
>> And maybe node for those who are interested in it:
>>    http://hydra.gnu.org:3000/build/1326189
>>
>> And there are a few failures of python packages.
>
> I’m afraid these problems are still there, but overall we’re doing
> pretty good, so I merged ‘core-updates’ into ‘master’!
>
> Noteworthy changes include the upgrade to libc 2.23, GNU/Hurd and
> cross-compilation improvements, closure size reductions, and lots of
> things we’ve long forgotten.  :-)
>
> On GuixSD, to ease transition to libc 2.23, you’ll want to install libc
> 2.22 locale data globally (info "(guix) Locales"):
>
> --8<---------------cut here---------------start------------->8---
> (use-modules (gnu system locale))
>
> (operating-system
>   ;; …
>   (locale-libcs (cons glibc-2.22 %default-locale-libcs)))
> --8<---------------cut here---------------end--------------->8---
>
> Enjoy!
>
> Ludo’.
>

Thanks for pushing this. One question about the comment you made:
Why is this necessary? I rebuilt my systems with this and I get various
"invalid locale" errors now.
-- 
♥Ⓐ  ng0
Current Keys: https://we.make.ritual.n0.is/ng0.txt
For non-prism friendly talk find me on http://www.psyced.org

Re: core-updates merged!

[Ludovic Courtès]

Hello!

ng0 <ng0@we.make.ritual.n0.is> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> On GuixSD, to ease transition to libc 2.23, you’ll want to install libc
>> 2.22 locale data globally (info "(guix) Locales"):
>>
>> --8<---------------cut here---------------start------------->8---
>> (use-modules (gnu system locale))
>>
>> (operating-system
>>   ;; …
>>   (locale-libcs (cons glibc-2.22 %default-locale-libcs)))
>> --8<---------------cut here---------------end--------------->8---
>>
>> Enjoy!
>>
>> Ludo’.
>>
>
> Thanks for pushing this. One question about the comment you made:
> Why is this necessary?

This is necessary for binaries linked against libc 2.22, so they can
access libc 2.22 locale data:

  https://www.gnu.org/software/guix/manual/html_node/Locales.html#Locale-Data-Compatibility-Considerations

> I rebuilt my systems with this and I get various "invalid locale"
> errors now.

As discussed on IRC, SNAFU!  For reasons yet to be elucidated, the
glibc@2.23 package no longer honors /run/current-system/locale.

Commit ab3a64507a792e4da0527b423fbc28f8768e736a works around it by
setting GUIX_LOCPATH=/run/currrent-system/locale on GuixSD.  This is an
acceptable workaround, having no visible drawback.

Please report any other issues!

Ludo’.

Re: core-updates merged!

[Leo Famulari]

On Tue, Aug 02, 2016 at 07:32:23PM +0200, Ludovic Courtès wrote:
> As discussed on IRC, SNAFU!  For reasons yet to be elucidated, the
> glibc@2.23 package no longer honors /run/current-system/locale.

I believe that this commit in glibc@2.23 is the culprit:

http://repo.or.cz/glibc.git/commit/90fe682d3067163aa773feecf497ef599429457a

The variable 'libc_cv_localedir', which we set as
"/run/current-system/locale/" in the glibc/linux package definition, has
been renamed to 'libc_cv_complocaledir'.

Should it be enough to make the following change? This reverts ab3a6450
(system: Define 'GUIX_LOCPATH' to work around 'glibc' package defect.)
and changes the name of the variable.

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index a476837..bb1879a 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -535,8 +535,7 @@ store.")
             ;;
             ;; `--localedir' is not honored, so work around it.
             ;; See <http://sourceware.org/ml/libc-alpha/2013-03/msg00093.html>.
-            ;; FIXME: This hack no longer works on 2.23!
-            (string-append "libc_cv_localedir=/run/current-system/locale/"
+            (string-append "libc_cv_complocaledir=/run/current-system/locale/"
                            ,version)
 
             (string-append "--with-headers="
diff --git a/gnu/system.scm b/gnu/system.scm
index d6bf6c4..04dd7a8 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -545,12 +545,7 @@ use 'plain-file' instead~%")
 
     ;; By default, applications that use D-Bus, such as Emacs, abort at startup
     ;; when /etc/machine-id is missing.  Make sure these warnings are non-fatal.
-    ("DBUS_FATAL_WARNINGS" . "0")
-
-    ;; XXX: Normally we wouldn't need to do this, but our glibc@2.23 package
-    ;; looks things up in 'PREFIX/lib/locale' instead of
-    ;; '/run/current-system/locale' as was intended.
-    ("GUIX_LOCPATH" . "/run/current-system/locale")))
+    ("DBUS_FATAL_WARNINGS" . "0")))
 
 (define %setuid-programs
   ;; Default set of setuid-root programs.
diff --git a/gnu/tests/base.scm b/gnu/tests/base.scm
index 7170ab1..a6278b2 100644
--- a/gnu/tests/base.scm
+++ b/gnu/tests/base.scm
@@ -178,18 +178,6 @@ info --version")
              '(false-if-exception (getaddrinfo "does-not-exist"))
              marionette))
 
-          (test-equal "locale"
-            "en_US.utf8"
-            (marionette-eval '(begin
-                                ;; XXX: This 'setenv' call wouldn't be needed
-                                ;; but our glibc@2.23 currently ignores
-                                ;; /run/current-system/locale.
-                                (setenv "GUIX_LOCPATH"
-                                        "/run/current-system/locale")
-                                (let ((before (setlocale LC_ALL "en_US.utf8")))
-                                  (setlocale LC_ALL before)))
-                             marionette))
-
           (test-assert "screendump"
             (begin
               (marionette-control (string-append "screendump " #$output

Re: core-updates merged!

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Tue, Aug 02, 2016 at 07:32:23PM +0200, Ludovic Courtès wrote:
>> As discussed on IRC, SNAFU!  For reasons yet to be elucidated, the
>> glibc@2.23 package no longer honors /run/current-system/locale.
>
> I believe that this commit in glibc@2.23 is the culprit:
>
> http://repo.or.cz/glibc.git/commit/90fe682d3067163aa773feecf497ef599429457a
>
> The variable 'libc_cv_localedir', which we set as
> "/run/current-system/locale/" in the glibc/linux package definition, has
> been renamed to 'libc_cv_complocaledir'.

Good catch!

> diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
> index a476837..bb1879a 100644
> --- a/gnu/packages/base.scm
> +++ b/gnu/packages/base.scm
> @@ -535,8 +535,7 @@ store.")
>              ;;
>              ;; `--localedir' is not honored, so work around it.
>              ;; See <http://sourceware.org/ml/libc-alpha/2013-03/msg00093.html>.
> -            ;; FIXME: This hack no longer works on 2.23!
> -            (string-append "libc_cv_localedir=/run/current-system/locale/"
> +            (string-append "libc_cv_complocaledir=/run/current-system/locale/"
>                             ,version)
>  
>              (string-append "--with-headers="
> diff --git a/gnu/system.scm b/gnu/system.scm
> index d6bf6c4..04dd7a8 100644
> --- a/gnu/system.scm
> +++ b/gnu/system.scm
> @@ -545,12 +545,7 @@ use 'plain-file' instead~%")
>  
>      ;; By default, applications that use D-Bus, such as Emacs, abort at startup
>      ;; when /etc/machine-id is missing.  Make sure these warnings are non-fatal.
> -    ("DBUS_FATAL_WARNINGS" . "0")
> -
> -    ;; XXX: Normally we wouldn't need to do this, but our glibc@2.23 package
> -    ;; looks things up in 'PREFIX/lib/locale' instead of
> -    ;; '/run/current-system/locale' as was intended.
> -    ("GUIX_LOCPATH" . "/run/current-system/locale")))
> +    ("DBUS_FATAL_WARNINGS" . "0")))
>  
>  (define %setuid-programs
>    ;; Default set of setuid-root programs.
> diff --git a/gnu/tests/base.scm b/gnu/tests/base.scm
> index 7170ab1..a6278b2 100644
> --- a/gnu/tests/base.scm
> +++ b/gnu/tests/base.scm
> @@ -178,18 +178,6 @@ info --version")
>               '(false-if-exception (getaddrinfo "does-not-exist"))
>               marionette))
>  
> -          (test-equal "locale"
> -            "en_US.utf8"
> -            (marionette-eval '(begin
> -                                ;; XXX: This 'setenv' call wouldn't be needed
> -                                ;; but our glibc@2.23 currently ignores
> -                                ;; /run/current-system/locale.
> -                                (setenv "GUIX_LOCPATH"
> -                                        "/run/current-system/locale")
> -                                (let ((before (setlocale LC_ALL "en_US.utf8")))
> -                                  (setlocale LC_ALL before)))
> -                             marionette))

Here we should keep the test, but remove ‘setenv’:

  (marionette-eval '(let ((before (setlocale LC_ALL "en_US.utf8")))
                      (setlocale LC_ALL before))
                    marionette)

That will catch this regression in the future.

Otherwise LGTM; could you push it to core-updates-next?

Thank you for the fast investigation!

Ludo’.

Re: core-updates merged!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

On Tue, Aug 02, 2016 at 11:28:59PM +0200, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> Otherwise LGTM; could you push it to core-updates-next?

I merged master into core-updates-next and made the change in a
subsequent commit.

Unfortunately, I can't push the branch to Savannah because it contains
the following unsigned commits (one of them is mine, oops!):

f21403e2b6f5a9491937a0cc9f31fc113998ce5e
9bc84dfea9560c497c91863e7b5021860bd3c254
745ad37a780b28f72ac9dfcc8092317e577e51c9
2d74d94b17b23ab95072da68553d85ac0b3bfede
aebd383d04b351465cfb14e4fd0949b67d4b282e

What should we do?

In the meantime, you can inspect the updated branch on my personal Git
repo:
https://github.com/lfam/guix/commits/core-updates-next

And, since these big merge commits make me nervous, I made notes of all
the conflicts that I had to resolve. They are attached.

[-- Attachment #2: conflicts --]
[-- Type: text/plain, Size: 3041 bytes --]

Before merge, core-updates-next HEAD was
ef21276053b980c9eca8df027408ae85bd6af3d8

(gnu packages admin): No change on core-updates next, package di added
on master.

(gnu packages base): The "previous" glibc package, used to provide
locales for the previously packaged glibc, was named for 2.21 instead of
2.22.

(gnu packages bioinformatics): Package diamond was on version 0.8.7
instead of master's 0.8.17.

Package vsearch was at version 2.0.0 instead of master's 2.0.1.

(gnu packages commencement): New private packages bison-boot0 and
flex-boot0 present on core-updates-next but not on master.

Conflict between 'kernel-headers-boot0' and 'linux-libre-headers-boot0'.
Picked the former since it was deliberately introduced relatively
recently.

(gnu packages cross-base): Conflict between xheaders and xlinux-headers.
Chose xheaders.

(gnu packages crypto): Annoying conflict due to alphabetization of
module imports.
A few conflicts regarding license prefixes.

(gnu packages databases): Version conflict for sqlite.

(gnu packages dav): Version conflict for vdirsyncer.

(gnu packages emacs): New packages not found on HEAD (why is this a
conflict?)

(gnu packages game-development): New packages not found on HEAD.

(gnu packages games): New packages not found on HEAD.

(gnu packages guile): New packages not found on HEAD.

(gnu packages haskell): Period-at-end-of-synopsis conflict.

(gnu packages imagemagick): Version conflict for imagemagick.

(gnu packages linux): Version conflicts in linux-libres.

(gnu packages music): Version conflict in beets.

(gnu packages password-utils): Conflict in authorship lines.

(gnu packages perl): Conflict in authorship lines.

(gnu packages python): Conflict in authorship lines.
Many conflicts due to iyzsong's input rearranging blitz. Took all the
new arrangements.
Conflict in wrap-python3. Took HEAD, which wraps more things.

(gnu packages scheme): Conflict related to introduction of
ghostscript-gs. Took master.

(gnu packages tex): Version conflicts in texlive packages. Took latest.

(gnu packages tls): Version conflict in gnutls.

(gnu packages version-control): Version conflict in git.

(gnu packages video): New configure flags for mpv from master.

(gnu packages xorg): Conflict in authorship lines.

gnu/local.mk: HEAD did not have patch 'mysql-fix-failing-test.patch'

(gnu tests): master exports more marionette-related variables.

guix/config.scm.in: Related to 0b0086e (config: Export the raw
installation directories.). Took master.

(guix packages): Related to 1929fdba (packages: <origin> no longer has
an 'imported-modules' field.). Took master.

(gnu tests base): Related to d2fa61bc (tests: Add Avahi and NSS-mDNS
test.). Took master.
Took master for all conflicts, since no work had been done on
core-updates-next.

(gnu tests install): Took master for all conflicts, since no work had
been done on core-updates-next.

doc/guix.texi: The mcron documentation was duplicated for some reason.
Git did not report this conflict. I discovered it when `make` failed.

Re: core-updates merged!

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> I merged master into core-updates-next and made the change in a
> subsequent commit.
>
> Unfortunately, I can't push the branch to Savannah because it contains
> the following unsigned commits (one of them is mine, oops!):
>
> f21403e2b6f5a9491937a0cc9f31fc113998ce5e
> 9bc84dfea9560c497c91863e7b5021860bd3c254
> 745ad37a780b28f72ac9dfcc8092317e577e51c9
> 2d74d94b17b23ab95072da68553d85ac0b3bfede
> aebd383d04b351465cfb14e4fd0949b67d4b282e
>
> What should we do?

I think you should start from the pre-merge ‘core-updates-next’, sign
commits that are unsigned (I thought Manolis signed them all on the last
rebase?), then merge, and finally push.

Not ideal, but hey!

Ludo’.

Re: core-updates merged!

[Leo Famulari]

On Wed, Aug 03, 2016 at 06:42:34PM +0200, Ludovic Courtès wrote:
> I think you should start from the pre-merge ‘core-updates-next’, sign
> commits that are unsigned (I thought Manolis signed them all on the last
> rebase?), then merge, and finally push.

Unfortunately, signing old commits causes subsequent history to be
rewritten, and the subsequent signatures are lost. I would have to
re-sign commits all the way back to June (for aebd383). And Git users'
local history would become invalid.

By the way, all the commits that are rejected by the hook are from the
master branch.

> Not ideal, but hey!

Very! I'll wait for replies before taking this drastic action.

I think we are hitting something like the problem I warned about here:
http://lists.gnu.org/archive/html/guix-devel/2016-07/msg01220.html

Re: core-updates merged!

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Wed, Aug 03, 2016 at 06:42:34PM +0200, Ludovic Courtès wrote:
>> I think you should start from the pre-merge ‘core-updates-next’, sign
>> commits that are unsigned (I thought Manolis signed them all on the last
>> rebase?), then merge, and finally push.
>
> Unfortunately, signing old commits causes subsequent history to be
> rewritten, and the subsequent signatures are lost. I would have to
> re-sign commits all the way back to June (for aebd383). And Git users'
> local history would become invalid.

Yeah, but despite lacking the ‘wip-’ prefix as we usually do, this
branch was “rebaseable”, so I think it’s OK.

Hopefully that will no longer happen in the future.

> I think we are hitting something like the problem I warned about here:
> http://lists.gnu.org/archive/html/guix-devel/2016-07/msg01220.html

Yes, that’s annoying, but it’s a one-time transitional cost.

Thanks!

Ludo’.

Re: core-updates merged!

[Leo Famulari]

On Wed, Aug 03, 2016 at 07:56:05PM +0200, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > On Wed, Aug 03, 2016 at 06:42:34PM +0200, Ludovic Courtès wrote:
> >> I think you should start from the pre-merge ‘core-updates-next’, sign
> >> commits that are unsigned (I thought Manolis signed them all on the last
> >> rebase?), then merge, and finally push.
> >
> > Unfortunately, signing old commits causes subsequent history to be
> > rewritten, and the subsequent signatures are lost. I would have to
> > re-sign commits all the way back to June (for aebd383). And Git users'
> > local history would become invalid.
> 
> Yeah, but despite lacking the ‘wip-’ prefix as we usually do, this
> branch was “rebaseable”, so I think it’s OK.
> 
> Hopefully that will no longer happen in the future.
> 
> > I think we are hitting something like the problem I warned about here:
> > http://lists.gnu.org/archive/html/guix-devel/2016-07/msg01220.html
> 
> Yes, that’s annoying, but it’s a one-time transitional cost.

Just to clarify, I would be re-signing (with my own key) and rebasing
all commits that were made on the master branch and merged on
core-updates-next, going back to sometime in June 2016.

Whenever we merge this core-updates-next branch back into master, the
master branch's history will be rewritten, starting with aebd383d0.
Right? Or am I misunderstanding?

I tried it, to see what would happen.

$ git rebase aebd383d04b351465cfb14e4fd0949b67d4b282e^ --exec "git commit --amend --no-edit --gpg-sign" || git rebase --abort

But for some reason, it ends up trying to work on commits from February
(starting at "build-system/gnu: Do not patch symlinks"), and then fails
to apply the commit that updates Python 2 to 2.7.11. Nothing should fail
to apply, since I'm not changing any files. Am I doing it wrong, or is
it a bug in Git? Perhaps some complication rebasing through previous
merges?

Re: core-updates merged!

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

>> > I think we are hitting something like the problem I warned about here:
>> > http://lists.gnu.org/archive/html/guix-devel/2016-07/msg01220.html
>> 
>> Yes, that’s annoying, but it’s a one-time transitional cost.
>
> Just to clarify, I would be re-signing (with my own key) and rebasing
> all commits that were made on the master branch and merged on
> core-updates-next, going back to sometime in June 2016.

I think core-updates-next is just a dozen commits or so, right?

> Whenever we merge this core-updates-next branch back into master, the
> master branch's history will be rewritten, starting with aebd383d0.
> Right? Or am I misunderstanding?

I think we should remerge core-updates-next on top of master, making it
the new core-updates.

From there on, we will not rebase core-updates and only do merges in
that branch, as usual.

> I tried it, to see what would happen.
>
> $ git rebase aebd383d04b351465cfb14e4fd0949b67d4b282e^ --exec "git commit --amend --no-edit --gpg-sign" || git rebase --abort
>
> But for some reason, it ends up trying to work on commits from February
> (starting at "build-system/gnu: Do not patch symlinks"), and then fails
> to apply the commit that updates Python 2 to 2.7.11. Nothing should fail
> to apply, since I'm not changing any files. Am I doing it wrong, or is
> it a bug in Git? Perhaps some complication rebasing through previous
> merges?

Hmm.  Perhaps the explanation is the merged commit that I screwed, which
I’ll write about just now.

Ludo’.

Re: core-updates merged!

[Leo Famulari]

On Wed, Aug 03, 2016 at 10:01:48PM +0200, Ludovic Courtès wrote:
> I think core-updates-next is just a dozen commits or so, right?

> I think we should remerge core-updates-next on top of master, making it
> the new core-updates.

Do you mean `git checkout core-updates-next && git merge master`? That's
what I've done.

But, it means that some unsigned commits come to core-updates-next from
master, and I can't push the result to Savannah because of the new
signature hook.

Re: core-updates merged!

[Andreas Enge]

On Wed, Aug 03, 2016 at 05:01:29PM -0400, Leo Famulari wrote:
> Do you mean `git checkout core-updates-next && git merge master`? That's
> what I've done.

Another option would be the following:
git checkout master
git checkout -b core-updates
git cherry-pick "commit 1 from core-updates-next"
git cherry-pick "commit 2 from core-updates-next"
...

If there are only a dozen commits in core-updates-next, this could be
feasible, with the danger of forgetting some.

Andreas

Re: core-updates merged!

[Leo Famulari]

On Wed, Aug 03, 2016 at 11:27:28PM +0200, Andreas Enge wrote:
> On Wed, Aug 03, 2016 at 05:01:29PM -0400, Leo Famulari wrote:
> > Do you mean `git checkout core-updates-next && git merge master`? That's
> > what I've done.
> 
> Another option would be the following:
> git checkout master
> git checkout -b core-updates
> git cherry-pick "commit 1 from core-updates-next"
> git cherry-pick "commit 2 from core-updates-next"
> ...
> 
> If there are only a dozen commits in core-updates-next, this could be
> feasible, with the danger of forgetting some.

I'm not sure if this is the right way to check, but

$ git log --oneline master..core-updates-next | wc -l
161

Does anyone else want to test this? I just tried to copy the master
branch and push it as TEMP-signature-test, and it was rejected by
Savannah. Although, I didn't get any detail from Savannah about why it
was rejected, which I did get when I tried to do it with
core-updates-next earlier. So perhaps there is some other issue now...

Re: core-updates merged!

[Leo Famulari]

On Wed, Aug 03, 2016 at 12:04:46AM -0400, Leo Famulari wrote:
> On Tue, Aug 02, 2016 at 11:28:59PM +0200, Ludovic Courtès wrote:
> > Leo Famulari <leo@famulari.name> skribis:
> > Otherwise LGTM; could you push it to core-updates-next?
> 
> I merged master into core-updates-next and made the change in a
> subsequent commit.
> 
> Unfortunately, I can't push the branch to Savannah because it contains
> the following unsigned commits (one of them is mine, oops!):
> 
> f21403e2b6f5a9491937a0cc9f31fc113998ce5e
> 9bc84dfea9560c497c91863e7b5021860bd3c254
> 745ad37a780b28f72ac9dfcc8092317e577e51c9
> 2d74d94b17b23ab95072da68553d85ac0b3bfede
> aebd383d04b351465cfb14e4fd0949b67d4b282e
> 
> What should we do?

This is still preventing me from starting the new core-updates branch.

I guess that we should do the same thing we did when fixing the squashed
merge: disable the assert-commit-signed hook temporarily for the push.

It was also suggested to cherry-pick the 11 commits from
core-updates-next onto master — essentially rebasing them. I can try
that if everyone is okay with me re-signing those commits.

> In the meantime, you can inspect the updated branch on my personal Git
> repo:
> https://github.com/lfam/guix/commits/core-updates-next

Mark, if you want to do method that requires disabling the hook, feel
free to take my (signed) work from this link. It will save you from
doing a lot of merge conflict resolution.

Re: core-updates merged!

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 980 bytes --]

On Sat, Aug 06, 2016 at 10:42:22AM -0400, Leo Famulari wrote:
> It was also suggested to cherry-pick the 11 commits from
> core-updates-next onto master — essentially rebasing them. I can try
> that if everyone is okay with me re-signing those commits.

As discussed on #guix [0], I've pushed the result of this to
WIP-core-updates [1].

I rewrote an obsolete curl update patch to instead remove the curl graft
found on the master branch.

Since this required me to re-sign these commits, will somebody please
review the differences between WIP-core-updates and core-updates-next,
to make sure I've done the right thing?

When I get the "okay", I will rename the branch to core-updates, and
then it will be "open for business" :)

If all goes well, hopefully this is the last time we ever have to
re-sign each other's work.

[0]
https://gnunet.org/bot/log/guix/2016-08-10#T1099908

[1]
http://git.savannah.gnu.org/cgit/guix.git/log/?h=WIP-core-updates

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: core-updates merged!

[Manolis Ragkousis]

Hello Leo

On 08/10/16 22:49, Leo Famulari wrote:
> Since this required me to re-sign these commits, will somebody please
> review the differences between WIP-core-updates and core-updates-next,
> to make sure I've done the right thing?
> 
> When I get the "okay", I will rename the branch to core-updates, and
> then it will be "open for business" :)

Seems good to me. My local changes apply cleanly to the new branch and
my patches work as expected.

Thank you for taking care of this.
Manolis

Re: core-updates merged!

[Leo Famulari]

On Tue, Aug 02, 2016 at 11:28:59PM +0200, Ludovic Courtès wrote:
> Leo Famulari <leo@famulari.name> skribis:
> 
> > On Tue, Aug 02, 2016 at 07:32:23PM +0200, Ludovic Courtès wrote:
> >> As discussed on IRC, SNAFU!  For reasons yet to be elucidated, the
> >> glibc@2.23 package no longer honors /run/current-system/locale.

> > The variable 'libc_cv_localedir', which we set as
> > "/run/current-system/locale/" in the glibc/linux package definition, has
> > been renamed to 'libc_cv_complocaledir'.

> >              ;; See <http://sourceware.org/ml/libc-alpha/2013-03/msg00093.html>.
> > -            ;; FIXME: This hack no longer works on 2.23!
> > -            (string-append "libc_cv_localedir=/run/current-system/locale/"
> > +            (string-append "libc_cv_complocaledir=/run/current-system/locale/"
> >                             ,version)

> Otherwise LGTM; could you push it to core-updates-next?

I've rebuilt my GuixSD system with this change, and I can confirm that
it works.

Setting GUIX_LOCPATH does not work in all cases, at least for me.

For example, the remote shell program 'mosh' requires a UTF-8
environment on the host. Even with GUIX_LOCPATH set correctly in my
login profile, mosh does not find the locales.

Also, the calendar app 'ikhal' is no longer working, and setting
GUIX_LOCPATH does not work for me.

I think we should prioritize this fix.

core-updates merged!

[Federico Beffa]

ludo@gnu.org (Ludovic Courtès) writes:

>   • The ‘glibc’ package no longer has a “locale” output, making it
>     faster to build, and much smaller (the “locale” output was always
>     pulled, and it took 110 MiB.)

I notice that this breaks some packages like guile-ncurses and the
(all?) packages using python-sphinx to generate documentation like
matplotlib, numpy, scipy.

Guile-ncurses was fixed by adding a new phase to generate the
"en_US.UTF-8" locale during the generation of the package, but wouldn't
it be better to keep the "en_US.UTF-8" locale? That appears to be the
default used by most packages.

Regards,
Fede

Re: core-updates merged!

[Andreas Enge]

On Tue, Feb 10, 2015 at 05:38:26PM +0100, Federico Beffa wrote:
> Guile-ncurses was fixed by adding a new phase to generate the
> "en_US.UTF-8" locale during the generation of the package

This is interesting information! If I understand things correctly, I think
a solution would be to have tons of locale-XX packages, where XX stands for
the country code, the contents of which are created by calls to localedef.
And maybe a package locales-all for people who have space and do not wish
to choose. And a search path definition in glibc to set LOCPATH.

What do you think?

Andreas

Re: core-updates merged!

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On Tue, Feb 10, 2015 at 07:46:48PM +0100, Andreas Enge wrote:
     On Tue, Feb 10, 2015 at 05:38:26PM +0100, Federico Beffa wrote:
     > Guile-ncurses was fixed by adding a new phase to generate the
     > "en_US.UTF-8" locale during the generation of the package
     
     This is interesting information! If I understand things correctly, I think
     a solution would be to have tons of locale-XX packages, where XX stands for
     the country code, the contents of which are created by calls to localedef.
     And maybe a package locales-all for people who have space and do not wish
     to choose. And a search path definition in glibc to set LOCPATH.
     
     What do you think?
     
To me, it sounds like a good idea.    I have never liked the debian way where the
locales are compiled at install time.

J'


     

-- 
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: core-updates merged!

[Federico Beffa]

On Tue, Feb 10, 2015 at 7:46 PM, Andreas Enge <andreas@enge.fr> wrote:
> On Tue, Feb 10, 2015 at 05:38:26PM +0100, Federico Beffa wrote:
>> Guile-ncurses was fixed by adding a new phase to generate the
>> "en_US.UTF-8" locale during the generation of the package
>
> This is interesting information! If I understand things correctly, I think
> a solution would be to have tons of locale-XX packages, where XX stands for
> the country code, the contents of which are created by calls to localedef.
> And maybe a package locales-all for people who have space and do not wish
> to choose. And a search path definition in glibc to set LOCPATH.
>
> What do you think?

I like this idea as well.

Regards,
Fede

core-updates, wip-armhf, and 0.8.1

[Ludovic Courtès]

I’d like to merge core-updates to master Real Soon Now.  The last
missing bit for me is the Libtool upgrade, which shouldn’t be too hard
(surprisingly, Libtool has a number of test failures on MIPS:
<http://hydra.gnu.org/build/181662/nixlog/2/raw>; I’m tempted to disable
tests on MIPS altogether.)

The question is whether we should merge wip-armhf in core-updates
before.  I think it may be more reasonable to postpone that.  Mark?

Then I would like to make a primarily-bug-fix release, probably 0.8.1.

Thoughts?

Ludo’.

core-updates merged!

[Ludovic Courtès]

I’ve just merged core-updates in ‘master’, yay!

As a reminder, some of the cool things for packagers:

  • ‘search-path-specification’ can now specify files (instead of plain
    directories) or even file pattern (for example, ‘catalog.xml’ for
    libxml2, or maybe ‘\\.cmake$’ for CMake)  This should unlock a few
    things.

  • The ld wrapper now adds a ‘-rpath’ flag for libraries passed by file
    name (as in “/path/to/libfoo.so” instead of “-L/path/to -lfoo”.)
    This will be useful for CMake-generated makefiles, notably.

  • ‘libltdl’ and ‘libtool’ are now separate.

For users:

  • Documentation is now systematically in share/{man,info}, and it is
    compressed.

  • The ‘glibc’ package no longer has a “locale” output, making it
    faster to build, and much smaller (the “locale” output was always
    pulled, and it took 110 MiB.)

  • A bunch of updates, notably GCC 4.8.4, Binutils 2.25,
    xorg-server 1.16.3.

Enjoy!

Ludo’.

core-updates merge soon!

[Ludovic Courtès]

Now that Guile 2.0.10 is in core-updates, I’d like to merge it by the
end of the week if everything goes well.

So if you have any rebuild-the-world change in mind, this is the
last call.  :-)

Ludo’.

core-updates merged!

[Ludovic Courtès]

I’ve just merged core-updates in master, and Hydra has already built
most of it.  So that brings glibc 2.19, grep 2.18, libgc 7.4, guile
2.0.11, bash 6.3, the ability to use directories as package sources
(instead of tarballs), and a bunch of other updates and improvements.

It’s been more than 3 months between this merge and the previous one,
which is probably too much.

To avoid lingering branches, I think we should have a more formal,
time-based process.  For instance, we would let the branch live for N
months exactly, and freeze it at N months minus one week.

GCC and glibc are released roughly every 6 months, but typically with a
3 month offset.  So I would go for N = 2 or N = 3.  Maybe N = 2 is
better as it would allow us to make core improvements and small upgrades
more often.

Thoughts?

Thanks,
Ludo’.

Re: core-updates merged!

[Thompson, David]

On Wed, Mar 26, 2014 at 11:46 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> I've just merged core-updates in master, and Hydra has already built
> most of it.  So that brings glibc 2.19, grep 2.18, libgc 7.4, guile
> 2.0.11, bash 6.3, the ability to use directories as package sources
> (instead of tarballs), and a bunch of other updates and improvements.

bash 6.3?  Is this a typo or have I missed something?

>
> It's been more than 3 months between this merge and the previous one,
> which is probably too much.
>
> To avoid lingering branches, I think we should have a more formal,
> time-based process.  For instance, we would let the branch live for N
> months exactly, and freeze it at N months minus one week.
>
> GCC and glibc are released roughly every 6 months, but typically with a
> 3 month offset.  So I would go for N = 2 or N = 3.  Maybe N = 2 is
> better as it would allow us to make core improvements and small upgrades
> more often.
>
> Thoughts?
>
> Thanks,
> Ludo'.
>

Merging core-updates every 2 months sounds reasonable to me, fwiw.
What are the potential downsides to frequently merging core-updates?
Too much package rebuilding?   Unstable software?  Just curious if
there are any good reasons for a more conservative approach.

- Dave

Re: core-updates merged!

[Ludovic Courtès]

"Thompson, David" <dthompson2@worcester.edu> skribis:

> On Wed, Mar 26, 2014 at 11:46 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>> I've just merged core-updates in master, and Hydra has already built
>> most of it.  So that brings glibc 2.19, grep 2.18, libgc 7.4, guile
>> 2.0.11, bash 6.3, the ability to use directories as package sources
>> (instead of tarballs), and a bunch of other updates and improvements.
>
> bash 6.3?  Is this a typo or have I missed something?

4.3, indeed.  :-)

> Merging core-updates every 2 months sounds reasonable to me, fwiw.
> What are the potential downsides to frequently merging core-updates?
> Too much package rebuilding?   Unstable software?  Just curious if
> there are any good reasons for a more conservative approach.

The main issue is too much rebuilding, yes, and perhaps sometimes we’d
gather very few changes in 2 months.

Ludo’.

Re: core-updates merged!

[Thompson, David]

On Wed, Mar 26, 2014 at 12:14 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> "Thompson, David" <dthompson2@worcester.edu> skribis:
>
>> On Wed, Mar 26, 2014 at 11:46 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> I've just merged core-updates in master, and Hydra has already built
>>> most of it.  So that brings glibc 2.19, grep 2.18, libgc 7.4, guile
>>> 2.0.11, bash 6.3, the ability to use directories as package sources
>>> (instead of tarballs), and a bunch of other updates and improvements.
>>
>> bash 6.3?  Is this a typo or have I missed something?
>
> 4.3, indeed.  :-)

I didn't realize that bash had made a release last month.  Did we
influence this release?  I remember reading about the pile of patches
for 4.2.

>
>> Merging core-updates every 2 months sounds reasonable to me, fwiw.
>> What are the potential downsides to frequently merging core-updates?
>> Too much package rebuilding?   Unstable software?  Just curious if
>> there are any good reasons for a more conservative approach.
>
> The main issue is too much rebuilding, yes, and perhaps sometimes we'd
> gather very few changes in 2 months.
>
> Ludo'.

Could we skip a merge cycle if there haven't been many changes or
would that be too inconsistent?

- Dave

Re: core-updates merged!

[Ludovic Courtès]

"Thompson, David" <dthompson2@worcester.edu> skribis:

> On Wed, Mar 26, 2014 at 12:14 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> "Thompson, David" <dthompson2@worcester.edu> skribis:
>>
>>> On Wed, Mar 26, 2014 at 11:46 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>>>> I've just merged core-updates in master, and Hydra has already built
>>>> most of it.  So that brings glibc 2.19, grep 2.18, libgc 7.4, guile
>>>> 2.0.11, bash 6.3, the ability to use directories as package sources
>>>> (instead of tarballs), and a bunch of other updates and improvements.
>>>
>>> bash 6.3?  Is this a typo or have I missed something?
>>
>> 4.3, indeed.  :-)
>
> I didn't realize that bash had made a release last month.  Did we
> influence this release?  I remember reading about the pile of patches
> for 4.2.

I’m not sure we have this much influence yet.  ;-)

>>> Merging core-updates every 2 months sounds reasonable to me, fwiw.
>>> What are the potential downsides to frequently merging core-updates?
>>> Too much package rebuilding?   Unstable software?  Just curious if
>>> there are any good reasons for a more conservative approach.
>>
>> The main issue is too much rebuilding, yes, and perhaps sometimes we'd
>> gather very few changes in 2 months.
>>
>> Ludo'.
>
> Could we skip a merge cycle if there haven't been many changes or
> would that be too inconsistent?

Well yeah, we’ll see how it goes and adjust the process.  But we first
need to actually follow the process to get some hindsight.  :-)

Ludo’.