Ready for Guile 2.2!

Ready for Guile 2.2!

[Ludovic Courtès]

Hello Guix!

Exciting times!

As of commit 608e42e7c92114497e7908980424288079acee1e, Guix builds with
Guile 2.2 (to be released sometime within the next 24 hours) and the
whole test suite passes.

All the dependencies of Guix except Guile-SSH (optional; use for
offloading and for ‘guix copy’) are already compatible with Guile 2.2.

Guile 2.2 comes with a new compiler and a new virtual machine.  As a
result, one of the main user-visible changes is that things are faster.

Building Guix with 2.2 is the first migration step.  The next steps are:

  1. Renaming ‘guile-next’ to ‘guile’ and making 2.2 the default.  Some
     packages (notably Guile-SSH, GDB, LilyPond) don’t work with 2.2 yet
     so for these we’ll change ‘guile’ to ‘guile-2.0’.  This can start
     as soon as this week.  :-)

     We’ll replace ‘package-for-guile-2.2’ with ‘package-for-guile-2.0’,
     though I don’t think it’ll make sense to keep both a 2.0 and a 2.2
     version of all the packages.

  2. Use 2.2 to build derivation, as the default #:guile-for-build
     (aka. ‘default-guile’.)  This can be done on the next
     ‘core-updates’ cycle and if everything goes well, it will be
     completely transparent.

Happy hacking!

Ludo’.

Re: Ready for Guile 2.2!

[Pjotr Prins]

On Wed, Mar 15, 2017 at 05:13:01PM +0100, Ludovic Courtès wrote:
> Hello Guix!
> 
> Exciting times!
> 
> As of commit 608e42e7c92114497e7908980424288079acee1e, Guix builds with
> Guile 2.2 (to be released sometime within the next 24 hours) and the
> whole test suite passes.

Very cool. Hats off to you and Andy and others to make it work!

Pj.

Re: Ready for Guile 2.2!

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> Guile 2.2 comes with a new compiler and a new virtual machine.  As a
> result, one of the main user-visible changes is that things are faster.

An unscientific illustration of this point:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix scripts build)
scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
The following derivations would be built:

[...]

;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
scheme@(guile-user)> (version)
$1 = "2.0.13"
--8<---------------cut here---------------end--------------->8---

Compared to:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")

[...]

;; 2.224821s real time, 2.347463s run time.  0.374545s spent in GC.
--8<---------------cut here---------------end--------------->8---

Roughly an 18% speedup here.

Ludo’.

Re: Ready for Guile 2.2!

[Ludovic Courtès]

Hello!

ludo@gnu.org (Ludovic Courtès) skribis:

> Building Guix with 2.2 is the first migration step.  The next steps are:
>
>   1. Renaming ‘guile-next’ to ‘guile’ and making 2.2 the default.  Some
>      packages (notably Guile-SSH, GDB, LilyPond) don’t work with 2.2 yet
>      so for these we’ll change ‘guile’ to ‘guile-2.0’.  This can start
>      as soon as this week.  :-)
>
>      We’ll replace ‘package-for-guile-2.2’ with ‘package-for-guile-2.0’,
>      though I don’t think it’ll make sense to keep both a 2.0 and a 2.2
>      version of all the packages.

This first step has started: we have a bunch of Guile packages that are
2.2 by default, sometimes with an additional 2.0 variant.

I think we should keep incrementally moving things to 2.2-by-default,
along the lines of commit 7c86c0310939619cb37d4706ca3446ed342a2444 and
the previous ones.

Help welcome!  :-)

Ludo’.

Re: Ready for Guile 2.2!

[Ludovic Courtès]

Hello Guix!

ludo@gnu.org (Ludovic Courtès) skribis:

> An unscientific illustration of this point:
>
> scheme@(guile-user)> ,use(guix scripts build)
> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
> The following derivations would be built:
>
> [...]
>
> ;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
> scheme@(guile-user)> (version)
> $1 = "2.0.13"
>
>
> Compared to:
>
> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
>
> [...]
>
> ;; 2.224821s real time, 2.347463s run time.  0.374545s spent in GC.
>
> Roughly an 18% speedup here.

In Guile 2.2.1 Andy fixed an optimization of ‘thunk?’, which is called a
lot when we load packages (via ‘make-promise’, for record fields marked
as ‘delayed’).  Now we get… drum roll…

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix scripts build)
scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")

[...]

;; 1.826528s real time, 1.994426s run time.  0.382750s spent in GC.
scheme@(guile-user)> (version)
$1 = "2.2.1"
--8<---------------cut here---------------end--------------->8---

That’s a 33% speedup compared to 2.0.

Andy super-hero!

Ludo’.

Re: Ready for Guile 2.2!

[Andy Wingo]

On Thu 20 Apr 2017 14:35, ludo@gnu.org (Ludovic Courtès) writes:

> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> ;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
>> scheme@(guile-user)> (version)
>> $1 = "2.0.13"
>>
> scheme@(guile-user)> ,use(guix scripts build)
> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
>
> [...]
>
> ;; 1.826528s real time, 1.994426s run time.  0.382750s spent in GC.
> scheme@(guile-user)> (version)
> $1 = "2.2.1"
>
> That’s a 33% speedup compared to 2.0.

That is a 50% speedup compared to 2.0 :)  If we consider its speed as
being how many times you could do this per second, then 2.0 speed is
1/2.72, and 2.2.1 speed is 1/1.82.  Speed ratio is then
2.72/1.82=1.4945.  So 2.2.1 is 1.5x the speed of 2.0, or 50% faster :)

Andy, who is not looking for praise, but who likes perf numbers :)

Re: Ready for Guile 2.2!

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

Andy Wingo <wingo@igalia.com> writes:

> On Thu 20 Apr 2017 14:35, ludo@gnu.org (Ludovic Courtès) writes:
>
>> ludo@gnu.org (Ludovic Courtès) skribis:
>>
>>> ;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
>>> scheme@(guile-user)> (version)
>>> $1 = "2.0.13"
>>>
>> scheme@(guile-user)> ,use(guix scripts build)
>> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
>>
>> [...]
>>
>> ;; 1.826528s real time, 1.994426s run time.  0.382750s spent in GC.
>> scheme@(guile-user)> (version)
>> $1 = "2.2.1"
>>
>> That’s a 33% speedup compared to 2.0.
>
> That is a 50% speedup compared to 2.0 :)  If we consider its speed as
> being how many times you could do this per second, then 2.0 speed is
> 1/2.72, and 2.2.1 speed is 1/1.82.  Speed ratio is then
> 2.72/1.82=1.4945.  So 2.2.1 is 1.5x the speed of 2.0, or 50% faster :)
>
> Andy, who is not looking for praise, but who likes perf numbers :)

Impressive :). Is there a blog post/article/information somewhere about
what went in Guile to make it that faster?

Thank you for your work!

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Ready for Guile 2.2!

[Joshua Branson]

So are you building (aka compiling) libreoffice, certbot, and xmonad in 
essentially 2 seconds?  That sounds a little too fast...


On 04/20/2017 11:21 AM, Maxim Cournoyer wrote:
> Andy Wingo <wingo@igalia.com> writes:
>
>> On Thu 20 Apr 2017 14:35, ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> ludo@gnu.org (Ludovic Courtès) skribis:
>>>
>>>> ;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
>>>> scheme@(guile-user)> (version)
>>>> $1 = "2.0.13"
>>>>
>>> scheme@(guile-user)> ,use(guix scripts build)
>>> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
>>>
>>> [...]
>>>
>>> ;; 1.826528s real time, 1.994426s run time.  0.382750s spent in GC.
>>> scheme@(guile-user)> (version)
>>> $1 = "2.2.1"
>>>
>>> That’s a 33% speedup compared to 2.0.
>> That is a 50% speedup compared to 2.0 :)  If we consider its speed as
>> being how many times you could do this per second, then 2.0 speed is
>> 1/2.72, and 2.2.1 speed is 1/1.82.  Speed ratio is then
>> 2.72/1.82=1.4945.  So 2.2.1 is 1.5x the speed of 2.0, or 50% faster :)
>>
>> Andy, who is not looking for praise, but who likes perf numbers :)
> Impressive :). Is there a blog post/article/information somewhere about
> what went in Guile to make it that faster?
>
> Thank you for your work!
>
> Maxim

Re: Ready for Guile 2.2!

[Ludovic Courtès]

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Andy Wingo <wingo@igalia.com> writes:
>
>> On Thu 20 Apr 2017 14:35, ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> ludo@gnu.org (Ludovic Courtès) skribis:
>>>
>>>> ;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
>>>> scheme@(guile-user)> (version)
>>>> $1 = "2.0.13"
>>>>
>>> scheme@(guile-user)> ,use(guix scripts build)
>>> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
>>>
>>> [...]
>>>
>>> ;; 1.826528s real time, 1.994426s run time.  0.382750s spent in GC.
>>> scheme@(guile-user)> (version)
>>> $1 = "2.2.1"
>>>
>>> That’s a 33% speedup compared to 2.0.
>>
>> That is a 50% speedup compared to 2.0 :)  If we consider its speed as
>> being how many times you could do this per second, then 2.0 speed is
>> 1/2.72, and 2.2.1 speed is 1/1.82.  Speed ratio is then
>> 2.72/1.82=1.4945.  So 2.2.1 is 1.5x the speed of 2.0, or 50% faster :)
>>
>> Andy, who is not looking for praise, but who likes perf numbers :)
>
> Impressive :). Is there a blog post/article/information somewhere about
> what went in Guile to make it that faster?

I highly recommend posts from the last couple of years (or more!) at
<http://wingolog.org>.  :-)

<http://wingolog.org/archives/2017/03/15/guile-2-2-omg> has pointers to
the most important bits.

Ludo’.

Re: Ready for Guile 2.2!

[Ludovic Courtès]

Andy Wingo <wingo@igalia.com> skribis:

> On Thu 20 Apr 2017 14:35, ludo@gnu.org (Ludovic Courtès) writes:
>
>> ludo@gnu.org (Ludovic Courtès) skribis:
>>
>>> ;; 2.724686s real time, 3.117062s run time.  0.880827s spent in GC.
>>> scheme@(guile-user)> (version)
>>> $1 = "2.0.13"
>>>
>> scheme@(guile-user)> ,use(guix scripts build)
>> scheme@(guile-user)> ,time (guix-build "libreoffice" "certbot" "xmonad" "-n" "--no-substitutes" "--no-build-hook")
>>
>> [...]
>>
>> ;; 1.826528s real time, 1.994426s run time.  0.382750s spent in GC.
>> scheme@(guile-user)> (version)
>> $1 = "2.2.1"
>>
>> That’s a 33% speedup compared to 2.0.
>
> That is a 50% speedup compared to 2.0 :)  If we consider its speed as
> being how many times you could do this per second, then 2.0 speed is
> 1/2.72, and 2.2.1 speed is 1/1.82.  Speed ratio is then
> 2.72/1.82=1.4945.  So 2.2.1 is 1.5x the speed of 2.0, or 50% faster :)

Right!  What I meant to say is that:

  2.72 - 0.33×2.72 = 1.82

But maybe that’s not the preferred way to present things?

Anyway, it’s _way faster_.  :-)

Ludo’.

Re: Ready for Guile 2.2!

[Ludovic Courtès]

Joshua Branson <bransoj@hotmail.com> skribis:

> So are you building (aka compiling) libreoffice, certbot, and xmonad in 
> essentially 2 seconds?  That sounds a little too fast...

No no, they were already in /gnu/store.  What this test measures is the
time it takes to compute the derivations (those /gnu/store/….drv files.)

Ludo’.

‘guix pull’ vs. transition to Guile 2.2

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1930 bytes --]

Hello Guix!

ludo@gnu.org (Ludovic Courtès) skribis:

> As of commit 608e42e7c92114497e7908980424288079acee1e, Guix builds with
> Guile 2.2 (to be released sometime within the next 24 hours) and the
> whole test suite passes.
>
> All the dependencies of Guix except Guile-SSH (optional; use for
> offloading and for ‘guix copy’) are already compatible with Guile 2.2.

With the attached patch, the ‘guix’ package is built against Guile 2.2.
I’m running it on my GuixSD machine, and it works like a charm!

There’s a problem though, called “guix pull”.  ~/.config/guix/latest
currently contains 2.0 .go files.  Thus after reconfiguring GuixSD to
use Guix-for-2.2, running ‘guix’ typically gives loads of warnings like:

  ;;; WARNING: loading compiled file /home/ludo/.config/guix/latest/guix/derivations.go failed:
  ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory

The fix is for ‘guix pull’ to build with Guile 2.2 when that’s what
we’re running.  For that, build-self.scm must be sure it can get the
‘guile2.2-ssh’ package when we’re on 2.2, or it will fail to compile the
new Guix.  However, ‘guile2.2-ssh’ appeared a day ago, so it’s missing
from most installations…

In short, ‘guix pull’ is broken in a way that may practically prevent it
from handling the 2.0-to-2.2 transition.  The risk is that ‘guix pull’
will fail to upgrade, preventing users from upgrading Guix altogether.
To work around that, people will have to use a Git checkout of Guix.  Of
course that’s what many of us already do, but still.

Maybe the upcoming release is a good time to make that transition: at
least people installing GuixSD or Guix from that release will already be
on 2.2 and won’t have problems from there on.

Thoughts? Ideas?

We all know it, but it’s really really time to fix ‘guix pull’…

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: the patch --]
[-- Type: text/x-patch, Size: 1849 bytes --]

From e7577d6e2499596b2d802f1a5be1f229ce2ab67f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Sat, 22 Apr 2017 15:13:36 +0200
Subject: [PATCH 1/3] gnu: guix: Build with Guile 2.2.

* gnu/packages/package-management.scm (guix-devel)[inputs,
propagated-inputs]: New fields.
---
 gnu/packages/package-management.scm | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 952c268b0..bc2c1e0d3 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -52,6 +52,7 @@
   #:use-module (gnu packages tls)
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages vim)
+  #:use-module (srfi srfi-1)
   #:use-module (ice-9 match))
 
 (define (boot-guile-uri arch)
@@ -69,6 +70,8 @@
                         "/20131110/guile-2.0.9.tar.xz"))))
 
 (define-public guix-0.12.0
+  ;; TODO: On the next release, move the Guile 2.2 inputs from 'guix-devel'
+  ;; to here.
   (package
     (name "guix")
     (version "0.12.0")
@@ -238,6 +241,17 @@ the Nix package manager.")
                  (base32
                   "0p4rh0629j89v4ka5dsp70a1xrfhg7sxjjq54p68vw7x5dkann4a"))
                 (file-name (string-append "guix-" version "-checkout"))))
+
+      ;; Build with Guile 2.2.
+      ;; TODO: Move to the stable 'guix' package on the next release.
+      (inputs
+       `(("guile" ,guile-2.2)
+         ,@(alist-delete "guile" (package-inputs guix-0.12.0))))
+      (propagated-inputs
+       `(("gnutls" ,gnutls/guile-2.2)             ;for 'guix download' & co.
+         ("guile-json" ,guile2.2-json)
+         ("guile-ssh" ,guile2.2-ssh)))
+
       (arguments
        (substitute-keyword-arguments (package-arguments guix-0.12.0)
          ((#:configure-flags flags)
-- 
2.12.2

Re: ‘guix pull’ vs. transition to Guile 2.2

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> Thoughts? Ideas?

Great!  With this patch applied to master I get

============================================================================
Testsuite summary for GNU Guix 0.12.0
============================================================================
# TOTAL: 701
# PASS:  573
# SKIP:  34
# XFAIL: 0
# FAIL:  94
# XPASS: 0
# ERROR: 0

Are you [all] running core-updates?  Should I be?

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: Ready for Guile 2.2!

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

Hi Ludo,

ludo@gnu.org (Ludovic Courtès) writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
[...]
>>
>> Impressive :). Is there a blog post/article/information somewhere about
>> what went in Guile to make it that faster?
>
> I highly recommend posts from the last couple of years (or more!) at
> <http://wingolog.org>.  :-)
>
> <http://wingolog.org/archives/2017/03/15/guile-2-2-omg> has pointers to
> the most important bits.
>
> Ludo’.

Thanks for the pointers. Bookmarked :)

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: ‘guix pull’ vs. transition to Guile 2.2

[Ludovic Courtès]

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> Ludovic Courtès writes:
>
>> Thoughts? Ideas?
>
> Great!  With this patch applied to master I get
>
> ============================================================================
> Testsuite summary for GNU Guix 0.12.0
> ============================================================================
> # TOTAL: 701
> # PASS:  573
> # SKIP:  34
> # XFAIL: 0
> # FAIL:  94
> # XPASS: 0
> # ERROR: 0
>
> Are you [all] running core-updates?  Should I be?

‘core-updates’ is still in its early stages, I bet nobody runs it.  :-)

I haven’t checked whether I get those test failures though.

Ludo’.

Heads-up: transition to Guile 2.2

[Ludovic Courtès]

Hello Guix!

ludo@gnu.org (Ludovic Courtès) skribis:

> There’s a problem though, called “guix pull”.  ~/.config/guix/latest
> currently contains 2.0 .go files.  Thus after reconfiguring GuixSD to
> use Guix-for-2.2, running ‘guix’ typically gives loads of warnings like:
>
>   ;;; WARNING: loading compiled file /home/ludo/.config/guix/latest/guix/derivations.go failed:
>   ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory

With commit 838ba73d6e49bd2b1f2d4ed9329b65cc4e8c1f54, ‘guix pull’ builds
with the currently used Guile, be it 2.0 or 2.2.  To achieve that, it
tries hard to pick 2.0 or 2.2 packages for the dependencies of Guix.

If it cannot find a “guile-ssh” package for the Guile being used (for
instance, you’re using Guile 2.2 but lacking the “guile-ssh” package
built with 2.2), then it simply skips modules that depend on Guile-SSH;
they are optional anyway.  For Guile-JSON, it just assumes that there is
a matching Guile-JSON, which should be the case anyway.

As I wrote before, ‘guix pull’ in its current form is pretty fragile;
please report any problems you have.


In addition, with commit 7561881f2a5d2dc463c24713745eca03e67044bf, the
“guix” package is now built with Guile 2.2 instead of 2.0.  Thus, next
time you update GuixSD, /run/current-system/profile/bin/guix will be
using Guile 2.2.  At that point, if ~/.config/guix/latest contains .go
files for 2.0, you’ll get warnings as shown above, and you’ll have to
“guix pull” to get rid of them.


At this point, we’re still using Guile 2.0 (1) to build derivations, and
(2) in the Shepherd, mcron, and modules imported inside the Shepherd.
In ‘core-updates’ we have addressed #1 and we’ll address #2 as well.

Ludo’.

Re: Heads-up: transition to Guile 2.2

[myglc2]

On 05/09/2017 at 23:22 Ludovic Courtès writes:

> Hello Guix!
>
> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> There’s a problem though, called “guix pull”.  ~/.config/guix/latest
>> currently contains 2.0 .go files.  Thus after reconfiguring GuixSD to
>> use Guix-for-2.2, running ‘guix’ typically gives loads of warnings like:
>>
>>   ;;; WARNING: loading compiled file /home/ludo/.config/guix/latest/guix/derivations.go failed:
>>   ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
>
> With commit 838ba73d6e49bd2b1f2d4ed9329b65cc4e8c1f54, ‘guix pull’ builds
> with the currently used Guile, be it 2.0 or 2.2.  To achieve that, it
> tries hard to pick 2.0 or 2.2 packages for the dependencies of Guix.
>
> If it cannot find a “guile-ssh” package for the Guile being used (for
> instance, you’re using Guile 2.2 but lacking the “guile-ssh” package
> built with 2.2), then it simply skips modules that depend on Guile-SSH;
> they are optional anyway.  For Guile-JSON, it just assumes that there is
> a matching Guile-JSON, which should be the case anyway.
>
> As I wrote before, ‘guix pull’ in its current form is pretty fragile;
> please report any problems you have.
>
>
> In addition, with commit 7561881f2a5d2dc463c24713745eca03e67044bf, the
> “guix” package is now built with Guile 2.2 instead of 2.0.  Thus, next
> time you update GuixSD, /run/current-system/profile/bin/guix will be
> using Guile 2.2.  At that point, if ~/.config/guix/latest contains .go
> files for 2.0, you’ll get warnings as shown above, and you’ll have to
> “guix pull” to get rid of them.
>
>
> At this point, we’re still using Guile 2.0 (1) to build derivations, and
> (2) in the Shepherd, mcron, and modules imported inside the Shepherd.
> In ‘core-updates’ we have addressed #1 and we’ll address #2 as well.
>
> Ludo’.

Hi Ludo, this sounds great!

I run GuixSD from git checkout this way ...

cd ~/src/guix
guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
    -c 4
git pull
make clean-go
./bootstrap
./configure --localstatedir=/var --sysconfdir=/etc
make -j 10 check
exit
sudo guix system build /root/con/55.scm -M 4 -c 4
sudo guix system reconfigure /root/con/55.scm -M 4 -c 4
sudo reboot

Should this just work? Or is there something that needs to be done
differently for the 2.0 >>> 2.2 change?

TIA - George

Re: Heads-up: transition to Guile 2.2

[Ludovic Courtès]

Hi,

myglc2 <myglc2@gmail.com> skribis:

> I run GuixSD from git checkout this way ...
>
> cd ~/src/guix
> guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
>     -c 4
> git pull
> make clean-go
> ./bootstrap
> ./configure --localstatedir=/var --sysconfdir=/etc
> make -j 10 check
> exit
> sudo guix system build /root/con/55.scm -M 4 -c 4
> sudo guix system reconfigure /root/con/55.scm -M 4 -c 4
> sudo reboot
>
> Should this just work? Or is there something that needs to be done
> differently for the 2.0 >>> 2.2 change?

That should just work.

The only issue you might have is warnings about invalid .go files if
~/.config/guix/latest contains 2.0 .go files and you’re running Guile
2.2.  The fix is to run ‘guix pull’.

Ludo’.

Re: Heads-up: transition to Guile 2.2

[myglc2]

On 05/10/2017 at 14:05 Ludovic Courtès writes:

> Hi,
>
> myglc2 <myglc2@gmail.com> skribis:
>
>> I run GuixSD from git checkout this way ...
>>
>> cd ~/src/guix
>> guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
>>     -c 4
>> git pull
>> make clean-go
>> ./bootstrap
>> ./configure --localstatedir=/var --sysconfdir=/etc
>> make -j 10 check
>> exit
>> sudo guix system build /root/con/55.scm -M 4 -c 4
>> sudo guix system reconfigure /root/con/55.scm -M 4 -c 4
>> sudo reboot
>>
>> Should this just work? Or is there something that needs to be done
>> differently for the 2.0 >>> 2.2 change?
>
> That should just work.
>
> The only issue you might have is warnings about invalid .go files if
> ~/.config/guix/latest contains 2.0 .go files and you’re running Guile
> 2.2.  The fix is to run ‘guix pull’.

Thanks, I did that and, as you suggested, I got warnings ...

g1@g1 ~/src$ guix package -m g1.scm
;;; WARNING: loading compiled file /home/g1/.config/guix/latest/guix/ui.go failed:
;;; ERROR: In procedure load-thunk-from-memory: not an ELF file
;;; note: source file /home/g1/.config/guix/latest/guix/ui.scm
;;;       newer than compiled /gnu/store/d4wwx93gqizx132zjk7h1ir7rzph0pig-guix-0.12.0-10.ba2260d/lib/guile/2.2/site-ccache/guix/ui.go

... but I really didn't want to do a git pull. So I re-built guix.  

Bottom line: FWIW, I upgraded from 2.0 to 2.2 without 'guix pull' like
this way ...

cd ~/src/guix
guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
    -c 4
git pull
make clean-go
./bootstrap
./configure --localstatedir=/var --sysconfdir=/etc
make -j 10 check
exit
sudo guix system build /root/con/55.scm -M 4 -c 4
sudo guix system reconfigure /root/con/55.scm -M 4 -c 4
sudo reboot
<log in>
cd ~/src/guix
guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
    -c 4
make clean-go
./bootstrap
./configure --localstatedir=/var --sysconfdir=/etc
make -j 10 check

;-) George

Re: Heads-up: transition to Guile 2.2

[myglc2]

On 05/10/2017 at 13:11 myglc2 writes:

> On 05/10/2017 at 14:05 Ludovic Courtès writes:
>
>> Hi,
>>
>> myglc2 <myglc2@gmail.com> skribis:
>>
>>> I run GuixSD from git checkout this way ...
>>>
>>> cd ~/src/guix
>>> guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
>>>     -c 4
>>> git pull
>>> make clean-go
>>> ./bootstrap
>>> ./configure --localstatedir=/var --sysconfdir=/etc
>>> make -j 10 check
>>> exit
>>> sudo guix system build /root/con/55.scm -M 4 -c 4
>>> sudo guix system reconfigure /root/con/55.scm -M 4 -c 4
>>> sudo reboot
>>>
>>> Should this just work? Or is there something that needs to be done
>>> differently for the 2.0 >>> 2.2 change?
>>
>> That should just work.
>>
>> The only issue you might have is warnings about invalid .go files if
>> ~/.config/guix/latest contains 2.0 .go files and you’re running Guile
>> 2.2.  The fix is to run ‘guix pull’.
>
> Thanks, I did that and, as you suggested, I got warnings ...
>
> g1@g1 ~/src$ guix package -m g1.scm
> ;;; WARNING: loading compiled file /home/g1/.config/guix/latest/guix/ui.go failed:
> ;;; ERROR: In procedure load-thunk-from-memory: not an ELF file
> ;;; note: source file /home/g1/.config/guix/latest/guix/ui.scm
> ;;;       newer than compiled /gnu/store/d4wwx93gqizx132zjk7h1ir7rzph0pig-guix-0.12.0-10.ba2260d/lib/guile/2.2/site-ccache/guix/ui.go
>
> ... but I really didn't want to do a git pull. So I re-built guix.  
>
> Bottom line: FWIW, I upgraded from 2.0 to 2.2 without 'guix pull' like
> this way ...
>
> cd ~/src/guix
> guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
>     -c 4
> git pull
> make clean-go
> ./bootstrap
> ./configure --localstatedir=/var --sysconfdir=/etc
> make -j 10 check
> exit
> sudo guix system build /root/con/55.scm -M 4 -c 4
> sudo guix system reconfigure /root/con/55.scm -M 4 -c 4
> sudo reboot
> <log in>
> cd ~/src/guix
> guix environment -e "(@ (gnu packages package-management) guix)" -M 4 \
>     -c 4
> make clean-go
> ./bootstrap
> ./configure --localstatedir=/var --sysconfdir=/etc
> make -j 10 check

Or maybe not. With your recent change, guix says it's 85 days old ...

g1@g1 ~/src/guix$ guix package -i icecat
guix package: warning: Your Guix installation is 85 days old.
guix package: warning: Consider running 'guix pull' followed by
'guix package -u' to get up-to-date packages and security updates.
[...]

But I just did a git pull ...

g1@g1 ~/src/guix$ git -C ~/.config/guix/latest describe
v0.12.0-3681-gbc0e6c931

... and my ~/.config/guix/latest points to it ...

g1@g1 ~/src/guix$ stat ~/.config/guix/latest | grep File:
  File: /home/g1/.config/guix/latest -> /home/g1/src/guix/

... and I did 'make' and 'git system reconfigure' & reboot so that guix
claims it is the latest version ...

g1@g1 ~$ guix --version guix
(GNU Guix) 0.12.0.3681-bc0e6

So...  is some part of my guix 85 days old?

Or... there a problem with the new "days old" message?

TIA - George


  

Re: Heads-up: transition to Guile 2.2

[Ludovic Courtès]

myglc2 <myglc2@gmail.com> skribis:

> g1@g1 ~/src/guix$ guix package -i icecat
> guix package: warning: Your Guix installation is 85 days old.
> guix package: warning: Consider running 'guix pull' followed by
> 'guix package -u' to get up-to-date packages and security updates.
> [...]
>
> But I just did a git pull ...
>
> g1@g1 ~/src/guix$ git -C ~/.config/guix/latest describe
> v0.12.0-3681-gbc0e6c931
>
> ... and my ~/.config/guix/latest points to it ...

This is a special setup.  The warning above just stats
~/.config/guix/latest.  For someone using ‘guix pull’, it’s a symlink
that’s updated each time ‘guix pull’ is invoked.

In your case it’s a symlink that’s never updated… but your warranty is
void anyway.  ;-)

Ludo’.

Re: Heads-up: transition to Guile 2.2

[Vincent Legoll]

> This is a special setup.  The warning above just stats
> ~/.config/guix/latest.  For someone using ‘guix pull’, it’s a symlink
> that’s updated each time ‘guix pull’ is invoked.

Would there be a downside to stating the pointed-to file rather than
the link itself ?

-- 
Vincent Legoll

Re: Heads-up: transition to Guile 2.2

[myglc2]

On 05/11/2017 at 10:29 Ludovic Courtès writes:

> myglc2 <myglc2@gmail.com> skribis:
>
>> g1@g1 ~/src/guix$ guix package -i icecat
>> guix package: warning: Your Guix installation is 85 days old.
>> guix package: warning: Consider running 'guix pull' followed by
>> 'guix package -u' to get up-to-date packages and security updates.
>> [...]
>>
>> But I just did a git pull ...
>>
>> g1@g1 ~/src/guix$ git -C ~/.config/guix/latest describe
>> v0.12.0-3681-gbc0e6c931
>>
>> ... and my ~/.config/guix/latest points to it ...
>
> This is a special setup.  The warning above just stats
> ~/.config/guix/latest.  For someone using ‘guix pull’, it’s a symlink
> that’s updated each time ‘guix pull’ is invoked.
>
> In your case it’s a symlink that’s never updated… but your warranty is
> void anyway.  ;-)
>
> Ludo’.

Oh! Thanks! I can force a new symlink each time I re-make guix.

Maybe this is a situation where it would be beneficial to embed a
date. ISTM a canonical date can be determined for any guix version so it
would not lead to un-reproducibility. And it would be a nice gift to
researchers doing an archelogical dig 1000 years from now ;-)

Re: Heads-up: transition to Guile 2.2

[ng0]

With a normal setup this has lead to:

;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/scribus.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/scsi.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/sdcc.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/selinux.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/serveez.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/shellutils.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/simh.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/skarnet.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/skribilo.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/smalltalk.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/stalonetray.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/sync.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/syndication.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/synergy.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/task-management.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/tmux.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/tv.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/uml.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/unrtf.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/uucp.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/vtk.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/wdiff.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/web-browsers.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/wine.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/xnee.go failed:
;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
guix pull: error: libtiff-CVE-2017-7593.patch: patch not found

in the user profile and normal working conditions in the root profile.

Recreating the account is no option and it is minimalistic, no build
software installed in profile.
-- 
https://pragmatique.xyz
PGP: https://people.pragmatique.xyz/ng0/

Re: Heads-up: transition to Guile 2.2

[ng0]

ng0 transcribed 4.6K bytes:
> With a normal setup this has lead to:
> 
> ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
> ;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/xnee.go failed:
> ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
> guix pull: error: libtiff-CVE-2017-7593.patch: patch not found
> 
> in the user profile and normal working conditions in the root profile.
> 
> Recreating the account is no option and it is minimalistic, no build
> software installed in profile.
> -- 
> https://pragmatique.xyz
> PGP: https://people.pragmatique.xyz/ng0/
> 

… and the fix is to delete the symlink and run guix pull.
-- 
https://pragmatique.xyz
PGP: https://people.pragmatique.xyz/ng0/

Re: Heads-up: transition to Guile 2.2

[Adonay Felipe Nogueira]

Thanks! This also fixes the problem for me. :)

Re: Heads-up: transition to Guile 2.2

[Ludovic Courtès]

Vincent Legoll <vincent.legoll@gmail.com> skribis:

>> This is a special setup.  The warning above just stats
>> ~/.config/guix/latest.  For someone using ‘guix pull’, it’s a symlink
>> that’s updated each time ‘guix pull’ is invoked.
>
> Would there be a downside to stating the pointed-to file rather than
> the link itself ?

I can already tell you that the mtime of the symlink target is 1.
:-)

Ludo’.

Re: ‘guix pull’ vs. transition to Guile 2.2

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2545 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Hello Guix!
>
> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> As of commit 608e42e7c92114497e7908980424288079acee1e, Guix builds with
>> Guile 2.2 (to be released sometime within the next 24 hours) and the
>> whole test suite passes.
>>
>> All the dependencies of Guix except Guile-SSH (optional; use for
>> offloading and for ‘guix copy’) are already compatible with Guile 2.2.
>
> With the attached patch, the ‘guix’ package is built against Guile 2.2.
> I’m running it on my GuixSD machine, and it works like a charm!
>
> There’s a problem though, called “guix pull”.  ~/.config/guix/latest
> currently contains 2.0 .go files.  Thus after reconfiguring GuixSD to
> use Guix-for-2.2, running ‘guix’ typically gives loads of warnings like:
>
>   ;;; WARNING: loading compiled file
> /home/ludo/.config/guix/latest/guix/derivations.go failed:
>   ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
>
> The fix is for ‘guix pull’ to build with Guile 2.2 when that’s what
> we’re running.  For that, build-self.scm must be sure it can get the
> ‘guile2.2-ssh’ package when we’re on 2.2, or it will fail to compile the
> new Guix.  However, ‘guile2.2-ssh’ appeared a day ago, so it’s missing
> from most installations…
>
> In short, ‘guix pull’ is broken in a way that may practically prevent it
> from handling the 2.0-to-2.2 transition.  The risk is that ‘guix pull’
> will fail to upgrade, preventing users from upgrading Guix altogether.
> To work around that, people will have to use a Git checkout of Guix.  Of
> course that’s what many of us already do, but still.
>
> Maybe the upcoming release is a good time to make that transition: at
> least people installing GuixSD or Guix from that release will already be
> on 2.2 and won’t have problems from there on.
>
> Thoughts? Ideas?
>
> We all know it, but it’s really really time to fix ‘guix pull’…

For what it's worth, I ran guix pull on my GuixSD machine today (as root
and as my normal user).  Everything seems to work just fine.  I'm
currently running 'guix system reconfigure', but since substitutes
aren't available, it's building a lot of stuff from source, so who knows
how long it'll take.  I'll let you know if I encounter any problems.

I've also run guix pull (again, as root and also as my normal user) on
my Ubuntu machine, and again everything seems fine.

So far so good! :-)

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Heads-up: transition to Guile 2.2

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1282 bytes --]

ng0 <ng0@pragmatique.xyz> writes:

> ng0 transcribed 4.6K bytes:
>> With a normal setup this has lead to:
>> 
>> ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
>> ;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/xnee.go failed:
>> ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
>> guix pull: error: libtiff-CVE-2017-7593.patch: patch not found
>> 
>> in the user profile and normal working conditions in the root profile.
>> 
>> Recreating the account is no option and it is minimalistic, no build
>> software installed in profile.
>> -- 
>> https://pragmatique.xyz
>> PGP: https://people.pragmatique.xyz/ng0/
>> 
>
> … and the fix is to delete the symlink and run guix pull.

I also encountered this, and I fixed it simply by running "guix pull" a
second time.  I believe this is because, as Ludo mentioned earlier in
the thread, guile2.2-ssh is not present the first time you run "guix
pull", so things get compiled for Guile 2.0.  The second time you run
"guix pull", guile2.2-ssh IS present in the available packages on the
system (thanks to the first "guix pull"), so things get compiled for
Guile 2.2.  It's a bit round-about, but it worked for me.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Heads-up: transition to Guile 2.2

[Pjotr Prins]

On Tue, May 09, 2017 at 11:22:05PM +0200, Ludovic Court??s wrote:
> Hello Guix!
> 
> ludo@gnu.org (Ludovic Court??s) skribis:
> 
> > There???s a problem though, called ???guix pull???.  ~/.config/guix/latest
> > currently contains 2.0 .go files.  Thus after reconfiguring GuixSD to
> > use Guix-for-2.2, running ???guix??? typically gives loads of warnings like:
> >
> >   ;;; WARNING: loading compiled file /home/ludo/.config/guix/latest/guix/derivations.go failed:
> >   ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
> 
> With commit 838ba73d6e49bd2b1f2d4ed9329b65cc4e8c1f54, ???guix pull??? builds
> with the currently used Guile, be it 2.0 or 2.2.  To achieve that, it
> tries hard to pick 2.0 or 2.2 packages for the dependencies of Guix.

I have just wasted a few hours trying to find a way to
bootstrap the latest tree. And I got it to work haphazardly - that
means I can't reproduce what I did.

Starting from running 'guix pull' twice and essentially following the section
'Building GNU Guix from source (using Guix)' in

  https://gitlab.com/pjotrp/guix-notes/blob/master/INSTALL.org#building-gnu-guix-from-source-using-guix

which used to work reliably. It all has to do with the guile upgrade. Even from
a clean git clone it won't work as expected.

Typical errors during build are

  Backtrace:
    GUILEC   gnu/packages/fcitx.go
  Exception thrown while printing backtrace:
    GUILEC   ERROR: gnu/packages/figlet.go
    In procedure public-lookup: Module named (system repl debug) does
    not exist

But I got it somehow to build. guix now lacks a version number:

  ./pre-inst-env guix --version
  guile: warning: failed to install locale
    warning: failed to install locale: Invalid argument
    guix (GNU Guix) UNKNOWN

probably because bootstrap never did the right thing. Bootstrap passes, but 

  ./configure --localstatedir=/var

complains with 

  configure: error: C preprocessor "/lib/cpp" fails sanity check

And during installation:

  ERROR: In procedure stat:
  ERROR: In procedure stat: No such file or directory:
  "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"

(I remember that one from earlier days, it is a missing symlink)

In all, the system feels flaky at this point. I wish we had found a
way of upgrading guile with backward compatibility. Maybe temporarily
naming it guile2.2 with matching paths would have been better.

Pj.

Re: Heads-up: transition to Guile 2.2

[Pjotr Prins]

I rolled back all the way to March to get a reproducable stable
system again. 

At least we can do that, unlike almost any other software distribution
:). 

We'll be on guile 2.0 for the time being until we can get a reliable
upgrade path. Maybe it is the combination with the major gcc update to
7.10 that makes things unstable. I could not figure it out, just see
the result.

Pj.

Re: Heads-up: transition to Guile 2.2

[Pjotr Prins]

I rolled back all the way to March to get a reproducable stable
system again. 

At least we can do that, unlike almost any other software distribution
:). 

We'll be on guile 2.0 for the time being until we can get a reliable
upgrade path. Maybe it is the combination with the major gcc update to
7.10 that makes things unstable. I could not figure it out, just see
the result.

Pj.



-- 

Re: Heads-up: transition to Guile 2.2

[Jan Nieuwenhuizen]

Pjotr Prins writes:

> I rolled back all the way to March to get a reproducable stable
> system again. 

Ugh...I'm not sure what troubles you encountered.  I have been using
guile-2.2 for two years now; and have been "fighting" Guix's use of
guile-2.0...

Most troublesome is that some packages set GUILE_LOAD_PATH and
GUILE_LOAD_COMPILED_PATH and mixing guile-2.0 and guile-2.2 packages
that do so is still troublesome.  It would be nice (TM) if we could
find a way not to rely on setting those environment variables.

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Heads-up: transition to Guile 2.2

[pjotr.public12]

On Sun, May 14, 2017 at 06:28:01PM +0200, Jan Nieuwenhuizen wrote:
> Most troublesome is that some packages set GUILE_LOAD_PATH and
> GUILE_LOAD_COMPILED_PATH and mixing guile-2.0 and guile-2.2 packages
> that do so is still troublesome.  It would be nice (TM) if we could
> find a way not to rely on setting those environment variables.

One thing I noticed was that gnutls was in the 2.2 path and json was
in 2.0. Providing both paths did help compilation. It was one thing I
did. But it does not feel quite right :)

It appears to me that we shoud have a reproducable bootstrap path at
every stage of the tree. I mean for building from the checked out git
tree of guix.

Pj.

Creating a reliable bootstrap for building from source

[Pjotr Prins]

Thinking about it, what I want to achieve is that we can take the
latest git tree and bootstrap by building guix and packages. This
should be easy, since I have guix running, but it is not. And the main
trouble is that the underlying build packages can differ over time. I
am looking at gcc versions and guile versions.  I.e., we are building
on shifting sands. How unguixy!

The combination of 'guix pull' held a promise, were it not that pull is
also iffy. Probably for pretty much the same reason.

The bootstrap+configure scripts try to work that, but actually
address a wider case. I.e. people who want to bootstrap in Debian etc.
I don't think we need al that. I write Makefile.guix for my projects
and they tend to be simple! Once you can assume Guix is there life
gets simple as a developer - except when you try to bootstrap :0

The instruction I would like to write for others is:

1. Install the latest bootstrap-guix-from-source package after a guix pull
2. git clone guix && cd guix
3. run make -f Makefile.guix  

(no configure is needed in guix!)

4. ./pre-inst guix etc. etc.

Now the bootstrap-guix-from-source package is easy to create and needs
one single test: build guix from the git tree. It should have a
dependency on the git tree itself, so it gets forced every time on the
build farm.

Once we have that we can even bootstrap using guix pack and such.

I realise potluck could be an alternative, but a working bootstrap
would make guix pull also reliable and we can test it effectively on
the build farm.

Pj. 

(unguixy, I like that. It applies to many deployment statements. The
world is an unguixy place)

RE: Creating a reliable bootstrap for building from source

[Jeremiah]

> Thinking about it, what I want to achieve is that we can take the
> latest git tree and bootstrap by building guix and packages. This
> should be easy, since I have guix running, but it is not. And the main
> trouble is that the underlying build packages can differ over time. I
> am looking at gcc versions and guile versions.  I.e., we are building
> on shifting sands. How unguixy!

It is worse than that, Janneke and I are still trying to build out a
full source bootstrap.

Now mind you we have gotten quite a bit down that rabbit hole.
( I've build from a hex monitor to a Lexically scoped garbage
compacting/collecting lisp and Janneke built his rather impressive MES
which already supports large parts of the C language and enough to
bootstrap some rather important pieces)

But there still are many gaps left to close (how to bootstrap a 280 byte
hex monitor without a hex monitor or hex assembler, stage0-vm
downstrapping, MES tinycc bootstrapping, MES lisp bootstrapping, etc)

but ultimately shifting sands are the only grounds we can be certain
will be there.

So we better get comfortable minimizing our assumptions.

-Jeremiah

Re: Creating a reliable bootstrap for building from source

[Pjotr Prins]

On Sun, May 14, 2017 at 03:54:27PM -0400, Jeremiah@pdp10.guru wrote:
> > Thinking about it, what I want to achieve is that we can take the
> > latest git tree and bootstrap by building guix and packages. This
> > should be easy, since I have guix running, but it is not. And the main
> > trouble is that the underlying build packages can differ over time. I
> > am looking at gcc versions and guile versions.  I.e., we are building
> > on shifting sands. How unguixy!
> 
> It is worse than that, Janneke and I are still trying to build out a
> full source bootstrap.
> 
> Now mind you we have gotten quite a bit down that rabbit hole.
> ( I've build from a hex monitor to a Lexically scoped garbage
> compacting/collecting lisp and Janneke built his rather impressive MES
> which already supports large parts of the C language and enough to
> bootstrap some rather important pieces)
> 
> But there still are many gaps left to close (how to bootstrap a 280 byte
> hex monitor without a hex monitor or hex assembler, stage0-vm
> downstrapping, MES tinycc bootstrapping, MES lisp bootstrapping, etc)
> 
> but ultimately shifting sands are the only grounds we can be certain
> will be there.
> 
> So we better get comfortable minimizing our assumptions.

What you are trying to do is even more heroic - bootstrapping all of
guix :)

What I want is achievable, simply have the build system as a tested
binary available. A tested builder package that is know to work
against the current tree. We pretty much have it, it is only not
formalized and tested on the build farm.

Pj.
-- 

Re: Heads-up: transition to Guile 2.2

[Ludovic Courtès]

Hi Pjotr,

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> I have just wasted a few hours trying to find a way to
> bootstrap the latest tree. And I got it to work haphazardly - that
> means I can't reproduce what I did.

Oh.

> Starting from running 'guix pull' twice and essentially following the section
> 'Building GNU Guix from source (using Guix)' in
>
>   https://gitlab.com/pjotrp/guix-notes/blob/master/INSTALL.org#building-gnu-guix-from-source-using-guix
>
> which used to work reliably. It all has to do with the guile upgrade. Even from
> a clean git clone it won't work as expected.

Apparently I cannot access that page without logging in.  For
developers, the instructions at
<https://www.gnu.org/software/guix/manual/html_node/Building-from-Git.html>
are still valid, AFAIK.

> Typical errors during build are
>
>   Backtrace:
>     GUILEC   gnu/packages/fcitx.go
>   Exception thrown while printing backtrace:
>     GUILEC   ERROR: gnu/packages/figlet.go
>     In procedure public-lookup: Module named (system repl debug) does
>     not exist

Weird.  Was it with Guile 2.0 or 2.2?  What was on GUILE_LOAD_PATH?

> But I got it somehow to build. guix now lacks a version number:
>
>   ./pre-inst-env guix --version
>   guile: warning: failed to install locale
>     warning: failed to install locale: Invalid argument
>     guix (GNU Guix) UNKNOWN
>
> probably because bootstrap never did the right thing. Bootstrap passes, but 

That’s because build-aux/git-version-gen didn’t find ‘git’ in $PATH.

>   ./configure --localstatedir=/var
>
> complains with 
>
>   configure: error: C preprocessor "/lib/cpp" fails sanity check

What does config.log say?

> And during installation:
>
>   ERROR: In procedure stat:
>   ERROR: In procedure stat: No such file or directory:
>   "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"

During installation of what?

> In all, the system feels flaky at this point. I wish we had found a
> way of upgrading guile with backward compatibility. Maybe temporarily
> naming it guile2.2 with matching paths would have been better.

IIUC this does not relate to ‘guix pull’ since you’re building from Git.

FWIW I had been running Guix in my checkout with Guile 2.2 long before
this ‘guix pull’ transition.  It required making sure that all the 2.2
dependencies were in the environment (not the 2.0 dependencies), but
apart from that that went fine.

I’m afraid I don’t have enough info to debug the issues you mention here
though.  :-/

Ludo’.

Re: Creating a reliable bootstrap for building from source

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> The combination of 'guix pull' held a promise, were it not that pull is
> also iffy. Probably for pretty much the same reason.
>
> The bootstrap+configure scripts try to work that, but actually
> address a wider case. I.e. people who want to bootstrap in Debian etc.
> I don't think we need al that. I write Makefile.guix for my projects
> and they tend to be simple! Once you can assume Guix is there life
> gets simple as a developer - except when you try to bootstrap :0
>
> The instruction I would like to write for others is:
>
> 1. Install the latest bootstrap-guix-from-source package after a guix pull
> 2. git clone guix && cd guix
> 3. run make -f Makefile.guix  
>
> (no configure is needed in guix!)
>
> 4. ./pre-inst guix etc. etc.

I think there are two very different use cases.

As a user I want something like ‘apt-get update’, which is what ‘guix
pull’ tries to do.

For Guix developers, I think it’s reasonable to have a traditional GNU
build system.  After all, Guix is also a regular software package that
people can build from source with “./configure && make && make install”.

My 2¢,
Ludo’.

Re: Creating a reliable bootstrap for building from source

[Jeremiah]

> What you are trying to do is even more heroic - bootstrapping all of
> guix :)

The real heros are those that check our work, report bugs and give us
insight into solving our most difficult problems.

And we LOVE patches and pull requests ;-D

> What I want is achievable, simply have the build system as a tested
> binary available. A tested builder package that is know to work
> against the current tree. We pretty much have it, it is only not
> formalized and tested on the build farm

If what you want is achievable and going to help, then please do it.
No need to ask, we always love people who take time to make things
better.

-Jeremiah

Re: Creating a reliable bootstrap for building from source

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

Heya,

> As a user I want something like ‘apt-get update’, which is what ‘guix
> pull’ tries to do.

For end-users, cool stuff would be:

	nguix	[ guix-ncurse

		u	[ update the aptitude cached list of all packages,
			[ installed or not... just package descriptions, nothing
			[ has been downloaded yet
		U	[ mark as 'wish to update'
		g	[ list the above marked for update packages, then fine tune
			[ using =, -, + ...
		g	ok, download what I really selected to be updated
			and update...

		note that the most important aptitude ncurse menu is 'Cancel pending
		actions' :)

	or using emacs, 'à la magit'

	M-x guix-status
		u
		U
		...

> For Guix developers, I think it’s reasonable to have a traditional GNU
> build system.  After all, Guix is also a regular software package that
> people can build from source with “./configure && make && make install”.

If I could just grab guile-gnutls and  “./configure && make && make install”
then I could compile, play, use, learn, contribute with/to guix...

Cheers,
David

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Heads-up: transition to Guile 2.2

[Pjotr Prins]

Hi Ludo,

On Sun, May 14, 2017 at 11:28:17PM +0200, Ludovic Court??s wrote:
> > Starting from running 'guix pull' twice and essentially following the section
> > 'Building GNU Guix from source (using Guix)' in
> >
> >   https://gitlab.com/pjotrp/guix-notes/blob/master/INSTALL.org#building-gnu-guix-from-source-using-guix
> >
> > which used to work reliably. It all has to do with the guile upgrade. Even from
> > a clean git clone it won't work as expected.
> 
> Apparently I cannot access that page without logging in.  For

Sorry. I am migrating repositories from github and while gitlab imports even
the issue trackers (cool!) it set repos to private by default. Go figure.

Fixed access. But don't look at it now, I need to fix the text.

> developers, the instructions at
> <https://www.gnu.org/software/guix/manual/html_node/Building-from-Git.html>
> are still valid, AFAIK.

The good news is that I got a build of a recent guix with

  env -i /bin/bash --login --noprofile --norc
  ~/.guix-profile/bin/guix environment guix --ad-hoc help2man git strace pkg-config
  rm -rf autom4te.cache/ # to be sure
  make clean
  ./bootstrap
  ./configure --localstatedir=/var
  make clean # to be sure
  make

which is *nice*. It is clear my environment is somewhat unstable - a
combination of PATH pollution and (I think) recent tools mixed into the
build process.

And this is what my message is about, the tooling in my guix environment is
old. Checking inside above environment

  gcc --version
    gcc (GCC) 5.4.0

 guile --version
   guile (GNU Guile) 2.0.14

So, what is happening is that I am building a recent tree with an old
toolset. This is exactly what I meant! I don't dare do a guix pull
now, to avoid the tree build failing again(!). Heh.

Everyone, I mean everyone, is building guix with different toolsets, i.e,
combinations of tools and versions.

This is wrong and unguixy.

> > Typical errors during build are
> >
> >   Backtrace:
> >     GUILEC   gnu/packages/fcitx.go
> >   Exception thrown while printing backtrace:
> >     GUILEC   ERROR: gnu/packages/figlet.go
> >     In procedure public-lookup: Module named (system repl debug) does
> >     not exist
> 
> Weird.  Was it with Guile 2.0 or 2.2?  What was on GUILE_LOAD_PATH?

I know, can't reproduce it now.

> > But I got it somehow to build. guix now lacks a version number:
> >
> >   ./pre-inst-env guix --version
> >   guile: warning: failed to install locale
> >     warning: failed to install locale: Invalid argument
> >     guix (GNU Guix) UNKNOWN
> >
> > probably because bootstrap never did the right thing. Bootstrap passes, but 
> 
> That???s because build-aux/git-version-gen didn???t find ???git??? in $PATH.

Git was in the path, for sure. But obviously my system was unsettled.

> >   ./configure --localstatedir=/var
> >
> > complains with 
> >
> >   configure: error: C preprocessor "/lib/cpp" fails sanity check
> 
> What does config.log say?

It was a whole range of gcc 7.10 errors, looking for limit.h etc. Lost the log in anger ;)

> > And during installation:
> >
> >   ERROR: In procedure stat:
> >   ERROR: In procedure stat: No such file or directory:
> >   "/gnu/store/q5kdj7gpawi94pqd15x3wizjq0nx4zhx-python-2.7.13/share/man/man1/python.1"
> 
> During installation of what?

It was during profile path resolving of Guix. I happen to have this in
a screen. The full trace is

  warning: collision encountered: /gnu/store/b11lvv9x75jgiiw7rpyb53vj8j57jrw6-mysql-5.7.17/share/man/man1/mysqltest.1.gz 
  /gnu/store/vdvwj57w1rnay7khvi0c4wp05f35gqcl-mysql-5.6.25/share/man/man1/mysqltest.1.gz
  warning: arbitrarily choosing /gnu/store/b11lvv9x75jgiiw7rpyb53vj8j57jrw6-mysql-5.7.17/share/man/man1/mysqltest.1.gz
  Backtrace:
  In ice-9/boot-9.scm:
   160: 17 [catch #t #<catch-closure 8cac60> ...]
  In unknown file:
     ?: 16 [apply-smob/1 #<catch-closure 8cac60>]
  In ice-9/boot-9.scm:
    66: 15 [call-with-prompt prompt0 ...]
  In ice-9/eval.scm:
   432: 14 [eval # #]
  In ice-9/boot-9.scm:
  2404: 13 [save-module-excursion #<procedure 8ea7c0 at ice-9/boot-9.scm:4051:3 ()>]
  4056: 12 [#<procedure 8ea7c0 at ice-9/boot-9.scm:4051:3 ()>]
  1727: 11 [%start-stack load-stack #<procedure 8fcae0 at ice-9/boot-9.scm:4047:10 ()>]
  1732: 10 [#<procedure 8fd6f0 ()>]
  In unknown file:
     ?: 9 [primitive-load "/gnu/store/v9h4yaza43hi1780piyvvjsn5ba43hbn-profile-builder"]
  In ./guix/build/profiles.scm:
   133: 8 [build-profile "/gnu/store/2qnn7divxnh5phd1k8sq7g9p29y1vapc-profile" # ...]
  In unknown file:
     ?: 7 [hash-for-each #<procedure d3f3f0 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
     ?: 6 [hash-for-each #<procedure b92b10 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
     ?: 5 [hash-for-each #<procedure eabd80 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
     ?: 4 [hash-for-each #<procedure f24720 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
  In ./guix/build/union.scm:
   110: 3 [union "/gnu/store/2qnn7divxnh5phd1k8sq7g9p29y1vapc-profile/share/man/man1/python.1" ...]
  In unknown file:
     ?: 2 [partition #<procedure file-is-directory? (file)> #]
  In ./guix/build/union.scm:
    49: 1 [file-is-directory? "/gnu/store/q6rbp7s542jkhrhz04hsp2i60gw0h4as-python-2.7.12/share/man/man1/python.1"]
  In unknown file:
     ?: 0 [stat "/gnu/store/q6rbp7s542jkhrhz04hsp2i60gw0h4as-python-2.7.12/share/man/man1/python.1" ...]

  ERROR: In procedure stat:
  ERROR: In procedure stat: No such file or directory: "/gnu/store/q6rbp7s542jkhrhz04hsp2i60gw0h4as-python-2.7.12/share/man/man1/python.1"
  builder for `/gnu/store/ssqb8qmdna28yjq2ggy5bxiy2wn5q1k7-profile.drv' failed with exit code 1
  guix package: error: build failed: build of `/gnu/store/ssqb8qmdna28yjq2ggy5bxiy2wn5q1k7-profile.drv' failed

but, lets not focus on that right now.

> > In all, the system feels flaky at this point. I wish we had found a
> > way of upgrading guile with backward compatibility. Maybe temporarily
> > naming it guile2.2 with matching paths would have been better.
> 
> IIUC this does not relate to ???guix pull??? since you???re building from Git.

The tooling I am using to build from git is coming from a guix pull.
 
> FWIW I had been running Guix in my checkout with Guile 2.2 long before
> this ???guix pull??? transition.  It required making sure that all the 2.2
> dependencies were in the environment (not the 2.0 dependencies), but
> apart from that that went fine.
> 
> I???m afraid I don???t have enough info to debug the issues you mention here
> though.  :-/

No need. What I am suggesting is that we create an environment with a recent
and *tested* set of tools to build the source tree. If that build is
reproducible with recent tooling it will be easy to hunt down bugs. Currently
we can pull a guix environment, but point me where we have an automated build
of that to prove that it is working. Tell me how *you* can recreate
the exact same build environment that I am using. Above environment
isolation feature is useful, but does not guarantee reproducible
builds from source. The point is that we *can* do it.

I suggest to create a guix-build package which is tested against every commit
that updates guix itself on the build farm. We don't have to trigger for
updated packages. Just for commits to the core.

Do you get what I mean? I think it is fairly trivial to achieve, If you agree
this is important I may even have a go at it at some point. 

Pj.

Re: Creating a reliable bootstrap for building from source

[Pjotr Prins]

On Sun, May 14, 2017 at 11:32:06PM +0200, Ludovic Court??s wrote:
> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> 
> > The combination of 'guix pull' held a promise, were it not that pull is
> > also iffy. Probably for pretty much the same reason.
> >
> > The bootstrap+configure scripts try to work that, but actually
> > address a wider case. I.e. people who want to bootstrap in Debian etc.
> > I don't think we need al that. I write Makefile.guix for my projects
> > and they tend to be simple! Once you can assume Guix is there life
> > gets simple as a developer - except when you try to bootstrap :0
> >
> > The instruction I would like to write for others is:
> >
> > 1. Install the latest bootstrap-guix-from-source package after a guix pull
> > 2. git clone guix && cd guix
> > 3. run make -f Makefile.guix  
> >
> > (no configure is needed in guix!)
> >
> > 4. ./pre-inst guix etc. etc.
> 
> I think there are two very different use cases.
> 
> As a user I want something like 'apt-get update', which is what 'guix
> pull' tries to do.

Sure. But from my previous E-mail you can see we are effectively using
pull to bootstrap the source tree build.

> For Guix developers, I think it's reasonable to have a traditional GNU
> build system.  After all, Guix is also a regular software package that
> people can build from source with './configure && make && make install'.

My point is that we can simplify. I like simple. Simple is good.

We can have both the configure and a simple Makefile.guix option. That
is what I do with my projects.

We do not need bootstrap, autoconf and configure on a running Guix
system. We do need it for other distributions.

Anyway, feel free to ignore this idea.

Pj.
-- 

Re: Creating a reliable bootstrap for building from source

[Ludovic Courtès]

Hi David,

David Pirotte <david@altosw.be> skribis:

>> As a user I want something like ‘apt-get update’, which is what ‘guix
>> pull’ tries to do.
>
> For end-users, cool stuff would be:
>
> 	nguix	[ guix-ncurse
>
> 		u	[ update the aptitude cached list of all packages,
> 			[ installed or not... just package descriptions, nothing
> 			[ has been downloaded yet
> 		U	[ mark as 'wish to update'
> 		g	[ list the above marked for update packages, then fine tune
> 			[ using =, -, + ...
> 		g	ok, download what I really selected to be updated
> 			and update...
>
> 		note that the most important aptitude ncurse menu is 'Cancel pending
> 		actions' :)
>
> 	or using emacs, 'à la magit'
>
> 	M-x guix-status
> 		u
> 		U
> 		...

How would that differ from what Emacs-Guix provides?

>> For Guix developers, I think it’s reasonable to have a traditional GNU
>> build system.  After all, Guix is also a regular software package that
>> people can build from source with “./configure && make && make install”.
>
> If I could just grab guile-gnutls and  “./configure && make && make install”
> then I could compile, play, use, learn, contribute with/to guix...

David, if your distro does not provide a guile-gnutls package, you can
always install Guix via the binary installation method:

  https://www.gnu.org/software/guix/manual/html_node/Binary-Installation.html

HTH!

Ludo’.

Re: Creating a reliable bootstrap for building from source

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Sun, May 14, 2017 at 11:32:06PM +0200, Ludovic Court??s wrote:
>> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>> 
>> > The combination of 'guix pull' held a promise, were it not that pull is
>> > also iffy. Probably for pretty much the same reason.
>> >
>> > The bootstrap+configure scripts try to work that, but actually
>> > address a wider case. I.e. people who want to bootstrap in Debian etc.
>> > I don't think we need al that. I write Makefile.guix for my projects
>> > and they tend to be simple! Once you can assume Guix is there life
>> > gets simple as a developer - except when you try to bootstrap :0
>> >
>> > The instruction I would like to write for others is:
>> >
>> > 1. Install the latest bootstrap-guix-from-source package after a guix pull
>> > 2. git clone guix && cd guix
>> > 3. run make -f Makefile.guix  
>> >
>> > (no configure is needed in guix!)
>> >
>> > 4. ./pre-inst guix etc. etc.
>> 
>> I think there are two very different use cases.
>> 
>> As a user I want something like 'apt-get update', which is what 'guix
>> pull' tries to do.
>
> Sure. But from my previous E-mail you can see we are effectively using
> pull to bootstrap the source tree build.

Sure.

>> For Guix developers, I think it's reasonable to have a traditional GNU
>> build system.  After all, Guix is also a regular software package that
>> people can build from source with './configure && make && make install'.
>
> My point is that we can simplify. I like simple. Simple is good.
>
> We can have both the configure and a simple Makefile.guix option. That
> is what I do with my projects.
>
> We do not need bootstrap, autoconf and configure on a running Guix
> system. We do need it for other distributions.

I agree.  build-aux/build-self.scm, which is what ‘guix pull’ runs to
build Guix, is close to what you’re suggesting, IIUC: a pure-Guile build
script.  WDYT?

Ludo’.

Re: Heads-up: transition to Guile 2.2

[Ludovic Courtès]

Hi,

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> The good news is that I got a build of a recent guix with
>
>   env -i /bin/bash --login --noprofile --norc
>   ~/.guix-profile/bin/guix environment guix --ad-hoc help2man git strace pkg-config
>   rm -rf autom4te.cache/ # to be sure
>   make clean
>   ./bootstrap
>   ./configure --localstatedir=/var
>   make clean # to be sure
>   make
>
> which is *nice*. It is clear my environment is somewhat unstable - a
> combination of PATH pollution and (I think) recent tools mixed into the
> build process.
>
> And this is what my message is about, the tooling in my guix environment is
> old. Checking inside above environment
>
>   gcc --version
>     gcc (GCC) 5.4.0
>
>  guile --version
>    guile (GNU Guile) 2.0.14
>
> So, what is happening is that I am building a recent tree with an old
> toolset. This is exactly what I meant! I don't dare do a guix pull
> now, to avoid the tree build failing again(!). Heh.
>
> Everyone, I mean everyone, is building guix with different toolsets, i.e,
> combinations of tools and versions.
>
> This is wrong and unguixy.

Yes, I kinda agree, though again, we need to distinguish between
developers and users (I know I know, the deficiencies of ‘guix pull’
gives users an incentive to do use the
checkout/autoreconf/configure/make dance, but let’s put that aside for
now.)

As a developer, it’s important to be able to choose the tools that you
use.  For instance, to “port” Guix to Guile 2.2, I needed to build it
and run it with 2.2, but of course I couldn’t force it to users before
it was ready.  Thus, I believe the normal ./configure && make approach
makes sense in this developer context.

As a user though, I clearly don’t want to worry about these
shenanigans.  I want to be able to get the latest Guix and package set,
and I don’t care about what it involves behind the scenes.

>> >   ./configure --localstatedir=/var
>> >
>> > complains with 
>> >
>> >   configure: error: C preprocessor "/lib/cpp" fails sanity check
>> 
>> What does config.log say?
>
> It was a whole range of gcc 7.10 errors, looking for limit.h etc. Lost the log in anger ;)

Lack of limits.h, could it be that you had ‘gcc’ in your profile instead
of ‘gcc-toolchain’ (the latter brings in the libc and Linux headers,
including <limits.h>, while the former does not)?

> It was during profile path resolving of Guix. I happen to have this in
> a screen. The full trace is
>
>   warning: collision encountered: /gnu/store/b11lvv9x75jgiiw7rpyb53vj8j57jrw6-mysql-5.7.17/share/man/man1/mysqltest.1.gz 
>   /gnu/store/vdvwj57w1rnay7khvi0c4wp05f35gqcl-mysql-5.6.25/share/man/man1/mysqltest.1.gz
>   warning: arbitrarily choosing /gnu/store/b11lvv9x75jgiiw7rpyb53vj8j57jrw6-mysql-5.7.17/share/man/man1/mysqltest.1.gz
>   Backtrace:
>   In ice-9/boot-9.scm:
>    160: 17 [catch #t #<catch-closure 8cac60> ...]
>   In unknown file:
>      ?: 16 [apply-smob/1 #<catch-closure 8cac60>]
>   In ice-9/boot-9.scm:
>     66: 15 [call-with-prompt prompt0 ...]
>   In ice-9/eval.scm:
>    432: 14 [eval # #]
>   In ice-9/boot-9.scm:
>   2404: 13 [save-module-excursion #<procedure 8ea7c0 at ice-9/boot-9.scm:4051:3 ()>]
>   4056: 12 [#<procedure 8ea7c0 at ice-9/boot-9.scm:4051:3 ()>]
>   1727: 11 [%start-stack load-stack #<procedure 8fcae0 at ice-9/boot-9.scm:4047:10 ()>]
>   1732: 10 [#<procedure 8fd6f0 ()>]
>   In unknown file:
>      ?: 9 [primitive-load "/gnu/store/v9h4yaza43hi1780piyvvjsn5ba43hbn-profile-builder"]
>   In ./guix/build/profiles.scm:
>    133: 8 [build-profile "/gnu/store/2qnn7divxnh5phd1k8sq7g9p29y1vapc-profile" # ...]
>   In unknown file:
>      ?: 7 [hash-for-each #<procedure d3f3f0 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
>      ?: 6 [hash-for-each #<procedure b92b10 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
>      ?: 5 [hash-for-each #<procedure eabd80 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
>      ?: 4 [hash-for-each #<procedure f24720 at ./guix/build/union.scm:143:21 (file dirs-with-file)> ...]
>   In ./guix/build/union.scm:
>    110: 3 [union "/gnu/store/2qnn7divxnh5phd1k8sq7g9p29y1vapc-profile/share/man/man1/python.1" ...]
>   In unknown file:
>      ?: 2 [partition #<procedure file-is-directory? (file)> #]
>   In ./guix/build/union.scm:
>     49: 1 [file-is-directory? "/gnu/store/q6rbp7s542jkhrhz04hsp2i60gw0h4as-python-2.7.12/share/man/man1/python.1"]
>   In unknown file:
>      ?: 0 [stat "/gnu/store/q6rbp7s542jkhrhz04hsp2i60gw0h4as-python-2.7.12/share/man/man1/python.1" ...]
>
>   ERROR: In procedure stat:
>   ERROR: In procedure stat: No such file or directory: "/gnu/store/q6rbp7s542jkhrhz04hsp2i60gw0h4as-python-2.7.12/share/man/man1/python.1"
>   builder for `/gnu/store/ssqb8qmdna28yjq2ggy5bxiy2wn5q1k7-profile.drv' failed with exit code 1
>   guix package: error: build failed: build of `/gnu/store/ssqb8qmdna28yjq2ggy5bxiy2wn5q1k7-profile.drv' failed
>
> but, lets not focus on that right now.

Hmm that looks like a genuine bug.  Could you email the details to
bug-guix if possible, ideally with a simple way to reproduce it (like
take commit XYZ, run “./pre-inst-env guix package -p new -i python …”)?

> No need. What I am suggesting is that we create an environment with a recent
> and *tested* set of tools to build the source tree. If that build is
> reproducible with recent tooling it will be easy to hunt down bugs. Currently
> we can pull a guix environment, but point me where we have an automated build
> of that to prove that it is working. Tell me how *you* can recreate
> the exact same build environment that I am using. Above environment
> isolation feature is useful, but does not guarantee reproducible
> builds from source. The point is that we *can* do it.
>
> I suggest to create a guix-build package which is tested against every commit
> that updates guix itself on the build farm. We don't have to trigger for
> updated packages. Just for commits to the core.
>
> Do you get what I mean? I think it is fairly trivial to achieve, If you agree
> this is important I may even have a go at it at some point. 

Yeah, I get what you mean.

Currently we have the ‘current-guix’ package, which is the ‘guix’
package built from the current Git checkout.  I think it’s pretty much
what you have in mind?  It’s fully specified in a “guixy” way, which is
to say that the ‘guix’ package specifies precisely the dependency graph.

The GuixSD installation tests are the only things that use
‘current-guix’ right now.

The future pull/channel command, though, will probably use something
similar to ‘current-guix’, as opposed to the terrible dependency mixture
that ‘guix pull’ uses.

Thanks for your feedback,
Ludo’.

Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[myglc2]

On 05/15/2017 at 17:48 Ludovic Courtès writes:

> Hi,
>
> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>
[...]
>> Everyone, I mean everyone, is building guix with different toolsets, i.e,
>> combinations of tools and versions.
>>
>> This is wrong and unguixy.
>
> Yes, I kinda agree, though again, we need to distinguish between
> developers and users (I know I know, the deficiencies of ‘guix pull’
> gives users an incentive to do use the
> checkout/autoreconf/configure/make dance, but let’s put that aside for
> now.)

I believe there will be many guix "users" that want to run from git
checkout. They just may not know it, yet. They will want it not just
because they are trying to get around the limitations of git pull.  For
example, even users that don't do development have occasion to...

- select non-stock package config options

- need/apply/use patches from guix developers

- need/apply/use patches from package developers

- need/apply/use some crazy stuff from a friend

Furthermore, ISTM that current guix packaging is a rather nasty tease:
Our user installs guix/guixSD, does 'guix pull' & 'guix-edit', and what
do they get?

READ-ONLY source in an editor that BEEPS when they TRY TO CHANGE IT :-(

Right out of the box this seem contrary to the Guix promise of _freedom_
and _hackability_.

Now, if our user thinks they really do want to try making a change, they
first have to wade thru various and confusing config options and set
guix up in a _non-standard_ way. This is currently a large hurdle and
_far from liberating_.

Now there are no doubt different "classes" of guix users that need to be
accommodated. We should think about and agree on a model of what these
classes are and the services that guix provides in each class. Maybe some
of the discussions of channels, potluck, etc can also be fit into such a
framework.

Thinking about these issues I ended up wondering: why don't we use git
checkout in "guix pull" for all classes of user? Then there won't be a
big hurdle to get over when a user decides they want to try editing a
source. In this approach, different user classes would be accommodated
in the documentation structure and with a config option or two. As an
optimization, compiled .go modules can be pulled to minimize user-side
guix source compilation.

ISTM there would be no downside. WDYT?

- George

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[Ricardo Wurmus]

myglc2 <myglc2@gmail.com> writes:

> I believe there will be many guix "users" that want to run from git
> checkout. They just may not know it, yet. They will want it not just
> because they are trying to get around the limitations of git pull.  For
> example, even users that don't do development have occasion to...
>
> - select non-stock package config options
>
> - need/apply/use patches from guix developers
>
> - need/apply/use patches from package developers
>
> - need/apply/use some crazy stuff from a friend

For those things I believe it to be better to use the package rewriting
framework, i.e. *adding* package variants that have additional patches
or modified configuration options.

Using the git checkout directly comes with too many disadvantages for
users who are not developers.  Git’s UI is famously weird, and there
hardly is anyone who really loves it.

Using the git checkout also requires the use of “make clean-go” whenever
the ABI changes, and occasionally even necessitates re-bootstrapping, or
in the case of upgrading to Guile 2.2 re-configuration.

> Furthermore, ISTM that current guix packaging is a rather nasty tease:
> Our user installs guix/guixSD, does 'guix pull' & 'guix-edit', and what
> do they get?
>
> READ-ONLY source in an editor that BEEPS when they TRY TO CHANGE IT :-(
>
> Right out of the box this seem contrary to the Guix promise of _freedom_
> and _hackability_.

True, “guix edit” could do better, e.g. create a new package variant in
a user module that inherits from the given package and puts the cursor
in the right spot to make changes right there.  Upon saving, the package
could be used right away with GUIX_PACKAGE_PATH.

I think that a first step towards improving the experience with “guix
pull” is to build Guix continuously on Hydra and tell “guix pull” to
download that.  Users would no longer have to wait for compilation on
their local machines, removing a big incentive to go with a git
checkout.  Maybe we should do this first to stop the worst problems,
buying us a bit more time to implement channels.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[Maxim Cournoyer]

On Tue, May 16, 2017 at 8:55 AM, Ricardo Wurmus <rekado@elephly.net> wrote:
>
> myglc2 <myglc2@gmail.com> writes:
>
>> I believe there will be many guix "users" that want to run from git
>> checkout. They just may not know it, yet. They will want it not just
>> because they are trying to get around the limitations of git pull.  For
>> example, even users that don't do development have occasion to...
>>
>> - select non-stock package config options
>>
>> - need/apply/use patches from guix developers
>>
>> - need/apply/use patches from package developers
>>
>> - need/apply/use some crazy stuff from a friend
>
> For those things I believe it to be better to use the package rewriting
> framework, i.e. *adding* package variants that have additional patches
> or modified configuration options.
>
> Using the git checkout directly comes with too many disadvantages for
> users who are not developers.  Git’s UI is famously weird, and there
> hardly is anyone who really loves it.
>
> Using the git checkout also requires the use of “make clean-go” whenever
> the ABI changes, and occasionally even necessitates re-bootstrapping, or
> in the case of upgrading to Guile 2.2 re-configuration.
>
>> Furthermore, ISTM that current guix packaging is a rather nasty tease:
>> Our user installs guix/guixSD, does 'guix pull' & 'guix-edit', and what
>> do they get?
>>
>> READ-ONLY source in an editor that BEEPS when they TRY TO CHANGE IT :-(
>>
>> Right out of the box this seem contrary to the Guix promise of _freedom_
>> and _hackability_.
>
> True, “guix edit” could do better, e.g. create a new package variant in
> a user module that inherits from the given package and puts the cursor
> in the right spot to make changes right there.  Upon saving, the package
> could be used right away with GUIX_PACKAGE_PATH.
>
> I think that a first step towards improving the experience with “guix
> pull” is to build Guix continuously on Hydra and tell “guix pull” to
> download that.  Users would no longer have to wait for compilation on
> their local machines, removing a big incentive to go with a git
> checkout.  Maybe we should do this first to stop the worst problems,
> buying us a bit more time to implement channels.

+1

Re: Creating a reliable bootstrap for building from source

[Pjotr Prins]

On Mon, May 15, 2017 at 03:28:58PM +0200, Ludovic Court??s wrote:
> >> For Guix developers, I think it's reasonable to have a traditional GNU
> >> build system.  After all, Guix is also a regular software package that
> >> people can build from source with './configure && make && make install'.
> >
> > My point is that we can simplify. I like simple. Simple is good.
> >
> > We can have both the configure and a simple Makefile.guix option. That
> > is what I do with my projects.
> >
> > We do not need bootstrap, autoconf and configure on a running Guix
> > system. We do need it for other distributions.
> 
> I agree.  build-aux/build-self.scm, which is what ???guix pull??? runs to
> build Guix, is close to what you???re suggesting, IIUC: a pure-Guile build
> script.  WDYT?

It is interesting - especially the hoops jumping around guile
versioning ;) - but it still looks like it invokes the traditional
gnu-build-system. I think we can do without that too. 

Pj.
-- 

Re: Creating a reliable bootstrap for building from source

[Pjotr Prins]

On Sun, May 14, 2017 at 11:32:06PM +0200, Ludovic Court??s wrote:
> For Guix developers, I think it???s reasonable to have a traditional GNU
> build system.  After all, Guix is also a regular software package that
> people can build from source with ???./configure && make && make install???.

This is the procedure I have now for creating a reliable bootstrap
from source:

The safest route is by using guix environment after starting
a clean shell

  screen -S guix-build # I tend to build in screen
  env -i /bin/bash --login --noprofile --norc
  ~/.guix-profile/bin/guix environment guix --ad-hoc help2man git strace \
    pkg-config less vim binutils coreutils grep --no-grafts
  bash # you may want this shell

In fact pick the most recent guix you have got, see 'ls
/gnu/store/*guix*/bin/guix' and run that command. Use the --no-grafts
switch if you have built packages that way before.

Note that you can start guix by installing the binary tar ball, or
copying it from another machine using the rather useful guix archive
or [[https://www.gnu.org/software/guix/news/creating-bundles-with-guix-pack.html][guix pack]] commands.

You may want to take a note of these running versions

  gcc --version
  guile --version

Next in the source tree

  rm -rf autom4te.cache/ # to be sure
  make clean
  ./bootstrap
  ./configure --localstatedir=/var
  make clean # to be really sure
  time make

It works reliably on all my setups :). Note that typically gcc is v4 or v5 and guile 2.0.x,
but never the same.

I am pleased with this until it stops working ... I think it should go in the manual and
we should have an automated test on this procedure.

Pj.
-- 

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> I think that a first step towards improving the experience with “guix
> pull” is to build Guix continuously on Hydra and tell “guix pull” to
> download that.  Users would no longer have to wait for compilation on
> their local machines, removing a big incentive to go with a git
> checkout.  Maybe we should do this first to stop the worst problems,
> buying us a bit more time to implement channels.

That’s probably what we’ll do next.  But remember that users will still
have to compute the derivation for that Guix, which in turn should allow
them to get the corresponding substitute, and computing that derivation
is currently relatively costly since you need to compile or interpret a
subset of the Guix code.

Ludo’.

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[myglc2]

On 05/16/2017 at 17:55 Ricardo Wurmus writes:

> myglc2 <myglc2@gmail.com> writes:
>
>> I believe there will be many guix "users" that want to run from git
>> checkout. They just may not know it, yet. They will want it not just
>> because they are trying to get around the limitations of git pull.  For
>> example, even users that don't do development have occasion to...
>>
>> - select non-stock package config options
>>
>> - need/apply/use patches from guix developers
>>
>> - need/apply/use patches from package developers
>>
>> - need/apply/use some crazy stuff from a friend
>
> For those things I believe it to be better to use the package rewriting
> framework, i.e. *adding* package variants that have additional patches
> or modified configuration options.

I don't follow you. Are you talking about (guix) Defining Packages ...

 -- Scheme Procedure: package-input-rewriting REPLACEMENTS
          [REWRITE-NAME]

... or something else?

> Using the git checkout directly comes with too many disadvantages for
> users who are not developers.  Git’s UI is famously weird, and there
> hardly is anyone who really loves it.

Agreed. But I am not saying you should force guix users to learn git. I
mean you should leverage git to make guix better. For example, Homebrew
(https://github.com/Homebrew) users don't know anything about git.

> Using the git checkout also requires the use of “make clean-go” whenever
> the ABI changes, and occasionally even necessitates re-bootstrapping, or
> in the case of upgrading to Guile 2.2 re-configuration.

These are shortcomings in how guix uses git, not problems with git
itself. Changes in the way guix uses git can overcome these.

>> Furthermore, ISTM that current guix packaging is a rather nasty tease:
>> Our user installs guix/guixSD, does 'guix pull' & 'guix-edit', and what
>> do they get?
>>
>> READ-ONLY source in an editor that BEEPS when they TRY TO CHANGE IT :-(
>>
>> Right out of the box this seem contrary to the Guix promise of _freedom_
>> and _hackability_.
>
> True, “guix edit” could do better, e.g. create a new package variant in
> a user module that inherits from the given package and puts the cursor
> in the right spot to make changes right there.  Upon saving, the package
> could be used right away with GUIX_PACKAGE_PATH.

OK, you are jumping to an implementation that uses GUIX_PACKAGE_PATH. I
am saying you should consider whether an implementation of guix-edit
based on git may be better.

Confession: I don't use GUIX_PACKAGE_PATH and probably never will: The
GUIX_PACKAGE_PATH as described in the doc has multiple issues. The fact
that it is mentioned 7 times in guix.texi is one indication of the
problem. But the killer is the inherent problem of consistency between
guix and code on the GUIX_PACKAGE_PATH. Someone trying to use
GUIX_PACKAGE_PATH to share work with someone else is in for a word or
hurt. 

IMO a better alternative is git submodules and private git branches.

> I think that a first step towards improving the experience with “guix
> pull” is to build Guix continuously on Hydra and tell “guix pull” to
> download that.  Users would no longer have to wait for compilation on
> their local machines, removing a big incentive to go with a git
> checkout.  Maybe we should do this first to stop the worst problems,
> buying us a bit more time to implement channels.

I have been actively using guix pull and git pull for the last year. My
"hillbilly impression" is that the problem with hydra that it is almost
always either

- "ahead" of the user (e.g., files that the user needs are no longer in
   hydra's store) or

- "behind" the user (e.g. a developer that has just pulled someone
   else's commit, which aren't built yet).

Can't the "ahead" problem be fixed by just saving everything hydra has
ever built? Disks are cheap aren't they?

As regards the "behind" problem, can't this be fixed by having 'guix
pull' deliver only the git commits that are completely built?

- George

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[myglc2]

Oops, sent before done, please see corrections ...

On 05/17/2017 at 12:06 myglc2 writes:

> On 05/16/2017 at 17:55 Ricardo Wurmus writes:
>
>> myglc2 <myglc2@gmail.com> writes:
>>
>>> I believe there will be many guix "users" that want to run from git
>>> checkout. They just may not know it, yet. They will want it not just
>>> because they are trying to get around the limitations of git pull.  For
>>> example, even users that don't do development have occasion to...
>>>
>>> - select non-stock package config options
>>>
>>> - need/apply/use patches from guix developers
>>>
>>> - need/apply/use patches from package developers
>>>
>>> - need/apply/use some crazy stuff from a friend
>>
>> For those things I believe it to be better to use the package rewriting
>> framework, i.e. *adding* package variants that have additional patches
>> or modified configuration options.
>
> I don't follow you. Are you talking about (guix) Defining Packages ...
>
>  -- Scheme Procedure: package-input-rewriting REPLACEMENTS
>           [REWRITE-NAME]
>
> ... or something else?
>
>> Using the git checkout directly comes with too many disadvantages for
>> users who are not developers.  Git’s UI is famously weird, and there
>> hardly is anyone who really loves it.
>
> Agreed. But I am not saying you should force guix users to learn git. I
> mean you should leverage git to make guix better. For example, Homebrew
> (https://github.com/Homebrew) users don't know anything about git.
>
>> Using the git checkout also requires the use of “make clean-go” whenever
>> the ABI changes, and occasionally even necessitates re-bootstrapping, or
>> in the case of upgrading to Guile 2.2 re-configuration.
>
> These are shortcomings in how guix uses git, not problems with git
> itself. Changes in the way guix uses git can overcome these.
>
>>> Furthermore, ISTM that current guix packaging is a rather nasty tease:
>>> Our user installs guix/guixSD, does 'guix pull' & 'guix-edit', and what
>>> do they get?
>>>
>>> READ-ONLY source in an editor that BEEPS when they TRY TO CHANGE IT :-(
>>>
>>> Right out of the box this seem contrary to the Guix promise of _freedom_
>>> and _hackability_.
>>
>> True, “guix edit” could do better, e.g. create a new package variant in
>> a user module that inherits from the given package and puts the cursor
>> in the right spot to make changes right there.  Upon saving, the package
>> could be used right away with GUIX_PACKAGE_PATH.
>
> OK, you are jumping to an implementation that uses GUIX_PACKAGE_PATH. I
> am saying you should consider whether an implementation of guix-edit
> based on git may be better.
>
> Confession: I don't use GUIX_PACKAGE_PATH and probably never will: The
> GUIX_PACKAGE_PATH as described in the doc has multiple issues. The fact
> that it is mentioned 7 times in guix.texi is one indication of the
> problem. But the killer is the inherent problem of consistency between
> guix and code on the GUIX_PACKAGE_PATH. Someone trying to use
> GUIX_PACKAGE_PATH to share work with someone else is in for a word or
> hurt. 
>
> IMO a better alternative is git submodules and private git branches.
>
>> I think that a first step towards improving the experience with “guix
>> pull” is to build Guix continuously on Hydra and tell “guix pull” to
>> download that.  Users would no longer have to wait for compilation on
>> their local machines, removing a big incentive to go with a git
>> checkout.  Maybe we should do this first to stop the worst problems,
>> buying us a bit more time to implement channels.

Are you saying that users currently use the git checkout so they can
"stay put" on an old commit?

Are you saying that the version delivered by guix pull should be delayed
WRT the most current guix git repo master commit to allow hydra to
complete the build? If so +1

> I have been actively using guix pull and git pull for the last year. My
> "hillbilly impression" is that the problem with hydra that it is almost
> always either
[...]
- behind: user guix version is old and no longer in hydra's store

- ahead: user just did 'guix pull' but hydra hasn't build it.  I think
  you addressed this above

- new-commit: a developer has done git pull and hydra hasn't looked at
  it yet

- patch: a developer applies a patch from another developer

ISTM these can be addressed ...

- behind: save everything hydra builds, or at least the last few
  installation images. Can't we just accomplish this by never doing
  "guix gc" on hydra?

- new-commit: Here we want hydra to build every commit that lands in the
  guix git repo, prioritizing some branches (e.g. master) over others.

An approach that splits hydra's resources between building for ahead and
new-commit branches would be very nice to have, EH? ;-) 

WDYT? - George

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[myglc2]

On 05/17/2017 at 14:45 Ludovic Courtès writes:

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> I think that a first step towards improving the experience with “guix
>> pull” is to build Guix continuously on Hydra and tell “guix pull” to
>> download that.  Users would no longer have to wait for compilation on
>> their local machines, removing a big incentive to go with a git
>> checkout.  Maybe we should do this first to stop the worst problems,
>> buying us a bit more time to implement channels.
>
> That’s probably what we’ll do next.  But remember that users will still
> have to compute the derivation for that Guix, which in turn should allow
> them to get the corresponding substitute, and computing that derivation
> is currently relatively costly since you need to compile or interpret a
> subset of the Guix code.
>
> Ludo’.

This hints at the weirdly "re-entrant nature" of guix. And I mean that
in the most friendly way possible ;-)

I just wish I understood this aspect of guix better. Do we have a
diagram? Could we produce one? I sure would like to see it, or even help
make it.  Any suggestions along this line would be welcome.

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[Leo Famulari]

On Wed, May 17, 2017 at 01:46:16PM -0400, myglc2 wrote:
> ISTM these can be addressed ...
> 
> - behind: save everything hydra builds, or at least the last few
>   installation images. Can't we just accomplish this by never doing
>   "guix gc" on hydra?

We need more storage!

> - new-commit: Here we want hydra to build every commit that lands in the
>   guix git repo, prioritizing some branches (e.g. master) over others.

We need more computational power, especially in non-Intel-compatible
architectures (ARM 32 and 64-bit, and MIPS). Also, if we did something
like this, we would not want to build from every commit. Perhaps every
push. And only for master; branches like core-updates tend to collect
untested patches for a while before we start trying to build the branch.
There's no point trying to build every package from branches in that
state.

Re: Heads-up: transition to Guile 2.2

[ng0]

Chris Marusich transcribed 2.2K bytes:
> ng0 <ng0@pragmatique.xyz> writes:
> 
> > ng0 transcribed 4.6K bytes:
> >> With a normal setup this has lead to:
> >> 
> >> ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
> >> ;;; WARNING: loading compiled file /home/ng0/.config/guix/latest/gnu/packages/xnee.go failed:
> >> ;;; ERROR: In procedure load-thunk-from-memory: No such file or directory
> >> guix pull: error: libtiff-CVE-2017-7593.patch: patch not found
> >> 
> >> in the user profile and normal working conditions in the root profile.
> >> 
> >> Recreating the account is no option and it is minimalistic, no build
> >> software installed in profile.
> >> -- 
> >> https://pragmatique.xyz
> >> PGP: https://people.pragmatique.xyz/ng0/
> >> 
> >
> > … and the fix is to delete the symlink and run guix pull.
> 
> I also encountered this, and I fixed it simply by running "guix pull" a
> second time.  I believe this is because, as Ludo mentioned earlier in
> the thread, guile2.2-ssh is not present the first time you run "guix
> pull", so things get compiled for Guile 2.0.  The second time you run
> "guix pull", guile2.2-ssh IS present in the available packages on the
> system (thanks to the first "guix pull"), so things get compiled for
> Guile 2.2.  It's a bit round-about, but it worked for me.
> 
> -- 
> Chris

I did not mention that I ran guix pull more than once and
that /root/ guix in .config is symlinked to the /home/user guix .config...
so deleting was the only option.

-- 
https://pragmatique.xyz
PGP: https://people.pragmatique.xyz/ng0/

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[Katherine Cox-Buday]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

>> I think that a first step towards improving the experience with “guix
>> pull” is to build Guix continuously on Hydra and tell “guix pull” to
>> download that.  Users would no longer have to wait for compilation on
>> their local machines, removing a big incentive to go with a git
>> checkout.  Maybe we should do this first to stop the worst problems,
>> buying us a bit more time to implement channels.
>
> +1

+1. I have recently been considering setting up a mirror so that my
various machines running Guix don't all have to compile the same
thing. If pull worked like this, I probably wouldn't have to.

One question: for packages I really care about, how would I get the
latest/greatest? guix pull; guix package --fallback -u foo?

-- 
Katherine

Re: Rethinking guix pull [was Re: Heads-up: transition to Guile 2.2]

[Ludovic Courtès]

Hi,

Katherine Cox-Buday <cox.katherine.e@gmail.com> skribis:

> One question: for packages I really care about, how would I get the
> latest/greatest? guix pull; guix package --fallback -u foo?

Yes, that should work.

Normally you won’t even need --fallback, which papers over issues with
our infrastructure, which should be fixed now.

Ludo’.

Re: Creating a reliable bootstrap for building from source

[Ludovic Courtès]

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Mon, May 15, 2017 at 03:28:58PM +0200, Ludovic Court??s wrote:
>> >> For Guix developers, I think it's reasonable to have a traditional GNU
>> >> build system.  After all, Guix is also a regular software package that
>> >> people can build from source with './configure && make && make install'.
>> >
>> > My point is that we can simplify. I like simple. Simple is good.
>> >
>> > We can have both the configure and a simple Makefile.guix option. That
>> > is what I do with my projects.
>> >
>> > We do not need bootstrap, autoconf and configure on a running Guix
>> > system. We do need it for other distributions.
>> 
>> I agree.  build-aux/build-self.scm, which is what ???guix pull??? runs to
>> build Guix, is close to what you???re suggesting, IIUC: a pure-Guile build
>> script.  WDYT?
>
> It is interesting - especially the hoops jumping around guile
> versioning ;) - but it still looks like it invokes the traditional
> gnu-build-system.

No, (guix scripts pull) really just traverses the list of .scm files and
compiles them.

Ludo’.