bug#25775: Can't install packages after guix pull

bug#25775: Can't install packages after guix pull

[sirgazil]

[-- Attachment #1: Type: text/plain, Size: 2761 bytes --]

Hi,

I've been using the binary installation of Guix on Debian 8 for some time now. However, after a "guix pull", I can't install packages anymore.


Steps to reproduce
===============


- Run "guix pull" as a user

- Run "guix package -i guile"




Expected result
============

The latest stable version of GNU Guile is installed correctly.


Unexpected result
==============

I get the following error:

warning: failed to install locale: Invalid argument
Backtrace:
In guix/packages.scm:
982: 19 [bag-grafts # #]
966: 18 [fold-bag-dependencies #<procedure bbfa4f0 at guix/packages.scm:982:29 (package grafts)> ...]
983: 17 [#<procedure bbfa4f0 at guix/packages.scm:982:29 (package grafts)> # ()]
786: 16 [cache! #<weak-key-hash-table a01b060 0/223> # "i686-linux" ...]
910: 15 [thunk]
786: 14 [cache! #<weak-key-hash-table a01b280 167/223> # # ...]
1092: 13 [thunk]
1024: 12 [bag->derivation # # #]
In srfi/srfi-1.scm:
578: 11 [map #<procedure b73c510 at guix/packages.scm:1026:30 (t-1321339)> #]
In guix/packages.scm:
846: 10 [expand-input # # # ...]
In guix/store.scm:
1249: 9 [run-with-store # ...]
In guix/packages.scm:
480: 8 [#<procedure b7591e0 at guix/packages.scm:480:2 (state)> #]
485: 7 [#<procedure bbc9a80 at guix/packages.scm:485:34 (state)> #]
In gnu/packages/bash.scm:
249: 6 [#<procedure bbc9aa0 at gnu/packages/bash.scm:249:2 (state)> #]
In guix/download.scm:
421: 5 [#<procedure bafce38 at guix/download.scm:421:2 (state)> #]
In guix/store.scm:
1169: 4 [#<procedure b759138 at guix/store.scm:1168:28 (store)> #]
In guix/packages.scm:
786: 3 [cache! #<weak-key-hash-table a01b280 167/223> #f ...]
1092: 2 [thunk]
786: 1 [cache! #<weak-key-hash-table a01b100 169/223> #f ...]
871: 0 [thunk]

guix/packages.scm:871:27: In procedure thunk:
guix/packages.scm:871:27: In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f


System information
===============

Guix (after pull): guix (GNU Guix) 20170216.20

Foreign distro:
Debian 8 (Linux makina 3.16.0-4-686-pae #1 SMP Debian 3.16.39-1 (2016-12-30) i686 GNU/Linux)


Additional notes
=============

There is a related conversation about this problem in #guix log (https://gnunet.org/bot/log/guix/2017-02-17#T1295254) where amz3` reported the same problem on Ubuntu 16.10 (amd64). lfam couldn't reproduce the problem on x86_64 Guix on Debian.

It was suggested by lfam in the same conversation that the problem might be due to a very old guix-daemon, which is true for my system. However, there was no way for me to update the daemon because I get the same error described above when I run the second step of the following procedure (run as root):


- Run "guix pull"

- Run "guix package --upgrade"

- Restart guix-daemon





---
https://sirgazil.bitbucket.io/

[-- Attachment #2: Type: text/html, Size: 4229 bytes --]

bug#25775: Can't install packages after guix pull

[Pjotr Prins]

I am seeing the same on a fresh checkout and build:

penguin2:~/genenetwork/guix$ ./pre-inst-env guix package -p $HOME/opt/guix-build-system --install autoconf
warning: failed to install locale: Invalid argument
Backtrace:
In guix/packages.scm:
 982: 19 [bag-grafts # #]
 966: 18 [fold-bag-dependencies #<procedure 3cc5a20 at guix/packages.scm:982:29 (package grafts)> ...]
 983: 17 [#<procedure 3cc5a20 at guix/packages.scm:982:29 (package grafts)> # ()]
 786: 16 [cache! #<weak-key-hash-table 190de00 0/223> # "x86_64-linux" ...]
 910: 15 [thunk]
 786: 14 [cache! #<weak-key-hash-table 18fc520 167/223> # # ...]
1092: 13 [thunk]
1024: 12 [bag->derivation # # #]
In srfi/srfi-1.scm:
 578: 11 [map #<procedure 43a1960 at guix/packages.scm:1026:30 (t-1699962)> #]
In guix/packages.scm:
 846: 10 [expand-input # # # ...]
In guix/store.scm:
1249: 9 [run-with-store # ...]
In guix/packages.scm:
 480: 8 [#<procedure 43f9f60 at guix/packages.scm:480:2 (state)> #]
 485: 7 [#<procedure 3f5ff40 at guix/packages.scm:485:34 (state)> #]
In gnu/packages/bash.scm:
 249: 6 [#<procedure 3f5ff80 at gnu/packages/bash.scm:249:2 (state)> #]
In guix/download.scm:
 421: 5 [#<procedure 418ea00 at guix/download.scm:421:2 (state)> #]
In guix/store.scm:
1169: 4 [#<procedure 43f9e10 at guix/store.scm:1168:28 (store)> #]
In guix/packages.scm:
 786: 3 [cache! #<weak-key-hash-table 18fc520 167/223> #f ...]
1092: 2 [thunk]
 786: 1 [cache! #<weak-key-hash-table 1900640 169/223> #f ...]
 871: 0 [thunk]

@ commit 42e721a13d6f47397e585f5909d140c4bafb4ad0

Installation with --no-grafts passes fine.

Pj.

bug#25775: Can't install packages after guix pull

[Pjotr Prins]

And this appears to be related:

./pre-inst-env guix import cpan Time::ParseDate

Starting download of /tmp/guix-file.4UM1ZV
From http://mirror.ibcp.fr/pub/CPAN/authors/id/M/MU/MUIR/modules/Time-ParseDate-2015.103.tar.gz...
 ...2015.103.tar.gz  26KiB          214KiB/s 00:00 [####################] 100.0%
Backtrace:
In guix/packages.scm:
 982: 19 [bag-grafts # #]
 966: 18 [fold-bag-dependencies #<procedure 54aef20 at guix/packages.scm:982:29 (package grafts)> ...]
 983: 17 [#<procedure 54aef20 at guix/packages.scm:982:29 (package grafts)> # ()]
 786: 16 [cache! #<weak-key-hash-table 2be7100 0/223> # "x86_64-linux" ...]
 910: 15 [thunk]
 786: 14 [cache! #<weak-key-hash-table 2be7540 167/223> # # ...]
1092: 13 [thunk]
1024: 12 [bag->derivation # # #]
In srfi/srfi-1.scm:
 578: 11 [map #<procedure 5b16ba0 at guix/packages.scm:1026:30 (t-23184)> #]
In guix/packages.scm:
 846: 10 [expand-input # # # ...]
In guix/store.scm:
1249: 9 [run-with-store # ...]
In guix/packages.scm:
 480: 8 [#<procedure 5b3b870 at guix/packages.scm:480:2 (state)> #]
 485: 7 [#<procedure 420b600 at guix/packages.scm:485:34 (state)> #]
In gnu/packages/bash.scm:
 249: 6 [#<procedure 420b640 at gnu/packages/bash.scm:249:2 (state)> #]
In guix/download.scm:
 421: 5 [#<procedure 597a500 at guix/download.scm:421:2 (state)> #]
In guix/store.scm:
1169: 4 [#<procedure 5b3b720 at guix/store.scm:1168:28 (store)> #]
In guix/packages.scm:
 786: 3 [cache! #<weak-key-hash-table 2be7540 167/223> #f ...]
1092: 2 [thunk]
 786: 1 [cache! #<weak-key-hash-table 2be7240 170/223> #f ...]
 871: 0 [thunk]

guix/packages.scm:871:27: In procedure thunk:
guix/packages.scm:871:27: In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f

bug#25775: Can't install packages after guix pull

[Ricardo Wurmus]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> I am seeing the same on a fresh checkout and build:

Sirgazil wrote that an old version of the guix-daemon is in use.  What
version of the guix-daemon are you using, Pjotr?

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25775: Can't install packages after guix pull

[Pjotr Prins]

On Sat, Feb 18, 2017 at 04:35:06PM +0100, Ricardo Wurmus wrote:
> 
> Pjotr Prins <pjotr.public12@thebird.nl> writes:
> 
> > I am seeing the same on a fresh checkout and build:
> 
> Sirgazil wrote that an old version of the guix-daemon is in use.  What
> version of the guix-daemon are you using, Pjotr?

/gnu/store/wvj056mfbqid88q2l119kb3mwicii0i6-guix-0.11.0-1.4420/bin/guix-daemon

I'll try a later version soon. At least it builds without grafts.

bug#25775: Can't install packages after guix pull

[Pjotr Prins]

Yes, using the latest guix-daemon and guix client fixed it:

/gnu/store/175nlv448nk5kagwwl3zyy2w4726qfz6-guix-0.12.0-4.d9da/bin/guix-daemon

/home/pjotr/genenetwork/guix/scripts/guix --version
guix (GNU Guix) 20170218.09


./pre-inst-env guix import cpan Time::ParseDate

Starting download of /tmp/guix-file.hSC5MT
From http://mirror.ibcp.fr/pub/CPAN/authors/id/M/MU/MUIR/modules/Time-ParseDate-2015.103.tar.gz...
 ...2015.103.tar.gz  26KiB          201KiB/s 00:00 [####################] 100.0%
(package
  (name "perl-time-parsedate")
  (version "2015.103")
  (source
    (origin
      (method url-fetch)
      (uri (string-append
             "mirror://cpan/authors/id/M/MU/MUIR/modules/Time-ParseDate-"
             version
             ".tar.gz"))
      (sha256
        (base32
          "1lgfr87j4qwqnln0hyyzgik5ixqslzdaksn9m8y824gqbcihc6ic"))))
  (build-system perl-build-system)
  (native-inputs
    `(("perl-extutils-makemaker"
       ,perl-extutils-makemaker)
      ("perl-time-piece" ,perl-time-piece)))
  (home-page
    "http://search.cpan.org/dist/Time-ParseDate")
  (synopsis "Parse and format time values")
  (description fill-in-yourself!)
  (license #f))

I also confirm 'guix pull' failed with the older daemon. Whatever system we
find to replace 'guix pull' should be more conservative in preventing this type of 
breakage.

-- 

bug#25775: Can't install packages after guix pull

[pelzflorian (Florian Pelz)]

On 02/18/2017 12:11 PM, Pjotr Prins wrote:
> I am seeing the same on a fresh checkout and build:
> 
> penguin2:~/genenetwork/guix$ ./pre-inst-env guix package -p $HOME/opt/guix-build-system --install autoconf
> warning: failed to install locale: Invalid argument
> Backtrace:
> In guix/packages.scm:
>  982: 19 [bag-grafts # #]
>  966: 18 [fold-bag-dependencies #<procedure 3cc5a20 at guix/packages.scm:982:29 (package grafts)> ...]
>  983: 17 [#<procedure 3cc5a20 at guix/packages.scm:982:29 (package grafts)> # ()]
>  786: 16 [cache! #<weak-key-hash-table 190de00 0/223> # "x86_64-linux" ...]
>  910: 15 [thunk]
>  786: 14 [cache! #<weak-key-hash-table 18fc520 167/223> # # ...]
> 1092: 13 [thunk]
> 1024: 12 [bag->derivation # # #]
> In srfi/srfi-1.scm:
>  578: 11 [map #<procedure 43a1960 at guix/packages.scm:1026:30 (t-1699962)> #]
> In guix/packages.scm:
>  846: 10 [expand-input # # # ...]
> In guix/store.scm:
> 1249: 9 [run-with-store # ...]
> In guix/packages.scm:
>  480: 8 [#<procedure 43f9f60 at guix/packages.scm:480:2 (state)> #]
>  485: 7 [#<procedure 3f5ff40 at guix/packages.scm:485:34 (state)> #]
> In gnu/packages/bash.scm:
>  249: 6 [#<procedure 3f5ff80 at gnu/packages/bash.scm:249:2 (state)> #]
> In guix/download.scm:
>  421: 5 [#<procedure 418ea00 at guix/download.scm:421:2 (state)> #]
> In guix/store.scm:
> 1169: 4 [#<procedure 43f9e10 at guix/store.scm:1168:28 (store)> #]
> In guix/packages.scm:
>  786: 3 [cache! #<weak-key-hash-table 18fc520 167/223> #f ...]
> 1092: 2 [thunk]
>  786: 1 [cache! #<weak-key-hash-table 1900640 169/223> #f ...]
>  871: 0 [thunk]
> 
> @ commit 42e721a13d6f47397e585f5909d140c4bafb4ad0
> 
> Installation with --no-grafts passes fine.
> 
> Pj.
> 

I had the same error yesterday on Arch after `guix pull` when I tried to
`guix system init` or `guix package -u`. I tried again with a newly
compiled guix from git, but the same error occurred. I did not try
--no-grafts.

Today I don’t even get that far:

$ guix package -u
guix package: error: failed to connect to
`/var/guix/daemon-socket/socket': Connection refused

`systemctl status -l guix-daemon` shows no errors. Another reboot of
course does not help.

Regards,
Florian

bug#25775: Can't install packages after guix pull

[Ricardo Wurmus]

pelzflorian (Florian Pelz) <pelzflorian@pelzflorian.de> writes:

> Today I don’t even get that far:
>
> $ guix package -u
> guix package: error: failed to connect to
> `/var/guix/daemon-socket/socket': Connection refused

This indicates that the daemon is not running.  The daemon (when
running) listens for connections on that socket file.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25775: Can't install packages after guix pull

[pelzflorian (Florian Pelz)]

A full uninstall and reinstall as described on the Arch wiki solved the
issue for me; `guix pull` and everything works again. systemctl had
reported the daemon to be running; I still have no idea what happened.
Some network thing? No idea.

bug#25775: Can't install packages after guix pull

[Tomas Cech]

On Sat, 18 Feb 2017 17:30:34 +0100,
Pjotr Prins wrote:
> 
> Yes, using the latest guix-daemon and guix client fixed it:
> 
> /gnu/store/175nlv448nk5kagwwl3zyy2w4726qfz6-guix-0.12.0-4.d9da/bin/guix-daemon
> 
> /home/pjotr/genenetwork/guix/scripts/guix --version
> guix (GNU Guix) 20170218.09
> 
> 
> ./pre-inst-env guix import cpan Time::ParseDate
> 
> Starting download of /tmp/guix-file.hSC5MT
> >From http://mirror.ibcp.fr/pub/CPAN/authors/id/M/MU/MUIR/modules/Time-ParseDate-2015.103.tar.gz...
>  ...2015.103.tar.gz  26KiB          201KiB/s 00:00 [####################] 100.0%
> (package
>   (name "perl-time-parsedate")
>   (version "2015.103")
>   (source
>     (origin
>       (method url-fetch)
>       (uri (string-append
>              "mirror://cpan/authors/id/M/MU/MUIR/modules/Time-ParseDate-"
>              version
>              ".tar.gz"))
>       (sha256
>         (base32
>           "1lgfr87j4qwqnln0hyyzgik5ixqslzdaksn9m8y824gqbcihc6ic"))))
>   (build-system perl-build-system)
>   (native-inputs
>     `(("perl-extutils-makemaker"
>        ,perl-extutils-makemaker)
>       ("perl-time-piece" ,perl-time-piece)))
>   (home-page
>     "http://search.cpan.org/dist/Time-ParseDate")
>   (synopsis "Parse and format time values")
>   (description fill-in-yourself!)
>   (license #f))
> 
> I also confirm 'guix pull' failed with the older daemon. Whatever system we
> find to replace 'guix pull' should be more conservative in preventing this type of 
> breakage.


So this means that users of Guix can't make `guix pull' continuously
and will eventually need to use GIT tree?

If so, shouldn't we make new quick release to fill the gap?

Or split release of Guix and guix-daemon? That could be helpful...

S_W

bug#25775: Can't install packages after guix pull

[Ricardo Wurmus]

This problem was exposed by commit 50b8a527e, which adds a graft for
bash and along with it a download method for bash patches.  What happens
is that for a reason I haven’t yet understood the download method
procedure is called with “#f” instead of a package object for guile.

This is no problem when the built-in downloader is used, because by
design it doesn’t need Guile or GnuTLS.  Older daemons do not have a
built-in downloader and fall back to the “in-band-download”, which needs
to be passed a Guile package.  Because this doesn’t happen in this case,
Guix at this commit or later abort when an old daemon is used.

Since I lack an understanding of why the “guile” argument could ever
purposefully be set to “#f” I cannot come up with a nice fix.  Instead
of a nice fix I have found a really gross fix:

--8<---------------cut here---------------start------------->8---
diff --git a/guix/download.scm b/guix/download.scm
index 813f51f48..86f859881 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -418,7 +418,10 @@ GnuTLS itself and its dependencies.  See <http://bugs.gnu.org/22774>."
                        ;; hash of the expected result.
                        #:verify-certificate? #f)))))

-  (mlet %store-monad ((guile (package->derivation guile system)))
+  (mlet %store-monad ((guile (package->derivation
+                              (or guile
+                                  (@@ (gnu packages bootstrap) %bootstrap-guile))
+                              system)))
     (gexp->derivation file-name builder
                       #:guile-for-build guile
                       #:system system
--8<---------------cut here---------------end--------------->8---

With this patch installed I can use the latest Guix with an ancient
guix-daemon (I tested with 0.10.0).

For those of you who are using Guix on top of a foreign distribution:
you may find the following steps helpful in upgrading the daemon to
a compatible version.

--8<---------------cut here---------------start------------->8---
# become root
su -

# update to the last version of guix that works
# with older daemons
guix pull --url=http://git.savannah.gnu.org/cgit/guix.git/snapshot/afd532337.tar.gz

# update root's profile to get a new "guix" package
# including a new daemon
guix package -u

# edit the systemd unit to use the new daemon
sed -i "s|ExecStart=.*|ExecStart=/root/.guix-profile/bin/guix-daemon --build-users-group=guixbuild" /etc/systemd/system/guix-daemon.service

# reload the unit file
systemctl daemon-reload

# restart the daemon
systemctl restart guix-daemon

# now become a regular user and run "guix pull"
exit
guix pull
--8<---------------cut here---------------end--------------->8---

Ideally, this wouldn’t be necessary, but I cannot install this ugly
patch (or a prettier variant thereof) before someone who is more
familiar with the code has looked over it.

I hope this helps!

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25775: Can't install packages after guix pull

[Ben Sturmfels]

On Sat, 18 Feb 2017 16:30:34 +0000
Pjotr Prins <pjotr.public12@thebird.nl> wrote:

> Yes, using the latest guix-daemon and guix client fixed it:
> 
> /gnu/store/175nlv448nk5kagwwl3zyy2w4726qfz6-guix-0.12.0-4.d9da/bin/guix-daemon

Could I trouble you to walk through the steps for manually installing a
different version of guix-daemon? I'm stuck in the same situation where
I can't `guix package --install`.

Regards,
Ben

bug#25775: Can't install packages after guix pull

[Ben Sturmfels]

On Tue, 21 Feb 2017 09:44:03 +1100
Ben Sturmfels <ben@stumbles.id.au> wrote:

> On Sat, 18 Feb 2017 16:30:34 +0000
> Pjotr Prins <pjotr.public12@thebird.nl> wrote:
> 
> > Yes, using the latest guix-daemon and guix client fixed it:
> > 
> > /gnu/store/175nlv448nk5kagwwl3zyy2w4726qfz6-guix-0.12.0-4.d9da/bin/guix-daemon  
> 
> Could I trouble you to walk through the steps for manually installing
> a different version of guix-daemon? I'm stuck in the same situation
> where I can't `guix package --install`.

(Also can't `guix pull` without an error)

bug#25775: Can't install packages after guix pull

[Ben Sturmfels]

On Tue, 21 Feb 2017 09:52:11 +1100
Ben Sturmfels <ben@stumbles.id.au> wrote:

> > Could I trouble you to walk through the steps for manually
> > installing a different version of guix-daemon? I'm stuck in the
> > same situation where I can't `guix package --install`.  
> 
> (Also can't `guix pull` without an error)

I'm attempting to fix my GuixSD system by downloading Guix
0.12.0 Binary, extracting and running guix-daemon, then running `guix
pull` against that. The daemon runs ok, but `guix pull` is expecting
its dependencies to be at /gnu, when they're actually at
~/Downloads/gnu. Any suggestions?

$ sudo
Downloads/gnu/store/gj2i5xs7nm80mwrb139pa8iqfccw08zz-guix-0.12.0/bin/guix-daemon
--build-users-group guixbuild --substitute-urls
https://mirror.hydra.gnu.org

$
~/Downloads/gnu/store/gj2i5xs7nm80mwrb139pa8iqfccw08zz-guix-0.12.0/bin/guix
pull
--url=http://git.savannah.gnu.org/cgit/guix.git/snapshot/guix-0.12.0.tar.gz 
/home/ben/Downloads/gnu/store/gj2i5xs7nm80mwrb139pa8iqfccw08zz-guix-0.12.0/bin/guix:
line
4: /gnu/store/gj2i5xs7nm80mwrb139pa8iqfccw08zz-guix-0.12.0/bin/.guix-real:
No such file or directory

bug#25775: Can't install packages after guix pull

[Pjotr Prins]

On Tue, Feb 21, 2017 at 09:44:03AM +1100, Ben Sturmfels wrote:
> On Sat, 18 Feb 2017 16:30:34 +0000
> Pjotr Prins <pjotr.public12@thebird.nl> wrote:
> 
> > Yes, using the latest guix-daemon and guix client fixed it:
> > 
> > /gnu/store/175nlv448nk5kagwwl3zyy2w4726qfz6-guix-0.12.0-4.d9da/bin/guix-daemon
> 
> Could I trouble you to walk through the steps for manually installing a
> different version of guix-daemon? I'm stuck in the same situation where
> I can't `guix package --install`.

Since all the paths are hardcoded inside guix binaries you can move a
version of ./gnu/store/*-guix-0.12.* into the /gnu/store and run it
from there. The store does not care what you move there as long as the
path is unique (a feature of isolation) and it will actually clean up
when you garbage collect. You can even use the binaries inside the
binary tarball distribution of Guix by copying that one directory in.

Note that you should not overwrite files in /var/guix - where the
database resides as it will bring your whole system into a different
state.

I guess this is somewhat of a dirty fix, but it can fast track things
when you are stuck. Guix archive works in a similar fashion, but also
updates the database.

Pj.
-- 

bug#25775: Can't install packages after guix pull

[Ricardo Wurmus]

Ben Sturmfels <ben@stumbles.id.au> writes:

> On Sat, 18 Feb 2017 16:30:34 +0000
> Pjotr Prins <pjotr.public12@thebird.nl> wrote:
>
>> Yes, using the latest guix-daemon and guix client fixed it:
>> 
>> /gnu/store/175nlv448nk5kagwwl3zyy2w4726qfz6-guix-0.12.0-4.d9da/bin/guix-daemon
>
> Could I trouble you to walk through the steps for manually installing a
> different version of guix-daemon? I'm stuck in the same situation where
> I can't `guix package --install`.

I have posted a list of steps to update the daemon here:

    https://lists.gnu.org/archive/html/bug-guix/2017-02/msg00086.html

Do you need anything beyond that?

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25775: Attempts to fix bootstrap Guile bug

[Andy Wingo]

Hi,

It seems that this bug is related to the introduction of
url-fetch/reset-patch-level.  It takes a #:guile kwarg but defaults to
#f; if not given #:guile, that #f propagates through instead of a
package object.

So one fix is here:

  diff --git a/gnu/packages/bootstrap.scm b/gnu/packages/bootstrap.scm
  index c8d94c8..d070cca 100644
  --- a/gnu/packages/bootstrap.scm
  +++ b/gnu/packages/bootstrap.scm
  @@ -76,8 +76,10 @@
   
     (let ((orig-method (origin-method source)))
       (origin (inherit source)
  -      (method (cond ((eq? orig-method url-fetch)
  -                     (boot url-fetch))
  +      (method (cond ((or (eq? orig-method url-fetch)
  +                         (eq? orig-method
  +                              (@@ (gnu packages bash) url-fetch/reset-patch-level)))
  +                     (boot orig-method))
                       (else orig-method)))
         (patch-guile %bootstrap-guile)
         (patch-inputs %bootstrap-patch-inputs)

I think anyway.  If you look at this code, the "boot" helper decorates
the url-fetch call with a #:guile.  But I don't think this function is
called for bash because I still see an error.

Another fix is here I think:

  diff --git a/gnu/packages/bash.scm b/gnu/packages/bash.scm
  index c121fd8..fdb3aa8 100644
  --- a/gnu/packages/bash.scm
  +++ b/gnu/packages/bash.scm
  @@ -250,7 +250,8 @@ can apply to a patch-level 0 Bash."
                          (patch (url-fetch url hash-algo hash
                                            (string-append name ".orig")
                                            #:system system
  -                                         #:guile guile)))
  +                                         #:guile guile))
  +                       (guile (package->derivation guile system)))
       (gexp->derivation name
                         (with-imported-modules '((guix build utils))
                           #~(begin

Here this "guile" propagates below to #:guile-for-build, which AFAIU
should be a derivation, not a package.

But even with this, I still get an error.  Add (unless guile (error
"what")) to the top of url-fetch/reset-patch-level, and I get:

  [6h8mzf08ld7r69yy9gdbxw9vba0l9sr2-profile] wingo@clucks:~/src/guix$ guix build guile-next
  Backtrace:
  In srfi/srfi-1.scm:
   575: 19 [map #<procedure 1a7fb40 at guix/scripts/build.scm:629:16 (expr)> (#)]
  In guix/scripts/build.scm:
   634: 18 [#<procedure 1a7fb40 at guix/scripts/build.scm:629:16 (expr)> #]
  In guix/packages.scm:
   786: 17 [cache! #<weak-key-hash-table 18a8a40 318/443> # # ...]
  1095: 16 [thunk]
   982: 15 [bag-grafts # #]
   966: 14 [fold-bag-dependencies #<procedure 501b0a0 at guix/packages.scm:982:29 (package grafts)> ...]
   983: 13 [#<procedure 501b0a0 at guix/packages.scm:982:29 (package grafts)> # #]
   786: 12 [cache! #<weak-key-hash-table 18a8600 3/223> # "x86_64-linux" ...]
   910: 11 [thunk]
   786: 10 [cache! #<weak-key-hash-table 18a8a40 318/443> # # ...]
  1092: 9 [thunk]
  1024: 8 [bag->derivation # # #]
  In srfi/srfi-1.scm:
   575: 7 [map #<procedure 4bce810 at guix/packages.scm:1026:30 (t-32174)> #]
  In guix/packages.scm:
   846: 6 [expand-input # # # ...]
  In guix/store.scm:
  1249: 5 [run-with-store # ...]
  In guix/packages.scm:
  1200: 4 [#<procedure 4cee600 at guix/packages.scm:1194:5 (state)> #]
   486: 3 [patch-and-repack # # # ...]
  In srfi/srfi-1.scm:
   575: 2 [map #<procedure instantiate-patch (expr)> (#)]
  In gnu/packages/bash.scm:
   249: 1 [url-fetch/reset-patch-level "mirror://gnu/bash/bash-4.4-patches/bash44-007" ...]
  In unknown file:
     ?: 0 [scm-error misc-error #f "~A" ("what") #f]

  ERROR: In procedure scm-error:
  ERROR: what

That indicates to me that somehow the origin for this bash (or is it
bash/fixed?) isn't getting its url-fetch call appropriately decorated.

I think in the meantime it would be acceptable to insert an

  (unless guile
    (error "update your guix daemon from git and restart it (#25775)"))

to url-fetch/reset-patch-level, or something like that.  WDYT?

Andy

bug#25775: Attempts to fix bootstrap Guile bug

[Leo Famulari]

On Tue, Feb 21, 2017 at 09:52:09AM +0100, Andy Wingo wrote:
> I think in the meantime it would be acceptable to insert an
> 
>   (unless guile
>     (error "update your guix daemon from git and restart it (#25775)"))
> 
> to url-fetch/reset-patch-level, or something like that.  WDYT?

I think it would be better if users didn't have to set up a Guix
development environment to work around this bug.

My hunch is that the people who never updated root's packages and thus
experienced the bug are the people who are least comfortable with such a
solution.

What about Ricardo's "really gross fix"? [0] I don't understand it enough to
know why it's gross. If it works and does not break other things, at
least it could protect people who have not yet hit the bug, but will hit
it the next time they run `guix pull`.
 
[0]
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25775#35

bug#25775: Attempts to fix bootstrap Guile bug

[Ricardo Wurmus]

Leo Famulari <leo@famulari.name> writes:

> What about Ricardo's "really gross fix"? [0] I don't understand it enough to
> know why it's gross. If it works and does not break other things, at
> least it could protect people who have not yet hit the bug, but will hit
> it the next time they run `guix pull`.

It’s gross because it’s just a band aid.  I don’t know if it might break
other things (in my short tests it didn’t).  It shouldn’t affect people
with new daemons, because they will use the daemon’s download feature.

I guess we could just push it to fix “guix pull” now and revert later
once we have a good fix (or if it causes trouble).

What do you think?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25775: Can't install packages after guix pull

[Ricardo Wurmus]

Ben Sturmfels <ben@stumbles.id.au> writes:

> On Tue, 21 Feb 2017 09:52:11 +1100
> Ben Sturmfels <ben@stumbles.id.au> wrote:
>
>> > Could I trouble you to walk through the steps for manually
>> > installing a different version of guix-daemon? I'm stuck in the
>> > same situation where I can't `guix package --install`.  
>> 
>> (Also can't `guix pull` without an error)
>
> I'm attempting to fix my GuixSD system by downloading Guix
> 0.12.0 Binary, extracting and running guix-daemon, then running `guix
> pull` against that. The daemon runs ok, but `guix pull` is expecting
> its dependencies to be at /gnu, when they're actually at
> ~/Downloads/gnu. Any suggestions?

How did you end up having dependencies outside of /gnu?  All Guix
packages depend only on things inside the store.  If you want to use
substitutes you cannot avoid /gnu.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25775: Attempts to fix bootstrap Guile bug

[Leo Famulari]

On Tue, Feb 21, 2017 at 11:34:55PM +0100, Ricardo Wurmus wrote:
> 
> Leo Famulari <leo@famulari.name> writes:
> 
> > What about Ricardo's "really gross fix"? [0] I don't understand it enough to
> > know why it's gross. If it works and does not break other things, at
> > least it could protect people who have not yet hit the bug, but will hit
> > it the next time they run `guix pull`.
> 
> It’s gross because it’s just a band aid.  I don’t know if it might break
> other things (in my short tests it didn’t).  It shouldn’t affect people
> with new daemons, because they will use the daemon’s download feature.
> 
> I guess we could just push it to fix “guix pull” now and revert later
> once we have a good fix (or if it causes trouble).
> 
> What do you think?

Since you know my relative ignorance, take this with a grain of salt,
but that sounds good to me. You should probably wait for Andy's opinion,
though :)

bug#25775: Can't install packages after guix pull

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

On Mon, Feb 20, 2017 at 05:06:48PM +0100, Ricardo Wurmus wrote:
> Since I lack an understanding of why the “guile” argument could ever
> purposefully be set to “#f” I cannot come up with a nice fix.  Instead
> of a nice fix I have found a really gross fix:
> 
> --8<---------------cut here---------------start------------->8---
> diff --git a/guix/download.scm b/guix/download.scm
> index 813f51f48..86f859881 100644
> --- a/guix/download.scm
> +++ b/guix/download.scm
> @@ -418,7 +418,10 @@ GnuTLS itself and its dependencies.  See <http://bugs.gnu.org/22774>."
>                         ;; hash of the expected result.
>                         #:verify-certificate? #f)))))
> 
> -  (mlet %store-monad ((guile (package->derivation guile system)))
> +  (mlet %store-monad ((guile (package->derivation
> +                              (or guile
> +                                  (@@ (gnu packages bootstrap) %bootstrap-guile))
> +                              system)))
>      (gexp->derivation file-name builder
>                        #:guile-for-build guile
>                        #:system system
> --8<---------------cut here---------------end--------------->8---

Based on discussion on #guix with Ricardo and Andy, I pushed this diff
as 9f05908fb1e3707cae593d94688748294717a546 on Ricardo's behalf.

I'm leaving the bug open until we find a better fix.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#25775: Can't install packages after guix pull

[Ricardo Wurmus]

Leo Famulari <leo@famulari.name> writes:

> On Mon, Feb 20, 2017 at 05:06:48PM +0100, Ricardo Wurmus wrote:
>> Since I lack an understanding of why the “guile” argument could ever
>> purposefully be set to “#f” I cannot come up with a nice fix.  Instead
>> of a nice fix I have found a really gross fix:
>>
>> --8<---------------cut here---------------start------------->8---
>> diff --git a/guix/download.scm b/guix/download.scm
>> index 813f51f48..86f859881 100644
>> --- a/guix/download.scm
>> +++ b/guix/download.scm
>> @@ -418,7 +418,10 @@ GnuTLS itself and its dependencies.  See <http://bugs.gnu.org/22774>."
>>                         ;; hash of the expected result.
>>                         #:verify-certificate? #f)))))
>>
>> -  (mlet %store-monad ((guile (package->derivation guile system)))
>> +  (mlet %store-monad ((guile (package->derivation
>> +                              (or guile
>> +                                  (@@ (gnu packages bootstrap) %bootstrap-guile))
>> +                              system)))
>>      (gexp->derivation file-name builder
>>                        #:guile-for-build guile
>>                        #:system system
>> --8<---------------cut here---------------end--------------->8---
>
> Based on discussion on #guix with Ricardo and Andy, I pushed this diff
> as 9f05908fb1e3707cae593d94688748294717a546 on Ricardo's behalf.

Thank you!

> I'm leaving the bug open until we find a better fix.

That’s fair.

For people who encounter this bug at a later point: please upgrade the
daemon; that’s better than trying to use the latest Guix with an
outdated daemon.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#25775: Attempts to fix bootstrap Guile bug

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

Hello,

Thanks Andy & Ricardo for the detailed explanations!

Andy Wingo <wingo@pobox.com> skribis:

> It seems that this bug is related to the introduction of
> url-fetch/reset-patch-level.  It takes a #:guile kwarg but defaults to
> #f; if not given #:guile, that #f propagates through instead of a
> package object.

Nasty.

To reproduce the problem reported here, one can:

  1. Revert the “band-aid commit”
     9f05908fb1e3707cae593d94688748294717a546.

  2. Change download.scm to force it to behave as when talking to an old
     daemon.

This gives this:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1256 bytes --]

diff --git a/guix/download.scm b/guix/download.scm
index 86f859881..811abe27b 100644
--- a/guix/download.scm
+++ b/guix/download.scm
@@ -418,10 +418,7 @@ GnuTLS itself and its dependencies.  See <http://bugs.gnu.org/22774>."
                        ;; hash of the expected result.
                        #:verify-certificate? #f)))))
 
-  (mlet %store-monad ((guile (package->derivation
-                              (or guile
-                                  (@@ (gnu packages bootstrap) %bootstrap-guile))
-                              system)))
+  (mlet %store-monad ((guile (package->derivation guile system)))
     (gexp->derivation file-name builder
                       #:guile-for-build guile
                       #:system system
@@ -472,7 +469,7 @@ in the store."
             (and uri (memq (uri-scheme uri) '(#f file))))
         (interned-file (if uri (uri-path uri) url)
                        (or name file-name))
-        (mlet* %store-monad ((builtins (built-in-builders*))
+        (mlet* %store-monad ((builtins -> '())
                              (download -> (if (member "download" builtins)
                                               built-in-download
                                               in-band-download)))

[-- Attachment #3: Type: text/plain, Size: 232 bytes --]


Then run something like:

  guix gc -d /gnu/store/*-bash-4.4.tar.xz
  ./pre-inst-env guix build bash -S --no-substitutes

~~~~~~

To mirror what ‘url-fetch’ does, we should change the default value of
#:guile here:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: Type: text/x-patch, Size: 697 bytes --]

diff --git a/gnu/packages/bash.scm b/gnu/packages/bash.scm
index c3b94391e..b4d0b6777 100644
--- a/gnu/packages/bash.scm
+++ b/gnu/packages/bash.scm
@@ -243,7 +243,8 @@ without modification.")
 
 (define* (url-fetch/reset-patch-level url hash-algo hash
                                       #:optional name
-                                      #:key (system (%current-system)) guile)
+                                      #:key (system (%current-system))
+                                      (guile (default-guile)))
   "Fetch the Bash patch from URL and reset its 'PATCHLEVEL' definition so it
 can apply to a patch-level 0 Bash."
   (mlet* %store-monad ((name -> (or name (basename url)))

[-- Attachment #5: Type: text/plain, Size: 277 bytes --]


However that leads to a stack overflow unless we patch
‘bootstrap-origin’ the way Andy suggests (which is not desirable IMO).

So, instead, we can simply force the use of the bootstrap Guile for
these derivations, which doesn’t make any difference functionally:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #6: Type: text/x-patch, Size: 1837 bytes --]

--- a/gnu/packages/bash.scm
+++ b/gnu/packages/bash.scm
@@ -21,6 +21,7 @@
 (define-module (gnu packages bash)
   #:use-module (guix licenses)
   #:use-module (gnu packages)
+  #:use-module (gnu packages bootstrap)
   #:use-module (gnu packages ncurses)
   #:use-module (gnu packages readline)
   #:use-module (gnu packages bison)
@@ -243,14 +244,17 @@ without modification.")
 
 (define* (url-fetch/reset-patch-level url hash-algo hash
                                       #:optional name
-                                      #:key (system (%current-system)) guile)
+                                      #:key (system (%current-system)))
   "Fetch the Bash patch from URL and reset its 'PATCHLEVEL' definition so it
 can apply to a patch-level 0 Bash."
+  ;; Note: Forcefully use %BOOTSTRAP-GUILE here to work around bootstrapping
+  ;; issues when using a daemon that lacks the "download" built-in.  See
+  ;; <https://bugs.gnu.org/25775>.
   (mlet* %store-monad ((name -> (or name (basename url)))
                        (patch (url-fetch url hash-algo hash
                                          (string-append name ".orig")
                                          #:system system
-                                         #:guile guile)))
+                                         #:guile %bootstrap-guile)))
     (gexp->derivation name
                       (with-imported-modules '((guix build utils))
                         #~(begin
@@ -259,7 +263,6 @@ can apply to a patch-level 0 Bash."
                             (substitute* #$output
                               (("PATCHLEVEL [0-6]+")
                                "PATCHLEVEL 0"))))
-                      #:guile-for-build guile
                       #:system system)))
 
 (define bash/fixed                        ;CVE-2017-5932 (RCE with completion)

[-- Attachment #7: Type: text/plain, Size: 138 bytes --]


And it does the job.

Pushed as 6c5b56f9fa01b7fe9034bac47b20e08a2fdb2629.  Let me know if
there are still fishy things!

Ludo’.

bug#25775: Can't install packages after guix pull

[Ben Sturmfels]

[-- Attachment #1: Type: text/plain, Size: 3073 bytes --]

On 22/02/17 09:36, Ricardo Wurmus wrote:
> 
> Ben Sturmfels <ben@stumbles.id.au> writes:
> 
>> On Tue, 21 Feb 2017 09:52:11 +1100
>> Ben Sturmfels <ben@stumbles.id.au> wrote:
>>
>>>> Could I trouble you to walk through the steps for manually
>>>> installing a different version of guix-daemon? I'm stuck in the
>>>> same situation where I can't `guix package --install`.  
>>>
>>> (Also can't `guix pull` without an error)
>>
>> I'm attempting to fix my GuixSD system by downloading Guix
>> 0.12.0 Binary, extracting and running guix-daemon, then running `guix
>> pull` against that. The daemon runs ok, but `guix pull` is expecting
>> its dependencies to be at /gnu, when they're actually at
>> ~/Downloads/gnu. Any suggestions?
> 
> How did you end up having dependencies outside of /gnu?  All Guix
> packages depend only on things inside the store.  If you want to use
> substitutes you cannot avoid /gnu.

Sorry for the late reply. I had dependencies outside of /gnu
because I was attempting to upgrade guix-daemon without messing with the
nice clean read-only mounted /gnu/store filesystem. Turns out I didn't
use that approach in the end and used "rollback" instead.

I realised that the recovery steps in
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25775#35 were more for
Guix on a foreign distro.

For anyone else on GuixSD who can't `guix pull` or `guix package
--install` due to older guix-daemon mismatched with newer guix installed
for root, here's how I fixed my system:

1. I had a number of system generations available so I rolled back to
one before I ran the problematic "sudo guix pull", with `guix system
list-generations` and `guix system switch-generations`.

2. Ran `sudo guix pull
--url=http://git.savannah.gnu.org/cgit/guix.git/snapshot/afd532337.tar.gz`
to get the last version of guix compatible with older daemons.

3. Ran `sudo guix system reconfigure myconfig.scm`, which upgrades
guix-daemon. Note that `guix package --upgrade` as root doesn't upgrade
guix-daemon on GuixSD. Note also that I ended up reducing the services
in this config to just "%base-services" to avoid hours of unnecessary
compiling, when I really just needed guix-daemon. Ended up having to do
it in a virtual terminal.

4. Reboot and verify that I had an 0.12 version of guix-daemon running.

5. Ran `sudo guix pull` and `sudo guix system reconfigure myconfig2.scm`
to restore my full desktop services and upgrade system infrastructure to
latest version.

6. For good measure, ran `sudo guix pull` and `sudo guix package
--upgrade` to bring root's guix tools up to date (actually had zero
packages installed for root).

7. Ran `guix pull` and `guix package --upgrade` as my normal user to
update that too.

My actual path took a lot more than 7 steps, so I hope I got those
instructions right. ;)

Reinstalling probably would have been faster, but I've learnt a lot more
about how GuixSD works, and it's satisfying and reassuring to be able to
fix things by rolling back!

Regards,
Ben






[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]