* Guix v1.1.0 (~2020) and Software Heritage & friends
@ 2023-02-28 19:51 Simon Tournier
2023-03-01 15:12 ` v1.1.0 (~2020) and SWH: Part II Simon Tournier
2023-03-06 20:40 ` Guix v1.1.0 (~2020) and Software Heritage & friends Ludovic Courtès
0 siblings, 2 replies; 4+ messages in thread
From: Simon Tournier @ 2023-02-28 19:51 UTC (permalink / raw)
To: Guix Devel; +Cc: Mathieu Othacehe
Hi,
Here I would like to report my attempt to simulate a “network collapse”
– as discussed with Mathieu at Guix Days. :-)
What is the minimal set of machines for running “guix time-machine” from
a current Guix to an older one?
Using this,
archive.softwareheritage.org
disarchive.guix.gnu.org
git.savannah.gnu.org
is not enough. It fails for many. For instance,
ed-1.15
gcc-core-2.95.3
ghostscript-9.27
guile-2.2.6
linux-libre-4.19.56-gnu
linux-libre-headers-stripped-4.14.67-i686-linux
mes-0.19
mes-minimal-stripped-0.19-i686-linux
do not seem backed up with SWH + Disarchive. Well, basically after
these, I stop because it’s boring and I add ftp.gnu.org to the list. It
still fails,
nyacc-0.86.0
static-binaries
First lesson: we have holes in sources.json and maybe Disarchive too; I
will give a look.
To be continued…
Cheers,
simon
- - -
Using Guix b8f6ead (~February 2023) and trying to go to v1.1.0 (~April
2020) corresponding to this commit,
--8<---------------cut here---------------start------------->8---
$ cat /tmp/v1.1.0.scm
(list (channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git")
(commit
"d62c9b2671be55ae0305bebfda17b595f33797f2")))
--8<---------------cut here---------------end--------------->8---
All is fine when substitutes is allowed and the network is fine. So let
garbage collect all and restart reaching the minimal set of network.
Now, let disable the network name resolution and only resolve for
archive.softwareheritage.org. Then using the exact same command line as
previously, I get:
--8<---------------cut here---------------start------------->8---
$ guix time-machine -C /tmp/v1.1.0.scm -- help
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: warning: ci.guix.gnu.org: host not found: Name or service not known
substitute:
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 0.0%guix substitute: warning: bordeaux.guix.gnu.org: host not found: Name or service not known
substitute:
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 0.0%
retrying download of '/gnu/store/8kv98fjwjp26ry1ixjdz28i1vjj9xfqi-config.scm' with other substitute URLs...
guix substitute: warning: bordeaux.guix.gnu.org: host not found: Name or service not known
guix substitute: error: failed to find alternative substitute for '/gnu/store/8kv98fjwjp26ry1ixjdz28i1vjj9xfqi-config.scm'
substitution of /gnu/store/8kv98fjwjp26ry1ixjdz28i1vjj9xfqi-config.scm failed
building /gnu/store/4a2p4vylpw1l0wz5szvyh4xgm0x5pl7f-config.scm.drv...
guix time-machine: error: some substitutes for the outputs of derivation `/gnu/store/lf70hiwqyz5qx418zjz7chn2lwq7nhic-module-import-compiled.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source
--8<---------------cut here---------------end--------------->8---
which is a bit unexpected. I miss why Guix does not fallback by itself
but that’s another story I guess.
Let re-run using --fallback as explained,
--8<---------------cut here---------------start------------->8---
$ guix time-machine -C /tmp/v1.1.0.scm --fallback -- help
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: warning: ci.guix.gnu.org: host not found: Name or service not known
substitute:
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 0.0%guix substitute: warning: bordeaux.guix.gnu.org: host not found: Name or service not known
substitute:
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 0.0%
retrying download of '/gnu/store/8kv98fjwjp26ry1ixjdz28i1vjj9xfqi-config.scm' with other substitute URLs...
guix substitute: warning: bordeaux.guix.gnu.org: host not found: Name or service not known
guix substitute: error: failed to find alternative substitute for '/gnu/store/8kv98fjwjp26ry1ixjdz28i1vjj9xfqi-config.scm'
substitution of /gnu/store/8kv98fjwjp26ry1ixjdz28i1vjj9xfqi-config.scm failed
building /gnu/store/4a2p4vylpw1l0wz5szvyh4xgm0x5pl7f-config.scm.drv...
building /gnu/store/0xpdpf1az8p2i1zppn4s1w407f5b2ngg-git.scm.drv...
building /gnu/store/c70kbbbw6v9vj10fab7jnbjl9yv3s23j-hash.scm.drv...
building /gnu/store/cq303xhw6g86b5nyn1xc1gqr2i5bgn91-module-import.drv...
building /gnu/store/z4f6cy8lp1c90wvjzh738kdznxbii2vf-module-import.drv...
building /gnu/store/wjbqf99lhpyx84c6nqdvvl1lyhgi7igw-module-import-compiled.drv...
building /gnu/store/lf70hiwqyz5qx418zjz7chn2lwq7nhic-module-import-compiled.drv...
building /gnu/store/mhmpdnzklfw2py384wbqa74avnsahg3q-compute-guix-derivation.drv...
WARNING: (guix build emacs-build-system): imported module (guix build utils) overrides core binding `delete'
Computing Guix derivation for 'x86_64-linux'... |@ substituter-started /gnu/store/l1iakyjw5lacjbnynm6z7b31clyh1llx-ghostscript-9.27-doc substitute
retrying download of '/gnu/store/l1iakyjw5lacjbnynm6z7b31clyh1llx-ghostscript-9.27-doc' with other substitute URLs...
guix substitute: warning: bordeaux.guix.gnu.org: host not found: Name or service not known
guix substitute: error: failed to find alternative substitute for '/gnu/store/l1iakyjw5lacjbnynm6z7b31clyh1llx-ghostscript-9.27-doc'
@ substituter-failed /gnu/store/l1iakyjw5lacjbnynm6z7b31clyh1llx-ghostscript-9.27-doc fetching path `/gnu/store/l1iakyjw5lacjbnynm6z7b31clyh1llx-ghostscript-9.27-doc' (empty status: '')
Backtrace:
13 (primitive-load "/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation")
In ice-9/eval.scm:
155:9 12 (_ _)
159:9 11 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#<directory (guile-user) 7fb6842d0c?> ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
In ./guix/store.scm:
1975:24 10 (run-with-store #<store-connection 256.99 7fb681ec7500> #<procedure 7fb67d5c8d80 at ./guix/self.scm:11?> ?)
1811:8 9 (_ #<store-connection 256.99 7fb681ec7500>)
In ./guix/gexp.scm:
959:2 8 (_ #<store-connection 256.99 7fb681ec7500>)
819:2 7 (_ #<store-connection 256.99 7fb681ec7500>)
In ./guix/store.scm:
1859:12 6 (_ #<store-connection 256.99 7fb681ec7500>)
1312:5 5 (map/accumulate-builds #<store-connection 256.99 7fb681ec7500> #<procedure 7fb67e8492c0 at ./guix/stor?> ?)
1323:15 4 (_ #<store-connection 256.99 7fb681ec7500> ("/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscrip?" ?) ?)
1323:15 3 (loop #f)
711:11 2 (process-stderr #<store-connection 256.99 7fb681ec7500> _)
In ./guix/serialization.scm:
101:11 1 (read-int #<input-output: file 10>)
79:6 0 (get-bytevector-n* #<input-output: file 10> 8)
./guix/serialization.scm:79:6: In procedure get-bytevector-n*:
ERROR:
1. &nar-error:
file: #f
port: #<input-output: file 10>
guix time-machine: error: You found a bug: the program '/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation'
failed to compute the derivation for Guix (version: "d62c9b2671be55ae0305bebfda17b595f33797f2"; system: "x86_64-linux";
host version: "b8f6ead5faac3c1b9a8fa6e060c00cf0917e884e"; pull-version: 1).
Please report it by email to <bug-guix@gnu.org>.
--8<---------------cut here---------------end--------------->8---
It’s more than before… but still. The backtrace is unexpected–another
story. Again, I do not understand why Guix does not run by itself
--no-substitutes, so let force it:
--8<---------------cut here---------------start------------->8---
$ guix time-machine -C /tmp/v1.1.0.scm --fallback --no-substitutes -- help
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... WARNING: (guix build emacs-build-system): imported module (guix build utils) overrides core binding `delete'
|@ build-started /gnu/store/z9gv2cv3g61ywd5xrjybradhazjp27m4-bzip2-1.0.6.tar.gz.drv - x86_64-linux /var/log/guix/drvs/z9//gv2cv3g61ywd5xrjybradhazjp27m4-bzip2-1.0.6.tar.gz.drv.gz 2206447
@ build-log 2206447 15
Starting downl@ build-log 2206447 166
oad of /gnu/store/bjpalrv9f49d2k40p7ga0y6dwdys9w5j-bzip2-1.0.6.tar.gz
From https://web.archive.org/web/20180624184806/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz...
@ build-log 2206447 52
In procedure getaddrinfo: Name or service not known
[...]
Starting download of /gnu/store/mzfkrxd4w8vqrmyrx169wj8wyw7r8i37-bash
From https://web.archive.org/web/20230228180751/https://git.savannah.gnu.org/cgit/guix.git/plain/gnu/packages/bootstrap/i686-linux/bash?id=44f07d1dc6806e97c4e9ee3e6be883cc59dc666e...
@ build-log 2206585 141
In procedure getaddrinfo: Name or service not known
Trying to use Disarchive to assemble /gnu/store/mzfkrxd4w8vqrmyrx169wj8wyw7r8i37-bash...
@ build-log 2206585 44
could not find its Disarchive specification
@ build-log 2206585 209
failed to download "/gnu/store/mzfkrxd4w8vqrmyrx169wj8wyw7r8i37-bash" from "https://git.savannah.gnu.org/cgit/guix.git/plain/gnu/packages/bootstrap/i686-linux/bash?id=44f07d1dc6806e97c4e9ee3e6be883cc59dc666e"
builder for `/gnu/store/y0bayja26kgi6whfzrwkx7rv88x25qw9-bash.drv' failed to produce output path `/gnu/store/mzfkrxd4w8vqrmyrx169wj8wyw7r8i37-bash'
@ build-failed /gnu/store/y0bayja26kgi6whfzrwkx7rv88x25qw9-bash.drv - 1 builder for `/gnu/store/y0bayja26kgi6whfzrwkx7rv88x25qw9-bash.drv' failed to produce output path `/gnu/store/mzfkrxd4w8vqrmyrx169wj8wyw7r8i37-bash'
@ build-started /gnu/store/sv095nqg5xfv3a9ii0788ylmvph08ll5-bash50-001.drv - x86_64-linux /var/log/guix/drvs/sv//095nqg5xfv3a9ii0788ylmvph08ll5-bash50-001.drv.gz 2206611
cannot build derivation `/gnu/store/9g3mfjpxajl69krfwvdiaznz5gh54af2-guile-bootstrap-2.0.drv': 1 dependencies couldn't be built
[...]
Backtrace:
10 (primitive-load "/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation")
In ice-9/eval.scm:
155:9 9 (_ _)
159:9 8 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#<directory (guile-user) 7f8d4d479c?> ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
In ./guix/store.scm:
1975:24 7 (run-with-store #<store-connection 256.99 7f8d4b0724b0> #<procedure 7f8d3b427580 at ./guix/self.scm:11?> ?)
1811:8 6 (_ #<store-connection 256.99 7f8d4b0724b0>)
In ./guix/gexp.scm:
959:2 5 (_ #<store-connection 256.99 7f8d4b0724b0>)
819:2 4 (_ #<store-connection 256.99 7f8d4b0724b0>)
In ./guix/store.scm:
1859:12 3 (_ #<store-connection 256.99 7f8d4b0724b0>)
1312:5 2 (map/accumulate-builds #<store-connection 256.99 7f8d4b0724b0> #<procedure 7f8d4a374900 at ./guix/stor?> ?)
1323:15 1 (_ #<store-connection 256.99 7f8d4b0724b0> ("/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscrip?" ?) ?)
1323:15 0 (loop #f)
./guix/store.scm:1323:15: In procedure loop:
ERROR:
1. &store-protocol-error:
message: "build of `/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscript-9.27.drv' failed"
status: 1
guix time-machine: error: You found a bug: the program '/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation'
failed to compute the derivation for Guix (version: "d62c9b2671be55ae0305bebfda17b595f33797f2"; system: "x86_64-linux";
host version: "b8f6ead5faac3c1b9a8fa6e060c00cf0917e884e"; pull-version: 1).
Please report it by email to <bug-guix@gnu.org>.
--8<---------------cut here---------------end--------------->8---
Well, the failure is expected but not the backtrace–again another
story. :-)
Now, let add git.savannah.gnu.org and repeat.
--8<---------------cut here---------------start------------->8---
$ guix time-machine -C /tmp/v1.1.0.scm --fallback --no-substitutes -- help
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... WARNING: (guix build emacs-build-system): imported module (guix build utils) overrides core binding `delete'
|@ build-started /gnu/store/z9gv2cv3g61ywd5xrjybradhazjp27m4-bzip2-1.0.6.tar.gz.drv - x86_64-linux /var/log/guix/drvs/z9//gv2cv3g61ywd5xrjybradhazjp27m4-bzip2-1.0.6.tar.gz.drv.gz 2231928
@ build-log 2231928 15
Starting downl@ build-log 2231928 166
oad of /gnu/store/bjpalrv9f49d2k40p7ga0y6dwdys9w5j-bzip2-1.0.6.tar.gz
From https://web.archive.org/web/20180624184806/http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz...
@ build-log 2231928 52
In procedure getaddrinfo: Name or service not known
[...]
failed to download "/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz" from "mirror://gnu/ed/ed-1.15.tar.lz"
builder for `/gnu/store/1bhz96rq2faiy8r7yk29g2ydrrs4pzfy-ed-1.15.tar.lz.drv' failed to produce output path `/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz'
@ build-failed /gnu/store/1bhz96rq2faiy8r7yk29g2ydrrs4pzfy-ed-1.15.tar.lz.drv - 1 builder for `/gnu/store/1bhz96rq2faiy8r7yk29g2ydrrs4pzfy-ed-1.15.tar.lz.drv' failed to produce output path `/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz'
cannot build derivation `/gnu/store/kdm7cyhlqqv3cri4qlymjrjfvd2imm24-ed-1.15.drv': 1 dependencies couldn't be built
@ build-started /gnu/store/xjs4kkxyqn79sys2jkbw26w522adkrqp-expat-2.2.7.tar.xz.drv - x86_64-linux /var/log/guix/drvs/xj//s4kkxyqn79sys2jkbw26w522adkrqp-expat-2.2.7.tar.xz.drv.gz 2232408
cannot build derivation `/gnu/store/i8nhiybq6a9w6fiz1vhnxc8r00mxyn3q-patch-2.7.6.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscript-9.27.drv': 1 dependencies couldn't be built
Backtrace:
10 (primitive-load "/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation")
In ice-9/eval.scm:
155:9 9 (_ _)
159:9 8 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#<directory (guile-user) 7f0edc744c?> ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
In ./guix/store.scm:
1975:24 7 (run-with-store #<store-connection 256.99 7f0eda33a500> #<procedure 7f0ecfdaa8c0 at ./guix/self.scm:11?> ?)
1811:8 6 (_ #<store-connection 256.99 7f0eda33a500>)
In ./guix/gexp.scm:
959:2 5 (_ #<store-connection 256.99 7f0eda33a500>)
819:2 4 (_ #<store-connection 256.99 7f0eda33a500>)
In ./guix/store.scm:
1859:12 3 (_ #<store-connection 256.99 7f0eda33a500>)
1312:5 2 (map/accumulate-builds #<store-connection 256.99 7f0eda33a500> #<procedure 7f0ed1f53340 at ./guix/stor?> ?)
1323:15 1 (_ #<store-connection 256.99 7f0eda33a500> ("/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscrip?" ?) ?)
1323:15 0 (loop #f)
./guix/store.scm:1323:15: In procedure loop:
ERROR:
1. &store-protocol-error:
message: "build of `/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscript-9.27.drv' failed"
status: 1
guix time-machine: error: You found a bug: the program '/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation'
failed to compute the derivation for Guix (version: "d62c9b2671be55ae0305bebfda17b595f33797f2"; system: "x86_64-linux";
host version: "b8f6ead5faac3c1b9a8fa6e060c00cf0917e884e"; pull-version: 1).
Please report it by email to <bug-guix@gnu.org>.
--8<---------------cut here---------------end--------------->8---
Well, that’s still expected. Now, let add disarchive.guix.gnu.org and
repeat.
--8<---------------cut here---------------start------------->8---
$ guix time-machine -C /tmp/v1.1.0.scm --fallback --no-substitutes -- help
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... WARNING: (guix build emacs-build-system): imported module (guix build utils) overrides core binding `delete'
|@ build-started /gnu/store/xjs4kkxyqn79sys2jkbw26w522adkrqp-expat-2.2.7.tar.xz.drv - x86_64-linux /var/log/guix/drvs/xj//s4kkxyqn79sys2jkbw26w522adkrqp-expat-2.2.7.tar.xz.drv.gz 2234548
@ build-log 2234548 19
Starting download @ build-log 2234548 152
of /gnu/store/m88np9f2215x7bvn2nfwj777src4imjq-expat-2.2.7.tar.xz
From http://downloads.sourceforge.net/project/expat/expat/2.2.7/expat-2.2.7.tar.xz...
@ build-log 2234548 142
In procedure getaddrinfo: Name or service not known
[...]
@ build-log 2234568 179
download failed "https://archive.softwareheritage.org/api/1/content/sha256:ad4489c0ad7a108c514262da28e6c2a426946fb408a3977ef1ed34308bdfd174/raw/" 404 "Not Found"
Starting downloa@ build-log 2234568 69
d of /gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz
From @ build-log 2234568 94
https://web.archive.org/web/20230228192114/https://ftpmirror.gnu.org/gnu/ed/ed-1.15.tar.lz...
@ build-log 2234568 52
In procedure getaddrinfo: Name or service not known
@ build-log 2234568 1
T@ build-log 2234568 6
rying @ build-log 2234568 3
to @ build-log 2234568 4
use @ build-log 2234568 3
Dis@ build-log 2234568 4
arch@ build-log 2234568 4
ive @ build-log 2234568 4
to a@ build-log 2234568 4
ssem@ build-log 2234568 3
ble@ build-log 2234568 1
@ build-log 2234568 62
/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz...
@ build-log 2234568 24
could not find its Disar@ build-log 2234568 20
chive specification
@ build-log 2234568 118
failed to download "/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz" from "mirror://gnu/ed/ed-1.15.tar.lz"
builder for `/gnu/store/1bhz96rq2faiy8r7yk29g2ydrrs4pzfy-ed-1.15.tar.lz.drv' failed to produce output path `/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz'
@ build-failed /gnu/store/1bhz96rq2faiy8r7yk29g2ydrrs4pzfy-ed-1.15.tar.lz.drv - 1 builder for `/gnu/store/1bhz96rq2faiy8r7yk29g2ydrrs4pzfy-ed-1.15.tar.lz.drv' failed to produce output path `/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz'
cannot build derivation `/gnu/store/kdm7cyhlqqv3cri4qlymjrjfvd2imm24-ed-1.15.drv': 1 dependencies couldn't be built
@ build-started /gnu/store/d3ixmflajlvg20xijm1y782srghbj4bi-file-5.33.tar.gz.drv - x86_64-linux /var/log/guix/drvs/d3//ixmflajlvg20xijm1y782srghbj4bi-file-5.33.tar.gz.drv.gz 2234588
cannot build derivation `/gnu/store/i8nhiybq6a9w6fiz1vhnxc8r00mxyn3q-patch-2.7.6.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscript-9.27.drv': 1 dependencies couldn't be built
Backtrace:
10 (primitive-load "/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation")
In ice-9/eval.scm:
155:9 9 (_ _)
159:9 8 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#<directory (guile-user) 7f6be7e71c?> ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
In ./guix/store.scm:
1975:24 7 (run-with-store #<store-connection 256.99 7f6be5a69500> #<procedure 7f6bdd3c9d40 at ./guix/self.scm:11?> ?)
1811:8 6 (_ #<store-connection 256.99 7f6be5a69500>)
In ./guix/gexp.scm:
959:2 5 (_ #<store-connection 256.99 7f6be5a69500>)
819:2 4 (_ #<store-connection 256.99 7f6be5a69500>)
In ./guix/store.scm:
1859:12 3 (_ #<store-connection 256.99 7f6be5a69500>)
1312:5 2 (map/accumulate-builds #<store-connection 256.99 7f6be5a69500> #<procedure 7f6bddcfa520 at ./guix/stor?> ?)
1323:15 1 (_ #<store-connection 256.99 7f6be5a69500> ("/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscrip?" ?) ?)
1323:15 0 (loop #f)
./guix/store.scm:1323:15: In procedure loop:
ERROR:
1. &store-protocol-error:
message: "build of `/gnu/store/36fgj9n3c8bmix2pd12kfaszi7bd5y7a-ghostscript-9.27.drv' failed"
status: 1
guix time-machine: error: You found a bug: the program '/gnu/store/lim79w7d43hr4rxnj6qkiijfbh30fkgv-compute-guix-derivation'
failed to compute the derivation for Guix (version: "d62c9b2671be55ae0305bebfda17b595f33797f2"; system: "x86_64-linux";
host version: "b8f6ead5faac3c1b9a8fa6e060c00cf0917e884e"; pull-version: 1).
Please report it by email to <bug-guix@gnu.org>.
--8<---------------cut here---------------end--------------->8---
Still failing and that’s not expected. ;-)
Let add ’ed’ manually by restoring the network and running,
guix build /gnu/store/1bhz96rq2faiy8r7yk29g2ydrrs4pzfy-ed-1.15.tar.lz.drv
then disallow again the network and run again “guix time-machine”. And
repeat for others.
/gnu/store/vxzq9glg4pck4va6aga791g04z6rckx3-gcc-core-2.95.3.tar.gz.drv
/gnu/store/g8bln8bi8l70ikkwpd4gpiqas2gp8x9n-ghostscript-9.27.tar.xz.drv
/gnu/store/pf9acfsb3zkmvfqcny4b8hrjif9a1kb8-guile-2.2.6.tar.xz.drv
/gnu/store/qwbmqzyqv8nl39pkmzyp268lcnjrhrvs-linux-libre-4.19.56-gnu.tar.xz.drv
/gnu/store/bdiwmsi227a9di29hvmw9x6r006jm0kk-linux-libre-headers-stripped-4.14.67-i686-linux.tar.xz.drv
/gnu/store/w4cprnzzz8siim5j1mqk0w6myrpigngb-mes-0.19.tar.gz.drv
/gnu/store/h54ichi7phb3ph07sxbclwdc56a8z7y1-mes-minimal-stripped-0.19-i686-linux.tar.xz.drv
Now, I add ftp.gnu.org to the list of allowed machines because I am
bored. Again, it fails for:
/gnu/store/85p96nq7i86cav03m5ahxlcldzd8wq8c-nyacc-0.86.0.tar.gz.drv
/gnu/store/sjf06i58f3bbhfggw6sj3n742zkn9632-static-binaries.tar.xz.drv
Manual addition. Then it starts to build. \o/
But then it fails with:
--8<---------------cut here---------------start------------->8---
command "sh" "check.sh" failed with status 1
builder for `/gnu/store/ssj7xx3q05l4b84iyzrc90rdw79c90b3-tcc-boot0-0.9.26-6.c004e9a.drv' failed with exit code 1
--8<---------------cut here---------------end--------------->8---
well, keep it for another investigation…
^ permalink raw reply [flat|nested] 4+ messages in thread
* v1.1.0 (~2020) and SWH: Part II
2023-02-28 19:51 Guix v1.1.0 (~2020) and Software Heritage & friends Simon Tournier
@ 2023-03-01 15:12 ` Simon Tournier
2023-03-06 20:40 ` Guix v1.1.0 (~2020) and Software Heritage & friends Ludovic Courtès
1 sibling, 0 replies; 4+ messages in thread
From: Simon Tournier @ 2023-03-01 15:12 UTC (permalink / raw)
To: Guix Devel
Hi,
Let continue the investigations. Now, assume that:
guix time-machine -C /tmp/v1.1.0cm -- help
passes, where /tmp/v1.1.0.scm is defined elsewhere in this thread.
We spot these issues:
+ Holes in the coverage
guile-json-3.2.0.tar.gz.drv
net-tools-1.60-0.479bb4a.zip.drv
hidapi-0.9.0-checkout.drv
+ Failures to re-build
python-minimal-3.7.4.drv
gnutls-3.6.A.drv
+ Hash mismatch (hum?!)
+ Probably from libgit2 not dealing correctly with crlf and .gitattribute,
Need more investigations…
+ Missing features in (guix swh)
+ lightweight tags
+ zip archive (Disarchive side :-))
+ SWH request rate limits
something to discuss for “Reproducible Science” at scale. :-)
To be continued…
Cheers,
simon
- - -
(Again, it appears to me weird to have to specify --no-substitutes where
the substitutes servers are unreachable. Well, maybe it had been
corrected since this old version v1.1.0. :-))
# Preliminary
Using only,
archive.softwareheritage.org
disarchive.guix.gnu.org
then
guix time-machine -C /tmp/v1.1.0.scm -- build hello --no-substitutes
fails because many missing dependencies. I will not list all since, let
say, it is part of the issues pointed by Part I. As said previously,
the addition of git.savannah.gnu.org helps which confirms that we have
holes with our coverage of SWH + Disarchive. Then, we hit ed and all so
let add ftp.gnu.org for building hello. Missing,
/gnu/store/qwbmqzyqv8nl39pkmzyp268lcnjrhrvs-linux-libre-4.19.56-gnu.tar.xz.drv
/gnu/store/85p96nq7i86cav03m5ahxlcldzd8wq8c-nyacc-0.86.0.tar.gz.drv
/gnu/store/sjf06i58f3bbhfggw6sj3n742zkn9632-static-binaries.tar.xz.drv
And note that
https://ftp.gnu.org/gnu/guix/bootstrap/x86_64-linux/20131110/static-binaries.tar.xz
is gone.
Well, let enable the network and run:
guix time-machine -C /tmp/v1.1.0.scm -- build hello
guix time-machine -C /tmp/v1.1.0.scm -- build hello --check --no-grafts
That’s said, let do some tests about the coverage. Without specific
mention, we consider only this:
archive.softwareheritage.org
disarchive.guix.gnu.org
git.savannah.gnu.org
ftp.gnu.org
# Github is down
Let build open-zwave which reads,
--8<---------------cut here---------------start------------->8---
(uri (git-reference
(url "https://github.com/OpenZWave/open-zwave/")
(commit (string-append "v" version))))
--8<---------------cut here---------------end--------------->8---
using,
guix time-machine -C /tmp/v1.1.0.scm -- build open-zwave --no-substitutes
and it fails with,
/gnu/store/mvmi5fsv42zv6dn4g5w7r3mw8mqrgzik-guile-json-3.2.0.tar.gz.drv
/gnu/store/9ry22bncalynd5is3icrlaib4570gya0-net-tools-1.60-0.479bb4a.zip.drv
/gnu/store/2s25w9k5bi3hkr5sgmvn1d4hl886bchp-hidapi-0.9.0-checkout.drv
/gnu/store/984hkbdjp9dskn9yxwha61q9i6nbl11v-python-minimal-3.7.4.drv <-- annoying failures!
/gnu/store/sshwk1xqaf1c1ms3cm93brzaizk15c7l-python-3.7.4.drv <--
/gnu/store/qm3l79ic89qpjjd8avqxd81425v4wvv5-gnutls-3.6.A.drv <--
And we hit a bug!
--8<---------------cut here---------------start------------->8---
fatal: unable to access 'https://github.com/libusb/hidapi.git/': Could not resolve host: github.com
Failed to do a shallow fetch; retrying a full fetch...
fatal: unable to access 'https://github.com/libusb/hidapi.git/': Could not resolve host: github.com
git-fetch: '/gnu/store/xzhkhidb2kqwvpv8b8zaqzgd3gcjs9nn-git-minimal-2.26.0/bin/git fetch origin' failed with exit code 128
Trying content-addressed mirror at berlin.guixsd.org...
Trying content-addressed mirror at berlin.guixsd.org...
Trying to download from Software Heritage...
SWH: found revision 7da5cc91fc0d2dbe4df4f08cd31f6ca1a262418f with directory at 'https://archive.softwareheritage.org/api/1/directory/c95078f3645abc050536d3aaf76cb5fb6eb6b288/'
SWH vault: requested bundle cooking, waiting for completion...
swh:1:dir:c95078f3645abc050536d3aaf76cb5fb6eb6b288/
[...]
swh:1:dir:c95078f3645abc050536d3aaf76cb5fb6eb6b288/windows/hidtest.vcproj
r:sha256 hash mismatch for /gnu/store/n6rvypb3z386v0ml6n0xyga2x4q9ij58-hidapi-0.9.0-checkout:
expected hash: 1p4g8lgwj4rki6lbn5l6rvwj0xlbn1xfh4d255bg5pvgczmwmc4i
actual hash: 1cbbg4rkfpjwkgqva2659zasqr5d5rz6bi71malnnzy0y2qin2pz
hash mismatch for store item '/gnu/store/n6rvypb3z386v0ml6n0xyga2x4q9ij58-hidapi-0.9.0-checkout'
--8<---------------cut here---------------end--------------->8---
Quoting, vlorentz from SWH:
<vlorentz> zimoun: so in short, the discrepency is probably because your git
doesn't honor the crlf setting in .gitattribute but used to, or
vice versa
So probably libgit2 incorrectly dealing with .gitattribute from hidapi
repository.
Last, another more annoying one:
--8<---------------cut here---------------start------------->8---
Initialized empty Git repository in /gnu/store/zlmgdc8c9mgb9l17fchkarsn7hf3s39r-open-zwave-1.6-checkout/.git/
fatal: unable to access 'https://github.com/OpenZWave/open-zwave/': Could not resolve host: github.com
Failed to do a shallow fetch; retrying a full fetch...
fatal: unable to access 'https://github.com/OpenZWave/open-zwave/': Could not resolve host: github.com
git-fetch: '/gnu/store/xzhkhidb2kqwvpv8b8zaqzgd3gcjs9nn-git-minimal-2.26.0/bin/git fetch origin' failed with exit code 128
Trying content-addressed mirror at berlin.guixsd.org...
Trying content-addressed mirror at berlin.guixsd.org...
Trying to download from Software Heritage...
builder for `/gnu/store/bcdsz16z454g756fsnrqrs92r7z1dqwa-open-zwave-1.6-checkout.drv' failed to produce output path `/gnu/store/zlmgdc8c9mgb9l17fchkarsn7hf3s39r-open-zwave-1.6-checkout'
build of /gnu/store/bcdsz16z454g756fsnrqrs92r7z1dqwa-open-zwave-1.6-checkout.drv failed
--8<---------------cut here---------------end--------------->8---
Well, this repository is saved in SWH and the release tag v1.6 is
ingested, as shown here:
https://archive.softwareheritage.org/browse/snapshot/93d83c96be2bbb2ea9a172c3fdacb80413bc276c/branches/?origin_url=https://github.com/OpenZWave/open-zwave
but, quoting olasd from SWH:
<olasd> zimoun: it is; but it's a lightweight tag so it only shows in the
"branches" list
It means we have some missing feature on our side.
Ok! Now, let check for all the packages with github.com as upstream,
using this manifest file:
--8<---------------cut here---------------start------------->8---
(use-modules (guix) (gnu)
(guix git-download)
(ice-9 match)
(srfi srfi-1)
(srfi srfi-26))
(define (that-url? package)
(define (that-string? str)
(string-contains str "github.com"))
(match (package-source package)
((? origin? o)
(match (origin-uri o)
((? string? url)
(that-string? url))
(((? string? urls) ...)
(any that-string? urls))
((? git-reference? ref)
(that-string? (git-reference-url ref)))
(_ #f)))
(_ #f)))
(manifest
(map package->manifest-entry
(fold-packages (lambda (package result)
(if (that-url? package)
(cons package result)
result))
'())))
--8<---------------cut here---------------end--------------->8---
and,
guix time-machine -C /tmp/v1.1.0.scm \
-- build -m /tmp/from-that-url.scm -S --no-substitutes -k
fails at some points. One is because we hit the SWH rate limit. Other
are, among many others,
/gnu/store/867xgw7syxkm32ryjcnrz7cxc1gklz4a-android-googletest-1.8.0-checkout.drv
/gnu/store/hrzhpb2xp682ikcgrnrl5y2651hr9wn6-android-make-stub-0.6.0-checkout.drv
/gnu/store/43mkyklyn9sqz3wnhlrp313xcbm57j0p-apache-arrow-0.10.0-checkout.drv
/gnu/store/5ybwii43g0nclcx527y2ws8s6ai81rsj-apfs-fuse-0.0.0-0.c7036a3-checkout.drv
/gnu/store/66xzcip9rbsig3y5dbyz5xyyaqzszpyp-arc-icon-theme-20161122-checkout.drv
/gnu/store/gyvhlabn6xn6zav76cpx7xg82kfisniq-arducopter-bbbmini-3.6.11-checkout.drv
/gnu/store/y8rajri7kykv1yh8ykk4a3ighswjiadj-arducopter-bebop-3.6.11-checkout.drv
/gnu/store/wg12wwb2yb6nzmg1ani6jm2dnmi7r8n5-arduplane-bbbmini-4.0.1-checkout.drv
[...]
/gnu/store/64zx0dviwm3jbkqaka3nrri1azgw9bsh-allegro-4.4.3.tar.gz.drv
/gnu/store/5l1g6n291s2m7fn4hypk5q66vh9nw51v-alot-0.5.1.tar.gz.drv
/gnu/store/x9mr8dkn4rirr8z3sqq9nfkh0gx5mdp0-apertium-3.5.2.tar.gz.drv
[...]
/gnu/store/p2izqfpw3k1jvq20m43f3bn5mpj8fxh9-apsw-3.28.0-r1.zip.drv
To be continued… for instance gitlab.com, bioconductor.org, etc.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Guix v1.1.0 (~2020) and Software Heritage & friends
2023-02-28 19:51 Guix v1.1.0 (~2020) and Software Heritage & friends Simon Tournier
2023-03-01 15:12 ` v1.1.0 (~2020) and SWH: Part II Simon Tournier
@ 2023-03-06 20:40 ` Ludovic Courtès
2023-03-07 10:27 ` Simon Tournier
1 sibling, 1 reply; 4+ messages in thread
From: Ludovic Courtès @ 2023-03-06 20:40 UTC (permalink / raw)
To: Simon Tournier; +Cc: Guix Devel, Mathieu Othacehe
Hello!
Simon Tournier <zimon.toutoune@gmail.com> skribis:
> Here I would like to report my attempt to simulate a “network collapse”
> – as discussed with Mathieu at Guix Days. :-)
Great experiment!
> Using this,
>
> archive.softwareheritage.org
> disarchive.guix.gnu.org
> git.savannah.gnu.org
>
> is not enough. It fails for many. For instance,
>
> ed-1.15
> gcc-core-2.95.3
> ghostscript-9.27
> guile-2.2.6
> linux-libre-4.19.56-gnu
> linux-libre-headers-stripped-4.14.67-i686-linux
> mes-0.19
> mes-minimal-stripped-0.19-i686-linux
>
> do not seem backed up with SWH + Disarchive.
It’d be great to identify the root cause for each of them, with a script
to do that. But note that 1.1.0 did *not* support Disarchive; it can
only look up SWH or one of the content-addressed mirrors.
That said, it’s true that disarchive.guix lacks coverage for some of
these. For ed, that’s because it’s lzip:
--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=v1.1.0 -- build -S ed
guile: warning: failed to install locale
0.1 MB will be downloaded:
/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz
substituting /gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz...
downloading from https://ci.guix.gnu.org/nar/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz...
ed-1.15.tar.lz 66KiB 1.3MiB/s 00:00 [##################] 100.0%
/gnu/store/0xgcp3mgq3nql22ak97dacvs3k18c4zg-ed-1.15.tar.lz
$ wget https://disarchive.guix.gnu.org/sha256/$(guix hash -f hex $(guix time-machine --commit=v1.1.0 -- build -S ed))
guile: warning: failed to install locale
--2023-03-06 21:27:53-- https://disarchive.guix.gnu.org/sha256/ad4489c0ad7a108c514262da28e6c2a426946fb408a3977ef1ed34308bdfd174
Resolving disarchive.guix.gnu.org (disarchive.guix.gnu.org)... 141.80.181.40
Connecting to disarchive.guix.gnu.org (disarchive.guix.gnu.org)|141.80.181.40|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-06 21:27:53 ERROR 404: Not Found.
--8<---------------cut here---------------end--------------->8---
For Guile it’s probably because it’s xz and predates xz support in
Disarchive:
--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=v1.1.0 -- build -Sd guile@2.2.6
guile: warning: failed to install locale
/gnu/store/19mwlfc1092vipdka1y0l6v92ahkhl52-guile-2.2.6.tar.xz.drv
$ guix gc --references /gnu/store/19mwlfc1092vipdka1y0l6v92ahkhl52-guile-2.2.6.tar.xz.drv |grep tar.xz.drv
/gnu/store/pf9acfsb3zkmvfqcny4b8hrjif9a1kb8-guile-2.2.6.tar.xz.drv
$ wget -O- https://disarchive.guix.gnu.org/sha256/$(guix hash -f hex $(guix build /gnu/store/pf9acfsb3zkmvfqcny4b8hrjif9a1kb8-guile-2.2.6.tar.xz.drv))
10.9 MB will be downloaded:
/gnu/store/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz
substituting /gnu/store/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz...
downloading from https://ci.guix.gnu.org/nar/zfp7d4wr5hbl5lrnzs8c15bsc3ygq74g-guile-2.2.6.tar.xz ...
guile-2.2.6.tar.xz 10.4MiB 24.3MiB/s 00:00 ▕██████████████████▏ 100.0%
--2023-03-06 21:30:41-- https://disarchive.guix.gnu.org/sha256/b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988
Resolving disarchive.guix.gnu.org (disarchive.guix.gnu.org)... 141.80.181.40
Connecting to disarchive.guix.gnu.org (disarchive.guix.gnu.org)|141.80.181.40|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-06 21:30:41 ERROR 404: Not Found.
$ wget -qO- https://disarchive.ngyro.com/sha256/$(guix hash -f hex $(guix build /gnu/store/pf9acfsb3zkmvfqcny4b8hrjif9a1kb8-guile-2.2.6.tar.xz.drv)) |head -4
(disarchive
(version 0)
(xz-file
(name "guile-2.2.6.tar.xz")
--8<---------------cut here---------------end--------------->8---
> Well, basically after these, I stop because it’s boring and I add
> ftp.gnu.org to the list. It still fails,
>
> nyacc-0.86.0
> static-binaries
>
> First lesson: we have holes in sources.json and maybe Disarchive too; I
> will give a look.
Some are known issues: lack of lzip support in Disarchive, lack of
SVN/SWH integration, missing entries in the Disarchive database for old
packages, etc.
One thing that’s easily done is retroactively populating the Disarchive
database. It’s a job that Timothy did on their side:
<https://disarchive.ngyro.com> has more “old” entries, which we should
copy at disarchive.guix.
We should also reuse the tools that Timothy wrote:
<https://git.ngyro.com/>. In particular, we should use project
infrastructure to compute a new Preservation of Guix report, and ideally
automate the whole process.
Maybe we should write down a road map and “recruit” on these items?
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Guix v1.1.0 (~2020) and Software Heritage & friends
2023-03-06 20:40 ` Guix v1.1.0 (~2020) and Software Heritage & friends Ludovic Courtès
@ 2023-03-07 10:27 ` Simon Tournier
0 siblings, 0 replies; 4+ messages in thread
From: Simon Tournier @ 2023-03-07 10:27 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: Guix Devel, Mathieu Othacehe
Hi Ludo,
Thanks for giving a look and for your insights.
On Mon, 6 Mar 2023 at 21:40, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
> Maybe we should write down a road map and “recruit” on these items?
Yeah, I totally agree. :-) For what it is worth, in that mood, for instance,
+ GSoC <https://libreplanet.org/wiki/Group:Guix/GSoC-2023>
+ Patch #62008 <https://issues.guix.gnu.org/62008>
+ SWH #4538< https://gitlab.softwareheritage.org/swh/meta/-/issues/4538>
Well, I will try to summarize the current status of Disarchive.
Cheers,
simon
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-03-07 10:28 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-28 19:51 Guix v1.1.0 (~2020) and Software Heritage & friends Simon Tournier
2023-03-01 15:12 ` v1.1.0 (~2020) and SWH: Part II Simon Tournier
2023-03-06 20:40 ` Guix v1.1.0 (~2020) and Software Heritage & friends Ludovic Courtès
2023-03-07 10:27 ` Simon Tournier
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).