* bug#69058: Failure when fallback to Disarchive+SWH
@ 2024-02-11 17:21 Simon Tournier
2024-02-11 18:11 ` Simon Tournier
0 siblings, 1 reply; 7+ messages in thread
From: Simon Tournier @ 2024-02-11 17:21 UTC (permalink / raw)
To: 69058
Hi,
Here an example where something is wrong:
+ Diarsarchive specificiation is ok.
<https://disarchive.guix.gnu.org/sha256/787014b29f0c5793ecc2d93e1109a049ff48ab0c29b851dab34f683ceef6b152>
+ SWH has it
<https://archive.softwareheritage.org/browse/directory/35e81cfbd7ca261e2ddebf372f38a4d1baa2f931>
But something is failing on Guix side.
--8<---------------cut here---------------start------------->8---
$ guix build /gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv
/gnu/store/58m53a8ygffj4l5kpqhyxby1ylhri9dr-xscreensaver-6.04.tar.gz
$ guix build /gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv --check
The following derivation will be built:
/gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv
building /gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv...
Starting download of /gnu/store/123bm8wnld70qnajw5crdw7a7lhbpa0x-xscreensaver-6.04.tar.gz
[...]
Trying to use Disarchive to assemble /gnu/store/123bm8wnld70qnajw5crdw7a7lhbpa0x-xscreensaver-6.04.tar.gz...
Retrieving Disarchive spec from https://disarchive.guix.gnu.org/sha256/787014b29f0c5793ecc2d93e1109a049ff48ab0c29b851dab34f683ceef6b152 ...
Assembling the directory xscreensaver-6.04
Downloading /gnu/store/123bm8wnld70qnajw5crdw7a7lhbpa0x-xscreensaver-6.04.tar.gz from Software Heritage...
SWH: object swh:1:dir:35e81cfbd7ca261e2ddebf372f38a4d1baa2f931 could not be fetched from the vault
Could not resolve directory reference
failed to download "/gnu/store/123bm8wnld70qnajw5crdw7a7lhbpa0x-xscreensaver-6.04.tar.gz" from "https://www.jwz.org/xscreensaver/xscreensaver-6.04.tar.gz"
builder for `/gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv' failed to produce output path `/gnu/store/58m53a8ygffj4l5kpqhyxby1ylhri9dr-xscreensaver-6.04.tar.gz'
build of /gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv failed
View build log at '/var/log/guix/drvs/mx/sfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv.gz'.
guix build: error: build of `/gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv' failed
--8<---------------cut here---------------end--------------->8---
Let’s do it manually:
1. fetch content and specification
2. assemble
3. compare the assembled with the one from content-addressed-mirrors
1.
--8<---------------cut here---------------start------------->8---
$ wget -O /tmp/spec https://disarchive.guix.gnu.org/sha256/787014b29f0c5793ecc2d93e1109a049ff48ab0c29b851dab34f683ceef6b152
/tmp/spec [ <=> ] 3,25M 512KB/s in 6,6s
$ wget -O /tmp/content https://archive.softwareheritage.org/api/1/vault/flat/swh:1:dir:35e81cfbd7ca261e2ddebf372f38a4d1baa2f931/raw
/tmp/content 100%[===================>] 24,86M 530KB/s in 49s
--8<---------------cut here---------------end--------------->8---
2.
--8<---------------cut here---------------start------------->8---
$ tar xf content
$ disarchive assemble swh:1:dir:35e81cfbd7ca261e2ddebf372f38a4d1baa2f931 spec > xscreensaver-6.04.tar.gz
Assembling the directory xscreensaver-6.04
Checking xscreensaver-6.04 digest... ok
Assembling the tarball xscreensaver-6.04.tar
Checking xscreensaver-6.04.tar digest... ok
Assembling the Gzip file xscreensaver-6.04.tar.gz
Checking xscreensaver-6.04.tar.gz digest... ok
Writing result to output port
--8<---------------cut here---------------end--------------->8---
3.
--8<---------------cut here---------------start------------->8---
$ disarchive assemble swh:1:dir:35e81cfbd7ca261e2ddebf372f38a4d1baa2f931 spec 2>/dev/null | guix hash -
0lmiyvp3qs2gngd53f191jmlizs9l04i2gnrqbn96mqckyr18w3q
$ guix hash $(guix build /gnu/store/mxsfblzy65v5y4gwrxqkihj7g4w5f5b1-xscreensaver-6.04.tar.gz.drv)
0lmiyvp3qs2gngd53f191jmlizs9l04i2gnrqbn96mqckyr18w3q
--8<---------------cut here---------------end--------------->8---
Cheers,
simon
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#69058: Failure when fallback to Disarchive+SWH
2024-02-11 17:21 bug#69058: Failure when fallback to Disarchive+SWH Simon Tournier
@ 2024-02-11 18:11 ` Simon Tournier
2024-02-12 10:28 ` Ludovic Courtès
0 siblings, 1 reply; 7+ messages in thread
From: Simon Tournier @ 2024-02-11 18:11 UTC (permalink / raw)
To: 69058
Hi,
Another example using Git. It is from v1.1.0.
--8<---------------cut here---------------start------------->8---
$ guix time-machine -q --commit=v1.1.0 -- build -S redkite
guile: warning: failed to install locale
The following derivation will be built:
/gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv
building /gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv...
guile: warning: failed to install locale
environment variable `PATH' set to `/gnu/store/i2cdl0hvrml8hjdqplqln8didnvxkgp5-gzip-1.10/bin:/gnu/store/jh17p4sns7dvbizwz58gdh953qpic144-tar-1.32/bin'
Initialized empty Git repository in /gnu/store/y7j54zw1fwhl95534zsjl6rl1rfp7z82-redkite-0.8.0-checkout/.git/
fatal: could not read Username for 'https://gitlab.com': No such device or address
Failed to do a shallow fetch; retrying a full fetch...
fatal: could not read Username for 'https://gitlab.com': No such device or address
git-fetch: '/gnu/store/xzhkhidb2kqwvpv8b8zaqzgd3gcjs9nn-git-minimal-2.26.0/bin/git fetch origin' failed with exit code 128
Trying content-addressed mirror at berlin.guixsd.org...
Trying content-addressed mirror at berlin.guixsd.org...
Trying to download from Software Heritage...
SWH: found revision 68276b9b88491198c33b03c4188dfe7d6e5dbf2a with directory at 'https://archive.softwareheritage.org/api/1/directory/d14c0eb6801bca920d1e1baac7a4b9e2a0589728/'
SWH: directory d14c0eb6801bca920d1e1baac7a4b9e2a0589728 could not be fetched from the vault
builder for `/gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv' failed to produce output path `/gnu/store/y7j54zw1fwhl95534zsjl6rl1rfp7z82-redkite-0.8.0-checkout'
build of /gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv failed
View build log at '/var/log/guix/drvs/x4/hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv.gz'.
guix build: error: build of `/gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv' failed
--8<---------------cut here---------------end--------------->8---
From its definition the package reads:
--8<---------------cut here---------------start------------->8---
(define-public redkite
[...]
(sha256
(base32
"1747w1kg8y9jbl11xi018d85dm38xk7843pz26sh0k5fdv87a10q"))))
--8<---------------cut here---------------end--------------->8---
Then if I manually download it:
--8<---------------cut here---------------start------------->8---
$ wget -O /tmp/content https://archive.softwareheritage.org/api/1/vault/flat/swh:1:dir:d14c0eb6801bca920d1e1baac7a4b9e2a0589728/raw
[...]
/tmp/content 100%[===================>] 129,38K 555KB/s in 0,2s
2024-02-11 19:00:33 (555 KB/s) - ‘/tmp/content’ saved [132488/132488]
$ tar xf content
--8<---------------cut here---------------end--------------->8---
And we get the expected value:
--8<---------------cut here---------------start------------->8---
$ guix hash -rx swh:1:dir:d14c0eb6801bca920d1e1baac7a4b9e2a0589728
1747w1kg8y9jbl11xi018d85dm38xk7843pz26sh0k5fdv87a10q
--8<---------------cut here---------------end--------------->8---
Cheers,
simon
^ permalink raw reply [flat|nested] 7+ messages in thread* bug#69058: Failure when fallback to Disarchive+SWH
2024-02-11 18:11 ` Simon Tournier
@ 2024-02-12 10:28 ` Ludovic Courtès
2024-02-12 11:24 ` Ludovic Courtès
0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2024-02-12 10:28 UTC (permalink / raw)
To: Simon Tournier; +Cc: 69058
Hi,
Simon Tournier <zimon.toutoune@gmail.com> skribis:
> Another example using Git. It is from v1.1.0.
>
> $ guix time-machine -q --commit=v1.1.0 -- build -S redkite
> guile: warning: failed to install locale
> The following derivation will be built:
> /gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv
> building /gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv...
> guile: warning: failed to install locale
> environment variable `PATH' set to `/gnu/store/i2cdl0hvrml8hjdqplqln8didnvxkgp5-gzip-1.10/bin:/gnu/store/jh17p4sns7dvbizwz58gdh953qpic144-tar-1.32/bin'
> Initialized empty Git repository in /gnu/store/y7j54zw1fwhl95534zsjl6rl1rfp7z82-redkite-0.8.0-checkout/.git/
> fatal: could not read Username for 'https://gitlab.com': No such device or address
> Failed to do a shallow fetch; retrying a full fetch...
> fatal: could not read Username for 'https://gitlab.com': No such device or address
> git-fetch: '/gnu/store/xzhkhidb2kqwvpv8b8zaqzgd3gcjs9nn-git-minimal-2.26.0/bin/git fetch origin' failed with exit code 128
> Trying content-addressed mirror at berlin.guixsd.org...
> Trying content-addressed mirror at berlin.guixsd.org...
> Trying to download from Software Heritage...
> SWH: found revision 68276b9b88491198c33b03c4188dfe7d6e5dbf2a with directory at 'https://archive.softwareheritage.org/api/1/directory/d14c0eb6801bca920d1e1baac7a4b9e2a0589728/'
> SWH: directory d14c0eb6801bca920d1e1baac7a4b9e2a0589728 could not be fetched from the vault
> builder for `/gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv' failed to produce output path `/gnu/store/y7j54zw1fwhl95534zsjl6rl1rfp7z82-redkite-0.8.0-checkout'
> build of /gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv failed
> View build log at '/var/log/guix/drvs/x4/hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv.gz'.
> guix build: error: build of `/gnu/store/x4hcwpmb3p8llbhjzxlfggpkpqc7n5ng-redkite-0.8.0-checkout.drv' failed
I can reproduce it with c4372f7ebfac17236299346fe60c6ca125336b1f:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (swh-download "https://gitlab.com/geontime/redkite.git" "v0.8.0" "/tmp/redkite")
SWH: found revision 68276b9b88491198c33b03c4188dfe7d6e5dbf2a with directory at 'https://archive.softwareheritage.org/api/1/directory/d14c0eb6801bca920d1e1baac7a4b9e2a0589728/'
SWH: object swh:1:dir:d14c0eb6801bca920d1e1baac7a4b9e2a0589728 could not be fetched from the vault
$5 = #f
--8<---------------cut here---------------end--------------->8---
The bug is that the Vault now returns 302 (redirect) in this case, which
is similar to the bug I fixed in <https://issues.guix.gnu.org/68741>.
So I’ll have to push that patch series and add a patch for ‘vault-fetch’
on top of it.
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#69058: Failure when fallback to Disarchive+SWH
2024-02-12 10:28 ` Ludovic Courtès
@ 2024-02-12 11:24 ` Ludovic Courtès
2024-02-12 12:59 ` Simon Tournier
0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2024-02-12 11:24 UTC (permalink / raw)
To: Simon Tournier; +Cc: 69058-done
Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
> I can reproduce it with c4372f7ebfac17236299346fe60c6ca125336b1f:
>
> scheme@(guile-user)> (swh-download "https://gitlab.com/geontime/redkite.git" "v0.8.0" "/tmp/redkite")
> SWH: found revision 68276b9b88491198c33b03c4188dfe7d6e5dbf2a with directory at 'https://archive.softwareheritage.org/api/1/directory/d14c0eb6801bca920d1e1baac7a4b9e2a0589728/'
> SWH: object swh:1:dir:d14c0eb6801bca920d1e1baac7a4b9e2a0589728 could not be fetched from the vault
> $5 = #f
>
> The bug is that the Vault now returns 302 (redirect) in this case, which
> is similar to the bug I fixed in <https://issues.guix.gnu.org/68741>.
In fact it’s the exact same problem.
Fixed in 1610a632d4b3097282d18af27ff3e9e178d7dfcb.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#69058: Failure when fallback to Disarchive+SWH
2024-02-12 11:24 ` Ludovic Courtès
@ 2024-02-12 12:59 ` Simon Tournier
2024-02-12 13:15 ` Ludovic Courtès
0 siblings, 1 reply; 7+ messages in thread
From: Simon Tournier @ 2024-02-12 12:59 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 69058-done
Hi Ludo,
On lun., 12 févr. 2024 at 12:24, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> I can reproduce it with c4372f7ebfac17236299346fe60c6ca125336b1f:
>>
>> scheme@(guile-user)> (swh-download "https://gitlab.com/geontime/redkite.git" "v0.8.0" "/tmp/redkite")
>> SWH: found revision 68276b9b88491198c33b03c4188dfe7d6e5dbf2a with directory at 'https://archive.softwareheritage.org/api/1/directory/d14c0eb6801bca920d1e1baac7a4b9e2a0589728/'
>> SWH: object swh:1:dir:d14c0eb6801bca920d1e1baac7a4b9e2a0589728 could not be fetched from the vault
>> $5 = #f
>>
>> The bug is that the Vault now returns 302 (redirect) in this case, which
>> is similar to the bug I fixed in <https://issues.guix.gnu.org/68741>.
>
> In fact it’s the exact same problem.
>
> Fixed in 1610a632d4b3097282d18af27ff3e9e178d7dfcb.
Cool! Thanks.
However, it means that it stays broken for all commits before this fix,
right? Somehow, if only SWH is available, then
guix time-machine -C channels.scm -- shell -m manifest.scm
will fail for all channels older than today. Right?
For instance, redoing your ReScience paper from 2020 using only SWH as I
did [1] some months ago is now broken (because a change on SWH side).
Well, I am fine to keep this bug closed but for the posterity we need
something like:
guix swh get -C channels.scm -m manifest.scm
as we discussed elsewhere (off-list IIRC :-)).
1: https://simon.tournier.info/posts/2023-06-23-hackathon-repro.html
Cheers,
simon
^ permalink raw reply [flat|nested] 7+ messages in thread* bug#69058: Failure when fallback to Disarchive+SWH
2024-02-12 12:59 ` Simon Tournier
@ 2024-02-12 13:15 ` Ludovic Courtès
2024-02-12 14:04 ` Simon Tournier
0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2024-02-12 13:15 UTC (permalink / raw)
To: Simon Tournier; +Cc: 69058-done
Hi,
Simon Tournier <zimon.toutoune@gmail.com> skribis:
>> Fixed in 1610a632d4b3097282d18af27ff3e9e178d7dfcb.
>
> Cool! Thanks.
>
> However, it means that it stays broken for all commits before this fix,
> right? Somehow, if only SWH is available, then
>
> guix time-machine -C channels.scm -- shell -m manifest.scm
>
> will fail for all channels older than today. Right?
Mostly, yes (“mostly” because derivations using “builtin:git-download”
will automatically be fixed once we’ve updated the ‘guix’ package, but
that only helps if you’re traveling to after September 2023 (commit
13b0cf85eb31e1b1ea674debbbfb0f35a32d1374).
> For instance, redoing your ReScience paper from 2020 using only SWH as I
> did [1] some months ago is now broken (because a change on SWH side).
Indeed. :-/
That’s a real bummer, but that’s the nature of software: we cannot
retroactively fix bugs.
> Well, I am fine to keep this bug closed but for the posterity we need
> something like:
>
> guix swh get -C channels.scm -m manifest.scm
Exactly! We have the demonstration of just how important that is.
Ludo’.
^ permalink raw reply [flat|nested] 7+ messages in thread
* bug#69058: Failure when fallback to Disarchive+SWH
2024-02-12 13:15 ` Ludovic Courtès
@ 2024-02-12 14:04 ` Simon Tournier
0 siblings, 0 replies; 7+ messages in thread
From: Simon Tournier @ 2024-02-12 14:04 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 69058-done
> That’s a real bummer, but that’s the nature of software: we cannot
> retroactively fix bugs.
The nature of service and not software appears to me more correct. :-)
(Here it is not the retroactive fix of a bug but the introduction of a
non-backward compatible change.)
Cheers,
simon
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2024-02-12 14:05 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-11 17:21 bug#69058: Failure when fallback to Disarchive+SWH Simon Tournier
2024-02-11 18:11 ` Simon Tournier
2024-02-12 10:28 ` Ludovic Courtès
2024-02-12 11:24 ` Ludovic Courtès
2024-02-12 12:59 ` Simon Tournier
2024-02-12 13:15 ` Ludovic Courtès
2024-02-12 14:04 ` Simon Tournier
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).