* bug#39575: guix time-machine fails when a tarball was modified in-place
@ 2020-02-12 13:40 Jan Nieuwenhuizen
2020-02-12 14:55 ` zimoun
2020-02-13 21:34 ` Ludovic Courtès
0 siblings, 2 replies; 32+ messages in thread
From: Jan Nieuwenhuizen @ 2020-02-12 13:40 UTC (permalink / raw)
To: 39575
Hi,
Trying to travel back to Sun Apr 7 22:07:14 2019 +0200 (commit
56e95d54d209c2428f970d65d9b27ae4168449ad) to re-create mcrl2-minimal by
doing
--8<---------------cut here---------------start------------->8---
guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- environment --ad-hoc mcrl2-minimal
--8<---------------cut here---------------end--------------->8---
fails with
--8<---------------cut here---------------start------------->8---
building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...
downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...
|offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'
- 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:
expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'
build of /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv failed
View build log at '/var/log/guix/drvs/cj/im33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv.bz2'.
cannot build derivation `/gnu/store/p6gfcdacjcqf2br0zwsyzx1chfvg9gxi-harfbuzz-2.4.0.drv': 1 dependencies couldn't be built
killing process 5083
--8<---------------cut here---------------end--------------->8---
The recipe for harfbuzz has a sha256 that used to be valid in April, but
hasn't been valid anymore since May, as this fix
--8<---------------cut here---------------start------------->8---
commit a8bb8fccd82a10a46f127b2235675b4f6cbaaf98
Author: Marius Bakke <mbakke@fastmail.com>
Date: Sat May 4 18:01:12 2019 +0200
gnu: harfbuzz: Update source hash.
The previous tarball was modified in-place; see
<https://github.com/harfbuzz/harfbuzz/issues/1641>.
* gnu/packages/gtk.scm (harfbuzz)[source](sha256): Update.
--8<---------------cut here---------------end--------------->8---
shows. Thoughts?
Greetings,
janneke
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-12 13:40 bug#39575: guix time-machine fails when a tarball was modified in-place Jan Nieuwenhuizen
@ 2020-02-12 14:55 ` zimoun
2020-02-13 21:34 ` Ludovic Courtès
1 sibling, 0 replies; 32+ messages in thread
From: zimoun @ 2020-02-12 14:55 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 39575
Hi,
On Wed, 12 Feb 2020 at 14:44, Jan Nieuwenhuizen <janneke@gnu.org> wrote:
> Trying to travel back to Sun Apr 7 22:07:14 2019 +0200 (commit
> 56e95d54d209c2428f970d65d9b27ae4168449ad) to re-create mcrl2-minimal by
> doing
>
> --8<---------------cut here---------------start------------->8---
> guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- environment --ad-hoc mcrl2-minimal
> --8<---------------cut here---------------end--------------->8---
Even the simple:
--8<---------------cut here---------------start------------->8---
guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help
--8<---------------cut here---------------end--------------->8---
> fails with
>
> --8<---------------cut here---------------start------------->8---
> building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...
> downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...
> |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'
> - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:
> expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
> actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
> hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'
> build of /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv failed
> View build log at '/var/log/guix/drvs/cj/im33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv.bz2'.
> cannot build derivation `/gnu/store/p6gfcdacjcqf2br0zwsyzx1chfvg9gxi-harfbuzz-2.4.0.drv': 1 dependencies couldn't be built
> killing process 5083
> --8<---------------cut here---------------end--------------->8---
same for e.g., this commit:
--8<---------------cut here---------------start------------->8---
guix time-machine --commit=ae528aaf19f3828d3d7d204b15570800e1bbf100 -- help
--8<---------------cut here---------------end--------------->8---
> The recipe for harfbuzz has a sha256 that used to be valid in April, but
> hasn't been valid anymore since May, as this fix
>
> --8<---------------cut here---------------start------------->8---
> commit a8bb8fccd82a10a46f127b2235675b4f6cbaaf98
> Author: Marius Bakke <mbakke@fastmail.com>
> Date: Sat May 4 18:01:12 2019 +0200
>
> gnu: harfbuzz: Update source hash.
>
> The previous tarball was modified in-place; see
> <https://github.com/harfbuzz/harfbuzz/issues/1641>.
>
> * gnu/packages/gtk.scm (harfbuzz)[source](sha256): Update.
> --8<---------------cut here---------------end--------------->8---
>
> shows. Thoughts?
Therefore, all the commits between the introduction of harfbuzz with
the old sha256 (commit 2da9b81837fd1e6f08a10952784d3358be982855) and
the commit updating to the new sha256 should be broken.
Roughly speaking, all the commits between April, 7th and the May, 4th;
i.e., 1100+ commits, isn't it?
Well, this ask an interesting question: how Guix can fallback when
upstream is doing wrong?
Considering this 'harbuzz' issue, is it possible to rebuild the old
tarball and push it to SoftWare Heritage? Then when a sha mismatch
happens, fallback and try to fetch it from SWH?
WDYT?
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-12 13:40 bug#39575: guix time-machine fails when a tarball was modified in-place Jan Nieuwenhuizen
2020-02-12 14:55 ` zimoun
@ 2020-02-13 21:34 ` Ludovic Courtès
2020-02-14 1:05 ` zimoun
` (3 more replies)
1 sibling, 4 replies; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-13 21:34 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 39575
Hi,
Jan Nieuwenhuizen <janneke@gnu.org> skribis:
> building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...
> downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...
> |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'
> - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:
> expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
> actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
> hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'
The problem here is really that we fall back to content-addressed
mirrors instead of using them directly:
https://issues.guix.gnu.org/issue/28659
The file itself is still available on our machines though, and you can
get it with:
guix download -o harfbuzz-2.4.0.tar.bz2 \
https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
guix download file://$PWD/harfbuzz-2.4.0.tar.bz2
After that, re-running ‘guix time-machine’ should work.
Using ci.guix.gnu.org for substitutes should have the same effect.
HTH,
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-13 21:34 ` Ludovic Courtès
@ 2020-02-14 1:05 ` zimoun
2020-02-14 10:03 ` Giovanni Biscuolo
` (2 subsequent siblings)
3 siblings, 0 replies; 32+ messages in thread
From: zimoun @ 2020-02-14 1:05 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575
Hi Ludo,
On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
> The problem here is really that we fall back to content-addressed
> mirrors instead of using them directly:
>
> https://issues.guix.gnu.org/issue/28659
Thank you for the pointer. Good to see that the problem is almost addressed.
I will try to understand the discussion and see what is the status of
the proposed patch.
> The file itself is still available on our machines though, and you can
> get it with:
It is an half cooked solution because the Guix project cannot archive
all; for example in term of store resources.
The content-addressed mirror should be SWH, IMHO.
Well, once sources.json will be up, it should be almost done for the future.
But we still need to push all the correct sources that are on
ci.guix.gnu.org; at least all the url based source of package.
Do you have suggestion for a plan?
> guix download -o harfbuzz-2.4.0.tar.bz2 \
> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
>
> guix download file://$PWD/harfbuzz-2.4.0.tar.bz2
>
> After that, re-running ‘guix time-machine’ should work.
Thank you. This should fix the harfbuzz mismatch issue. Cool! :-)
> Using ci.guix.gnu.org for substitutes should have the same effect.
Hum? I thought that I used ci.guix.gnu.org as substitutes... soI need to check.
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-13 21:34 ` Ludovic Courtès
2020-02-14 1:05 ` zimoun
@ 2020-02-14 10:03 ` Giovanni Biscuolo
2020-02-14 10:56 ` zimoun
2020-02-14 10:47 ` zimoun
2020-02-14 13:45 ` Jan Nieuwenhuizen
3 siblings, 1 reply; 32+ messages in thread
From: Giovanni Biscuolo @ 2020-02-14 10:03 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575
[-- Attachment #1: Type: text/plain, Size: 1102 bytes --]
Hello Ludo'
Ludovic Courtès <ludo@gnu.org> writes:
[...]
> The problem here is really that we fall back to content-addressed
> mirrors instead of using them directly:
>
> https://issues.guix.gnu.org/issue/28659
Given the natute (AFAIU) of this issue is the very same of the bug you
mention, shouldn't this bug be merged (forcemerged?) whith #28659?
If you agree and have no time, I can try doing this
> The file itself is still available on our machines though, and you can
> get it with:
>
> guix download -o harfbuzz-2.4.0.tar.bz2 \
> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
>
> guix download file://$PWD/harfbuzz-2.4.0.tar.bz2
>
> After that, re-running ‘guix time-machine’ should work.
Does make it sense to make this workaround more general and add this as
a Cookbook or blog entry? If the patch you propose in #28659 solve any
issue and is ready to be merget of course it's not worth the effort
[...]
Thanks! Gio'
--
Giovanni Biscuolo
Xelera IT Infrastructures
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-13 21:34 ` Ludovic Courtès
2020-02-14 1:05 ` zimoun
2020-02-14 10:03 ` Giovanni Biscuolo
@ 2020-02-14 10:47 ` zimoun
2020-02-14 12:26 ` Ludovic Courtès
2020-02-14 12:45 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2020-02-14 13:45 ` Jan Nieuwenhuizen
3 siblings, 2 replies; 32+ messages in thread
From: zimoun @ 2020-02-14 10:47 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575
Hi Ludo,
On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hi,
>
> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
> > building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...
> > downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...
> > |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'
> > - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:
> > expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
> > actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
> > hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'
>
> The file itself is still available on our machines though, and you can
> get it with:
>
> guix download -o harfbuzz-2.4.0.tar.bz2 \
> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
Maybe I miss a point, but the file we need is the old one, not the new
one, i.e., the one with the expected hash
1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch. And I should do
wrong but ci.guix.gnu.org does not have this file -- otherwise it will
find it because of substitutes mechanism.
--8<---------------cut here---------------start------------->8---
$ guix download -o /tmp/harfbuzz-old.tar.bz2 \
https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
Starting download of /tmp/harfbuzz-old.tar.bz2
From https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch...
download failed
"https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch"
404 "Not Found"
failed to download "/tmp/harfbuzz-old.tar.bz2" from
"https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch"
guix download: error: open-file: No such file or directory:
"/tmp/harfbuzz-old.tar.bz2"
--8<---------------cut here---------------end--------------->8---
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 10:03 ` Giovanni Biscuolo
@ 2020-02-14 10:56 ` zimoun
0 siblings, 0 replies; 32+ messages in thread
From: zimoun @ 2020-02-14 10:56 UTC (permalink / raw)
To: Giovanni Biscuolo; +Cc: 39575
Hi Giovanni,
On Fri, 14 Feb 2020 at 11:07, Giovanni Biscuolo <g@xelera.eu> wrote:
> Ludovic Courtès <ludo@gnu.org> writes:
> > The problem here is really that we fall back to content-addressed
> > mirrors instead of using them directly:
> >
> > https://issues.guix.gnu.org/issue/28659
>
> Given the natute (AFAIU) of this issue is the very same of the bug you
> mention, shouldn't this bug be merged (forcemerged?) whith #28659?
AFAIU, there is 2 issues:
1. how to do with the particular case of harfbuzz -- bug#39575
2. how to solve the general case of unreliable sources -- bug#28659
So instead of merging the bugs (case 2.), I would like to solve 1.,
mark it as done and report to bug#28659. It will ease to follow
because the thread in bug#28659 is already heavy. :-)
All the best,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 10:47 ` zimoun
@ 2020-02-14 12:26 ` Ludovic Courtès
2020-02-14 13:24 ` Jan Nieuwenhuizen
2020-02-14 12:45 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
1 sibling, 1 reply; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-14 12:26 UTC (permalink / raw)
To: zimoun; +Cc: 39575
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
> On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
>>
>> Hi,
>>
>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>>
>> > building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...
>> > downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...
>> > |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'
>> > - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:
>> > expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
>> > actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
>> > hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'
>>
>> The file itself is still available on our machines though, and you can
>> get it with:
>>
>> guix download -o harfbuzz-2.4.0.tar.bz2 \
>> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
>
> Maybe I miss a point, but the file we need is the old one, not the new
> one, i.e., the one with the expected hash
> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch.
Oops, my bad.
> And I should do wrong but ci.guix.gnu.org does not have this file --
> otherwise it will find it because of substitutes mechanism.
>
> $ guix download -o /tmp/harfbuzz-old.tar.bz2 \
> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
I checked on a bunch of machines and couldn’t find it.
Everyone, please check whether you have
/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2 and
so share!
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 10:47 ` zimoun
2020-02-14 12:26 ` Ludovic Courtès
@ 2020-02-14 12:45 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2020-02-14 13:14 ` Ludovic Courtès
1 sibling, 1 reply; 32+ messages in thread
From: Tobias Geerinckx-Rice via Bug reports for GNU Guix @ 2020-02-14 12:45 UTC (permalink / raw)
To: zimoun; +Cc: Ludovic Courtès, 39575
[-- Attachment #1: Type: text/plain, Size: 608 bytes --]
zimoun 写道：
> Maybe I miss a point, but the file we need is the old one, not
> the new
> one, i.e., the one with the expected hash
> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch. And I
> should do
~ λ guix download
https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2
Starting download of /tmp/guix-file.JSWxOl
From https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2...
...4.0.tar.bz2 17.1MiB
6.5MiB/s 00:03 [##################] 100.0%
/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2
1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
Enjoy,
T G-R
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 12:45 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
@ 2020-02-14 13:14 ` Ludovic Courtès
2020-02-15 15:51 ` zimoun
0 siblings, 1 reply; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-14 13:14 UTC (permalink / raw)
To: Tobias Geerinckx-Rice; +Cc: 39575
Hi,
Tobias Geerinckx-Rice <me@tobias.gr> skribis:
> zimoun 写道：
>> Maybe I miss a point, but the file we need is the old one, not the
>> new
>> one, i.e., the one with the expected hash
>> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch. And I should
>> do
>
> ~ λ guix download https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2
Thanks, you saved us!
Now we have it here:
https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
I’ve also registered a GC root.
Anyway, everything will be so much better when SWH archives tarballs!
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 12:26 ` Ludovic Courtès
@ 2020-02-14 13:24 ` Jan Nieuwenhuizen
2020-02-14 13:51 ` Ludovic Courtès
2020-02-17 18:32 ` zimoun
0 siblings, 2 replies; 32+ messages in thread
From: Jan Nieuwenhuizen @ 2020-02-14 13:24 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575
Ludovic Courtès writes:
> Hi,
>
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> On Thu, 13 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
>>>
>>> Hi,
>>>
>>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>>>
>>> > building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...
>>> > downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...
>>> > |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'
>>> > - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:
>>> > expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
>>> > actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
>>> > hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'
>>>
>>> The file itself is still available on our machines though, and you can
>>> get it with:
>>>
>>> guix download -o harfbuzz-2.4.0.tar.bz2 \
>>> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
>>
>> Maybe I miss a point, but the file we need is the old one, not the new
>> one, i.e., the one with the expected hash
>> 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch.
>
> Oops, my bad.
>
>> And I should do wrong but ci.guix.gnu.org does not have this file --
>> otherwise it will find it because of substitutes mechanism.
>>
>> $ guix download -o /tmp/harfbuzz-old.tar.bz2 \
>> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
>
> I checked on a bunch of machines and couldn’t find it.
>
> Everyone, please check whether you have
> /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2 and
> so share!
What about
https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
(The strange thing being here, that snapshot.debian.org does not provide
a copy of the the in-place rewritten upstream tarball, either on
2019-05-06 or later.)
So, this now becomes the recipe
wget -O harfbuzz-2.4.0.tar.bz2 https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
guix download $PWD/harfbuzz-2.4.0.tar.bz2
guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad --no-offload -- help
that i'm trying now, and for now it looks fine (lots of stuff to build,
i'll report success or failure when it's done).
It seems, however, that for offload builds to work the guix download
needs to be repeated on the offload build farm machines too?
janneke
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-13 21:34 ` Ludovic Courtès
` (2 preceding siblings ...)
2020-02-14 10:47 ` zimoun
@ 2020-02-14 13:45 ` Jan Nieuwenhuizen
2020-02-14 21:34 ` Ludovic Courtès
3 siblings, 1 reply; 32+ messages in thread
From: Jan Nieuwenhuizen @ 2020-02-14 13:45 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575
Ludovic Courtès writes:
> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> building /gnu/store/cjim33x0q1bv1ppkv3qijvr1pvsn4y0q-harfbuzz-2.4.0.tar.bz2.drv...
>> downloading from https://www.freedesktop.org/software/harfbuzz/release/harfbuzz-2.4.0.tar.bz2...
>> |offloading build of /gnu/store/6fgg1irkcvqyb4f9f8n0nzi5gknyqhfn-gcc-mesboot1-4.7.4.drv to 'kluit.dezyne.org'
>> - 'build' phasesha256 hash mismatch for /gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2:
>> expected hash: 1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
>> actual hash: 0vrkvdlmihdg62a4c6h5kx27khc33xmb95l50zgnwnavvpwyyw5l
>> hash mismatch for store item '/gnu/store/b4cdp9sp44848348lrpzbfafhmjqf8nr-harfbuzz-2.4.0.tar.bz2'
>
> The problem here is really that we fall back to content-addressed
> mirrors instead of using them directly:
>
> https://issues.guix.gnu.org/issue/28659
Wait, what happened here; you finally proposed a patch two years ago and
nothing happened/we all forgot to follow up?
I cannot determine if possibly we were hoping to "wait" for the guile
build daemon?
janneke
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 13:24 ` Jan Nieuwenhuizen
@ 2020-02-14 13:51 ` Ludovic Courtès
2020-02-15 15:32 ` zimoun
2020-02-17 18:32 ` zimoun
1 sibling, 1 reply; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-14 13:51 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 39575
Hi,
Jan Nieuwenhuizen <janneke@gnu.org> skribis:
> What about
>
> https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
Good idea.
> So, this now becomes the recipe
>
> wget -O harfbuzz-2.4.0.tar.bz2 https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
> guix download $PWD/harfbuzz-2.4.0.tar.bz2
> guix time-machine --commit=56e95d54d209c2428f970d65d9b27ae4168449ad --no-offload -- help
>
> that i'm trying now, and for now it looks fine (lots of stuff to build,
> i'll report success or failure when it's done).
OK!
> It seems, however, that for offload builds to work the guix download
> needs to be repeated on the offload build farm machines too?
No, I don’t think so, because the head node copies all the inputs to
build machines before it actually offloads the build.
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 13:45 ` Jan Nieuwenhuizen
@ 2020-02-14 21:34 ` Ludovic Courtès
2020-02-15 15:43 ` bug#28659: " zimoun
2020-09-09 14:31 ` bug#28659: Content-addressed mirror is not used upon invalid hash zimoun
0 siblings, 2 replies; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-14 21:34 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 39575, 28659
Jan Nieuwenhuizen <janneke@gnu.org> skribis:
> Ludovic Courtès writes:
[...]
>> The problem here is really that we fall back to content-addressed
>> mirrors instead of using them directly:
>>
>> https://issues.guix.gnu.org/issue/28659
>
> Wait, what happened here; you finally proposed a patch two years ago and
> nothing happened/we all forgot to follow up?
I think we forgot, indeed.
One thing I don’t quite like about the patch is the fact that ‘guix
substitutes’ connects to the daemon in ‘content-addressed-item?’.
Also, one could argue that we’d steer users towards downloading from our
server, which could be a privacy concern (probably not a strong argument
since one can easily change the substitute URLs.)
Thoughts?
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 13:51 ` Ludovic Courtès
@ 2020-02-15 15:32 ` zimoun
2020-02-15 20:01 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
0 siblings, 1 reply; 32+ messages in thread
From: zimoun @ 2020-02-15 15:32 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575
Hi,
On Fri, 14 Feb 2020 at 14:51, Ludovic Courtès <ludo@gnu.org> wrote:
> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
> > What about
> >
> > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
>
> Good idea.
Cool!
But how do you determine the "date", i.e., this reference '20190406T212022Z' ?
Could it be automated?
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#28659: bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 21:34 ` Ludovic Courtès
@ 2020-02-15 15:43 ` zimoun
2020-02-16 10:59 ` Ludovic Courtès
2020-09-09 14:31 ` bug#28659: Content-addressed mirror is not used upon invalid hash zimoun
1 sibling, 1 reply; 32+ messages in thread
From: zimoun @ 2020-02-15 15:43 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575, 28659
Hi,
On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
> Also, one could argue that we’d steer users towards downloading from our
> server, which could be a privacy concern (probably not a strong argument
> since one can easily change the substitute URLs.)
I am not following the privacy concern.
What do you mean?
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 13:14 ` Ludovic Courtès
@ 2020-02-15 15:51 ` zimoun
0 siblings, 0 replies; 32+ messages in thread
From: zimoun @ 2020-02-15 15:51 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575
Hi,
On Fri, 14 Feb 2020 at 14:14, Ludovic Courtès <ludo@gnu.org> wrote:
> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
> > ~ λ guix download https://www.tobias.gr/guix/harfbuzz-2.4.0.tar.bz2
>
> Thanks, you saved us!
Thank you! :-)
> Anyway, everything will be so much better when SWH archives tarballs!
The future will be better. :-)
Even some details need to be discussed: frequency of source.json
generation, frequency of the SWH crawler ingest it, etc.
How could all the archives living in ci.guix.gnu.org be sent to SWH?
Because IMHO the of Berlin (or other) is to archive the world but to
build it. :-)
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-15 15:32 ` zimoun
@ 2020-02-15 20:01 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2020-02-15 23:57 ` Bengt Richter
2020-02-17 8:47 ` zimoun
0 siblings, 2 replies; 32+ messages in thread
From: Tobias Geerinckx-Rice via Bug reports for GNU Guix @ 2020-02-15 20:01 UTC (permalink / raw)
To: zimoun, Jan Nieuwenhuizen; +Cc: 39575
[-- Attachment #1: Type: text/plain, Size: 1056 bytes --]
Jan, Simon,
Janneke 写道：
> https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
This is a wonderful resource! Thank you, Janneke (and Debian)!
zimoun 写道：
> Cool!
> But how do you determine the "date", i.e., this reference
> '20190406T212022Z' ?
You'd take the timestamp immediately preceding your desired (Guix)
commit's date, or something like that. The fact that git commit
dates aren't linear shouldn't hurt here.
> Could it be automated?
Not without parsing HTML to get the valid timestamps:
<https://snapshot.debian.org/archive/debian/?year=2020&month=2>.
Also, this doesn't seem to be a supported service yet[0]:
“This is an implementation for a possible snapshot.debian.org
service.
It's not yet finished, it's more a prototype/proof of concept
to show
and learn what we want and can provide. So far it seems to
actually work.”
Still really cool,
T G-R
[0]: https://salsa.debian.org/snapshot-team/snapshot
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-15 20:01 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
@ 2020-02-15 23:57 ` Bengt Richter
2020-02-17 8:47 ` zimoun
1 sibling, 0 replies; 32+ messages in thread
From: Bengt Richter @ 2020-02-15 23:57 UTC (permalink / raw)
To: Tobias Geerinckx-Rice; +Cc: 39575
On +2020-02-15 21:01:36 +0100, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
> Jan, Simon,
>
> Janneke 写道：
> > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
>
> This is a wonderful resource! Thank you, Janneke (and Debian)!
>
> zimoun 写道：
> > Cool!
> > But how do you determine the "date", i.e., this reference
> > '20190406T212022Z' ?
>
> You'd take the timestamp immediately preceding your desired (Guix) commit's
> date, or something like that. The fact that git commit dates aren't linear
> shouldn't hurt here.
>
> > Could it be automated?
>
> Not without parsing HTML to get the valid timestamps:
> <https://snapshot.debian.org/archive/debian/?year=2020&month=2>.
>
You may not need to parse the html fully if the part you need is
isolatable into delimited scopes that you can successively narrow.
For example, I while back I wanted a command I could type to get
the url of the latest linux kernel at kernel.org:
stable-kernel.scm -h
--8<---------------cut here---------------start------------->8---
Usage: stable-kernel-scm [ -h ]
-h for this message
(without args):
go to https://www.kernel.org/ to wget page,
extract URL of latest stable release tarball
and write that URL to stdout.
--8<---------------cut here---------------end--------------->8---
(oops, I see I din't use $0 in the usage text -- should be .scm, not -scm)
I offer it below [1], with the thought that you could probably
modify (not to mention improve :-) it to get the timestamps you want.
Especially if you could get them to make the narrow context unique enough
that it's delimiters can delimit it in one shot.
The page at kernel.org is apparently stable enough that this still works,
but YMMV until the snapshot page is similarly stable. (You could ask
them to make it easy :)
> Also, this doesn't seem to be a supported service yet[0]:
>
> “This is an implementation for a possible snapshot.debian.org service.
> It's not yet finished, it's more a prototype/proof of concept to show
> and learn what we want and can provide. So far it seems to actually
> work.”
>
> Still really cool,
>
> T G-R
>
> [0]: https://salsa.debian.org/snapshot-team/snapshot
HTH or is useful some way.
--
Regards,
Bengt Richter
[1]
--8<---------------cut here---------------start------------->8---
#!/usr/bin/bash
exec guile -e main -s "$0" "$@"
!#
;;;; stable-kernel.scm
;;;; goes to https://www.kernel.org/ to wget page, then
;;;; extracts name of latest stable release tarball to stdout
;;;;
(define (usage)
(format (current-error-port)
(string-join
'(
"Usage: stable-kernel-scm [ -h ]"
" -h for this message"
" (without args):"
" go to https://www.kernel.org/ to wget page,"
" extract URL of latest stable release tarball"
" and write that URL to stdout."
"")
"\n")))
(use-modules (ice-9 format))
(use-modules (ice-9 rdelim))
(use-modules (ice-9 popen))
(use-modules (ice-9 textual-ports))
(use-modules (ice-9 and-let-star))
(use-modules (ice-9 regex))
(define (extract-delimited str s-beg s-end)
(and-let*
((ix-beg (string-contains str s-beg))
(ix-post-beg (+ ix-beg (string-length s-beg)))
(ix-end (string-contains str s-end ix-post-beg)))
(substring str ix-post-beg ix-end)))
(define kernel-url "https://www.kernel.org/")
(define (get-kern-name)
(let*((cmd-kern (string-append "wget -q -O - " kernel-url))
(p-inp (open-input-pipe cmd-kern))
(wgot-pinp-str (get-string-all p-inp))
(extracted-table-releases
(extract-delimited wgot-pinp-str
"<table id=\"releases\">"
"</table>"))
(extracted-stable-tarball-anchor
(extract-delimited extracted-table-releases
"<td>stable:</td>"
">tarball<"))
(extracted-stable-href
(extract-delimited extracted-stable-tarball-anchor
"<a href=\""
"\"")))
(begin
extracted-stable-href)))
(define (main args)
(begin
(set! args (cdr args)) ;; always dump callee arg
(if (not (pair? args))
(set! args '("-do-default") ))
;; simple opts...
(cond
((string-prefix? "-h" (car args))
(begin (usage)
(exit)))
((string-prefix? "-do-default" (car args))
(let ((kern-name (get-kern-name)))
(display kern-name)(newline)))
(#t
(begin
(format (current-error-port) "Error: bad opt: '~a'\n" (car args))
(usage)
(exit))))))
--8<---------------cut here---------------end--------------->8---
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#28659: bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-15 15:43 ` bug#28659: " zimoun
@ 2020-02-16 10:59 ` Ludovic Courtès
2020-02-17 10:18 ` zimoun
0 siblings, 1 reply; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-16 10:59 UTC (permalink / raw)
To: zimoun; +Cc: 39575, 28659
Hi!
zimoun <zimon.toutoune@gmail.com> skribis:
> On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Also, one could argue that we’d steer users towards downloading from our
>> server, which could be a privacy concern (probably not a strong argument
>> since one can easily change the substitute URLs.)
>
> I am not following the privacy concern.
> What do you mean?
I mean that by default, someone who’s disabled substitutes (presumably
out of security or privacy concerns) would find themself downloading
source code from ci.guix.gnu.org instead of various upstream sites.
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-15 20:01 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2020-02-15 23:57 ` Bengt Richter
@ 2020-02-17 8:47 ` zimoun
2020-02-17 13:26 ` Efraim Flashner
2020-02-17 15:02 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
1 sibling, 2 replies; 32+ messages in thread
From: zimoun @ 2020-02-17 8:47 UTC (permalink / raw)
To: Tobias Geerinckx-Rice; +Cc: 39575
Hi,
On Sat, 15 Feb 2020 at 21:01, Tobias Geerinckx-Rice <me@tobias.gr> wrote:
> Janneke 写道：
> > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
>
> This is a wonderful resource! Thank you, Janneke (and Debian)!
>
> zimoun 写道：
> > Cool!
> > But how do you determine the "date", i.e., this reference
> > '20190406T212022Z' ?
>
> You'd take the timestamp immediately preceding your desired (Guix)
> commit's date, or something like that. The fact that git commit
> dates aren't linear shouldn't hurt here.
You assume that Debian packs packages as fast as Guix, I mean on the
same schedule which is a strong assumption IMHO.
For example, if it was the contrary and the "new" release of harfbuzz
2.4.0 were missing, then would Debian be helpful?
> Also, this doesn't seem to be a supported service yet[0]:
>
> “This is an implementation for a possible snapshot.debian.org
> service.
> It's not yet finished, it's more a prototype/proof of concept
> to show
> and learn what we want and can provide. So far it seems to
> actually work.”
>
> Still really cool,
Yes, still cool! :-)
Thanks,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-16 10:59 ` Ludovic Courtès
@ 2020-02-17 10:18 ` zimoun
2020-02-17 14:40 ` bug#28659: " Ludovic Courtès
0 siblings, 1 reply; 32+ messages in thread
From: zimoun @ 2020-02-17 10:18 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575, 28659
Hi Ludo,
On Sun, 16 Feb 2020 at 11:59, Ludovic Courtès <ludo@gnu.org> wrote:
> zimoun <zimon.toutoune@gmail.com> skribis:
> > On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
> >> Also, one could argue that we’d steer users towards downloading from our
> >> server, which could be a privacy concern (probably not a strong argument
> >> since one can easily change the substitute URLs.)
> >
> > I am not following the privacy concern.
> > What do you mean?
>
> I mean that by default, someone who’s disabled substitutes (presumably
> out of security or privacy concerns) would find themself downloading
> source code from ci.guix.gnu.org instead of various upstream sites.
I do not see the difference between mirroring and traveling back in
time with missing upstream sources.
And because it is content-addressed, it seems even more secure than
downloading from a upstream URL, IMHO.
If one trusts Guix, then an attacker needs to corrupt in the same time
the Guix history and Berlin (and/or any other farm).
If one does not trust Guix, why does they use the recipe coming from
Guix? To be precise, this person has to check all the recipes of all
the dependencies.
Well, I do not see a security concern because we are talking about
serving the sources.
It is another story when the substitutes serve the results of the
build (binaries); because one does not have any strong guarantee that
the substitute serves the expected binaries.
By privacy concern, do you mean that Guix could collect who downloads
what; in a central fashion? Which is not the case when one downloads
from several distributed upstream sources. Right?
Well, I am not convinced because the case of missing upstream source
is rare. And it is easy to protect against such collecting data
process.
In paranoid mode, traveling back in time is becoming difficult because
of the reliability of the sources; I mean if the sources were
reliable, SWH would not exist. ;-) The solution should be an IPFS /
GNUnet / full distributed archive... which is not ready... yet! :-)
Well, maybe for the TODO list of the time-machine: add an option to
allow substitutes *only* for the sources (substitutes meaning
ci.guix.gnu.org and/or SWH). If this option does not exist yet. ;-)
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-17 8:47 ` zimoun
@ 2020-02-17 13:26 ` Efraim Flashner
2020-02-17 15:02 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
1 sibling, 0 replies; 32+ messages in thread
From: Efraim Flashner @ 2020-02-17 13:26 UTC (permalink / raw)
To: zimoun; +Cc: 39575
[-- Attachment #1: Type: text/plain, Size: 1845 bytes --]
On Mon, Feb 17, 2020 at 09:47:41AM +0100, zimoun wrote:
> Hi,
>
> On Sat, 15 Feb 2020 at 21:01, Tobias Geerinckx-Rice <me@tobias.gr> wrote:
>
> > Janneke 写道：
> > > https://snapshot.debian.org/archive/debian/20190406T212022Z/pool/main/h/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
> >
> > This is a wonderful resource! Thank you, Janneke (and Debian)!
> >
> > zimoun 写道：
> > > Cool!
> > > But how do you determine the "date", i.e., this reference
> > > '20190406T212022Z' ?
> >
> > You'd take the timestamp immediately preceding your desired (Guix)
> > commit's date, or something like that. The fact that git commit
> > dates aren't linear shouldn't hurt here.
>
> You assume that Debian packs packages as fast as Guix, I mean on the
> same schedule which is a strong assumption IMHO.
> For example, if it was the contrary and the "new" release of harfbuzz
> 2.4.0 were missing, then would Debian be helpful?
>
>
We could first try
mirror://debian/pool/main/harfbuzz/harfbuzz_2.4.0.orig.tar.bz2
and then scrape https://snapshot.debian.org/package/harfbuzz/ for
2.4.0-1 and then parse the website for harfbuzz_2.4.0.orig.tar.bz2. Or
for just 'orig.tar'
> > Also, this doesn't seem to be a supported service yet[0]:
> >
> > “This is an implementation for a possible snapshot.debian.org
> > service.
> > It's not yet finished, it's more a prototype/proof of concept
> > to show
> > and learn what we want and can provide. So far it seems to
> > actually work.”
> >
> > Still really cool,
>
> Yes, still cool! :-)
>
>
> Thanks,
> simon
>
>
>
--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#28659: bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-17 10:18 ` zimoun
@ 2020-02-17 14:40 ` Ludovic Courtès
2020-02-17 15:04 ` zimoun
0 siblings, 1 reply; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-17 14:40 UTC (permalink / raw)
To: zimoun; +Cc: 39575, 28659
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
> On Sun, 16 Feb 2020 at 11:59, Ludovic Courtès <ludo@gnu.org> wrote:
>> zimoun <zimon.toutoune@gmail.com> skribis:
>> > On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> >> Also, one could argue that we’d steer users towards downloading from our
>> >> server, which could be a privacy concern (probably not a strong argument
>> >> since one can easily change the substitute URLs.)
>> >
>> > I am not following the privacy concern.
>> > What do you mean?
>>
>> I mean that by default, someone who’s disabled substitutes (presumably
>> out of security or privacy concerns) would find themself downloading
>> source code from ci.guix.gnu.org instead of various upstream sites.
[...]
> By privacy concern, do you mean that Guix could collect who downloads
> what; in a central fashion? Which is not the case when one downloads
> from several distributed upstream sources. Right?
Exactly. But like I wrote above, I don’t think it’s a strong argument.
What remains is the issue with ‘content-addressed-item?’, then.
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-17 8:47 ` zimoun
2020-02-17 13:26 ` Efraim Flashner
@ 2020-02-17 15:02 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
1 sibling, 0 replies; 32+ messages in thread
From: Tobias Geerinckx-Rice via Bug reports for GNU Guix @ 2020-02-17 15:02 UTC (permalink / raw)
To: zimoun; +Cc: 39575
[-- Attachment #1: Type: text/plain, Size: 162 bytes --]
zimoun 写道：
> You assume that Debian packs packages as fast as Guix
Indeed I do! :-D
Efraim's solution sounds reasonable.
Kind regards,
T G-R
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#28659: bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-17 14:40 ` bug#28659: " Ludovic Courtès
@ 2020-02-17 15:04 ` zimoun
0 siblings, 0 replies; 32+ messages in thread
From: zimoun @ 2020-02-17 15:04 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 39575, 28659
On Mon, 17 Feb 2020 at 15:40, Ludovic Courtès <ludo@gnu.org> wrote:
> Exactly. But like I wrote above, I don’t think it’s a strong argument.
I agree and the big picture depends on the audience.
Scientific communities would be fine with centralized archives such as
SWH. And only centralized archives IMHO can provide a reliable "long
term" support which is the point for that communities. (Quote because
not clearly defined what it is. :-))
Other communities would prefer distributed archive such as IPFS or
GNUnet but 1. it still needs some work and 2. the "long term" is not
guarantee by nature, IMHO. But it is probably not an issue for that
communities.
> What remains is the issue with ‘content-addressed-item?’, then.
I agree.
The bridge with SWH is in good shape, IMHO.
And the pending IPFS patch would deserve more love. :-) Maybe soon...
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-14 13:24 ` Jan Nieuwenhuizen
2020-02-14 13:51 ` Ludovic Courtès
@ 2020-02-17 18:32 ` zimoun
2020-02-19 11:58 ` Jan Nieuwenhuizen
1 sibling, 1 reply; 32+ messages in thread
From: zimoun @ 2020-02-17 18:32 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 39575
HI Jan,
On Fri, 14 Feb 2020 at 14:24, Jan Nieuwenhuizen <janneke@gnu.org> wrote:
This command
> >> $ guix download -o /tmp/harfbuzz-old.tar.bz2 \
> >> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
now works.
However, this command
$ guix time-machine \
--commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help
still fails for me with the message:
--8<---------------cut here---------------start------------->8---
[...]
building /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv...
- 'check' phasebuilder for
`/gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv' failed
with exit code 1
build of /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv failed
View build log at
'/var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2'.
cannot build derivation
`/gnu/store/yqpxm07zm0mirrdvl2c4qvf8biyzg468-guix-56e95d54d.drv': 1
dependencies couldn't be built
cannot build derivation
`/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv': 1
dependencies couldn't be built
guix time-machine: error: build of
`/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv' failed
--8<---------------cut here---------------end--------------->8---
The log /var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2
is not meaningful for me... but I can report it here.
> that i'm trying now, and for now it looks fine (lots of stuff to build,
> i'll report success or failure when it's done).
Well, is it a success or a failure for you?
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-17 18:32 ` zimoun
@ 2020-02-19 11:58 ` Jan Nieuwenhuizen
2020-02-21 15:58 ` zimoun
0 siblings, 1 reply; 32+ messages in thread
From: Jan Nieuwenhuizen @ 2020-02-19 11:58 UTC (permalink / raw)
To: zimoun; +Cc: 39575
zimoun writes:
Hi Simon,
> On Fri, 14 Feb 2020 at 14:24, Jan Nieuwenhuizen <janneke@gnu.org> wrote:
>
> This command
>
>> >> $ guix download -o /tmp/harfbuzz-old.tar.bz2 \
>> >> https://ci.guix.gnu.org/file/harfbuzz-2.4.0.tar.bz2/sha256/1mpah6kwqid1kxsj4rwqsniivqbrx231j65v51yncx6s0dch0dch
>
> now works.
>
>
> However, this command
>
> $ guix time-machine \
> --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help
>
> still fails for me with the message:
>
> [...]
> building /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv...
> - 'check' phasebuilder for
> `/gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv' failed
> with exit code 1
> build of /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv failed
> View build log at
> '/var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2'.
> cannot build derivation
> `/gnu/store/yqpxm07zm0mirrdvl2c4qvf8biyzg468-guix-56e95d54d.drv': 1
> dependencies couldn't be built
> cannot build derivation
> `/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv': 1
> dependencies couldn't be built
> guix time-machine: error: build of
> `/gnu/store/7z7p0m7abi246gzigw8as2q3w33k1n31-profile.drv' failed
>
> The log /var/log/guix/drvs/gg/lbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv.bz2
> is not meaningful for me... but I can report it here.
>
>
>> that i'm trying now, and for now it looks fine (lots of stuff to build,
>> i'll report success or failure when it's done).
>
> Well, is it a success or a failure for you?
For me, pythohn-minimal fails to build
build-started /gnu/store/s0lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv - x86_64-linux /var/log/guix/drvs/s0//lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv.bz2 20827
======================================================================
FAIL: test_register_chain (test.test_faulthandler.FaultHandlerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_faulthandler.py", line 724, in test_register_chain
self.check_register(chain=True)
File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_faulthandler.py", line 702, in check_register
self.assertEqual(exitcode, 0)
AssertionError: -11 != 0
----------------------------------------------------------------------
Ran 42 tests in 18.782s
FAILED (failures=1, skipped=4)
1 test failed again:
test_faulthandler
== Tests result: FAILURE then FAILURE ==
382 tests OK.
1 test failed:
test_faulthandler
Not sure what to do here. Could this be a (harmless) coincident?
Greetings
janneke
--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-19 11:58 ` Jan Nieuwenhuizen
@ 2020-02-21 15:58 ` zimoun
2020-02-21 20:48 ` Ludovic Courtès
0 siblings, 1 reply; 32+ messages in thread
From: zimoun @ 2020-02-21 15:58 UTC (permalink / raw)
To: Jan Nieuwenhuizen; +Cc: 39575
Hi,
Some follow ups.
For me, now, the command
> > $ guix time-machine \
> > --commit=56e95d54d209c2428f970d65d9b27ae4168449ad -- help
does not fail anymore at the Guile step:
--8<---------------cut here---------------start------------->8---
building /gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv...
- 'check' phasebuilder for
`/gnu/store/gglbrs8j0iq8ygh55inwfvpwb5z2x254-guile-2.2.4.drv' failed
--8<---------------cut here---------------end--------------->8---
because it was about "FAIL: test-out-of-memory".
Well, now, I am failing at the Python 3.7.3 step too:
--8<---------------cut here---------------start------------->8---
/gnu/store/s0lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv failed
--8<---------------cut here---------------end--------------->8---
However, the python error seems about TLS:
--8<---------------cut here---------------start------------->8---
test.test_asyncio.test_windows_utils (unittest.loader.ModuleSkipped)
... test test_asyncio failed
skipped 'Windows only'
======================================================================
ERROR: test_start_tls_server_1
(test.test_asyncio.test_sslproto.SelectorStartTLSTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",
line 507, in test_start_tls_server_1
self.loop.run_until_complete(run_main())
File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/base_events.py",
line 584, in run_until_complete
return future.result()
File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",
line 502, in run_main
loop=self.loop, timeout=self.TIMEOUT)
File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/tasks.py",
line 423, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
--8<---------------cut here---------------end--------------->8---
> Not sure what to do here. Could this be a (harmless) coincident?
Me neither.
Cheers,
simon
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#39575: guix time-machine fails when a tarball was modified in-place
2020-02-21 15:58 ` zimoun
@ 2020-02-21 20:48 ` Ludovic Courtès
0 siblings, 0 replies; 32+ messages in thread
From: Ludovic Courtès @ 2020-02-21 20:48 UTC (permalink / raw)
To: zimoun; +Cc: 39575
Hi,
zimoun <zimon.toutoune@gmail.com> skribis:
> Well, now, I am failing at the Python 3.7.3 step too:
>
> /gnu/store/s0lw23myd3hvpw28sffkhz8b30x1hcz0-python-minimal-3.7.3.drv failed
>
>
>
> However, the python error seems about TLS:
>
> test.test_asyncio.test_windows_utils (unittest.loader.ModuleSkipped)
> ... test test_asyncio failed
> skipped 'Windows only'
>
> ======================================================================
> ERROR: test_start_tls_server_1
> (test.test_asyncio.test_sslproto.SelectorStartTLSTests)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",
> line 507, in test_start_tls_server_1
> self.loop.run_until_complete(run_main())
> File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/base_events.py",
> line 584, in run_until_complete
> return future.result()
> File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/test/test_asyncio/test_sslproto.py",
> line 502, in run_main
> loop=self.loop, timeout=self.TIMEOUT)
> File "/tmp/guix-build-python-minimal-3.7.3.drv-0/Python-3.7.3/Lib/asyncio/tasks.py",
> line 423, in wait_for
> raise futures.TimeoutError()
> concurrent.futures._base.TimeoutError
It seems to be timing-sensitive, is it deterministic?
One lesson here is that we should keep substitutes for a longer amount
of time—we actually have a lot of storage space on berlin so we should
investigate what happened. (The NixOS folks currently keep substitutes
forever but they found it’s starting to be expensive…)
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#28659: Content-addressed mirror is not used upon invalid hash
2020-02-14 21:34 ` Ludovic Courtès
2020-02-15 15:43 ` bug#28659: " zimoun
@ 2020-09-09 14:31 ` zimoun
2020-09-10 8:14 ` Ludovic Courtès
1 sibling, 1 reply; 32+ messages in thread
From: zimoun @ 2020-09-09 14:31 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 28659
Hi,
On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
> One thing I don’t quite like about the patch is the fact that ‘guix
> substitutes’ connects to the daemon in ‘content-addressed-item?’.
What is the status of this patch [1] following the recent discussion about
tar “disarchive” and SWH?
Related:
- http://issues.guix.gnu.org/issue/39575
- http://issues.guix.gnu.org/42162
- https://git.ngyro.com/disarchive/
All the best,
simon
[1] http://issues.guix.gnu.org/issue/28659#26
^ permalink raw reply [flat|nested] 32+ messages in thread
* bug#28659: Content-addressed mirror is not used upon invalid hash
2020-09-09 14:31 ` bug#28659: Content-addressed mirror is not used upon invalid hash zimoun
@ 2020-09-10 8:14 ` Ludovic Courtès
0 siblings, 0 replies; 32+ messages in thread
From: Ludovic Courtès @ 2020-09-10 8:14 UTC (permalink / raw)
To: zimoun; +Cc: 28659
Hello,
zimoun <zimon.toutoune@gmail.com> skribis:
> On Fri, 14 Feb 2020 at 22:34, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> One thing I don’t quite like about the patch is the fact that ‘guix
>> substitutes’ connects to the daemon in ‘content-addressed-item?’.
>
> What is the status of this patch [1] following the recent discussion about
> tar “disarchive” and SWH?
>
> Related:
> - http://issues.guix.gnu.org/issue/39575
> - http://issues.guix.gnu.org/42162
> - https://git.ngyro.com/disarchive/
Thanks for the reminder. I don’t think Timothy’s work changes anything
wrt. to this issue: it would still need to be addressed.
Ludo’.
^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2020-09-10 8:18 UTC | newest]
Thread overview: 32+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-02-12 13:40 bug#39575: guix time-machine fails when a tarball was modified in-place Jan Nieuwenhuizen
2020-02-12 14:55 ` zimoun
2020-02-13 21:34 ` Ludovic Courtès
2020-02-14 1:05 ` zimoun
2020-02-14 10:03 ` Giovanni Biscuolo
2020-02-14 10:56 ` zimoun
2020-02-14 10:47 ` zimoun
2020-02-14 12:26 ` Ludovic Courtès
2020-02-14 13:24 ` Jan Nieuwenhuizen
2020-02-14 13:51 ` Ludovic Courtès
2020-02-15 15:32 ` zimoun
2020-02-15 20:01 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2020-02-15 23:57 ` Bengt Richter
2020-02-17 8:47 ` zimoun
2020-02-17 13:26 ` Efraim Flashner
2020-02-17 15:02 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2020-02-17 18:32 ` zimoun
2020-02-19 11:58 ` Jan Nieuwenhuizen
2020-02-21 15:58 ` zimoun
2020-02-21 20:48 ` Ludovic Courtès
2020-02-14 12:45 ` Tobias Geerinckx-Rice via Bug reports for GNU Guix
2020-02-14 13:14 ` Ludovic Courtès
2020-02-15 15:51 ` zimoun
2020-02-14 13:45 ` Jan Nieuwenhuizen
2020-02-14 21:34 ` Ludovic Courtès
2020-02-15 15:43 ` bug#28659: " zimoun
2020-02-16 10:59 ` Ludovic Courtès
2020-02-17 10:18 ` zimoun
2020-02-17 14:40 ` bug#28659: " Ludovic Courtès
2020-02-17 15:04 ` zimoun
2020-09-09 14:31 ` bug#28659: Content-addressed mirror is not used upon invalid hash zimoun
2020-09-10 8:14 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).