bug#17150: Stale bootstrap/*/guile-2.0.9.tar.xz files are not detected

bug#17150: Stale bootstrap/*/guile-2.0.9.tar.xz files are not detected

[Mark H Weaver]

I just realized that my x86_64 and Loongson 3A systems have spent an
enormous amount of time building the new guix master branch based on
outdated bootstrap/*/guile-2.0.9.tar.xz.

The issue is that if you simply "git pull" from a build directory with
older versions of bootstrap/*/guile-2.0.9.tar.xz, although the various
places where the hashes are stored are updated, those new hashes are
never checked against the existing files.  Therefore, you can proceed to
build an entire system based on an outdated bootstrap guile, and with
hashes that don't match what's on hydra and what other people are
building.

It would be good if the hashes were checked even if they are already
present in the build directory.

     Mark

bug#17150: Stale bootstrap/*/guile-2.0.9.tar.xz files are not detected

[Mark H Weaver]

I wrote:
> I just realized that my x86_64 and Loongson 3A systems have spent an
> enormous amount of time building the new guix master branch based on
> outdated bootstrap/*/guile-2.0.9.tar.xz.

Upon further investigation, I see that only MIPS was affected by this
problem in the recent merge of core-updates.  The reason is that the
bootstrap guile for MIPS was updated without changing its version
number, whereas the Intel ones were 2.0.7 before the update.

     Mark

bug#17150: Stale bootstrap/*/guile-2.0.9.tar.xz files are not detected

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> I just realized that my x86_64 and Loongson 3A systems have spent an
> enormous amount of time building the new guix master branch based on
> outdated bootstrap/*/guile-2.0.9.tar.xz.
>
> The issue is that if you simply "git pull" from a build directory with
> older versions of bootstrap/*/guile-2.0.9.tar.xz, although the various
> places where the hashes are stored are updated, those new hashes are
> never checked against the existing files.  Therefore, you can proceed to
> build an entire system based on an outdated bootstrap guile, and with
> hashes that don't match what's on hydra and what other people are
> building.

Right, ‘guix pull’ doesn’t survive updates of the bootstrap Guile
tarballs, because it doesn’t try to download it (see ‘build-guix’ in
guix/build/pull.scm.)  That’s rare in practice, but still a serious
limitation as you note.  :-/

There are other things to do in ‘guix pull’, such as authentication, and
improved bandwidth usage.  For the latter an option would be to resort
to git, and perhaps for the former too.

Ludo’.

bug#17150: Stale bootstrap/*/guile-2.0.9.tar.xz files are not detected

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> I just realized that my x86_64 and Loongson 3A systems have spent an
>> enormous amount of time building the new guix master branch based on
>> outdated bootstrap/*/guile-2.0.9.tar.xz.
>>
>> The issue is that if you simply "git pull" from a build directory with
>> older versions of bootstrap/*/guile-2.0.9.tar.xz, although the various
>> places where the hashes are stored are updated, those new hashes are
>> never checked against the existing files.  Therefore, you can proceed to
>> build an entire system based on an outdated bootstrap guile, and with
>> hashes that don't match what's on hydra and what other people are
>> building.
>
> Right, ‘guix pull’ doesn’t survive updates of the bootstrap Guile
> tarballs, because it doesn’t try to download it (see ‘build-guix’ in
> guix/build/pull.scm.)  That’s rare in practice, but still a serious
> limitation as you note.  :-/

Hmm, yes, I suppose that "guix pull" is more relevant for typical users,
but actually that's not what I was talking about above.  I was talking
about "git pull" followed by "make".

> There are other things to do in ‘guix pull’, such as authentication, and
> improved bandwidth usage.  For the latter an option would be to resort
> to git, and perhaps for the former too.

Yes, it seems to me that git is a good tool for this job.

      Mark

bug#17150: Stale bootstrap/*/guile-2.0.9.tar.xz files are not detected

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> I just realized that my x86_64 and Loongson 3A systems have spent an
>>> enormous amount of time building the new guix master branch based on
>>> outdated bootstrap/*/guile-2.0.9.tar.xz.
>>>
>>> The issue is that if you simply "git pull" from a build directory with
>>> older versions of bootstrap/*/guile-2.0.9.tar.xz, although the various
>>> places where the hashes are stored are updated, those new hashes are
>>> never checked against the existing files.  Therefore, you can proceed to
>>> build an entire system based on an outdated bootstrap guile, and with
>>> hashes that don't match what's on hydra and what other people are
>>> building.
>>
>> Right, ‘guix pull’ doesn’t survive updates of the bootstrap Guile
>> tarballs, because it doesn’t try to download it (see ‘build-guix’ in
>> guix/build/pull.scm.)  That’s rare in practice, but still a serious
>> limitation as you note.  :-/
>
> Hmm, yes, I suppose that "guix pull" is more relevant for typical users,
> but actually that's not what I was talking about above.  I was talking
> about "git pull" followed by "make".

Ah, sorry!  Ah yes, I see what the problem is.  Only
build-aux/download.scm checks the hash, so indeed, if the file is stale
or modified later, Guix doesn’t notice.

Perhaps we should add a ‘check-hash’ rule or something in the makefile,
that automatically triggers before installation or something?

Ludo’.