* bug#22423: git-fetch does not update checked out tree when commit hash changes
@ 2016-01-21 6:54 Pjotr Prins
2016-01-21 8:50 ` Ludovic Courtès
0 siblings, 1 reply; 6+ messages in thread
From: Pjotr Prins @ 2016-01-21 6:54 UTC (permalink / raw)
To: 22423
I can reliably reproduce this using a recent version of GNU Guix.
When updating the commit hash to a different commit the git-fetch
derivation *does* change (I checked in guile), but the checked out git
tree in the store does not change - it gets shared between the
commits. I am not sure why the tree gets shared, but the effect is
that the same package gets installed using the same
/gnu/store/xxx-git-checkout.
Removing the git-checkout dir and updating the Hash gives a missing
dir error (as expected when they use the same).
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#22423: git-fetch does not update checked out tree when commit hash changes
2016-01-21 6:54 bug#22423: git-fetch does not update checked out tree when commit hash changes Pjotr Prins
@ 2016-01-21 8:50 ` Ludovic Courtès
2016-01-21 9:08 ` Pjotr Prins
0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2016-01-21 8:50 UTC (permalink / raw)
To: Pjotr Prins; +Cc: 22423-done
Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> When updating the commit hash to a different commit the git-fetch
> derivation *does* change (I checked in guile), but the checked out git
> tree in the store does not change - it gets shared between the
> commits. I am not sure why the tree gets shared, but the effect is
> that the same package gets installed using the same
> /gnu/store/xxx-git-checkout.
This is expected: origins are fixed-output derivations, meaning that it
does not matter how we perform them (using Git, over HTTP, or thanks to
an avian carrier), as long as the result has the specified sha256.
Thus, when you change, say, the Git commit ID or origin ‘method’ without
changing the ‘sha256’ field, nothing happens: the daemon says “OK, I
already have a store item with that ‘sha256’, so I don’t do anything.”
Clearly, one has to be cautious with this, it’s easy to mistakenly use
the old source.
Hope this clarifies things!
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#22423: git-fetch does not update checked out tree when commit hash changes
2016-01-21 8:50 ` Ludovic Courtès
@ 2016-01-21 9:08 ` Pjotr Prins
2016-01-21 10:10 ` Ludovic Courtès
0 siblings, 1 reply; 6+ messages in thread
From: Pjotr Prins @ 2016-01-21 9:08 UTC (permalink / raw)
To: 22423-done
On Thu, Jan 21, 2016 at 09:50:18AM +0100, Ludovic Courtès wrote:
> This is expected: origins are fixed-output derivations, meaning that it
> does not matter how we perform them (using Git, over HTTP, or thanks to
> an avian carrier), as long as the result has the specified sha256.
>
> Thus, when you change, say, the Git commit ID or origin ‘method’ without
> changing the ‘sha256’ field, nothing happens: the daemon says “OK, I
> already have a store item with that ‘sha256’, so I don’t do anything.”
>
> Clearly, one has to be cautious with this, it’s easy to mistakenly use
> the old source.
Hmmm. I thought the sha256 was calculated over the derivation +
sources, so any relevant change would trigger a build. Apparently it
is triggered by the sha256 field only. Good to know.
Pj.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#22423: git-fetch does not update checked out tree when commit hash changes
2016-01-21 9:08 ` Pjotr Prins
@ 2016-01-21 10:10 ` Ludovic Courtès
2016-01-21 11:50 ` Jookia
0 siblings, 1 reply; 6+ messages in thread
From: Ludovic Courtès @ 2016-01-21 10:10 UTC (permalink / raw)
To: Pjotr Prins; +Cc: 22423-done
Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> On Thu, Jan 21, 2016 at 09:50:18AM +0100, Ludovic Courtès wrote:
>> This is expected: origins are fixed-output derivations, meaning that it
>> does not matter how we perform them (using Git, over HTTP, or thanks to
>> an avian carrier), as long as the result has the specified sha256.
>>
>> Thus, when you change, say, the Git commit ID or origin ‘method’ without
>> changing the ‘sha256’ field, nothing happens: the daemon says “OK, I
>> already have a store item with that ‘sha256’, so I don’t do anything.”
>>
>> Clearly, one has to be cautious with this, it’s easy to mistakenly use
>> the old source.
>
> Hmmm. I thought the sha256 was calculated over the derivation +
> sources
What you’re saying is true of the hash that appears in /gnu/store file
name, but I was referring to the ‘sha256’ field of origins, which is a
different thing.
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#22423: git-fetch does not update checked out tree when commit hash changes
2016-01-21 10:10 ` Ludovic Courtès
@ 2016-01-21 11:50 ` Jookia
2016-01-21 16:25 ` Ludovic Courtès
0 siblings, 1 reply; 6+ messages in thread
From: Jookia @ 2016-01-21 11:50 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 22423-done
On Thu, Jan 21, 2016 at 11:10:14AM +0100, Ludovic Courtès wrote:
> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>
> > On Thu, Jan 21, 2016 at 09:50:18AM +0100, Ludovic Courtès wrote:
> >> This is expected: origins are fixed-output derivations, meaning that it
> >> does not matter how we perform them (using Git, over HTTP, or thanks to
> >> an avian carrier), as long as the result has the specified sha256.
> >>
> >> Thus, when you change, say, the Git commit ID or origin ‘method’ without
> >> changing the ‘sha256’ field, nothing happens: the daemon says “OK, I
> >> already have a store item with that ‘sha256’, so I don’t do anything.”
> >>
> >> Clearly, one has to be cautious with this, it’s easy to mistakenly use
> >> the old source.
> >
> > Hmmm. I thought the sha256 was calculated over the derivation +
> > sources
>
> What you’re saying is true of the hash that appears in /gnu/store file
> name, but I was referring to the ‘sha256’ field of origins, which is a
> different thing.
>
> Ludo’.
I think this is a bit of a problem even if it's expected: Often times we can't
calculate the hash until it's downloaded and get a hash mismatch. The other day
I rebuilt NixOS almost entirely on my machine and changed the revision on
Firefox to a new branch but didn't change the hash since I expected a mismatch.
Needles to say I realized what happened when I checked Firefox's version. I
think it'd be great to have a 'INVALID' hash we can use for development that
just prints a mismatch and errors out like usual. Maybe this is possible in
Guix, but it didn't seem documented and it's not possible in NixOS.
Cheers,
Jookia.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#22423: git-fetch does not update checked out tree when commit hash changes
2016-01-21 11:50 ` Jookia
@ 2016-01-21 16:25 ` Ludovic Courtès
0 siblings, 0 replies; 6+ messages in thread
From: Ludovic Courtès @ 2016-01-21 16:25 UTC (permalink / raw)
To: Jookia; +Cc: 22423-done
Jookia <166291@gmail.com> skribis:
> On Thu, Jan 21, 2016 at 11:10:14AM +0100, Ludovic Courtès wrote:
>> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>>
>> > On Thu, Jan 21, 2016 at 09:50:18AM +0100, Ludovic Courtès wrote:
>> >> This is expected: origins are fixed-output derivations, meaning that it
>> >> does not matter how we perform them (using Git, over HTTP, or thanks to
>> >> an avian carrier), as long as the result has the specified sha256.
>> >>
>> >> Thus, when you change, say, the Git commit ID or origin ‘method’ without
>> >> changing the ‘sha256’ field, nothing happens: the daemon says “OK, I
>> >> already have a store item with that ‘sha256’, so I don’t do anything.”
>> >>
>> >> Clearly, one has to be cautious with this, it’s easy to mistakenly use
>> >> the old source.
>> >
>> > Hmmm. I thought the sha256 was calculated over the derivation +
>> > sources
>>
>> What you’re saying is true of the hash that appears in /gnu/store file
>> name, but I was referring to the ‘sha256’ field of origins, which is a
>> different thing.
>>
>> Ludo’.
>
> I think this is a bit of a problem even if it's expected: Often times we can't
> calculate the hash until it's downloaded and get a hash mismatch. The other day
> I rebuilt NixOS almost entirely on my machine and changed the revision on
> Firefox to a new branch but didn't change the hash since I expected a mismatch.
>
> Needles to say I realized what happened when I checked Firefox's version. I
> think it'd be great to have a 'INVALID' hash we can use for development that
> just prints a mismatch and errors out like usual. Maybe this is possible in
> Guix, but it didn't seem documented and it's not possible in NixOS.
The problem is that Guix cannot guess that the ‘sha256’ the developer
provided in the ‘origin’ form is actually wrong. Only the developer
knows.
This is mitigated in Guix by the fact that we usually include the
version string or Git commit ID in the origin’s file name, with patterns
like:
(define foo
(let ((commit "deadbeef"))
(package
(name "foo")
(version commit)
(source (origin
(method git-fetch)
(uri (git-reference (commit commit) …))
(file-name (string-append name "-" commit)) ;<– !
(sha256 …)
…))
…)))
The good thing with this pattern is that, when you modify the value of
‘commit’, you also end up modifying ‘file-name’, thus triggering a
re-download. So if you had forgotten to update ‘sha256’, you
immediately get a hash mismatch error.
In practice, I’ve found this to be rather helpful.
Additionally, ‘guix lint’ emits warnings for packages that do not follow
this pattern, hopefully making the problem that Pjotr describes less
likely.
Ludo’.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-01-21 16:26 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-21 6:54 bug#22423: git-fetch does not update checked out tree when commit hash changes Pjotr Prins
2016-01-21 8:50 ` Ludovic Courtès
2016-01-21 9:08 ` Pjotr Prins
2016-01-21 10:10 ` Ludovic Courtès
2016-01-21 11:50 ` Jookia
2016-01-21 16:25 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).