Re: On raw strings in <origin> commit field

[Timothy Sample <samplet@ngyro.com>]

Hey,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Since you are our expert on preservation, would you mind if I ask you
> for some estimates on how painful it is to track down such commits in
> general, if it could be made easier were you to record tag → commit
> (alternatively file-name x sha256 → SWHID) maps periodically (or if you
> already have such a map and those arise while creating it), and how
> many “Tricking Peer Review”-style problems you think are currently
> around?

I haven’t been keeping a detailed log of issues or anything, but I have
some notes and recollections.  As of last month [1], we have 9,554 Git
sources (fixed-output derivations lowered from ‘git-reference’ origins).
Of those, 186 could not be recovered automatically (by simply cloning
the repo or, for about 100 cases with the commit hash, checking SWH).
Most of the 186 have ‘(recursive? #t)’, which is something I haven’t
implemented yet (there’s no Guix fallback support for it either).
However, there are 51 of those that should just work but don’t.

It turns out that most of these are due to my scripts ignoring the wrong
kind of VCS files (like ignoring “.hg”) when hashing.  My scripts follow
the logic of ‘guix hash -S nar -x .’, but Guix actually just deletes the
Git metadata: ‘rm -rf .git; guix hash -S nar .’.  :)

Another couple are <https://issues.guix.gnu.org/48540>.

There were a handful of mutated tags (around a dozen).  Some of them
were deleted, but the tag name referred to the commit hash (as if the
tag was named by ‘git describe’).  Some of them were changed, but it was
clear that the original tag was just a few commits back.  There was only
one I couldn’t figure out:

    https://github.com/jurplel/qView.git
    at tag 2.0
    with hash 1s29hz44rb5dwzq8d4i4bfg77dr0v3ywpvidpa6xzg7hnnv3mhi5

A similar problem is when the repo URL changes, but the tags are still
the same when you track down another copy of the repo.  I encountered
this a few times.

Another handful (again around a dozen) were hash mistakes in the style
of tricking peer review.  In most cases our commit messages were clear
enough to figure out what the hash was actually for.  There are two
mysterious cases:

    https://git.umaneti.net/flycheck-grammalecte/
    at tag v1.3
    with hash 1f1gapvs9j89qr474103dqgsiyb96phlnsmq5hiv4ba242blg9lb
    (see Guix commit ca5a791f6285b08506ccd662d5911ccf0c4d1ece)

    https://github.com/fdik/libetpan
    at commit 210ba2b3b310b8b7a6ee4a4e35e50f7fa379643f
    with hash 00000nij3ray7nssvq0lzb352wmnab8ffzk7dgff2c68mvjbh1l6
    (the hash kinda looks fake, but it was like that for a long time)

There are two other cases that are basically “typos” in the hash.  One
is clearly just an edit to the hash to make the build fail and print the
correct hash (see commits 618df2e335acb49a27ca014b555ede34f79503f3 and
bdc7f72fe4391ede313a0388ddd17cbb053931c9).  The other one is commit
c0dc4179091f85fe4b8a2bbdb07c154a7f0408ed, which changes the hash of the
package ‘zimg’ without mentioning anything about it in the commit
message.  This is fixed in b08c4f5fceff6064baedea3385703689b8a72e47
(back to the original hash).  Tobias might remember what happened there,
but it looks like an honest mistake to me.  I have no clue what that
other hash was for.

Note for all of this that my scripts treat the SHA256 hash as *the*
identifier for a source.  That is, if a tag is mutated and a someone
adjusts the origin URI to point to the commit that the tag used to refer
to, I would not notice.  Similarly, for tricking peer review: fixing the
URI to match the hash is invisible to me.  It’s only when we fix the
hash to match the URI that I notice.

See also zimoun’s analysis of the same thing, but with older data:
<https://lists.gnu.org/archive/html/guix-devel/2021-12/msg00032.html>.

[1] https://ngyro.com/pog-reports/2021-12-06/

> Am Samstag, dem 01.01.2022 um 12:45 -0500 schrieb Timothy Sample:
>
>> Given what I wrote above, maybe we could start by updating the linter
>> so that ‘check-source’ actually checks that it gets the right result.
>> Right now it uses a few heuristics to check that the result looks
>> okay (for instance, it checks if the result is suspiciously small). 
>> Maybe it should just go through the whole download process and verify
>> the hash?  Alternatively (or additionally), the CI “source”
>> specification could be configured to avoid using our servers as a
>> fallback when checking sources.
>
> I think substitutes should be disabled for the source download of a
> "check-source".  Even if a substitute or SWH fallback exists, that's
> not what we want to check here, no?

Exactly.  It should just fetch the source as naïvely as possible, akin
to ‘GUIX_DOWNLOAD_FALLBACK_TEST=none guix build --check -S ...’ (or with
‘--substitute-urls=""’ or whatever).

>> I agree that adding more identifiers (commit hashes or whatever)
>> makes things more robust, but the cost is more work when creating,
>> updating, and reviewing packages.  I think we should start by
>> verifying the identifiers we already have (i.e., checking that the
>> URI and method of the origin produce the right output).  It would
>> solve many existing problems and would serve as a nice foundation for
>> future improvements.
>
> Is this something we can reasonably expect our current CI or CI in
> general to handle (assuming we tweaked the linter to behave as you
> intend?)  Or would it make more sense to implement this as a
> weekly/monthly cronjob?

I really only mentioned the CI because I had to explain to myself why it
didn’t notice the problem.  I think the linter is probably the better
place to improve things here.  It’s something I’m willing to work on,
but I would need to understand why it doesn’t check the hash already.
It seems like one of those things someone may have already thought about
and decided against.


-- Tim