Re: On raw strings in <origin> commit field

[Liliana Marie Prikler <liliana.prikler@gmail.com>]

Am Freitag, dem 31.12.2021 um 20:41 -0500 schrieb Mark H Weaver:
> I disagree with the last line above.  What makes you think that I'm
> presupposing that the tag does change?
> 
> There's a difference between "presupposing that the tag does change"
> and "not assuming that the tag will not change".  Do you see the
> difference?
I'm pretty sure ¬assume(¬X) = assume(¬¬X) in this concept.  You have to
start with some assumptions and while ideally we'd like to encode "I
don't care", we do not have a system that allows us to do so.

> > However, if we are always talking about more than one possible
> > "1.2.3" (with the included future tag that we have yet to witness),
> > we lose the basis by which we currently assign "1.2.3" as the
> > version 
> 
> I see what you're getting at here, but still I disagree.  Our basis
> for associating version "1.2.3" with commit XYZ is simply that
> upstream had indicated that version "1.2.3" was commit XYZ.  That
> historical fact is immutable.
History is a social construct, it's not immutable.

> If upstream later indicates that version "1.2.3" is now commit YYZ, I
> don't think that invalidates our basis for continuing to associate
> version "1.2.3" with commit XYZ.  The aforementioned immutable
> historical fact still remains our basis and justification for making
> that association.
I'm pretty sure it does, particularly to a future observer who may not
have the luxury of a history to distinguish that record from one in
which a malicious committer linked those versions and tag together and
then no one bothered to check.

> Perhaps some people would prefer to use a distro where version
> "1.2.3" of package FOO could mean a different thing tomorrow than it
> means today.  Personally, that's not what I want.
> 
> If upstream changes their mind about the meaning of version "1.2.3",
> I want that to correspond to a different version number in Guix,
> perhaps "1.2.3a" or something, as Taylan suggested.  Incidentally, I
> vaguely recall that we've done that in the past, but I don't know if
> we've done it consistently.
The entire point here is to use git-version in combination with let-
bound commit hashes, yes.

> > As pointed out elsewhere, SWH keeps a history of the tags that we
> > could look up until one matches,
> 
> Only if SWH took a snapshot at the right time.  I would guess that
> mutations of release tags usually happen within a few days after the
> release tag is first created.  Relying on SWH to take a snapshot
> within that possibly quite small time interval doesn't sound very
> robust to me.
If the scenario is "a few days within release" vs "literally forever",
there would for one only be a relatively short range of bad Guix
versions having a broken reference (limiting impact) and for another,
the majority of the audience would also associate the latter with said
version.  There's really no good argument from the robustness side to
be had here.

> > and there'd also be the option to keep a secondary index ourselves
> > (or have a third party do it).
> 
> That's true, but then we'd be adding another piece of centralized
> infrastructure that users would need to rely upon in order to
> reliably reproduce their systems.  That infrastructure would have to
> be maintained indefinitely.  If we failed to keep up maintenance, the
> users could run into problems reproducing their older systems.
> 
> It seems to me clearly better to avoid relying on a piece of
> centralized infrastructure if it can be easily avoided, no?
We can make that a key-value store for which you write a distributed
MapReduce function in Erlang if it makes you happier.

> > > On the other hand, if we refer to git _commit hashes_, then it
> > > *is* feasible for us to fetch the archived source from SWH,
> > > regardless of what upstream has done to its tags in the meantime.
> > > 
> > > For that reason alone, I think that the way Ricardo wrote the
> > > guile-aiscm package definition is clearly the right approach,
> > > given Guix's longstanding goals.
> > To me, it rather sounds like a workaround for longstanding bugs [1,
> > 2].
> [...]
> > [1] https://issues.guix.gnu.org/28659
> > [2] https://issues.guix.gnu.org/39575
> 
> I don't understand how it's a workaround for those bugs.  Even if
> those bugs were fixed, we'd still need a reliable way to find the git
> commit that matches the one expected by a git-fetch <origin> record,
> i.e. the one that will produce a source checkout with the expected
> SHA256 hash.
> 
> Am I missing something?
We are working on the base assumption here, that we have an (array of)
reachable fallbacks in any case, I don't think it's too big of a leap
to assume that we can keep a mapping 
  (origin-file-name x origin-hash) → canonicalized-uri
around either as part of said fallbacks or in parallel.

> Regarding "Tricking Peer Review": I think it would be ideal for
> package definitions to include both the git tag _and_ the git commit
> hash, and to teach our linter to raise an alarm when the expected
> tags are missing or fail to match the expected commit hash.
That is among the solutions I've proposed here, so naturally I'd be
fine with it.

> For similar reasons, it would also be good to include the
> fingerprints of upstream PGP signing keys in our package definitions,
> and to teach our linter to check those signatures and that they match
> the SHA256 hashes in our recipes.
> 
> What do you think?
I think that is one of the main things we could import over from Guix
for Racket users (previously Xiden, currently denxi).  I.e. we could
have 

  (origin
    ...
    (sha256 some-hash)
    (sha512 some-other-hash)
    (pgp-signature sig)
    [other validation forms...]
    [patches and snippet])

We would have to break record ABI for that, but imo with field
sanitizers that's something we could code up.  If at some time in the
future SHA-2 is broken, we can then still rely on the robustness that
breaking all of these hashes would be difficult and perhaps not worth
it for GNU Hello.

Now obviously, there is a performance tradeoff here.  You don't want to
only check signatures all the time for a relatively minor build
(particularly with Rust where the build phase is literally copy-paste
for 90% of the packages).  So we'd have to add a configuration option
on the sliding scale between "only check the weakest" over "only check
the strongest" over "check at least N at random or all of them" to
"check everything always".

Cheers