Tricking peer review

Tricking peer review

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 74 bytes --]

Hello,

Consider this file as if it were a patch you’re reviewing:


[-- Attachment #2: source --]
[-- Type: text/plain, Size: 1094 bytes --]

(define-module (content-addressed))
(use-modules (guix)
             (guix build-system gnu)
             (guix licenses)
             (gnu packages perl))

(define-public sed
  (package
   (name "sed")
   (version "4.8")
   (source (origin
            (method url-fetch)
            (uri (string-append "mirror://gnu/zed/sed-" version
                                ".tar.gz"))
            (sha256
             (base32
              "1yy33kiwrxrwj2nxa4fg15bvmwyghqbs8qwkdvy5phm784f7brjq"))))
   (build-system gnu-build-system)
   (synopsis "Stream editor")
   (native-inputs
    `(("perl" ,perl)))                            ;for tests
   (description
    "Sed is a non-interactive, text stream editor.  It receives a text
input from a file or from standard input and it then applies a series of text
editing commands to the stream and prints its output to standard output.  It
is often used for substituting text patterns in a stream.  The GNU
implementation offers several extensions over the standard utility.")
   (license gpl3+)
   (home-page "https://www.gnu.org/software/sed/")))

sed

[-- Attachment #3: Type: text/plain, Size: 4238 bytes --]


It builds just fine:

--8<---------------cut here---------------start------------->8---
$ guix build -f /tmp/content-addressed.scm  
/gnu/store/lpais26sjwxcyl7y7jqns6f5qrbrnb34-sed-4.8
$ guix build -f /tmp/content-addressed.scm -S --check -v0
/gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz
--8<---------------cut here---------------end--------------->8---

Did you spot a problem?

…

\f
So, what did we just build?

--8<---------------cut here---------------start------------->8---
$ ls $(guix build -f /tmp/content-addressed.scm)/bin
egrep  fgrep  grep
--8<---------------cut here---------------end--------------->8---

Oh oh!  This ‘sed’ package is giving us ‘grep’!  How come?

The trick is easy: we give a URL that’s actually 404, with the hash of a
file that can be found on Software Heritage (in this case, that of
‘grep-3.4.tar.xz’).  When downloading the source, the automatic
content-addressed fallback kicks in, and voilà:

--8<---------------cut here---------------start------------->8---
$ guix build -f /tmp/content-addressed.scm  -S --check 
La jena derivaĵo estos konstruata:
   /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
building /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv...

Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-4.8.tar.gz
From https://ftpmirror.gnu.org/gnu/zed/sed-4.8.tar.gz...
following redirection to `https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz'...
download failed "https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz" 404 "Not Found"

[...]

Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-4.8.tar.gz
From https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/...
downloading from https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/ ...

warning: rewriting hashes in `/gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz'; cross fingers
successfully built /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
--8<---------------cut here---------------end--------------->8---

It’s nothing new, it’s what I do when I want to test the download
fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it could
somehow be abused to have malicious packages pass review.

Granted, ‘guix lint’ immediately flags the issue:

--8<---------------cut here---------------start------------->8---
$ guix lint -L /tmp/p sed
guix lint: warning: plursenca pak-specifigo 'sed'
guix lint: warning: ni elektas sed@4.8 el /tmp/content-addressed.scm:8:2
/tmp/content-addressed.scm:11:11: sed@4.8: all the source URIs are unreachable:
/tmp/content-addressed.scm:11:11: sed@4.8: URI https://ftpmirror.gnu.org/gnu/zed/sed-4.8.tar.gz ne estas alirebla: 404 ("Not Found")
/tmp/content-addressed.scm:11:11: sed@4.8: URI ftp://ftp.cs.tu-berlin.de/pub/gnu/zed/sed-4.8.tar.gz domajno ne trovita: Nomo aŭ servo ne konatas
/tmp/content-addressed.scm:11:11: sed@4.8: URI ftp://ftp.funet.fi/pub/mirrors/ftp.gnu.org/gnu/zed/sed-4.8.tar.gz ne estas alirebla: 550 ("Can't change directory to zed: No such file or directory")
/tmp/content-addressed.scm:11:11: sed@4.8: URI http://ftp.gnu.org/pub/gnu/zed/sed-4.8.tar.gz ne estas alirebla: 404 ("Not Found")
--8<---------------cut here---------------end--------------->8---

Also, just because a URL looks nice and is reachable doesn’t mean the
source is trustworthy either.  An attacker could submit a package for an
obscure piece of software that happens to be malware.  The difference
here is that the trick above would allow targeting a high-impact
package.

On the plus side, such an attack would be recorded forever in Git
history.

Also on the plus side, it turns out our origin URLs are currently
(unintentionally) limited to ASCII, so I couldn’t write “/ṡed” in the
URL.


All in all, it’s probably not as worrisome as it first seems.  However,
it’s worth keeping in mind when reviewing a package.

Thoughts?

Ludo’.

Re: Tricking peer review

[Liliana Marie Prikler]

Hi,

Am Freitag, den 15.10.2021, 20:54 +0200 schrieb Ludovic Courtès:
> Hello,
> 
> Consider this file as if it were a patch you’re reviewing:

> (define-module (content-addressed))
> (use-modules (guix)
>              (guix build-system gnu)
>              (guix licenses)
>              (gnu packages perl))
> 
> (define-public sed
>   (package
>    (name "sed")
>    (version "4.8")
>    (source (origin
>             (method url-fetch)
>             (uri (string-append "mirror://gnu/zed/sed-" version
>                                 ".tar.gz"))
To be fair, gnu/zed sounds wonky, but you could try inserting a version
that does not exist (e.g. 1+ the current latest version) and as a
committer thereby bypass review entirely.  However, given that we trust
committers in this aspect, I'd say they should be able to verify both
URI and version field.  This is trivially possible with most schemes
safe for the mirror:// one.
>             (sha256
>              (base32
>               "1yy33kiwrxrwj2nxa4fg15bvmwyghqbs8qwkdvy5phm784f7brjq")
> )))
>    (build-system gnu-build-system)
>    (synopsis "Stream editor")
>    (native-inputs
>     `(("perl" ,perl)))                            ;for tests
>    (description
>     "Sed is a non-interactive, text stream editor.  It receives a
> text
> input from a file or from standard input and it then applies a series
> of text
> editing commands to the stream and prints its output to standard
> output.  It
> is often used for substituting text patterns in a stream.  The GNU
> implementation offers several extensions over the standard utility.")
>    (license gpl3+)
>    (home-page "https://www.gnu.org/software/sed/")))
> 
> sed

> It builds just fine:
> 
> --8<---------------cut here---------------start------------->8---
> $ guix build -f /tmp/content-addressed.scm  
> /gnu/store/lpais26sjwxcyl7y7jqns6f5qrbrnb34-sed-4.8
> $ guix build -f /tmp/content-addressed.scm -S --check -v0
> /gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz
> --8<---------------cut here---------------end--------------->8---
> 
> Did you spot a problem?
> 
> …
> 
> 
> So, what did we just build?
> 
> --8<---------------cut here---------------start------------->8---
> $ ls $(guix build -f /tmp/content-addressed.scm)/bin
> egrep  fgrep  grep
> --8<---------------cut here---------------end--------------->8---
> 
> Oh oh!  This ‘sed’ package is giving us ‘grep’!  How come?
> 
> The trick is easy: we give a URL that’s actually 404, with the hash
> of a file that can be found on Software Heritage (in this case, that
> of ‘grep-3.4.tar.xz’).  When downloading the source, the automatic
> content-addressed fallback kicks in, and voilà:
> 
> --8<---------------cut here---------------start------------->8---
> $ guix build -f /tmp/content-addressed.scm  -S --check 
> La jena derivaĵo estos konstruata:
>    /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
> building /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-
> 4.8.tar.gz.drv...
> 
> Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-
> 4.8.tar.gz
> > From https://ftpmirror.gnu.org/gnu/zed/sed-4.8.tar.gz...
> following redirection to `
> https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz'...
> download failed "https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz"
> 404 "Not Found"
> 
> [...]
> 
> Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-
> 4.8.tar.gz
> > From 
> > https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/
> > ...
> downloading from 
> https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/
> ...
> 
> warning: rewriting hashes in
> `/gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz'; cross
> fingers
> successfully built /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-
> 4.8.tar.gz.drv
> --8<---------------cut here---------------end--------------->8---
> 
> It’s nothing new, it’s what I do when I want to test the download
> fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
> c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it
> could somehow be abused to have malicious packages pass review.
I don't think this is much of a problem for packages where we have
another source of truth (in this case mirrors/archives of sed), but it
does point at a bigger problem when SWH is our only source of truth. 
I.e. when trying to conserve such software for the future, when other
archives might fail and perhaps SHA256 itself might be broken, we can
no longer be sure that the Guix time-machine indeed does what it
promises.

> Also, just because a URL looks nice and is reachable doesn’t mean the
> source is trustworthy either.  An attacker could submit a package for
> an obscure piece of software that happens to be malware.  The
> difference here is that the trick above would allow targeting a high-
> impact package.
Again, less of an issue w.r.t. review because the reviewers can at
review time check that the tarball matches their expectations.  I
personally find "I can't find this source anywhere but on SWH" to be a
perfect reason to reject software in the main Guix channel, though
perhaps that rule is a bit softer in Guix Past.

> On the plus side, such an attack would be recorded forever in Git
> history.
On the minus side, time-machine makes said record a landmine to step
into.

> Also on the plus side, it turns out our origin URLs are currently
> (unintentionally) limited to ASCII, so I couldn’t write “/ṡed” in the
> URL.
Couldn't one circumvent that with percent encoding and a nice enough
file-name, however?

> All in all, it’s probably not as worrisome as it first
> seems.  However, it’s worth keeping in mind when reviewing a package.
> 
> Thoughts?
I agree, that cross-checking “guix download” might be good praxis for
review.  Perhaps in light of this we should extend it to Git/SVN/other
VCS?

Regards,
Liliana

Re: Tricking peer review

[Ryan Prior]

On Friday, October 15th, 2021 at 10:03 PM, Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:

> > On the plus side, such an attack would be recorded forever in Git
> >
> > history.
>
> On the minus side, time-machine makes said record a landmine to step
>
> into.

I've suggested this before and this seems like a good time to bring it up again: can we create a database of known "bad" Guix commit hashes, and make time-machine fetch the list and warn before it'll visit one of those hashes? This would resolve the land-mine problem and generally de-risk our git tree, which is maintained by fallible volunteers who will occasionally push tragic commits.

Re: Tricking peer review

[Liliana Marie Prikler]

Am Freitag, den 15.10.2021, 22:28 +0000 schrieb Ryan Prior:
> On Friday, October 15th, 2021 at 10:03 PM, Liliana Marie Prikler <
> liliana.prikler@gmail.com> wrote:
> 
> > > On the plus side, such an attack would be recorded forever in Git
> > > 
> > > history.
> > 
> > On the minus side, time-machine makes said record a landmine to
> > step
> > 
> > into.
> 
> I've suggested this before and this seems like a good time to bring
> it up again: can we create a database of known "bad" Guix commit
> hashes, and make time-machine fetch the list and warn before it'll
> visit one of those hashes? This would resolve the land-mine problem
> and generally de-risk our git tree, which is maintained by fallible
> volunteers who will occasionally push tragic commits.
I don't think things would be quite as simple.  A "bad" commit might
still be perfectly fine to fetch certain things from if they're
unaffected by it, plus you're now tasked with the job of keeping the
list of bad commits safe somehow.  In some situations resetting a
branch might work, but obviously not for months old sleeper commits.

Re: Tricking peer review

[Ryan Prior]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> A "bad" commit might still be perfectly fine to fetch certain things from if they're unaffected by it

The database could store a comment with each "bad" commit hash to help people decide if they're affected. It could even go further and include a list of tainted packages, so you could programmatically determine whether one of them is in your dependency tree.

> you're now tasked with the job of keeping the list of bad commits safe somehow.

Right now afaik Ludovic's key is the root of trust (is this still true?) so I imagine we'd sign the list too, with that key or some other key signed by it.

> In some situations resetting a branch might work, but obviously not for months old sleeper commits.

Not sure what you mean by this, can you explain?

Re: Tricking peer review

[Thiago Jung Bauermann]

Hello,

Em sexta-feira, 15 de outubro de 2021, às 19:03:22 -03, Liliana Marie 
Prikler escreveu:
> Am Freitag, den 15.10.2021, 20:54 +0200 schrieb Ludovic Courtès:
> > Consider this file as if it were a patch you’re reviewing:
> > 
> > (define-module (content-addressed))
> > (use-modules (guix)
> > 
> >              (guix build-system gnu)
> >              (guix licenses)
> >              (gnu packages perl))
> > 
> > (define-public sed
> > 
> >   (package
> >   
> >    (name "sed")
> >    (version "4.8")
> >    (source (origin
> >    
> >             (method url-fetch)
> >             (uri (string-append "mirror://gnu/zed/sed-" version
> >             
> >                                 ".tar.gz"))
> 
> To be fair, gnu/zed sounds wonky, but you could try inserting a version
> that does not exist (e.g. 1+ the current latest version) and as a
> committer thereby bypass review entirely.  However, given that we trust
> committers in this aspect, I'd say they should be able to verify both
> URI and version field.  This is trivially possible with most schemes
> safe for the mirror:// one.
> 
> >             (sha256
> >             
> >              (base32
> >              
> >               "1yy33kiwrxrwj2nxa4fg15bvmwyghqbs8qwkdvy5phm784f7brjq")
> > 
> > )))
> > 
> >    (build-system gnu-build-system)
> >    (synopsis "Stream editor")
> >    (native-inputs
> >    
> >     `(("perl" ,perl)))                            ;for tests
> >    
> >    (description
> >    
> >     "Sed is a non-interactive, text stream editor.  It receives a
> > 
> > text
> > input from a file or from standard input and it then applies a series
> > of text
> > editing commands to the stream and prints its output to standard
> > output.  It
> > is often used for substituting text patterns in a stream.  The GNU
> > implementation offers several extensions over the standard utility.")
> > 
> >    (license gpl3+)
> >    (home-page "https://www.gnu.org/software/sed/")))
> > 
> > sed
> > 
> > It builds just fine:
> > 
> > --8<---------------cut here---------------start------------->8---
> > $ guix build -f /tmp/content-addressed.scm
> > /gnu/store/lpais26sjwxcyl7y7jqns6f5qrbrnb34-sed-4.8
> > $ guix build -f /tmp/content-addressed.scm -S --check -v0
> > /gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz
> > --8<---------------cut here---------------end--------------->8---
> > 
> > Did you spot a problem?
> > 
> > …
> > 
> > 
> > So, what did we just build?
> > 
> > --8<---------------cut here---------------start------------->8---
> > $ ls $(guix build -f /tmp/content-addressed.scm)/bin
> > egrep  fgrep  grep
> > --8<---------------cut here---------------end--------------->8---
> > 
> > Oh oh!  This ‘sed’ package is giving us ‘grep’!  How come?
> > 
> > The trick is easy: we give a URL that’s actually 404, with the hash
> > of a file that can be found on Software Heritage (in this case, that
> > of ‘grep-3.4.tar.xz’).  When downloading the source, the automatic
> > content-addressed fallback kicks in, and voilà:
> > 
> > --8<---------------cut here---------------start------------->8---
> > $ guix build -f /tmp/content-addressed.scm  -S --check
> > 
> > La jena derivaĵo estos konstruata:
> >    /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
> > 
> > building /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-
> > 4.8.tar.gz.drv...
> > 
> > Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-
> > 4.8.tar.gz
> > 
> > > From https://ftpmirror.gnu.org/gnu/zed/sed-4.8.tar.gz...
> > 
> > following redirection to `
> > https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz'...
> > download failed "https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz"
> > 404 "Not Found"
> > 
> > [...]
> > 
> > Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-
> > 4.8.tar.gz
> > 
> > > From
> > > https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a
> > > 7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/ ...
> > 
> > downloading from
> > https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c
> > 25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/ ...
> > 
> > warning: rewriting hashes in
> > `/gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz'; cross
> > fingers
> > successfully built /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-
> > 4.8.tar.gz.drv
> > --8<---------------cut here---------------end--------------->8---
> > 
> > It’s nothing new, it’s what I do when I want to test the download
> > fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
> > c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it
> > could somehow be abused to have malicious packages pass review.
> 
> I don't think this is much of a problem for packages where we have
> another source of truth (in this case mirrors/archives of sed), but it
> does point at a bigger problem when SWH is our only source of truth.
> I.e. when trying to conserve such software for the future, when other
> archives might fail and perhaps SHA256 itself might be broken, we can
> no longer be sure that the Guix time-machine indeed does what it
> promises.

I’ve been thinking lately that Guix {sh,c}ould have a new ’release-signing-
keys’ field in the package record which would list the keys that are known 
to sign official releases of the package. Then Guix would check the tarball/
git commit/git tag when downloading it. It would be an additional (and IMHO 
important) source of truth.

There are details that would need to be hashed out such as how to deal with 
revoked keys or whether to store the keys themselves on the Guix repo or 
anywhere else in Guix’s infrastructure, but I think it’s possible to arrive 
at a reasonable solution.

Not all projects sign their release artifacts, but some do and it would be 
nice to take advantage of that.

-- 
Thanks,
Thiago

Re: Tricking peer review

[Ludovic Courtès]

Moin!

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> Am Freitag, den 15.10.2021, 20:54 +0200 schrieb Ludovic Courtès:

[...]

>> It’s nothing new, it’s what I do when I want to test the download
>> fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
>> c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it
>> could somehow be abused to have malicious packages pass review.
> I don't think this is much of a problem for packages where we have
> another source of truth (in this case mirrors/archives of sed), but it
> does point at a bigger problem when SWH is our only source of truth. 
> I.e. when trying to conserve such software for the future, when other
> archives might fail and perhaps SHA256 itself might be broken, we can
> no longer be sure that the Guix time-machine indeed does what it
> promises.

At the time a package is committed, its source is normally not
downloaded from SWH—at least that’s what we aim for, and ‘guix lint’
warns against 404 source URLs.  So when the package is reviewed and
committed, people can check the origin of the source, verify it against
published signatures when possible, and so on.

>> Also, just because a URL looks nice and is reachable doesn’t mean the
>> source is trustworthy either.  An attacker could submit a package for
>> an obscure piece of software that happens to be malware.  The
>> difference here is that the trick above would allow targeting a high-
>> impact package.
> Again, less of an issue w.r.t. review because the reviewers can at
> review time check that the tarball matches their expectations.  I
> personally find "I can't find this source anywhere but on SWH" to be a
> perfect reason to reject software in the main Guix channel, though
> perhaps that rule is a bit softer in Guix Past.

Right.  SWH is a fallback, meaning that, eventually, most source gets
downloaded from there (because the original hosting sites vanish); but
again, at the time of review, source must be available elsewhere.

>> On the plus side, such an attack would be recorded forever in Git
>> history.
> On the minus side, time-machine makes said record a landmine to step
> into.

That’s one way to look at it; the same could be said of unpatched
vulnerabilities found in old versions.  It remains that deploying from a
pinned Guix revision has its uses.

[...]

> I agree, that cross-checking “guix download” might be good praxis for
> review.

Reviewing includes at least building the package, thus downloading its
source, and running running ‘guix lint’.  So there’s nothing really new
here I guess,

> Perhaps in light of this we should extend it to Git/SVN/other VCS?

Thanks,
Ludo’.

Re: Tricking peer review

[Ludovic Courtès]

Hi Ryan,

Ryan Prior <rprior@protonmail.com> skribis:

> I've suggested this before and this seems like a good time to bring it
> up again: can we create a database of known "bad" Guix commit hashes,
> and make time-machine fetch the list and warn before it'll visit one
> of those hashes? This would resolve the land-mine problem and
> generally de-risk our git tree, which is maintained by fallible
> volunteers who will occasionally push tragic commits.

How would we define “bad” though?

By definition, packages found in past Guix revisions are missing
features, have unfixed bugs and unpatched security vulnerabilities.
Yet, pinning the Guix revision or re-deploying a past revision as-is has
its uses and I’d say it’s a key feature, even.

I don’t think there are “tragic” commits either.  Usually, one records a
revision that works for them and use the same months later.

Thanks,
Ludo’.

Re: Tricking peer review

[Ludovic Courtès]

Hello,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

> I’ve been thinking lately that Guix {sh,c}ould have a new ’release-signing-
> keys’ field in the package record which would list the keys that are known 
> to sign official releases of the package. Then Guix would check the tarball/
> git commit/git tag when downloading it. It would be an additional (and IMHO 
> important) source of truth.

Yes, it’s been discussed a few times and I agree it’d be nice.

The difficulty here is that it’s “silent” metadata: it’s not used, or at
least not necessarily used as part of the download process.  But maybe
that’s OK: we can have the download process check signatures when
possible.

Right now we rely on ‘guix refresh -u’ or contributors/reviewers do
perform this check.

> There are details that would need to be hashed out such as how to deal with 
> revoked keys or whether to store the keys themselves on the Guix repo or 
> anywhere else in Guix’s infrastructure, but I think it’s possible to arrive 
> at a reasonable solution.

Perhaps a first step would be to download keys opportunistically.

We have (guix openpgp) which can be used to verify signatures without
taking revocation into account.

Thanks,
Ludo’.

Re: Tricking peer review

[Ryan Prior]

On Monday, October 18th, 2021 at 7:40 AM, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> Hi Ryan,
> How would we define “bad” though?

A definition isn't necessary, this can be an "I know it when I see it" thing. If we have an oops or discover an issue, and say oh darn that lives in the repo forever now, we'd be able to leave a note to all who try in the future to visit impacted commits that all was not well.

Some of this is already captured by our CVE scanning feature, but other things (like your hypothetical "somebody snuck a bad `sed` in!") would benefit from yet more explanation. We don't need a perfect and complete definition of "bad" to agree that any commit where `sed` is actually `grep` (or malware) is a bad commit & merits a warning. This should not interfere with people who want to keep using their pinned version of Guix & aren't impacted by the bad package, which remains useful as you note.

Re: Tricking peer review

[zimoun]

Hi,

On Fri, 15 Oct 2021 at 20:54, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> --8<---------------cut here---------------start------------->8---
> $ guix build -f /tmp/content-addressed.scm  -S --check 
> La jena derivaĵo estos konstruata:
>    /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
> building /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv...
>
> Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-4.8.tar.gz
> From https://ftpmirror.gnu.org/gnu/zed/sed-4.8.tar.gz...
> following redirection to `https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz'...
> download failed "https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz" 404 "Not Found"
>
> [...]
>
> Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-4.8.tar.gz
> From https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/...
> downloading from https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/ ...
>
> warning: rewriting hashes in `/gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz'; cross fingers
> successfully built /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
> --8<---------------cut here---------------end--------------->8---

The message can be even more “spottable” than the URL
’archive.softwareheritage.org’ if the vault requires a cooking.  One
will see «SWH vault: requested bundle cooking, waiting for
completion...» or «SWH vault: retrying...».

Yeah, it is hidden with the option ’v0’ but it is the user’s
responsibility to check, IMHO.

> It’s nothing new, it’s what I do when I want to test the download
> fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
> c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it could
> somehow be abused to have malicious packages pass review.

Yes, it is nothing new.  Somehow, it is an issue with any
content-addressed system.  Here, we need to split the issue depending on
the origins:

 - url-fetch: the attacker has to introduce the tarballs into SWH.
   There is not so much means, from my understanding: SWH ingests
   tarballs via loaders, for instance gnu.org or sources.json or
   debian.org etc. Therefore the attacker has to introduce the malicious
   code to these platforms.

 - url-fetch without metadata (as your example), indeed, the reviewer
   could be abused; mitigated by the fact that “guix lint” spots the
   potential issue.

 - url-fetch with metadata: the attacker have to also corrupt
   Diasarchive-DB.   Otherwise, the tarball returned by SWH will not
   match the checksum.

 - svn-fetch, hg-fetch, cvs-fetch: no attack possible, yet.

 - git-fetch: it is the *real* issue.  Because it is easy for the
   attacker to introduce malicious code into SWH (create a repo on
   GitHub, click Save, done).  Then submit a package using it as you
   did.  It is the same case as url-fetch without metadata but easier
   for the attacker.  It is mitigated by “guix lint”.

That’s said, if I am an attacker and I would like to corrupt Guix, then
I would create a fake project mimicking a complex software.  For
instance, Gmsh is a complex C++ scientific software.  The correct URL is
<https://gmsh.info> and the source at
<https://gitlab.onelab.info/gmsh/gmsh>.  Then, as an attacker, I buy the
domain say gmsh.org and put a malicious code there.  Last, I send for
inclusion a package using this latter URL.  The reviewer would be
abused.

That’s why more eyes, less issues. :-)
   

> Also, just because a URL looks nice and is reachable doesn’t mean the
> source is trustworthy either.  An attacker could submit a package for an
> obscure piece of software that happens to be malware.  The difference
> here is that the trick above would allow targeting a high-impact
> package.

I agree.

> On the plus side, such an attack would be recorded forever in Git
> history.

I agree again. :-)

> All in all, it’s probably not as worrisome as it first seems.  However,
> it’s worth keeping in mind when reviewing a package.

I agree with a minor comment.  From my opinion, not enough patches are
going via guix-patches and are pushed directly.

For instance, the «Commit policy» section says «For patches that just
add a new package, and a simple one, it’s OK to commit, if you’re
confident (which means you successfully built it in a chroot setup, and
have done a reasonable copyright and license auditing).»

And from my point of view, new packages should *always* go via
guix-patches, wait 15 days, then push if no remark.  It lets the time
for the community to chime in.  And if not, it just slows down for 2
weeks.

Cheers,
simon

Re: Tricking peer review

[zimoun]

Hi Ludo,

On Mon, 18 Oct 2021 at 09:40, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> I don’t think there are “tragic” commits either.  Usually, one records a
> revision that works for them and use the same months later.

Drifting from the initial comment.  One could name “tragic” commits are
commits which break “guix pull”.  It is rare these days but there are
some reachable ones via “guix time-machine”.


Cheers,
simon

Re: Tricking peer review

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> I agree with a minor comment.  From my opinion, not enough patches are
> going via guix-patches and are pushed directly.
>
> For instance, the «Commit policy» section says «For patches that just
> add a new package, and a simple one, it’s OK to commit, if you’re
> confident (which means you successfully built it in a chroot setup, and
> have done a reasonable copyright and license auditing).»
>
> And from my point of view, new packages should *always* go via
> guix-patches, wait 15 days, then push if no remark.  It lets the time
> for the community to chime in.  And if not, it just slows down for 2
> weeks.

Three comments: (1) two weeks for a trivial Python/R/C package (the
“simple one” rule above) can be a lot, (2) committers are by definition
trusted to not mess up and to clean up behind them, and (3) when a
simple package is pushed there’s after-the-fact peer review in practice
(sometimes I reply to guix-commits notifications, for example.)

I think it’s about finding the right balance to be reasonably efficient
while not compromising on quality.

Ludo’.

Re: Tricking peer review

[zimoun]

Hi Ludo,

On Tue, 19 Oct 2021 at 14:56, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> I agree with a minor comment.  From my opinion, not enough patches are
>> going via guix-patches and are pushed directly.
>>
>> For instance, the «Commit policy» section says «For patches that just
>> add a new package, and a simple one, it’s OK to commit, if you’re
>> confident (which means you successfully built it in a chroot setup, and
>> have done a reasonable copyright and license auditing).»
>>
>> And from my point of view, new packages should *always* go via
>> guix-patches, wait 15 days, then push if no remark.  It lets the time
>> for the community to chime in.  And if not, it just slows down for 2
>> weeks.
>
> Three comments: (1) two weeks for a trivial Python/R/C package (the
> “simple one” rule above) can be a lot, (2) committers are by definition
> trusted to not mess up and to clean up behind them, and (3) when a
> simple package is pushed there’s after-the-fact peer review in practice
> (sometimes I reply to guix-commits notifications, for example.)

While I agree for updates or various fixes, I disagree when adding new
packages.  I do not see what is so urgent, I mean, if it was so urgent,
why the package had not been added before?  Therefore, I disagree with
(1) when adding new packages.

About (2), yes I agree that committers are by definition trusted.  No
question here.  However, (a) more eyes prevent mistakes and (b) some
ramblings.

One question is “encouragement” for reviewing, somehow.  Asking for new
package additions to go via guix-patches is a call making kind of
equality between contributors.  As someone without commit access, I can
tell you that it is often demotivating to send a trivial addition, wait
forever, ping people (aside I know who I have to ping :-)).  Usually, it
means people are busy elsewhere, so I try to help to reduce the workload
by reviewing stuff or by doing bug triage.  However, in the same time, I
see committers push their own trivial additions.  It appears to me
“unfair”.  Why are committer’s trivial additions more “urgent” than
mine?

I do not blame anyone, obviously not! I just comment here from my side
the current process in order to improve it.

About (3), if peer review before pushing can be used, how is a
after-the-fact peer review a justification?  Mistakes happen and I am
fine with that.  And hopefully they can be fixed.  However, they remain
forever in the Git history tree – therefore, here we have avoidable
“tragic” commits.

(This tricking peer review is a corollary of a more general
rambling about review. :-))

> I think it’s about finding the right balance to be reasonably efficient
> while not compromising on quality.

I totally agree.  And I do not see nor understand where is the
inefficiency here when asking to go via guix-patches and wait two weeks
for adding a new package.

Could you provide one concrete example of an urgent trivial package
addition?


Cheers,
simon

Incentives for review

[Ludovic Courtès]

zimoun <zimon.toutoune@gmail.com> skribis:

> On Tue, 19 Oct 2021 at 14:56, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

[...]

> One question is “encouragement” for reviewing, somehow.  Asking for new
> package additions to go via guix-patches is a call making kind of
> equality between contributors.  As someone without commit access, I can
> tell you that it is often demotivating to send a trivial addition, wait
> forever, ping people (aside I know who I have to ping :-)).  Usually, it
> means people are busy elsewhere, so I try to help to reduce the workload
> by reviewing stuff or by doing bug triage.  However, in the same time, I
> see committers push their own trivial additions.  It appears to me
> “unfair”.

I understand and sympathize (I also see us slipping off-topic :-)).

> Why are committer’s trivial additions more “urgent” than mine?

Yeah, I see what you mean.

I would like to see us committers do more review work.  But I also view
things from a different angle: everyone contributes in their own way,
and each contribution is a gift.  We can insist on community
expectations (reviewing other people’s work), but we should also welcome
contributions as they come.

There’s a balance to be found between no formal commitment on behalf of
committers, and a strict and codified commitment similar to what is
required for participation in the distros list¹.

A good middle ground may be to provide incentives for review.  How?  I’m
not sure exactly, but first by making it clear that review is makes the
project move forward and is invaluable.  You once proposed having
‘Reviewed-By’ tags to acknowledge non-committer reviews, and I think
that would be one step in that direction.  Perhaps there are other
things we could do?

[...]

>> I think it’s about finding the right balance to be reasonably efficient
>> while not compromising on quality.
>
> I totally agree.  And I do not see nor understand where is the
> inefficiency here when asking to go via guix-patches and wait two weeks
> for adding a new package.

It’s not about urgency but rather about not contributing to the growth
of our patch backlog, which is a real problem.

Thanks,
Ludo’.

¹ https://oss-security.openwall.org/wiki/mailing-lists/distros

Re: Incentives for review

[zimoun]

On Tue, 19 Oct 2021 at 17:41, Ludovic Courtès <ludo@gnu.org> wrote:

> I would like to see us committers do more review work.  But I also view
> things from a different angle: everyone contributes in their own way,
> and each contribution is a gift.  We can insist on community
> expectations (reviewing other people’s work), but we should also welcome
> contributions as they come.

I am aligned with these words: «everyone contributes in their own way,
and each contribution is a gift».  I truly agree.

> There’s a balance to be found between no formal commitment on behalf of
> committers, and a strict and codified commitment similar to what is
> required for participation in the distros list¹.

I do not know (yet?) if I agree on that.  I would just say that many
more potential contributors are floating around.  We are already
selecting a small part of them using the email workflow (using another
workflow would select another part; who knows what is best ;-)).  Then,
the question is how to keep them?  Obviously, I do not have the
answer. :-) And sadly, I do not have number for backing an intuition.
From my experience when digging into the bug tracker, I see many
submissions without an answer and people who seem to have given up.
Sadly, it is impossible to know why.

> A good middle ground may be to provide incentives for review.  How?  I’m
> not sure exactly, but first by making it clear that review is makes the
> project move forward and is invaluable.  You once proposed having
> ‘Reviewed-By’ tags to acknowledge non-committer reviews, and I think
> that would be one step in that direction.  Perhaps there are other
> things we could do?

Yeah, I think that non-committer doing review deserve rewards. :-) For
instance, the Linux project uses a lot of various tags [2].  The Guix
project could borrow some. ;-)

For instance, the Guix project uses Reported-by for bugs, somehow.  One
can imagine a Reviewed-by or Tested-by.

2: <https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html>

>>> I think it’s about finding the right balance to be reasonably efficient
>>> while not compromising on quality.
>>
>> I totally agree.  And I do not see nor understand where is the
>> inefficiency here when asking to go via guix-patches and wait two weeks
>> for adding a new package.
>
> It’s not about urgency but rather about not contributing to the growth
> of our patch backlog, which is a real problem.

Thus, why wait two weeks before pushing is an issue?


Cheers,
simon

Re: Incentives for review

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> A good middle ground may be to provide incentives for review. 
> How?  I’m
> not sure exactly, but first by making it clear that review is 
> makes the
> project move forward and is invaluable.  You once proposed 
> having
> ‘Reviewed-By’ tags to acknowledge non-committer reviews, and I 
> think
> that would be one step in that direction.  Perhaps there are 
> other
> things we could do?

I was thinking in the opposite direction: not incentives to 
recognize reviewers but a closer relationship to the patch 
submitters, i.e. “patch buddies” or mentorship.  If I made myself 
officially responsible for reviewing commits by Simon and all 
commits touching R then I’m more likely to actually do the review 
for these patches.

Reviews done by people who are not committers could also be 
acknowledged, of course, but applying the patch (sometimes 
manually because of conflicts) is still manual work that can feel 
like a chore if the committer doesn’t feel a connection to the 
patch or the person who submitted it.

-- 
Ricardo

Re: Incentives for review

[Christine Lemmer-Webber]

Ricardo Wurmus <rekado@elephly.net> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> A good middle ground may be to provide incentives for review. How?
>> I’m
>> not sure exactly, but first by making it clear that review is makes
>> the
>> project move forward and is invaluable.  You once proposed having
>> ‘Reviewed-By’ tags to acknowledge non-committer reviews, and I think
>> that would be one step in that direction.  Perhaps there are other
>> things we could do?
>
> I was thinking in the opposite direction: not incentives to recognize
> reviewers but a closer relationship to the patch submitters,
> i.e. “patch buddies” or mentorship.  If I made myself officially
> responsible for reviewing commits by Simon and all commits touching R
> then I’m more likely to actually do the review for these patches.

I think this sounds good.  Maybe in conjunction with "guix days", etc?

> Reviews done by people who are not committers could also be
> acknowledged, of course, but applying the patch (sometimes 
> manually because of conflicts) is still manual work that can feel like
> a chore if the committer doesn’t feel a connection to the patch or the
> person who submitted it.

Re: Incentives for review

[Joshua Branson]

Ricardo Wurmus <rekado@elephly.net> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> A good middle ground may be to provide incentives for review. How?
>> I’m
>> not sure exactly, but first by making it clear that review is makes
>> the
>> project move forward and is invaluable.  You once proposed having
>> ‘Reviewed-By’ tags to acknowledge non-committer reviews, and I think
>> that would be one step in that direction.  Perhaps there are other
>> things we could do?
>
> I was thinking in the opposite direction: not incentives to recognize
> reviewers but a closer relationship to the patch submitters,
> i.e. “patch buddies” or mentorship.  If I made myself officially
> responsible for reviewing commits by Simon and all commits touching R
> then I’m more likely to actually do the review for these patches.
>
> Reviews done by people who are not committers could also be
> acknowledged, of course, but applying the patch (sometimes 
> manually because of conflicts) is still manual work that can feel like
> a chore if the committer doesn’t feel a connection to the patch or the
> person who submitted it.

If you're looking for an occassional guix contributor friend to
mentor...I'll gladly volunteer!  :)

-- 
Joshua Branson (jab in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar
  

Re: Tricking peer review

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]

Hi Simon and Ludovic,

very interesting thread, thank you!

I think the "final" result of this discussion should be condensed in a
few (one?) additional paragraphs in the Contributing section of the Guix
manual

zimoun <zimon.toutoune@gmail.com> writes:

[...]

>  - url-fetch: the attacker has to introduce the tarballs into SWH.
>    There is not so much means, from my understanding: SWH ingests
>    tarballs via loaders, for instance gnu.org or sources.json or
>    debian.org etc. Therefore the attacker has to introduce the malicious
>    code to these platforms.
>
>  - url-fetch without metadata (as your example), indeed, the reviewer
>    could be abused; mitigated by the fact that “guix lint” spots the
>    potential issue.
>
>  - url-fetch with metadata: the attacker have to also corrupt
>    Diasarchive-DB.   Otherwise, the tarball returned by SWH will not
>    match the checksum.
>
>  - svn-fetch, hg-fetch, cvs-fetch: no attack possible, yet.
>
>  - git-fetch: it is the *real* issue.  Because it is easy for the
>    attacker to introduce malicious code into SWH (create a repo on
>    GitHub, click Save, done).  Then submit a package using it as you
>    did.  It is the same case as url-fetch without metadata but easier
>    for the attacker.  It is mitigated by “guix lint”.

Well done Simon: AFAIU this is a complete analisys of the possible
"source" attacks, or is something missing?

> That’s said, if I am an attacker and I would like to corrupt Guix, then
> I would create a fake project mimicking a complex software.  For
> instance, Gmsh is a complex C++ scientific software.  The correct URL is
> <https://gmsh.info> and the source at
> <https://gitlab.onelab.info/gmsh/gmsh>.  Then, as an attacker, I buy the
> domain say gmsh.org

or onelab.org, onehab.info and also set up a https://onehab.info web
site identical to the legitimate one just to trick people

> and put a malicious code there.  Last, I send for
> inclusion a package using this latter URL.  The reviewer would be
> abused.
> That’s why more eyes, less issues. :-)

I agree, but eyes should also be aware of this class of possible attacks

>> Also, just because a URL looks nice and is reachable doesn’t mean the
>> source is trustworthy either.  An attacker could submit a package for an
>> obscure piece of software that happens to be malware.  The difference
>> here is that the trick above would allow targeting a high-impact
>> package.
>
> I agree.

I also agree (obviously) and I think this kind of attack should also be
documented in the manual (if not already done)

[...]

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

patches for new packages proper workflow (Re: Tricking peer review)

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 1002 bytes --]

Hi,

zimoun <zimon.toutoune@gmail.com> writes:

[...]

>> All in all, it’s probably not as worrisome as it first seems.  However,
>> it’s worth keeping in mind when reviewing a package.
>
> I agree with a minor comment.  From my opinion, not enough patches are
> going via guix-patches and are pushed directly.
>
> For instance, the «Commit policy» section says «For patches that just
> add a new package, and a simple one, it’s OK to commit, if you’re
> confident (which means you successfully built it in a chroot setup, and
> have done a reasonable copyright and license auditing).»
>
> And from my point of view, new packages should *always* go via
> guix-patches, wait 15 days, then push if no remark.  It lets the time
> for the community to chime in.  And if not, it just slows down for 2
> weeks.

I agree with Simon, the policy to add new packages should be changed as
he proposes.

Thanks! Giovanni

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Tricking peer review

[zimoun]

Hi,

On Wed, 20 Oct 2021 at 10:22, Giovanni Biscuolo <g@xelera.eu> wrote:

> I think the "final" result of this discussion should be condensed in a
> few (one?) additional paragraphs in the Contributing section of the Guix
> manual

Run “guix lint” is already listed.  What do you have in mind about more
additions?


> Well done Simon: AFAIU this is a complete analisys of the possible
> "source" attacks, or is something missing?

To my knowledge, yes it is exhaustive with the current situation about
tricking the content-addressed system.

On the top of that, it is addressed by hash functions; it is thus
vulnerable to preimage attack of such hash functions.  SWH uses SHA-1 to
address and I do not know how they address potential collisions.

For instance, the cost for SHA-1 [1] is still really expensive.  Well,
for interested reader, one can read the discussion here [2].  SHA-1 is
2^160 (~10^48.2) and compare to 10^50 which is the estimated number of
atoms in Earth.  Speaking about content-addressability, SHA-1 seems
fine.  We are speaking about content-addressability not about using
SHA-1 as hash function for security, IMHO.  It is the same situation as
Git, for instance.

The surface of attack is very low because:

 a) SWH is an archive and not a forge,
 b) a chosen-prefix attack [3] could no work if review is correctly done;
    which means run “guix lint”,
 c) an attacker has to trick the checksum (SHA-256) and the address
    (SHA-1); at various locations: Guix history (now signed), SWH,
     Disarchive-DB.

1: <https://shattered.it/>
2: <http://issues.guix.gnu.org/issue/44187#4>
3: <https://sha-mbles.github.io/>

>>> Also, just because a URL looks nice and is reachable doesn’t mean the
>>> source is trustworthy either.  An attacker could submit a package for an
>>> obscure piece of software that happens to be malware.  The difference
>>> here is that the trick above would allow targeting a high-impact
>>> package.
>>
>> I agree.
>
> I also agree (obviously) and I think this kind of attack should also be
> documented in the manual (if not already done)

Well, nothing new here, IMHO.  A distribution relies on content, i.e.,
any distribution points to that content.  Whatever the nature of the
pointing arrow (URL, Git commit, hash, etc.), the pointed material must
be carefully checked at package time; as explained by «Submitting
Patches» [4]. :-) That’s why I am advocating [5] that:

         new packages should *always* go via guix-patches, wait 15 days,
        then push if no remark.  It lets the time for the community to
        chime in.  And if not, it just slows down for 2 weeks.


4: <https://guix.gnu.org/manual/devel/en/guix.html#Submitting-Patches>
5: <https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00110.html>

Cheers,
simon

Re: Incentives for review

[Thiago Jung Bauermann]

Hello,

Em terça-feira, 19 de outubro de 2021, às 12:41:23 -03, Ludovic Courtès 
escreveu:
> zimoun <zimon.toutoune@gmail.com> skribis:
> > On Tue, 19 Oct 2021 at 14:56, Ludovic Courtès 
<ludovic.courtes@inria.fr> wrote:
> [...]
> 
> I would like to see us committers do more review work.  But I also view
> things from a different angle: everyone contributes in their own way,
> and each contribution is a gift.  We can insist on community
> expectations (reviewing other people’s work), but we should also welcome
> contributions as they come.

Thank you for viewing it from that angle. On a personal note, I’m aware 
that my ratio of patches reviewed / patches posted approaches zero and this 
makes me a bit uncomfortable every time I type `git send-email`.

Sometimes I try to review patches, but it’s not a very productive endeavour 
for a few reasons:

1. In many cases, I don’t see anything wrong with the patch I’m looking at. 
In those cases I could reply saying so, but I refrain from doing that 
because if such message comes from someone who doesn’t have much experience 
in the part of Guix that the patch touches (which is almost always the case 
for me when reviewing patches), then how much values does that really add?

2. Going through the guix-patches mailing list looking for submissions that 
touch the few areas of Guix where I have at least some experience. I don’t 
think I found an effective method yet (in part the problem is on my side 
because the search function of the email client I use isn’t very reliable).

> There’s a balance to be found between no formal commitment on behalf of
> committers, and a strict and codified commitment similar to what is
> required for participation in the distros list¹.
> 
> A good middle ground may be to provide incentives for review.  How?  I’m
> not sure exactly, but first by making it clear that review is makes the
> project move forward and is invaluable.  You once proposed having
> ‘Reviewed-By’ tags to acknowledge non-committer reviews, and I think
> that would be one step in that direction.

I like the ‘Reviewed-by’ idea and I agree that it provides a tangible 
incentive. A ‘Tested-by:’ tag would have the same effect as well, as 
suggested by simon.

> Perhaps there are other things we could do?

One thing that would help me would be some way to “subscribe” to changes in 
certain areas of Guix. That way, when a patch is submitted which touches 
those areas I would be automatically copied on the emails that go to the 
guix-patches mailing list. “areas of Guix” could be defined by paths in the 
repo, guile modules or regexps matching package names, for example.

-- 
Thanks,
Thiago

Re: Tricking peer review

[Leo Famulari]

On Tue, Oct 19, 2021 at 10:39:12AM +0200, zimoun wrote:
> Drifting from the initial comment.  One could name “tragic” commits are
> commits which break “guix pull”.  It is rare these days but there are
> some reachable ones via “guix time-machine”.

That's a good point. Is it a good idea to teach Guix to help (not force)
users to avoid these commits?

Re: Tricking peer review

[Leo Famulari]

On Fri, Oct 15, 2021 at 08:54:09PM +0200, Ludovic Courtès wrote:
> The trick is easy: we give a URL that’s actually 404, with the hash of a
> file that can be found on Software Heritage (in this case, that of
> ‘grep-3.4.tar.xz’).  When downloading the source, the automatic
> content-addressed fallback kicks in, and voilà:
[...]
> Thoughts?

It's a real risk... another illustration that our security model trusts
committers implicitly (not saying that's a bad thing or even avoidable).

In years past I mentioned a similar technique but based on using
old/vulnerable versions of security-critical packages like OpenSSL. The
same approach would have worked since we started using Nix's
content-addressed mirror.

> It’s nothing new, it’s what I do when I want to test the download
> fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
> c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it could
> somehow be abused to have malicious packages pass review.

Nice feature! Sorry if this was already suggested, but is it possible to
create an argument to this variable that disallows use of the fallback
mechanisms? I would certainly use that while reviewing and testing my
own patches.

Re: Tricking peer review

[Ludovic Courtès]

Hi,

Leo Famulari <leo@famulari.name> skribis:

> On Fri, Oct 15, 2021 at 08:54:09PM +0200, Ludovic Courtès wrote:
>> The trick is easy: we give a URL that’s actually 404, with the hash of a
>> file that can be found on Software Heritage (in this case, that of
>> ‘grep-3.4.tar.xz’).  When downloading the source, the automatic
>> content-addressed fallback kicks in, and voilà:
> [...]
>> Thoughts?
>
> It's a real risk... another illustration that our security model trusts
> committers implicitly (not saying that's a bad thing or even avoidable).
>
> In years past I mentioned a similar technique but based on using
> old/vulnerable versions of security-critical packages like OpenSSL. The
> same approach would have worked since we started using Nix's
> content-addressed mirror.

Right.  Like zimoun wrote, the SWH fallback makes this even more
stealthily exploitable.

>> It’s nothing new, it’s what I do when I want to test the download
>> fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
>> c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it could
>> somehow be abused to have malicious packages pass review.
>
> Nice feature! Sorry if this was already suggested, but is it possible to
> create an argument to this variable that disallows use of the fallback
> mechanisms? I would certainly use that while reviewing and testing my
> own patches.

Yes, you can do “GUIX_DOWNLOAD_FALLBACK_TEST=none” (added in
bd61d62182bfda4a695757ec66810b28e8e1a6d0).

Thanks,
Ludo’.

Re: Tricking peer review

[zimoun]

Hi,

On Wed, 20 Oct 2021 at 19:03, Leo Famulari <leo@famulari.name> wrote:
> On Tue, Oct 19, 2021 at 10:39:12AM +0200, zimoun wrote:
>> Drifting from the initial comment.  One could name “tragic” commits are
>> commits which break “guix pull”.  It is rare these days but there are
>> some reachable ones via “guix time-machine”.
>
> That's a good point. Is it a good idea to teach Guix to help (not force)
> users to avoid these commits?

Somehow, it means extend the mechanism behind
’channel-with-substitutes-available’ [1], I guess.

1: <https://guix.gnu.org/manual/devel/en/guix.html#Channels-with-Substitutes>


Cheers,
simon

Re: Incentives for review

[Artem Chernyak]

> On Wed, Oct 20, 2021 at 4:37 PM Thiago Jung Bauermann <bauermann@kolabnow.com> wrote:

[...]

> One thing that would help me would be some way to “subscribe” to changes in
> certain areas of Guix. That way, when a patch is submitted which touches
> those areas I would be automatically copied on the emails that go to the
> guix-patches mailing list. “areas of Guix” could be defined by paths in the
> repo, guile modules or regexps matching package names, for example.

You could handle it by filtering the email in your email client.

One thing that could help this, is to include a little more info in
the subject line for patches. Right now we include the broad area
(e.g. "gnu: ..."). But we could go on level deeper and include the
specific file (e.g. "gnu: docker: ..."). This becomes important
because gnu as an area of guix spans a lot of different packages and
languages. With the extra file level information, we can easily filter
it down to only the areas we know about.

Re: Incentives for review

[Katherine Cox-Buday]

Ludovic Courtès <ludo@gnu.org> writes:

>> On Tue, 19 Oct 2021 at 14:56, Ludovic Courtès <ludovic.courtes@inria.fr>

> But I also view things from a different angle: everyone contributes in their
> own way, and each contribution is a gift.

Maybe selfishly, but I really agree with this. I think this is just the nature of community-based projects: people are going to scratch their own itch, and when time allows, do altruistic things for other people.

Some people (e.g. me) don't have very much time at all to do the altruistic things (which gnaws at me). I do what I can, when I can, and hope that someone else benefits.

> A good middle ground may be to provide incentives for review.  How?  I’m
> not sure exactly, but first by making it clear that review is makes the
> project move forward and is invaluable.
>
>>> I think it’s about finding the right balance to be reasonably efficient
>>> while not compromising on quality.
>>
>> I totally agree.  And I do not see nor understand where is the
>> inefficiency here when asking to go via guix-patches and wait two weeks
>> for adding a new package.

Often I find that people on projects/teams have fundamentally different understandings of what reviews are for. Are they quality control? Mentoring opportunities? Opportunities to teach others how something new works? A way to encourage cohesiveness in the project?

It can help to publicly state the intent somewhere. I think the word "review" is mentioned in the manual 11 times, but nowhere does it say what the review's purpose is.

Large, public projects like Guix are a bit different, so I'm not sure this applies, but reviews meant to be gates on quality are my least favorite:

(Please note: these are general observations about the industry and not necessarily specific to Guix)

- The reviewers are human too, and there are various circumstances where they
  will miss things. Some of the most insidious forms of this is are: tragedy
  of the commons, i.e.

  > Submitter: They always do such a good job catching things. I think this is
  > good, but I know they'll catch any issues.

  > Reviewer: I feel bad this has sat for so long, this person usually does a
  > good job. +1 without a detailed review.

  > Submitter: A +1! It must not have had any issues.

- Unavoidably, because of human nature, groups form, and certain people
  experience less friction getting patches in. See the last point.

- There is a feedback loop present: those who have committed earlier have
  control and are more likely to reject later commits which don't do things as
  they would have. Sometimes "quality" is abused as a cover for opinion. Very
  few people are doing this maliciously, but it still happens.

- As I mentioned in another thread[1], trying to chase the ideal of quality
  may actually be worse in the end, or be a local maxima for quality or
  utility. Focusing on velocity may help achieve the global maxima for both.
  As always, there is a balance.

> It’s not about urgency but rather about not contributing to the growth
> of our patch backlog, which is a real problem.

I have often seen folks on various projects worried about the size of various backlogs: bugs, issues, etc. I think it is human to want to try and contain something that appears to be growing, unbounded.

I think the thing that bothers us is a sense that the backlog is becoming unmanageable, or too large to triage. I submit that this is actually a tooling and organizational issue, and not an intrinsic issue to be solved. Bugs may still be valid; patches may still have valuable bones to modify.

I think the real issue is that as a backlog grows, the tools we're used to using cannot answer the questions we want to ask: what is most relevant to me or the project right now?

To me, this sounds like a search and display problem.

[1] - https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00081.html

-- 
Katherine

Re: Incentives for review

[Ricardo Wurmus]

Thiago Jung Bauermann <bauermann@kolabnow.com> writes:

> 2. Going through the guix-patches mailing list looking for 
> submissions that 
> touch the few areas of Guix where I have at least some 
> experience. I don’t 
> think I found an effective method yet (in part the problem is on 
> my side 
> because the search function of the email client I use isn’t very 
> reliable).

If a browser fits into your workflow you may want to search for 
submissions on https://issues.guix.gnu.org.  It also shows 
“forgotten” issues that would likely benefit from some poking.

> One thing that would help me would be some way to “subscribe” to 
> changes in 
> certain areas of Guix. That way, when a patch is submitted which 
> touches 
> those areas I would be automatically copied on the emails that 
> go to the 
> guix-patches mailing list. “areas of Guix” could be defined by 
> paths in the 
> repo, guile modules or regexps matching package names, for 
> example.

Perhaps issues.guix.gnu.org could offer atom feeds for certain 
keywords (e.g. the name of the module touched by the commits?).

-- 
Ricardo

Re: Incentives for review

[Ricardo Wurmus]

Katherine Cox-Buday <cox.katherine.e@gmail.com> writes:

>> It’s not about urgency but rather about not contributing to the 
>> growth
>> of our patch backlog, which is a real problem.
>
> I have often seen folks on various projects worried about the 
> size of
> various backlogs: bugs, issues, etc. I think it is human to want 
> to
> try and contain something that appears to be growing, unbounded.
>
> I think the thing that bothers us is a sense that the backlog is
> becoming unmanageable, or too large to triage. I submit that 
> this is
> actually a tooling and organizational issue, and not an 
> intrinsic
> issue to be solved. Bugs may still be valid; patches may still 
> have
> valuable bones to modify.
>
> I think the real issue is that as a backlog grows, the tools 
> we're used to using cannot answer the questions we want to ask: 
> what is most relevant to me or the project right now?
>
> To me, this sounds like a search and display problem.

I agree with your analysis.  And with your earlier comments about 
the vagueness of what review means and how it can lead to a 
failure to communicate.

At least on the side of presenting submitted issues I’m sure we 
can do better.  One attempt in the past was to automatically bring 
up “forgotten” issues to the fore; Thiago’s idea to allow people 
to subscribe to certain *kinds* of issues when they are reported 
is also good.

I would be happy if people used this opportunity to change mumi 
(the tool behind issues.guix.gnu.org) to present the backlog in 
more helpful ways.

Here’s the code for reference: 
https://git.elephly.net/gitweb.cgi?p=software/mumi.git

-- 
Ricardo

Re: Incentives for review

[zimoun]

Hi,

On Thu, 21 Oct 2021 at 16:06, Ricardo Wurmus <rekado@elephly.net> wrote:

> Perhaps issues.guix.gnu.org could offer atom feeds for certain 
> keywords (e.g. the name of the module touched by the commits?).

For instance, let have some number over the last year and half:

        File:  gnu/packages/bioconductor.scm
           1103 Ricardo Wurmus
             24 Roel Janssen
              7 Tobias Geerinckx-Rice

        File:  gnu/packages/bioinformatics.scm
            450 Ricardo Wurmus
             58 Efraim Flashner
             28 Roel Janssen

        File:  gnu/packages/cran.scm
           1687 Ricardo Wurmus
             35 Lars-Dominik Braun
             34 Roel Janssen

Well, it means Ricardo you are the one who pushes modifications of these
files.  Therefore, it could be nice to automatically keep you in touch
to your convenience (instead of X-Debbugs-CC or annoy you via IRC ;-)).

For instance with a feed (Atom or RSS).  Because these days, the traffic
of guix-patches is more or less high so I am not convinced all
committers subscribe. :-)

I mean, such feed would help “specialized” reviewers to filter the
volume and would probably be more incentive.  (Well, for sure a good
email client with the right filter is already doing the same. :-))


Cheers,
simon

Re: Incentives for review

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2958 bytes --]

On 2021-10-19, Ludovic Courtès wrote:
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> On Tue, 19 Oct 2021 at 14:56, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

>> One question is “encouragement” for reviewing, somehow.  Asking for new
>> package additions to go via guix-patches is a call making kind of
>> equality between contributors.  As someone without commit access, I can
>> tell you that it is often demotivating to send a trivial addition, wait
>> forever, ping people (aside I know who I have to ping :-)).  Usually, it
>> means people are busy elsewhere, so I try to help to reduce the workload
>> by reviewing stuff or by doing bug triage.  However, in the same time, I
>> see committers push their own trivial additions.  It appears to me
>> “unfair”.
>
> I understand and sympathize (I also see us slipping off-topic :-)).
>
>> Why are committer’s trivial additions more “urgent” than mine?
>
> Yeah, I see what you mean.
>
> I would like to see us committers do more review work.  But I also view
> things from a different angle: everyone contributes in their own way,
> and each contribution is a gift.  We can insist on community
> expectations (reviewing other people’s work), but we should also welcome
> contributions as they come.

I must admit, I don't review patches unless they're in an area of
expertise (e.g. u-boot, arm-trusted-firmware, reproducible builds
tooling, etc.); I just don't have sufficient skill with guile to review
arbitrary packages in a meaningful way, other than the most trivial of
packages...

Before I was granted commit access, I *really* appreciated getting
review... but was also frustrated by how long it took to get a
contribution in; having limited time available for guix, spending that
energy checking if something I'd already "finished" was actually merged
was a bit demotivating.

I have added a small number of trivial packages without review; maybe I
shouldn't have... but it was also a bit of a sigh of relief once I could
push directly to no have to get caught up in the waiting game; I had
more time to actually contribute other improvements to guix.


> There’s a balance to be found between no formal commitment on behalf of
> committers, and a strict and codified commitment similar to what is
> required for participation in the distros list¹.

So yeah, it is a quite balancing act!


Would a workflow of pushing to a "wip-pending" branch in guix.git that
then gets merged and/or cherry-picked into master/staging/core-updates
help at all?

A cursory review could commit to "wip-pending", with the
plan/hope/expectation that it would get some other review and/or a
timeout before it gets merged.

I guess it would be hard to avoid having to constantly rebase with the
latest updates... "wip-pending" might just add more work to an already
needs-more-resources process...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Incentives for review

[Katherine Cox-Buday]

Ricardo Wurmus <rekado@elephly.net> writes:

> Katherine Cox-Buday <cox.katherine.e@gmail.com> writes:
>
>>> It’s not about urgency but rather about not contributing to the growth
>>> of our patch backlog, which is a real problem.
>>
>> I have often seen folks on various projects worried about the size of
>> various backlogs: bugs, issues, etc. I think it is human to want to
>> try and contain something that appears to be growing, unbounded.
>>
>> I think the thing that bothers us is a sense that the backlog is
>> becoming unmanageable, or too large to triage. I submit that this is
>> actually a tooling and organizational issue, and not an intrinsic
>> issue to be solved. Bugs may still be valid; patches may still have
>> valuable bones to modify.
>>
>> I think the real issue is that as a backlog grows, the tools we're used to
>> using cannot answer the questions we want to ask: what is most relevant to
>> me or the project right now?
>>
>> To me, this sounds like a search and display problem.

> I would be happy if people used this opportunity to change mumi (the tool
> behind issues.guix.gnu.org) to present the backlog in more helpful ways.

I don't have time to work on this, but here are some ideas. Some of these capabilities are present, but maybe not discoverable or a pre-built clickable link while viewing a patch/issue.

- Contextual search based on a path.
  - Show me issues/patches for this file/directory
  - Show me the rate of change of this file/directory

- Contextual search based on a patch
  - Show me bugs which mention any top-level public symbols changing in this
    patch, or if packages, the package name.
  - Show me patches which conflict with this one.
  
- Contextual search based on author.
  - Show me other patches by this author
  - Show me the median time-to-commit for this author's patches

- Show me patches/issues, grouped by author, sorted by median time-to-commit,
  descending.
- Show me the paths/files with the highest number of bugs reported.

A lot of this requires static analysis which may not be trivial to implement. Still, I think being able to say "we don't have time to build what would fix this" is a helpful progression from "we don't know how to manage this backlog".

Thanks for pointing out the source code to mumi!
  
-- 
Katherine

Re: Incentives for review

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 1395 bytes --]


Hi,

> Thiago’s idea to allow people to subscribe to certain *kinds* of
> issues when they are reported is also good.

I agree this is a great idea. Recently, I unsubscribed from
guix-patches. It's just too high volume. These days, I prefer to just
search for issues using emacs-debbugs and mumi.

Here's another idea for mumi: mumi should have a JSON API. Debbugs' SOAP
API is quite terrible, and doesn't even expose such things as the number
of emails in an issue. Mumi can offer its own API which does these
things properly. That way, we can write new clients (say, a CLI client)
for mumi, that can filter more intelligently. If we had a good CLI
client, our contributors wouldn't have to set up an email client or
emacs just to participate.

The way I see it, we are outgrowing general purpose bug trackers like
debbugs. We need a special purpose bug tracker specifically for Guix
with its special requirements. We are a big enough community for this to
be important.

I might be able to find some time to implement a simple JSON API for
mumi. Would there be interest in such a contribution?

Regarding, hacking on mumi, I understand that issues.guix.gnu.org is on
an IP whitelist with the GNU debbugs server. How do I hack on mumi if
simply running it on my local machine, and pulling data from GNU debbugs
would alarm the debbugs admins?

Regards,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Incentives for review

[Ludovic Courtès]

Hi!

Arun Isaac <arunisaac@systemreboot.net> skribis:

>> Thiago’s idea to allow people to subscribe to certain *kinds* of
>> issues when they are reported is also good.
>
> I agree this is a great idea. Recently, I unsubscribed from
> guix-patches. It's just too high volume. These days, I prefer to just
> search for issues using emacs-debbugs and mumi.
>
> Here's another idea for mumi: mumi should have a JSON API. Debbugs' SOAP
> API is quite terrible, and doesn't even expose such things as the number
> of emails in an issue. Mumi can offer its own API which does these
> things properly. That way, we can write new clients (say, a CLI client)
> for mumi, that can filter more intelligently. If we had a good CLI
> client, our contributors wouldn't have to set up an email client or
> emacs just to participate.
>
> The way I see it, we are outgrowing general purpose bug trackers like
> debbugs. We need a special purpose bug tracker specifically for Guix
> with its special requirements. We are a big enough community for this to
> be important.
>
> I might be able to find some time to implement a simple JSON API for
> mumi. Would there be interest in such a contribution?

Definitely, but I think the JSON API is a means, not an end, so what
matters is what we’ll build with it.

Example that comes to mind: debbugs.el could use it for features Debbugs
lacks; we could have a client Scheme module and a command-line tool to
perform certain query, with the goal of being able to do things like:

  guix review apply 1234
  guix review search bioinformatics
  …

That could be a game-changer.

Of course we should start small and focus on specific features such as
searching, listing, and retrieving.

> Regarding, hacking on mumi, I understand that issues.guix.gnu.org is on
> an IP whitelist with the GNU debbugs server. How do I hack on mumi if
> simply running it on my local machine, and pulling data from GNU debbugs
> would alarm the debbugs admins?

Ricardo may know the answer.  :-)

Thanks,
Ludo’.

Re: Incentives for review

[Ludovic Courtès]

Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

> I was thinking in the opposite direction: not incentives to recognize
> reviewers but a closer relationship to the patch submitters,
> i.e. “patch buddies” or mentorship.  If I made myself officially
> responsible for reviewing commits by Simon and all commits touching R
> then I’m more likely to actually do the review for these patches.

Yeah.  Apparently several people made similar comments: that identifying
who’s knowledgeable about a certain area of the code would help.

I’ve seen GitHub bots that automatically ping people who’ve touched the
same area of code that you’re submitting a pull request for; that’s a
similar idea (I learned about it because for years I’d receive an
occasional notification for Nixpkgs pull requests :-)).

A notification/subscription mechanism along these lines for mumi would
be a step forward in that direction.

Ludo’.

Re: Incentives for review

[Jonathan McHugh]

If I recall, you can request Debbugs content if you email them.

====================
Jonathan McHugh
indieterminacy@libre.brussels

October 21, 2021 8:22 PM, "Arun Isaac" <arunisaac@systemreboot.net> wrote:

> Hi,
> 
>> Thiago’s idea to allow people to subscribe to certain *kinds* of
>> issues when they are reported is also good.
> 
> I agree this is a great idea. Recently, I unsubscribed from
> guix-patches. It's just too high volume. These days, I prefer to just
> search for issues using emacs-debbugs and mumi.
> 
> Here's another idea for mumi: mumi should have a JSON API. Debbugs' SOAP
> API is quite terrible, and doesn't even expose such things as the number
> of emails in an issue. Mumi can offer its own API which does these
> things properly. That way, we can write new clients (say, a CLI client)
> for mumi, that can filter more intelligently. If we had a good CLI
> client, our contributors wouldn't have to set up an email client or
> emacs just to participate.
> 
> The way I see it, we are outgrowing general purpose bug trackers like
> debbugs. We need a special purpose bug tracker specifically for Guix
> with its special requirements. We are a big enough community for this to
> be important.
> 
> I might be able to find some time to implement a simple JSON API for
> mumi. Would there be interest in such a contribution?
> 
> Regarding, hacking on mumi, I understand that issues.guix.gnu.org is on
> an IP whitelist with the GNU debbugs server. How do I hack on mumi if
> simply running it on my local machine, and pulling data from GNU debbugs
> would alarm the debbugs admins?
> 
> Regards,
> Arun

Re: Incentives for review

[zimoun]

Hi,

I mainly agree with the words of the message I am replying and my
intent is to provide numbers about what we are speaking.

>> It’s not about urgency but rather about not contributing to the growth
>> of our patch backlog, which is a real problem.

While I disagree for submitting new package – I do not understand why or
how it is a problem to submit to guix-patches, wait 15 days, and then
push – I would like to put numbers about this backlog…

> I have often seen folks on various projects worried about the size of
> various backlogs: bugs, issues, etc. I think it is human to want to
> try and contain something that appears to be growing, unbounded. 

…about patches only.  Bug is another story. :-)


Patch 49993 is from Aug. 2021.  Between this patch and now (patch
51319), there are 164 patches still open.  Other said, 164 still open
submission for 2 months – I have not counted how many closed.

Patch 48999 is from Jun 2021.  Because the Debbugs numbering is shared
by many GNU projects, it is hard to know how many patches over this
thousand are Guix only.  However, today still 83 patches open on this
thousand range (49093–49993) for ~2months. And on this 83 still open
submission, there are 17 submissions that have not received any reply.
And I bet they will not receive one and they are falling in the cracks.

Patch 47997 is from Apr 2021. Still 75 patch opens on this thousand
range (48006–48999) for ~2months.

Therefore, from Apr 2021 to now (~6months), it is ~320 patches still
open.  From Dec. 1rst 2020 (patch 45000) to the bottom Mar 2017 (patch
25849), it is 282 still open patches.  And I do not count how many
without any reply.

Just pick a random patch, say 47932 proposing the addition of package
’xqilla’.  First, there is no reply, And second, the patch does not
apply, thus it requires manual work.

Ok, so many are “just” triage.  For instance, last year over one month,
we did a bug squashing [1,2].  And I closed one per day over the month;
something like between 5min per report to half hour.  Bug was easier
than patch.  Considering that many of these 282 still open submissions
require: look if it is compliant, apply (manual work), build, etc. say
half hour on average, it means 141 hours which is basically a full month
full time for one person – and not a funny work – only for triaging old
submissions.

Here I speak about 282 old submissions (before Dec 2020) and for recent
ones (after Jan 2021), it is 438 still open submissions.  Other said, it
is ~720 submissions to deal with.  Considering 50 active people, it
means deal with 14 submissions per person, assuming half hour per
submission, it means a full day working only on that per person.

On the top of that, there are bugs, systems and new features. :-)

Do not take me wrong, it is great to have these numbers!  It means Guix
is used and people contribute.  So no complaint! :-)

Just number to fix the idea about large backlog.


1: <https://lists.gnu.org/archive/html/guix-devel/2020-12/msg00001.html>
2: <https://lists.gnu.org/archive/html/guix-devel/2020-12/msg00257.html>


> I think the thing that bothers us is a sense that the backlog is
> becoming unmanageable, or too large to triage. I submit that this is
> actually a tooling and organizational issue, and not an intrinsic
> issue to be solved. Bugs may still be valid; patches may still have
> valuable bones to modify.

This is the point.  What do you do?  What could we improve about tooling
and organisation to better scale and deal with this “becoming
unmanageable backlog”?

From my point of view, it is good to have this issue.  It means that
Guix is becoming more popular.  And we – regular user, contributor,
committer – have to adapt to this increasing workload, IMHO.

The question is how.  And how to invite people to complete review. :-)


> I think the real issue is that as a backlog grows, the tools we're
> used to using cannot answer the questions we want to ask: what is most
> relevant to me or the project right now?

If it is relevant to the project then it is also relevant to me as an
user.  And vice-versa. ;-)

When something relevant to me is not making progress, it often means
people are busy elsewhere, so I try to comment (review?) about patches
or bugs.  It is a Sisyphean task because the workload never
decreases. :-)  Or maybe structured procrastination. ;-)


Cheers,
simon

Re: Incentives for review

[Ricardo Wurmus]

Hi Arun,

>> Thiago’s idea to allow people to subscribe to certain *kinds* 
>> of
>> issues when they are reported is also good.
>
> I agree this is a great idea. Recently, I unsubscribed from
> guix-patches. It's just too high volume. These days, I prefer to 
> just
> search for issues using emacs-debbugs and mumi.
>
> Here's another idea for mumi: mumi should have a JSON 
> API. Debbugs' SOAP
> API is quite terrible, and doesn't even expose such things as 
> the number
> of emails in an issue. Mumi can offer its own API which does 
> these
> things properly. That way, we can write new clients (say, a CLI 
> client)
> for mumi, that can filter more intelligently. If we had a good 
> CLI
> client, our contributors wouldn't have to set up an email client 
> or
> emacs just to participate.

These are all excellent ideas, and seeing you articulate them 
makes me happy, because in my experience there’s always good code 
around the corner whenever you have good ideas :)

> The way I see it, we are outgrowing general purpose bug trackers 
> like
> debbugs. We need a special purpose bug tracker specifically for 
> Guix
> with its special requirements. We are a big enough community for 
> this to
> be important.
>
> I might be able to find some time to implement a simple JSON API 
> for
> mumi. Would there be interest in such a contribution?

Absolutely, yes, please!

> Regarding, hacking on mumi, I understand that 
> issues.guix.gnu.org is on
> an IP whitelist with the GNU debbugs server. How do I hack on 
> mumi if
> simply running it on my local machine, and pulling data from GNU 
> debbugs
> would alarm the debbugs admins?

That’s correct, but mumi itself doesn’t directly talk to debbugs 
any longer.  We just periodically sync all debbugs data from the 
GNU debbugs server and have mumi work on these files locally.  So 
to hack on mumi I’d suggest downloading a copy of the raw debbugs 
data from issues.guix.gnu.org.  I could put an archive somewhere 
where you can fetch it.

-- 
Ricardo

Re: Incentives for review

[zimoun]

Hi Arun,

On Thu, 21 Oct 2021 at 23:51, Arun Isaac <arunisaac@systemreboot.net> wrote:

> The way I see it, we are outgrowing general purpose bug trackers like
> debbugs. We need a special purpose bug tracker specifically for Guix
> with its special requirements. We are a big enough community for this to
> be important.

For an example of such «special purpose bug tracker», give a look at
what Org-mode people do:

   <https://updates.orgmode.org/>

Well, it is currently down.  It looks like that:

   <https://web.archive.org/web/20210228064154/https://updates.orgmode.org/>

The code running it is Woof (lisp based):

   <https://github.com/bzg/woof>

Maybe there are some ideas to borrow. :-)


Cheers,
simon

Re: Incentives for review

[Jonathan McHugh]

Hi Simon,

I had noticed Woof and it does seem interesting and worthy of analysis.

I should point out that the Ogmode community have the same problems and complaints regarding the governance and efficacy of issue/bug tracking. This is a socio-technological problem, which hopefully has solutions which can cut across communities.

Its also worth citing some of the suggestions in this devel-guix thread concerning tooling or options:
https://lists.nongnu.org/archive/html/guix-devel/2021-08/msg00042.html

Suggestions included:
Bugs Everywhere https://bugs-everywhere.readthedocs.io/en/latest/
Git-Issue https://github.com/dspinellis/git-issue
Fossil https://fossil-scm.org/home/doc/trunk/www/bugtheory.wiki

IMHO, I though Bugs Everywhere was the most interesting, though it may have experienced some bitrot.


More recently I have been deliberating on the idea of using a MUD type tool as an interface for issue and bugtracking.

It seems that Powwow would be capable of using scripts in an interesting way.
https://www.hoopajoo.net/static/powwow-mirror/powwow/index.html
This interests me:
```
Powwow also implements the MUME remote editing protocol, which enables you to edit texts on the MUD using your own favourite editor, several texts at once if you have a windowing terminal. 
```
Im also investigating Tintin++
https://tintin.mudhalla.net/

I personally would find it interesting to create 'conversational models' for issue/bug tracking in such an environment, backed by:
* interoperability with computing environments
* interpreting infrastructure
* projectile style configurations
* supplementary protocols
* light writing formats
* configs and bindings to provide workflow

A MUD type inferface is perhaps only a speculative concern but Id be interested in hearing -/+ opinions and interest levels, if not practical suggestions.


As some may be aware, I have been considering the issue/bug tracking topic from the perspective of the Gemini protocol (though now with a wider scale than just comsidering Debbugs). 

That is proceeding, though I was impeeded by some physical setbacks (status:resolved) and completing wider responsibilities. Now back in the flow, I will provide updates once my projects start perculating.

@Arun please mail me privately regarding your plans, Id like to hear about where they may be intersections.

@Christine I look forward to replying to your (private) mail in due course - apoligies for radio silence. If you have suggestions regarding how Guix can go MUD Im sure we'd all be interested!

Kind regards,


====================
Jonathan McHugh
indieterminacy@libre.brussels

October 22, 2021 9:44 AM, "zimoun" <zimon.toutoune@gmail.com> wrote:

> Hi Arun,
> 
> On Thu, 21 Oct 2021 at 23:51, Arun Isaac <arunisaac@systemreboot.net> wrote:
> 
>> The way I see it, we are outgrowing general purpose bug trackers like
>> debbugs. We need a special purpose bug tracker specifically for Guix
>> with its special requirements. We are a big enough community for this to
>> be important.
> 
> For an example of such «special purpose bug tracker», give a look at
> what Org-mode people do:
> 
> <https://updates.orgmode.org>
> 
> Well, it is currently down. It looks like that:
> 
> <https://web.archive.org/web/20210228064154/https://updates.orgmode.org>
> 
> The code running it is Woof (lisp based):
> 
> <https://github.com/bzg/woof>
> 
> Maybe there are some ideas to borrow. :-)
> 
> Cheers,
> simon

Re: Incentives for review

[zimoun]

Hi Jonathan,

On Fri, 22 Oct 2021 at 08:37, "Jonathan McHugh" <indieterminacy@libre.brussels> wrote:

> Its also worth citing some of the suggestions in this devel-guix thread concerning tooling or options:
> https://lists.nongnu.org/archive/html/guix-devel/2021-08/msg00042.html

Thanks.

> Suggestions included:
> Bugs Everywhere https://bugs-everywhere.readthedocs.io/en/latest/
> Git-Issue https://github.com/dspinellis/git-issue
> Fossil https://fossil-scm.org/home/doc/trunk/www/bugtheory.wiki
>
> IMHO, I though Bugs Everywhere was the most interesting, though it may have experienced some bitrot.

Git based, it is possible to use public-inbox with:

<https://yhetil.org/guix-bugs/>


> More recently I have been deliberating on the idea of using a MUD type
> tool as an interface for issue and bugtracking.

Yeah, maybe gamify the review could help.  Maybe Libreadventure could
help.

1: <https://vcs.fsf.org/?p=libreadventure.git;a=tree>
2: <https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00208.html>


Cheers,
simon

Re: Incentives for review

[Jonathan McHugh]

Hi Simon,

October 22, 2021 11:22 AM, "zimoun" <zimon.toutoune@gmail.com> wrote:

> Hi Jonathan,
> 
> On Fri, 22 Oct 2021 at 08:37, "Jonathan McHugh" <indieterminacy@libre.brussels> wrote:
> 
>> Its also worth citing some of the suggestions in this devel-guix thread concerning tooling or
>> options:
>> https://lists.nongnu.org/archive/html/guix-devel/2021-08/msg00042.html
> 
> Thanks.
> 
>> Suggestions included:
>> Bugs Everywhere https://bugs-everywhere.readthedocs.io/en/latest
>> Git-Issue https://github.com/dspinellis/git-issue
>> Fossil https://fossil-scm.org/home/doc/trunk/www/bugtheory.wiki
>> 
>> IMHO, I though Bugs Everywhere was the most interesting, though it may have experienced some
>> bitrot.
> 
> Git based, it is possible to use public-inbox with:
> 
> <https://yhetil.org/guix-bugs>
> 

Yes, Arun has recommended public-inbox before, Im enthused by anything he uses or packages.

>> More recently I have been deliberating on the idea of using a MUD type
>> tool as an interface for issue and bugtracking.
> 
> Yeah, maybe gamify the review could help. Maybe Libreadventure could
> help.
> 
> 1: <https://vcs.fsf.org/?p=libreadventure.git;a=tree>
> 2: <https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00208.html>
> 
Thanks for the catch, I had forgotten that one (and your appeal for someebody to package it!).

The fact its been trialed with FSF and could be Guix Days compatible is a big plus.

TBH, having mastered lua/jit I would feel a bit dirty packaging NPN to satisfy something that Lua would be better suited for. What are the (technical/usability) advantages of choosing LibreAdventure?

I will reflect on this, though Im (personally) more interested in building worlds from GemText than JSON.

# One more suggestion
Has anybody ever considered or used psyc - Protocol for Synchronous Conferencing?

(its how I discovered Powwow)

https://psyc.eu/
``` description
Imagine smartly multicasted chat and conferencing, non-proprietary instant messaging, distributed social networking and data sharing. And now imagine all of this rolled into one. PSYC is an open source protocol and technology, bringing the useful and amazing aspects of several technologies, some of which have been proprietary too long, together. 
````

It has a curious mixture of bit ambition and long history but I cant (yet) tell how modern and functional it is.
Pscyc utilities seem interesting
https://perl.psyc.eu/
```
git2psyc	- report changes to a git repository into a developer chatroom
psycauth	- authenticate something with your PSYC identity
psyccat		- dump a file to a PSYC recipient
psyccmd		- remote control psycamp and whatever wants to be controlled
psycfilemonitor*- notify changes to the file system in realtime
psycion		- amazing console psyc client!
psyclisten	- receive messages and notify the user about them
psycmail	- report incoming emails to the recipient
psycamp*	- media player with PSYC notification and remote control
psycmsg		- send a message to a PSYC recipient
psycnotify	- send presence notification from the command line
psycsyncd	- interfaces PSYC SYNC protocol to DBI (SQL databases etc)
syslog2psyc	- daemon that receives events from syslog and forwards to PSYC
remotor 	- control console for Tor routers that can notify into PSYC
```

For my purposes, the PSYC Syntax Specification could potentially serve orthogonal aspects for complementing the design principles of the Gemini protocol
https://about.psyc.eu/Spec:Syntax
https://gemini.circumlunar.space/docs/specification.html



Jonathan

Re: Incentives for review

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]


Hi Jonathan,

> If I recall, you can request Debbugs content if you email them.

I think you are referring to https://debbugs.gnu.org/server-request.html
I haven't tried it myself, and I'm not sure, but it probably doesn't
give you access to the raw emails. I'm guessing it's just an email
interface to the usual SOAP API.

Regards,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Incentives for review

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]


Hi Ricardo,

> These are all excellent ideas, and seeing you articulate them 
> makes me happy, because in my experience there’s always good code 
> around the corner whenever you have good ideas :)

He he, thanks! :-)

> So to hack on mumi I’d suggest downloading a copy of the raw debbugs
> data from issues.guix.gnu.org.  I could put an archive somewhere where
> you can fetch it.

Yes, please! Some minimal archive along with a few lines of instructions
on how to import it into mumi would be great. It might also be more
generally useful to people who want to hack on mumi.

Thanks,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Incentives for review

[Jonathan McHugh]

October 22, 2021 12:48 PM, "Arun Isaac" <arunisaac@systemreboot.net> wrote:

> Hi Ricardo,
> 
>> These are all excellent ideas, and seeing you articulate them
>> makes me happy, because in my experience there’s always good code
>> around the corner whenever you have good ideas :)
> 
> He he, thanks! :-)
> 
>> So to hack on mumi I’d suggest downloading a copy of the raw debbugs
>> data from issues.guix.gnu.org. I could put an archive somewhere where
>> you can fetch it.
> 
> Yes, please! Some minimal archive along with a few lines of instructions
> on how to import it into mumi would be great. It might also be more
> generally useful to people who want to hack on mumi.
> 
> Thanks,
> Arun

Seconded

Re: Incentives for review

[Jonathan McHugh]

Hi Arun,

I wasnt thinking of that. Ill update you if I find a reference among my notes.


Jonathan

October 22, 2021 12:44 PM, "Arun Isaac" <arunisaac@systemreboot.net> wrote:

> Hi Jonathan,
> 
>> If I recall, you can request Debbugs content if you email them.
> 
> I think you are referring to https://debbugs.gnu.org/server-request.html
> I haven't tried it myself, and I'm not sure, but it probably doesn't
> give you access to the raw emails. I'm guessing it's just an email
> interface to the usual SOAP API.
> 
> Regards,
> Arun

Re: Incentives for review

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 331 bytes --]


Hi zimoun,

> The code running it is Woof (lisp based):
>
>    <https://github.com/bzg/woof>

Ah, woof looks good! :-) It feels a lot like patchwork, but better
integrated into email. I'm tempted to say it's better than our debbugs
bug tracker! :-P But, it may be hard to move out of debbugs now that we
are in it.

Regards,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Incentives for review

[zimoun]

Hi,

On Fri, 22 Oct 2021 at 12:49, Arun Isaac <arunisaac@systemreboot.net> wrote:

> > These are all excellent ideas, and seeing you articulate them
> > makes me happy, because in my experience there’s always good code
> > around the corner whenever you have good ideas :)

It remember me this wishlist: Use Message-ID for indexing instead numbering.

<https://yhetil.org/guix/86sgbhz3fe.fsf@gmail.com/>


Let take an example.  This is indexing #2,

<http://issues.guix.gnu.org/44625#2>

is fine when using a browser but it is pain from Emacs-Notmuch (or any
descent email client) where is it really easy to get the Message-ID.
For instance, I have to open my browser, scroll to find the relevant
message, copy the URL, go back to my email, paste the link for
publicly sharing it.  Message-ID is the natural email-indexing, thus
it appears to me natural to use when the workflow is email-based. :-)


The wishlist is to also allow this:

<http://issues.guix.gnu.org/87wnyo5x7v.fsf@kyleam.com>

as it is done with public-inbox, see
<https://yhetil.org/guix/86sgbhz3fe.fsf@gmail.com>.

Using a bit of Emacs Lisp glue (or submit it to debbugs.el or Gnus),
it becomes easier for both world.


Cheers,
simon

Re: Incentives for review

[zimoun]

Hi Jonathan,

On Fri, 22 Oct 2021 at 12:40, Jonathan McHugh
<indieterminacy@libre.brussels> wrote:

> > <https://yhetil.org/guix-bugs>
>
> Yes, Arun has recommended public-inbox before, Im enthused by anything he uses or packages.

For the Emacs-Notmuch user, give a look at:

<https://yhetil.org/guix/874kl5dh7j.fsf@ambrevar.xyz/>

And for the interested reader about how to use public-inbox, give a
look at the thread:

<http://issues.guix.gnu.org/44625>

It had been discussed several times to have an official public-inbox
instance.  One could imagine: <https://inbox.guix.gnu.org>.  Well,
Org-mode did it [1].  Time is flying and it is hard to burn the candle
at both ends. :-)

1: <https://yhetil.org/orgmode/87k10mt0n6.fsf@bzg.fr/>


> TBH, having mastered lua/jit I would feel a bit dirty packaging NPN to satisfy something that Lua would be better suited for. What are the (technical/usability) advantages of choosing LibreAdventure?

I do not play to game based on computers so I lack imagination on this
area. :-)
That's just because FSF used it for their LibrePlanet event.

Cheers,
simon

Re: Incentives for review

[Thiago Jung Bauermann]

Hello Artem,

Em quinta-feira, 21 de outubro de 2021, às 10:38:44 -03, Artem Chernyak 
escreveu:
> > On Wed, Oct 20, 2021 at 4:37 PM Thiago Jung Bauermann 
<bauermann@kolabnow.com> wrote:
> [...]
> 
> > One thing that would help me would be some way to “subscribe” to
> > changes in certain areas of Guix. That way, when a patch is submitted
> > which touches those areas I would be automatically copied on the
> > emails that go to the guix-patches mailing list. “areas of Guix” could
> > be defined by paths in the repo, guile modules or regexps matching
> > package names, for example.
> You could handle it by filtering the email in your email client.

Yes, that is true. Thank you for the tip. I’ll set up some message filters 
for guix-patches.

And also use a separate mail client just to read the mailing list folders 
which is more suited for that use case (I like my current mail client for 
handling personal email and low volume stuff).

> One thing that could help this, is to include a little more info in
> the subject line for patches. Right now we include the broad area
> (e.g. "gnu: ..."). But we could go on level deeper and include the
> specific file (e.g. "gnu: docker: ..."). This becomes important
> because gnu as an area of guix spans a lot of different packages and
> languages. With the extra file level information, we can easily filter
> it down to only the areas we know about.

That is a good idea. I agree that it would be helpful.

-- 
Thanks,
Thiago

Re: Incentives for review

[Thiago Jung Bauermann]

Hello,

Em quinta-feira, 21 de outubro de 2021, às 13:06:35 -03, Ricardo Wurmus 
escreveu:
> Thiago Jung Bauermann <bauermann@kolabnow.com> writes:
> > 2. Going through the guix-patches mailing list looking for
> > submissions that
> > touch the few areas of Guix where I have at least some
> > experience. I don’t
> > think I found an effective method yet (in part the problem is on
> > my side
> > because the search function of the email client I use isn’t very
> > reliable).
> 
> If a browser fits into your workflow you may want to search for
> submissions on https://issues.guix.gnu.org.  It also shows
> “forgotten” issues that would likely benefit from some poking.

I don’t have a browser currently in my workflow, but I don’t mind using it. 
Thank you for the tip, I’ll try it out. Also thanks for reminding about the 
“forgotten” issues. I’ll look into them.

> > One thing that would help me would be some way to “subscribe” to
> > changes in
> > certain areas of Guix. That way, when a patch is submitted which
> > touches
> > those areas I would be automatically copied on the emails that
> > go to the
> > guix-patches mailing list. “areas of Guix” could be defined by
> > paths in the
> > repo, guile modules or regexps matching package names, for
> > example.
> 
> Perhaps issues.guix.gnu.org could offer atom feeds for certain
> keywords (e.g. the name of the module touched by the commits?).

That is a great idea.

-- 
Thanks,
Thiago

Re: Incentives for review

[Kyle Meyer]

Thiago Jung Bauermann writes:

> Em quinta-feira, 21 de outubro de 2021, às 10:38:44 -03, Artem Chernyak 
> escreveu:
[...]
>> One thing that could help this, is to include a little more info in
>> the subject line for patches. Right now we include the broad area
>> (e.g. "gnu: ..."). But we could go on level deeper and include the
>> specific file (e.g. "gnu: docker: ..."). This becomes important
>> because gnu as an area of guix spans a lot of different packages and
>> languages. With the extra file level information, we can easily filter
>> it down to only the areas we know about.
>
> That is a good idea. I agree that it would be helpful.

Fwiw public-inbox (<http://public-inbox.org/>) indexes the file name
from diffs.  Searching with the "dfn:" prefix against
<https://yhetil.org/guix-patches>, you can get a feed of changes
touching particular paths by using the "dfn:" prefix .  For example:

  https://yhetil.org/guix-patches/?q=dfn:docker&x=A

Re: Incentives for review

[Thiago Jung Bauermann]

Hello Kyle,

Em sexta-feira, 22 de outubro de 2021, às 22:43:05 -03, Kyle Meyer 
escreveu:
> Thiago Jung Bauermann writes:
> > Em quinta-feira, 21 de outubro de 2021, às 10:38:44 -03, Artem Chernyak
> > escreveu:
> [...]
> 
> >> One thing that could help this, is to include a little more info in
> >> the subject line for patches. Right now we include the broad area
> >> (e.g. "gnu: ..."). But we could go on level deeper and include the
> >> specific file (e.g. "gnu: docker: ..."). This becomes important
> >> because gnu as an area of guix spans a lot of different packages and
> >> languages. With the extra file level information, we can easily filter
> >> it down to only the areas we know about.
> > 
> > That is a good idea. I agree that it would be helpful.
> 
> Fwiw public-inbox (<http://public-inbox.org/>) indexes the file name
> from diffs.  Searching with the "dfn:" prefix against
> <https://yhetil.org/guix-patches>, you can get a feed of changes
> touching particular paths by using the "dfn:" prefix .  For example:
> 
>   https://yhetil.org/guix-patches/?q=dfn:docker&x=A

That is very good! I didn’t know that public-inbox did that. Thank you for 
the tip!

-- 
Thanks,
Thiago

Re: Incentives for review

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 195 bytes --]


Hi zimoun,

> The wishlist is to also allow this:
>
> <http://issues.guix.gnu.org/87wnyo5x7v.fsf@kyleam.com>

I've never needed this, but this is definitely a good feature to have!

Cheers! :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Incentives for review

[zimoun]

Hi,

On Fri, 22 Oct 2021 at 21:43, Kyle Meyer <kyle@kyleam.com> wrote:

> Fwiw public-inbox (<http://public-inbox.org/>) indexes the file name
> from diffs.  Searching with the "dfn:" prefix against
> <https://yhetil.org/guix-patches>, you can get a feed of changes
> touching particular paths by using the "dfn:" prefix .  For example:
>
>   https://yhetil.org/guix-patches/?q=dfn:docker&x=A

Oh, that’s really cool!

Do you know a bridge from Elfeed to Message-mode?

I mean, using the feed you are referring, Alice gets:

--8<---------------cut here---------------start------------->8---
Title: [bug#50227] [PATCH] build-system/go: Trim store references using the native compiler option.
Author: Marius Bakke <marius@gnu.org>
Date: Fri, 27 Aug 2021 18:45:37 CEST
Feed: dfn:docker - search results
Link: https://yhetil.org/guix-patches/20210827164423.17109-1-marius@gnu.org/

* guix/build/go-build-system.scm (build): Add '-trimpath' to the 'go install'
invocation.
[…]
--8<---------------cut here---------------end--------------->8---

This is really nice for filtering and only reading what is of interest
(for Alice).

However, it is not handy for commenting.  It could be cool to have a way
to turn what I showed (above) into a reply message.  Does a bridge exist
somewhere?


Cheers,
simon

public-inbox/elfeed -> Maildir bridge (was: Incentives for review)

[Kyle Meyer]

zimoun writes:

> On Fri, 22 Oct 2021 at 21:43, Kyle Meyer <kyle@kyleam.com> wrote:
[...]
>>   https://yhetil.org/guix-patches/?q=dfn:docker&x=A
>
> Oh, that’s really cool!
>
> Do you know a bridge from Elfeed to Message-mode?
>
> I mean, using the feed you are referring, Alice gets:
>
> --8<---------------cut here---------------start------------->8---
> Title: [bug#50227] [PATCH] build-system/go: Trim store references using the native compiler option.
> Author: Marius Bakke <marius@gnu.org>
> Date: Fri, 27 Aug 2021 18:45:37 CEST
> Feed: dfn:docker - search results
> Link: https://yhetil.org/guix-patches/20210827164423.17109-1-marius@gnu.org/
>
> * guix/build/go-build-system.scm (build): Add '-trimpath' to the 'go install'
> invocation.
> […]
> --8<---------------cut here---------------end--------------->8---
>
> This is really nice for filtering and only reading what is of interest
> (for Alice).
>
> However, it is not handy for commenting.  It could be cool to have a way
> to turn what I showed (above) into a reply message.  Does a bridge exist
> somewhere?

Good question.  It does :)

With the link in the Elfeed buffer, we can grab the mbox for a message
or entire thread from a public-inbox instance.  So, for those that use a
Maildir locally, the steps are to

  1) download the message (or thread)
  2) convert the mbox into Maildir messages
  3) visit the message in your regular mail client
  4) proceed as usual

piem can take care of 1 and 2 (as well as 3, with some user
configuration) via its piem-inject-thread-into-maildir command:

  https://docs.kyleam.com/piem/Injecting-messages-into-a-Maildir-directory.html

This command isn't specific to Elfeed buffers.  It just needs to be in a
buffer where piem knows how to grab the public-inbox link:

  https://docs.kyleam.com/piem/Enabling-integration-libraries.html

The other supported modes that are interesting in this context are EWW
and Gnus.

Elfeed -> Notmuch
=================

zimoun, I know you're a Notmuch user, so here's how you could configure
things so that calling piem-inject-thread-into-maildir from the Elfeed
buffer above throws you into a Notmuch show buffer for the message.

 * add a guix-patches entry to piem-inboxes

     (add-to-list 'piem-inboxes
                  '("guix-patches" :url "https://yhetil.org/guix-patches/"))

 * point piem to your Maildir

     (setq piem-maildir-directory "/path/to/maildir/")

   Alternatively, messages for different projects can be sent to
   different Maildir directories using the :maildir keyword in the
   piem-inboxes entry.  (This feature was added by Xinglu Chen :>)

 * enable Elfeed integration

     (piem-elfeed-mode 1)

 * tell piem to visit the message in Notmuch after injecting

     (add-hook 'piem-after-mail-injection-functions
                 (lambda (mid)
                   (require 'notmuch-lib)
                   (message "Running notmuch new")
                   (call-process notmuch-command nil nil nil "new")
                   (notmuch-show (concat "id:"  mid))))

   You actually asked about ending up in a (Notmuch) message mode buffer
   rather than a Notmuch show buffer.  Perhaps tossing a
   notmuch-show-reply in there after notmuch-show will work as expected,
   though I haven't tested it.

lei
===

This email is already too long, but I should briefly mention that Eric
Wong (public-inbox's creator) has been working on a local command-client
client for public-inbox called lei (local email interface).

To continue with the original dfn example, you could do something like
this with lei to dump those results to a Maildir and then view those in
mutt:

  $ lei q -o /tmp/mdir --mua mutt \
    -I https://yhetil.org/guix-patches dfn:docker d:4.months.ago..
  # later: update with new results and visit in mutt
  $ lei up --mua mutt /tmp/mdir

Anyway, that's just a small piece of what lei can do, and IMO it's
really impressive and exciting.  It will be a part of the next
public-inbox release, v1.7.  (How this all ends up integrating with piem
is very much up in the air.)

For a high-level picture that includes public-inbox, lei, and b4:
Konstantin Ryabitsev, b4's creator, recently talked at the Linux
Plumbers Conference:

  https://linuxplumbersconf.org/event/11/contributions/983/attachments/759/1421/Doing%20more%20with%20lore%20and%20b4.pdf

The lei part starts on page 24.  I believe there's a video out there,
but I haven't watched it and don't have a link on hand.

Re: Incentives for review

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3621 bytes --]

On Thu, Oct 21, 2021 at 10:51:07AM -0700, Vagrant Cascadian wrote:
> On 2021-10-19, Ludovic Courtès wrote:
> > zimoun <zimon.toutoune@gmail.com> skribis:
> >
> >> On Tue, 19 Oct 2021 at 14:56, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
> 
> >> One question is “encouragement” for reviewing, somehow.  Asking for new
> >> package additions to go via guix-patches is a call making kind of
> >> equality between contributors.  As someone without commit access, I can
> >> tell you that it is often demotivating to send a trivial addition, wait
> >> forever, ping people (aside I know who I have to ping :-)).  Usually, it
> >> means people are busy elsewhere, so I try to help to reduce the workload
> >> by reviewing stuff or by doing bug triage.  However, in the same time, I
> >> see committers push their own trivial additions.  It appears to me
> >> “unfair”.
> >
> > I understand and sympathize (I also see us slipping off-topic :-)).
> >
> >> Why are committer’s trivial additions more “urgent” than mine?
> >
> > Yeah, I see what you mean.
> >
> > I would like to see us committers do more review work.  But I also view
> > things from a different angle: everyone contributes in their own way,
> > and each contribution is a gift.  We can insist on community
> > expectations (reviewing other people’s work), but we should also welcome
> > contributions as they come.
> 
> I must admit, I don't review patches unless they're in an area of
> expertise (e.g. u-boot, arm-trusted-firmware, reproducible builds
> tooling, etc.); I just don't have sufficient skill with guile to review
> arbitrary packages in a meaningful way, other than the most trivial of
> packages...
> 
> Before I was granted commit access, I *really* appreciated getting
> review... but was also frustrated by how long it took to get a
> contribution in; having limited time available for guix, spending that
> energy checking if something I'd already "finished" was actually merged
> was a bit demotivating.
> 
> I have added a small number of trivial packages without review; maybe I
> shouldn't have... but it was also a bit of a sigh of relief once I could
> push directly to no have to get caught up in the waiting game; I had
> more time to actually contribute other improvements to guix.
> 
> 
> > There’s a balance to be found between no formal commitment on behalf of
> > committers, and a strict and codified commitment similar to what is
> > required for participation in the distros list¹.
> 
> So yeah, it is a quite balancing act!
> 
> 
> Would a workflow of pushing to a "wip-pending" branch in guix.git that
> then gets merged and/or cherry-picked into master/staging/core-updates
> help at all?
> 
> A cursory review could commit to "wip-pending", with the
> plan/hope/expectation that it would get some other review and/or a
> timeout before it gets merged.
> 
> I guess it would be hard to avoid having to constantly rebase with the
> latest updates... "wip-pending" might just add more work to an already
> needs-more-resources process...
> 
> 
> live well,
>   vagrant

There is cbaines' guix-patches service (and other things), where you can
add the git remote https://git.guix-patches.cbaines.net/git/guix-patches
and then cherry-pick commits from there to the necessary branch. I
haven't used it much though.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: public-inbox/elfeed -> Maildir bridge (was: Incentives for review)

[Jonathan McHugh]

Hi Kyle,

Lei and B4 look like very cool tools, thanks for the reference!

====================
Jonathan McHugh
indieterminacy@libre.brussels

October 23, 2021 6:19 PM, "Kyle Meyer" <kyle@kyleam.com> wrote:

> zimoun writes:
> 
>> On Fri, 22 Oct 2021 at 21:43, Kyle Meyer <kyle@kyleam.com> wrote:
> 
> [...]
> 
>>> https://yhetil.org/guix-patches/?q=dfn:docker&x=A
>> 
>> Oh, that’s really cool!
>> 
>> Do you know a bridge from Elfeed to Message-mode?
>> 
>> I mean, using the feed you are referring, Alice gets:
>> 
>> --8<---------------cut here---------------start------------->8---
>> Title: [bug#50227] [PATCH] build-system/go: Trim store references using the native compiler option.
>> Author: Marius Bakke <marius@gnu.org>
>> Date: Fri, 27 Aug 2021 18:45:37 CEST
>> Feed: dfn:docker - search results
>> Link: https://yhetil.org/guix-patches/20210827164423.17109-1-marius@gnu.org
>> 
>> * guix/build/go-build-system.scm (build): Add '-trimpath' to the 'go install'
>> invocation.
>> […]
>> --8<---------------cut here---------------end--------------->8---
>> 
>> This is really nice for filtering and only reading what is of interest
>> (for Alice).
>> 
>> However, it is not handy for commenting. It could be cool to have a way
>> to turn what I showed (above) into a reply message. Does a bridge exist
>> somewhere?
> 
> Good question. It does :)
> 
> With the link in the Elfeed buffer, we can grab the mbox for a message
> or entire thread from a public-inbox instance. So, for those that use a
> Maildir locally, the steps are to
> 
> 1) download the message (or thread)
> 2) convert the mbox into Maildir messages
> 3) visit the message in your regular mail client
> 4) proceed as usual
> 
> piem can take care of 1 and 2 (as well as 3, with some user
> configuration) via its piem-inject-thread-into-maildir command:
> 
> https://docs.kyleam.com/piem/Injecting-messages-into-a-Maildir-directory.html
> 
> This command isn't specific to Elfeed buffers. It just needs to be in a
> buffer where piem knows how to grab the public-inbox link:
> 
> https://docs.kyleam.com/piem/Enabling-integration-libraries.html
> 
> The other supported modes that are interesting in this context are EWW
> and Gnus.
> 
> Elfeed -> Notmuch
> =================
> 
> zimoun, I know you're a Notmuch user, so here's how you could configure
> things so that calling piem-inject-thread-into-maildir from the Elfeed
> buffer above throws you into a Notmuch show buffer for the message.
> 
> * add a guix-patches entry to piem-inboxes
> 
> (add-to-list 'piem-inboxes
> '("guix-patches" :url "https://yhetil.org/guix-patches"))
> 
> * point piem to your Maildir
> 
> (setq piem-maildir-directory "/path/to/maildir/")
> 
> Alternatively, messages for different projects can be sent to
> different Maildir directories using the :maildir keyword in the
> piem-inboxes entry. (This feature was added by Xinglu Chen :>)
> 
> * enable Elfeed integration
> 
> (piem-elfeed-mode 1)
> 
> * tell piem to visit the message in Notmuch after injecting
> 
> (add-hook 'piem-after-mail-injection-functions
> (lambda (mid)
> (require 'notmuch-lib)
> (message "Running notmuch new")
> (call-process notmuch-command nil nil nil "new")
> (notmuch-show (concat "id:" mid))))
> 
> You actually asked about ending up in a (Notmuch) message mode buffer
> rather than a Notmuch show buffer. Perhaps tossing a
> notmuch-show-reply in there after notmuch-show will work as expected,
> though I haven't tested it.
> 
> lei
> ===
> 
> This email is already too long, but I should briefly mention that Eric
> Wong (public-inbox's creator) has been working on a local command-client
> client for public-inbox called lei (local email interface).
> 
> To continue with the original dfn example, you could do something like
> this with lei to dump those results to a Maildir and then view those in
> mutt:
> 
> $ lei q -o /tmp/mdir --mua mutt \
> -I https://yhetil.org/guix-patches dfn:docker d:4.months.ago..
> # later: update with new results and visit in mutt
> $ lei up --mua mutt /tmp/mdir
> 
> Anyway, that's just a small piece of what lei can do, and IMO it's
> really impressive and exciting. It will be a part of the next
> public-inbox release, v1.7. (How this all ends up integrating with piem
> is very much up in the air.)
> 
> For a high-level picture that includes public-inbox, lei, and b4:
> Konstantin Ryabitsev, b4's creator, recently talked at the Linux
> Plumbers Conference:
> 
> https://linuxplumbersconf.org/event/11/contributions/983/attachments/759/1421/Doing more with lore
> and b4.pdf
> 
> The lei part starts on page 24. I believe there's a video out there,
> but I haven't watched it and don't have a link on hand.

Re: Tricking peer review

[Christine Lemmer-Webber]

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> It builds just fine:
>
> $ guix build -f /tmp/content-addressed.scm  
> /gnu/store/lpais26sjwxcyl7y7jqns6f5qrbrnb34-sed-4.8
> $ guix build -f /tmp/content-addressed.scm -S --check -v0
> /gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz
>
>
> Did you spot a problem?
>
> …
>
> \f
> So, what did we just build?
>
> $ ls $(guix build -f /tmp/content-addressed.scm)/bin
> egrep  fgrep  grep
>
>
> Oh oh!  This ‘sed’ package is giving us ‘grep’!  How come?
>
> The trick is easy: we give a URL that’s actually 404, with the hash of a
> file that can be found on Software Heritage (in this case, that of
> ‘grep-3.4.tar.xz’).  When downloading the source, the automatic
> content-addressed fallback kicks in, and voilà:
>
> $ guix build -f /tmp/content-addressed.scm  -S --check 
> La jena derivaĵo estos konstruata:
>    /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
> building /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv...
>
> Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-4.8.tar.gz
>>From https://ftpmirror.gnu.org/gnu/zed/sed-4.8.tar.gz...
> following redirection to `https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz'...
> download failed "https://mirror.cyberbits.eu/gnu/zed/sed-4.8.tar.gz" 404 "Not Found"
>
> [...]
>
> Starting download of /gnu/store/1mlpazwwa2mi35v7jab5552lm3ssvn6r-sed-4.8.tar.gz
>>From https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/...
> downloading from https://archive.softwareheritage.org/api/1/content/sha256:58e6751c41a7c25bfc6e9363a41786cff3ba5709cf11d5ad903cf7cce31cc3fb/raw/ ...
>
> warning: rewriting hashes in `/gnu/store/mgais6lk92mm8n5kyx70knr11jbwgfhr-sed-4.8.tar.gz'; cross fingers
> successfully built /gnu/store/nq2jdzbv3nh9b1mglan54dcpfz4l7bli-sed-4.8.tar.gz.drv
>
>
> It’s nothing new, it’s what I do when I want to test the download
> fallbacks (see also ‘GUIX_DOWNLOAD_FALLBACK_TEST’ in commit
> c4a7aa82e25503133a1bd33148d17968c899a5f5).  Still, I wonder if it could
> somehow be abused to have malicious packages pass review.

Here's another way to think of it, borrowing from some ocap systems: the
hash is the actual, canonical identifier of this package revision.  The
URL to get the package is merely a "hint" as to where to get it.

Therefore, there can be many other "hints" as to where to get it too,
enabling mirrors to be elevated to the "same" priority as the original
source.

Re: Tricking peer review

[Ludovic Courtès]

Hi!

Christine Lemmer-Webber <cwebber@dustycloud.org> skribis:

> Here's another way to think of it, borrowing from some ocap systems: the
> hash is the actual, canonical identifier of this package revision.  The
> URL to get the package is merely a "hint" as to where to get it.

Yes, definitely.  It remains we need to educate ourselves to not give
too much credit to this hint.

Thanks,
Ludo’.

Re: Incentives for review

[Katherine Cox-Buday]

zimoun <zimon.toutoune@gmail.com> writes:

>> I have often seen folks on various projects worried about the size of
>> various backlogs: bugs, issues, etc. I think it is human to want to
>> try and contain something that appears to be growing, unbounded. 
>
> …about patches only.  Bug is another story. :-)

Sorry, I meant to speak to both and instead repeated bugs with a different word, issues! I think patches and bugs are similar in this context.

> Just number to fix the idea about large backlog.

I think it's really great that you went through the trouble to quantify this. Thank you.

>> I think the thing that bothers us is a sense that the backlog is
>> becoming unmanageable, or too large to triage. I submit that this is
>> actually a tooling and organizational issue, and not an intrinsic
>> issue to be solved. Bugs may still be valid; patches may still have
>> valuable bones to modify.
>
> This is the point.  What do you do?  What could we improve about tooling
> and organisation to better scale and deal with this “becoming
> unmanageable backlog”?

I tried to give some ideas here[1].

> From my point of view, it is good to have this issue.  It means that
> Guix is becoming more popular.  And we – regular user, contributor,
> committer – have to adapt to this increasing workload, IMHO.

I totally agree!

> The question is how.  And how to invite people to complete review. :-)

Humans usually enjoy community. I think the group activities are really great.

>> I think the real issue is that as a backlog grows, the tools we're
>> used to using cannot answer the questions we want to ask: what is most
>> relevant to me or the project right now?
>
> If it is relevant to the project then it is also relevant to me as an
> user.  And vice-versa. ;-)

I think I did not give the proper context. I meant relevant as in "I am working on this package. Is anyone else? What tickets might I update? What other trivial bugs might I fix while I'm looking at this?"

I.e. relevant in the temporal sense.

> When something relevant to me is not making progress, it often means
> people are busy elsewhere, so I try to comment (review?) about patches
> or bugs.  It is a Sisyphean task because the workload never
> decreases. :-)  Or maybe structured procrastination. ;-)

I find it helpful to not think of it as a discrete task, but work along a continuum -- a joyful habit of collectively helping everyone to have something nice :)

"A society grows great when old (wo)men plant trees whose shade they know they shall never sit in."

[1] - https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00158.html

-- 
Katherine