Re: Guix Days: Patch flow discussion

Re: Guix Days: Patch flow discussion

[Suhail]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> This list is missing one point - which has been discussed several
> times already without any result:
>
> The current mail-based workflow is too complicated for new and
> occasional committers.

Could you please share a reference where the key difficulties you
encountered wrt the "current mail-based workflow" are summarized.  Is
the difficulty regd. checking out the code at the right commit and
installing the patches, or something else?

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

Am 05.02.24 um 19:44 schrieb Suhail:
> Could you please share a reference where the key difficulties you
> encountered wrt the "current mail-based workflow" are summarized.  Is
> the difficulty regd. checking out the code at the right commit and
> installing the patches, or something else?

It's not only installing and testing the patches, but also

  * when has this issue/patch been worked on last - is somebody
    currently working on it
  * what issue/patches I started to review?
  * commenting on code requires to download the patch - strip out parts
    which are okay, comment, then mail the commented code to the correct
    issue number
  * Even when using the debbugs interface in emacs
      o It is is hard to use for occasional users
      o It is an insurmountable obstacle for those not using emacs.
      o It does not tell what issues/patches I've been working on
        already - and waiting for a reply
      o It does not tell which issues are stale

Anyhow, all of this has been discussed several times already. And as 
long as vocal (and active :-) members of the community insist on being 
able to work via e-mail — while also not adopting modern e-mail-capable 
forges — this situation will not change.

Regards
Hartmut Goebel

| Hartmut Goebel          |h.goebel@crazy-compilers.com                |
|www.crazy-compilers.com  | compilers which you thought are impossible |

[-- Attachment #2: Type: text/html, Size: 2090 bytes --]

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1977 bytes --]

Hi Hartmut,

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Anyhow, all of this has been discussed several times already. And as 
> long as vocal (and active :-) members of the community insist on being 
> able to work via e-mail — while also not adopting modern e-mail-capable 
> forges — this situation will not change.

While I agree with your assessment of what's missing in this workflow,
I'm not sure I know of any e-mail-capable forges that resolve those
particular pain points…  People often think that eg. sr.ht works better
for this, but I don't think the sr.ht MLs have any more structure than
what we have.  For smaller projects, it works okay, but once you get to
the scale of something like Guix, it doesn't offer anything more.

One thing I would like to get rid of though is debbugs.  It causes a lot
of pain for everyone, eg. when sending patchsets, it completely breaks
modern email because it insists on rewriting DMARC-protected headers,
thus needing to also rewrite "From:" to avoid DMARC errors.

I've been following the Linux kernel development a bit closer this past
year, and while there are things that need to improve (like knowing the
status of a patchset in a maintainer's tree), they at least have a lot
of tools that I think we should adopt more broadly:

b4/lei is a nice example (we already have yhetil.org as a back-end, but
maybe a more blessed one would be better) of a tool that lets you
completely automate applying a patchset to a branch.

patchwork is a nice tool to gather up and track patchsets, with status
indicators like "under review", "accepted", etc.  Chris already deploys
one as part of QA, more integration with it would be nice.

One advantage of this is that we would benefit from upstream's bigger
user-base and dev time, and also from the fact that these tools are made
to work together (b4 can automatically mark patchsets as accepted)!

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Debbugs update (Was: Guix Days: Patch flow discussion)

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Hartmut & Josselin,

On Mon, Feb 05 2024, Hartmut Goebel wrote:

> Am 05.02.24 um 10:39 schrieb Steve George:
>
> [order of quotations reversed]
>
> The current mail-based workflow is too complicated ... which has been
> discussed several times already without any result:

Well, that's not totally true. After hearing the plight last year, I
offered the FSF to assume responsibility for debbugs.gnu.org. [1]

I also packaged and deployed on GNU Guix

  (A) the fifteen-year old Debbugs version deployed at gnu.org [2][3][4]
  (B) the modern Debbugs version deployed at debian.org [5][6][7]
  (C) and a custom version of Mumi for my own bug fixes [8][9]

Together with the official debbugs.gnu.org, which runs on Debian 8, and
issues.guix.gnu.org, I am now working to fold all five deployments into
one.

On Tue, Feb 06 2024, Josselin Poiret wrote:

> One thing I would like to get rid of though is debbugs.

I can do little to appease the hardcore Debbugs haters, but I have a
vision for a bug tracking system that, written in GNU Guile, will
attract less wholesale criticism and more constructive contributions
from from the Emacs and Guix communities, which are the primary users of
debbugs.gnu.org.

Both groups are already in love with the lambda calculus.

Mumi made great steps in that direction but has so far not attracted the
contributions it deserves.

In another piece of the puzzle, Michael Albinus wrote and maintains an
excellent Emacs package called Debbugs.el. It allows bug work to take
place in Gnus and Org Modes [10] rather than in a web browser. With that
package, Emacs could become a favorite way to work on our bugs and
patches, similar to what Magit did for Git.

At Guix, folks also do not use Debbugs to its full potential. Git hooks
are an example. Mumi obscures some features. I know because I worked on
Debian's bugs for many years.

I am committed to Debbugs because I'm not sure interactions between
people can be audited properly on modern development forges. For many
years, I worked with Salsa, Debian's Gitlab instance. While convenient,
it was difficult to find past conversations and code snippets, although
like Josselin I'm watching what the kernel folks are doing.

The databases in forges are also complicated to maintain. Plus, projects
experience immediate vendor lock in.

My primary hurdle with modernizing Debbugs, if anyone cares, is that the
FSF is reluctant to deploy GNU Guix. They insist on Trisquel plus
Ansible [11] which is what they have been using for some time.

Thank you, everyone, for your hard work on GNU Guix!

Kind regards
Felix

[1] https://lists.nongnu.org/archive/html/help-debbugs/2023-10/msg00003.html
[2] https://debbugs.juix.org/cgi/bugreport.cgi?bug=66703
[3] https://codeberg.org/lechner/juix/src/commit/5fffdb0053b18b4b28adadbbebb79a1d9bfe2337/juix/deploy/debbugs.scm#L1046-L1184
[4] https://codeberg.org/lechner/debbugs-gnu
[5] https://better.juix.org/cgi/bugreport.cgi?bug=66703
[6] https://codeberg.org/lechner/juix/src/commit/5fffdb0053b18b4b28adadbbebb79a1d9bfe2337/juix/deploy/debbugs.scm#L1186-L1364
[7] https://salsa.debian.org/debbugs-team/debbugs
[8] https://mumi.juix.org/66703
[9] https://codeberg.org/lechner/mumi
[10] https://elpa.gnu.org/packages/doc/debbugs-ug.html
[11] https://lists.nongnu.org/archive/html/help-debbugs/2024-01/msg00049.html

Re: Debbugs update (Was: Guix Days: Patch flow discussion)

[Ricardo Wurmus]

Hi Felix,

> I also packaged and deployed on GNU Guix
>
>   (A) the fifteen-year old Debbugs version deployed at gnu.org [2][3][4]
>   (B) the modern Debbugs version deployed at debian.org [5][6][7]
>   (C) and a custom version of Mumi for my own bug fixes [8][9]
>
> Together with the official debbugs.gnu.org, which runs on Debian 8, and
> issues.guix.gnu.org, I am now working to fold all five deployments into
> one.

Let me just pop in to say thank you for that.  I appreciate you taking
the time to talk to Debbugs / GNU folks to work through the obstacles
that the current situation with the GNU fork of Debbugs presents us
with.

I don’t want to add weight to the scales on either side (a future with
or without Debbugs), but I think it is important to identify the actual
limitations and opportunities of the existing platform and chart the
shortest route around them.

It’s how we ended up with mumi in the first place, and I’m hopeful that
this approach can get us to a better place when it comes to tooling.

(The biggest chunk of the problem may well be a social issue, that can
only be overcome if we find a narrative we all can comfortably enact.)

-- 
Ricardo

on the bug tracker (Re: Guix Days: Patch flow discussion)

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 3048 bytes --]

Hi Josselin,

Josselin Poiret <dev@jpoiret.xyz> writes:

[...]

> One thing I would like to get rid of though is debbugs.

given that a significant part of the Guix infrastructure is provided by
the GNU project, including the bug/issue tracker, and I don't think that
GNU will replace https://debbugs.gnu.org/ (or the forge, Savannah) with
something else, I suggest to concentrate the Guix community efforts in
giving contributors better user interfaces to Debbugs, e.g Mumi (web and
CLI) instead of trying to get rid of it.

In other words: the "problem" it's not the tool, it's the *interface*.

Please also consider that if (I hope not) the Guix would decide to adopt
a different bug/issue tracker system then Someome™ will have to
administrate it, and currently there are other pieces of core
infrastructure that need more resources, e.g. QA.

Speaking of interface features, I simply *love* the email based
interface provided by Debbugs [1]; also the web UI is not bad, and the Mumi
one (https://issues.guix.gnu.org/) it's even better.

But I'm curious: what bug tracker would you suggest instead of Debbugs?

Let's see what some "big players" are using...

> It causes a lot of pain for everyone, eg. when sending patchsets, it
> completely breaks modern email because it insists on rewriting
> DMARC-protected headers, thus needing to also rewrite "From:" to avoid
> DMARC errors.

I don't understand what "completely breaks modern email" means: please
could you point me to a document where this issue/bug is documented?

> I've been following the Linux kernel development a bit closer this past
> year, and while there are things that need to improve (like knowing the
> status of a patchset in a maintainer's tree), they at least have a lot
> of tools that I think we should adopt more broadly:

you mention: b4/lei and patchwork but they are not bug/issue trackers.

the Linux kernel community is using https://bugzilla.kernel.org/;
RedHat, Fedora, OpenSUSE and SUSE are also using Bugzilla

Arch Linux have adopted GitLab issues

Other alternavives:
https://en.wikipedia.org/wiki/Comparison_of_issue-tracking_systems

...or:
https://en.wikipedia.org/wiki/Bug_tracking_system#Distributed_bug_tracking

I personally like the idea that the bug/issue tracker is _embedded_
(integrated?) in the DVCS used by the project, Git in Guix case.

For this reason I find Tissue https://tissue.systemreboot.net/ an
interesting project for *public* issue/bug tracking systems, also
because Tissue is _not_ discussion-oriented: this means that
discussions are managed "somewhere else", because «It's much better to
have a clear succinct actionable issue report. This way, the issue
tracker is a list of clear actionable items rather than a mess of
unreproducible issues.»  [2]

Happy hacking! Gio'

[...]

[1] https://debbugs.gnu.org/server-control.html

[2] https://tissue.systemreboot.net/manual/dev/en/#section11795

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: on the bug tracker (Re: Guix Days: Patch flow discussion)

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 4603 bytes --]


-- 

Giovanni Biscuolo <g@xelera.eu> writes:

> Hi Josselin,
>
> Josselin Poiret <dev@jpoiret.xyz> writes:
>
> [...]
>
>> One thing I would like to get rid of though is debbugs.
>
> given that a significant part of the Guix infrastructure is provided by
> the GNU project, including the bug/issue tracker, and I don't think that
> GNU will replace https://debbugs.gnu.org/ (or the forge, Savannah) with
> something else, I suggest to concentrate the Guix community efforts in
> giving contributors better user interfaces to Debbugs, e.g Mumi (web and
> CLI) instead of trying to get rid of it.

We can re-use some other tool as well, no need to code our own in Guile
though.  Also note that the choice of bug-tracking tool is not benign,
as most other tools (Mumi, but also QA) have to adapt to its
idiosyncrasies.

> In other words: the "problem" it's not the tool, it's the *interface*.

I don't agree: see below for the two examples of where the tool design
itself is problematic.

> Please also consider that if (I hope not) the Guix would decide to adopt
> a different bug/issue tracker system then Someome™ will have to
> administrate it, and currently there are other pieces of core
> infrastructure that need more resources, e.g. QA.
>
> Speaking of interface features, I simply *love* the email based
> interface provided by Debbugs [1]; also the web UI is not bad, and the Mumi
> one (https://issues.guix.gnu.org/) it's even better.

I prefer a public-inbox web frontend for browsing emails personally.

>> It causes a lot of pain for everyone, eg. when sending patchsets, it
>> completely breaks modern email because it insists on rewriting
>> DMARC-protected headers, thus needing to also rewrite "From:" to avoid
>> DMARC errors.
>
> I don't understand what "completely breaks modern email" means: please
> could you point me to a document where this issue/bug is documented?

I don't think there's a "public" discussion of this, but I've conversed
with the mailman maintainers in private about this: because Debbugs
relies on modifying the mail message itself, it clashes with modern
email features that authentify received emails.

DKIM authentifies a subset of mail headers by signing them with a
private key, among which the "From" header is mandatory.  DMARC is a
policy mechanism for domains that can enforce DKIM (and/or SPF) for all
mails with a corresponding "From:".  Together, they prevent email
impersonation.

So Debbugs modifies a protected header (e.g. Subject) -> DKIM fails ->
if DMARC is set to enforce DKIM, DMARC fails -> mail is not received.

To avoid this issue, Mailman (the mailing daemon overseeing the
debbugs-managed MLs) also rewrites the From: header of such mails to a
debbugs-controlled domain, so that DKIM headers can be re-signed and a
different DMARC policy be used.  This is why you often see "via
guix-patches" in From: headers, and if you look at the email it's a
generic one.


I've also mentioned this before, but not being able to simply send a
full patchset without having to go through hoops is, put simply, a
problem.  Even prospective contributors used to email-based workflows
will be turned off by this.


In general, the problem with Debbugs is that it insists on *taking over*
emails, rather than just consuming them.

>> I've been following the Linux kernel development a bit closer this past
>> year, and while there are things that need to improve (like knowing the
>> status of a patchset in a maintainer's tree), they at least have a lot
>> of tools that I think we should adopt more broadly:
>
> you mention: b4/lei and patchwork but they are not bug/issue trackers.

Yes, I was mentioning general tools.

> I personally like the idea that the bug/issue tracker is _embedded_
> (integrated?) in the DVCS used by the project, Git in Guix case.
>
> For this reason I find Tissue https://tissue.systemreboot.net/ an
> interesting project for *public* issue/bug tracking systems, also
> because Tissue is _not_ discussion-oriented: this means that
> discussions are managed "somewhere else", because «It's much better to
> have a clear succinct actionable issue report. This way, the issue
> tracker is a list of clear actionable items rather than a mess of
> unreproducible issues.»  [2]

I like this separation as well, and Tissue seemed interesting the last
time I looked at it.  I liked how it uses standard tools and can be
consulted/modified off-line.  It basically checks all of the boxes :)

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]