Guix Days: Patch flow discussion

Guix Days: Patch flow discussion

[Steve George]

Hi,

Our goal for the discussion:

	How do we double the number of patches that are *reviewed* and
	*applied* to Guix in the next six months?

Patch flow is a pipeline, to change it we could:

a. Increase the number of committers - more people to do the
work
b. Increase the efficiency of existing committers
c. Open the gates by decreasing the quality expected from patches

We essentially decided to focus our discussion on (b). We looked at
things that 'hinder' and 'help' patch review:


Hinders
========

- All our patch reviewers are volunteers doing it in their spare time.

- For a volunteer reviewing someone else's work is not very rewarding, 
most would prefer to use that precious time to scratch their own itch.

- Can feel like an Sisyphean task: no matter how many patches someone 
reviews there are more, exacerbated by the number of Guix packages.

- Sense of responsibility: the minute that a reviewer looks at the patch 
they are now stuck with it

- Repetitive and boring: often patches have minor issues, but it's the 
same sorts of issues time and time again.

- Risk of negative social interaction: having to tell someone that their 
patch is incorrect, or that their contribution cannot be used is 
difficult and draining. Some people felt it was better to say nothing, 
rather than to respond to a patch.


Helps
======

This led us to the focus on the fact that **reviewing and applying
patches can be different people**

We looked for ideas to create more reviewers, make reviewing easier and
more fun:


- Share in the work
--------------------

1. encourage new reviewers to step forward - making it more known that 
reviewing patches helps to get them applied. Anyone can review patches.

2. create directed 'how-to' documentation for reviewing and connect it 
to QA so that 'new reviewers' know what to do

3. create documentation about 'when' and 'how' it's appropriate to send 
a 'v2' version of a patch so that the QA system builds and accepts it. 
Sometimes, patches rot because non-committers don't want to be seen as 
'stealing' someone's work with a v2 patch - but making the small changes 
and resubmitting to QA is what is required.

4. Pay someone else to do it. Noted but out of scope.
5. Remove old packages overhead. Old untouched packages create mental 
overhead, and make the task of maintaining the repository in a good 
state more difficult. We could remove old 'untouched' packages and ones 
that no-longer compile. We have methods to hide and notify.


- Make it more fun
-------------------

1. do online sessions around reviews, some sprints or pairing - both 
social and a way to spread skills
2. find ways to recognise and appreciate reviewers - 'reviewer of the month'
3. make it a game - we could have a 'Guix London' vs 'Guix Paris' leader 
board for reviews. Make it a group goal 'can we beat januarys reviews 
number'
4. create some graphs / leaderboard so we know how many patches are 
being reviewed and we can recognise the contribution


- Automate it away
-------------------

1. Chris is continuing to try and automate away the boring work - 
general agreement in the group that QA has made a lot of difference.

2. general discussion about create a 'guix review' command (Nix has one) 
which would download a branch with the appropriate patch and build it 
locally. This is for instances where some adjustment is needed or to 
check a build. While this can be done today, it's a number of steps and 
quite involved.


Agreed Actions
==============

* [Chris]: continuing his work to improve QA automation. Implication was 
we'll need some reports / graphs - but these were not discussed in detail.

* [Futurile]: organise a **patch review online sessions**. To run every 
13 days (so it rotates through the week) - for 3 months to see if it has 
any traction. Co-ordinate with maintainers so that patches that are 
reviewed can be committed


Actions looking for someone - you?
====================================

* Carry forward the 'guix review' command idea

* Write an RFC and discuss the idea of removing older 'bit-rot' packages

* write 'how-to' documentation for reviews and when it's socially
acceptable to do a v2 patch. A checklist-like approach.


If you were in the discussion and I've misrepresented your point, or 
forgotten an important aspect please please reply and correct me.

Also, if you would like to help on any of the tasks please email back to 
the group so we can all co-ordinate.

Finally, thank-you to everyone who came along and put their shared brain 
power to the task - look forward to doing some patch reviews together 
online in the coming weeks!

Thanks,

Steve/Futurile

Re: Guix Days: Patch flow discussion

[Leo Famulari]

On Mon, Feb 5, 2024, at 04:39, Steve George wrote:
> Our goal for the discussion:
>
> 	How do we double the number of patches that are *reviewed* and
> 	*applied* to Guix in the next six months?
>
> Patch flow is a pipeline, to change it we could:
>
> a. Increase the number of committers - more people to do the
> work
> b. Increase the efficiency of existing committers
> c. Open the gates by decreasing the quality expected from patches

Hi George,

It's an important subject and, in my opinion, more important than any technical questions at this stage of the project.

However, I think the question assumes that all contributions should be accepted, and that the entire problem is that we are not accepting them efficiently enough. We should not unconsciously accept this assumption.

Guix can reject contributions, either in a general way (we don't want that type of thing in Guix), or due to specific reasons (the code is not idiomatic, the contributor can't work effectively with the rest of the group, etc).

Leo

Re: Guix Days: Patch flow discussion

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

On 2024-02-05 09:07:05 -0500, Leo Famulari wrote:
> However, I think the question assumes that all contributions should be accepted, and that the entire problem is that we are not accepting them efficiently enough. We should not unconsciously accept this assumption.
>
> Guix can reject contributions, either in a general way (we don't want that type of thing in Guix), or due to specific reasons (the code is not idiomatic, the contributor can't work effectively with the rest of the group, etc).

And I believe that is fine and completely within the rights of the committers.
However, in my experience contributing, the issue is not that people explicitly
tell me to go away, but that the patches are just ignored.

I would say that the outcome (for any specific patch) from the proposed "review
session" can very well be patch just being closed, or tagged as "need-more-work"
(or whatever tag is Guix using).

In ideal world, there would always be *some* reaction from the project, even if
that reaction would be "we do not want this change".  Even if it would be just
an auto-close (for the "contributor can't work effectively..." case).

Or, to put it in a different way: The problem is not that too few patches get
merged.  The problem is that too few patches get reviewed.

Have a nice day,
Tomas Volf

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

Hello,

On Mon, Feb 05 2024, Steve George wrote:

> Hi,
>
> Our goal for the discussion:
>
> 	How do we double the number of patches that are *reviewed* and
> 	*applied* to Guix in the next six months?
>
> Patch flow is a pipeline, to change it we could:
>
> a. Increase the number of committers - more people to do the
> work

I don't think reviewers have to be committers.  If a patch is marked as
‘reviewed’ by a non-commiter, another person can just apply the patch
and push.  Although the commiter will take some responsibility in doing
so, and make sure the review was done correctly, it still makes their
work way easier.

My point is that not being committer shouldn't discourage people to
review patches.

Clément

Re: Guix Days: Patch flow discussion

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Clément,

On Mon, Feb 05 2024, Clément Lassieur wrote:

> I don't think reviewers have to be committers.

How much more evidence does the project need to see in order to realize
that the plan is not working?

I'll spare the list a lengthy analysis of the social dynamics but the
delegation of competency to the bottom is an unsuccessful practice in
many places.

Maintainers must close bugs in service to the public. There is no other
way.

Kind regards
Felix

Re: Guix Days: Patch flow discussion

[Suhail]

Tomas Volf <~@wolfsden.cz> writes:

> In ideal world, there would always be *some* reaction from the
> project, even if that reaction would be "we do not want this change".

Agreed.  A reduction in the time between a patch (or patch revision)
submission and a review/reaction would be a positive change in my
opinion.

> Even if it would be just an auto-close (for the "contributor can't
> work effectively..." case).

Are there current instances of "contributor can't work effectively"?  If
not, I propose that decisions concerning those situations be deferred
till we have actual instances to guide us.

> Or, to put it in a different way: The problem is not that too few patches get
> merged.  The problem is that too few patches get reviewed.

Agreed.

I believe the below steps would help with the current situation, and
perhaps both should be pursued (among possibly others).

1. It would help to eliminate the need for review in certain kinds of
   patches.  Some version upgrades for a package $foo where there exists
   no package $bar that depends on $foo may fall in this category.  More
   generally, some "known to be safe with reasonable confidence" changes
   may be safe to be auto-committed without needing manual review.
   Recognizing such changes could be challenging, but we don't have to
   correctly label all such changes in order for this to be helpful.

2. It would help if it were easier for the community to be able to identify the next
   steps, given a patch submission.  It might help to (more?) clearly
   differentiate between the below states:
   - patch *needs review* (automatically set)
   - patch *needs revision* (set explicitly by the reviewer, say, by
     using a specific keyword in their email)
   - patch *needs followup review* (automatically set if there's a
     revised patch)
   - patch has a *satisfied reviewer* (set explicitly by the reviewer,
     say, by using a specific keyword in their email)

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Vivien Kraus]

Le lundi 05 février 2024 à 17:21 +0000, Suhail a écrit :
> > Even if it would be just an auto-close (for the "contributor can't
> > work effectively..." case).
> 
> Are there current instances of "contributor can't work effectively"? 
> If
> not, I propose that decisions concerning those situations be deferred
> till we have actual instances to guide us.
Maybe this encompasses the cases where the contributor sends a v1 and
never addresses the review? I don’t fully understand what this means.

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Mon, Feb 05 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:

> Hi Clément,
>
> On Mon, Feb 05 2024, Clément Lassieur wrote:
>
>> I don't think reviewers have to be committers.
>
> How much more evidence does the project need to see in order to realize
> that the plan is not working?

The best review I've had in years was done by a person who is not a
committer.  I see no evidence here.  And I'm unsure which plan you are
talking about (the plan?).

> I'll spare the list a lengthy analysis of the social dynamics but the
> delegation of competency to the bottom is an unsuccessful practice in
> many places.

What do you mean with "bottom"?

> Maintainers must close bugs in service to the public. There is no other
> way.

Reviewing != Closing

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Mon, Feb 05 2024, Suhail wrote:

> Tomas Volf <~@wolfsden.cz> writes:
>
>> In ideal world, there would always be *some* reaction from the
>> project, even if that reaction would be "we do not want this change".
>
> Agreed.  A reduction in the time between a patch (or patch revision)
> submission and a review/reaction would be a positive change in my
> opinion.
>
>> Even if it would be just an auto-close (for the "contributor can't
>> work effectively..." case).
>
> Are there current instances of "contributor can't work effectively"?  If
> not, I propose that decisions concerning those situations be deferred
> till we have actual instances to guide us.
>
>> Or, to put it in a different way: The problem is not that too few patches get
>> merged.  The problem is that too few patches get reviewed.
>
> Agreed.
>
> I believe the below steps would help with the current situation, and
> perhaps both should be pursued (among possibly others).
>
> 1. It would help to eliminate the need for review in certain kinds of
>    patches.  Some version upgrades for a package $foo where there exists
>    no package $bar that depends on $foo may fall in this category.  More
>    generally, some "known to be safe with reasonable confidence" changes
>    may be safe to be auto-committed without needing manual review.

I think "auto-commit" is a bit too much, but tagging a patch as "can be
pushed" could make it easy for committers to find them and push them.

>    Recognizing such changes could be challenging, but we don't have to
>    correctly label all such changes in order for this to be helpful.
>
> 2. It would help if it were easier for the community to be able to identify the next
>    steps, given a patch submission.  It might help to (more?) clearly
>    differentiate between the below states:
>    - patch *needs review* (automatically set)
>    - patch *needs revision* (set explicitly by the reviewer, say, by
>      using a specific keyword in their email)
>    - patch *needs followup review* (automatically set if there's a
>      revised patch)
>    - patch has a *satisfied reviewer* (set explicitly by the reviewer,
>      say, by using a specific keyword in their email)

+1

Ideally with debbugs tags.

(Patches that don't need review should be tagged as well.)

Re: Guix Days: Patch flow discussion

[Hartmut Goebel]

Am 05.02.24 um 10:39 schrieb Steve George:
> Hinders 

This list is missing one point - which has been discussed several times 
already without any result:

The current mail-based workflow is too complicated for new and 
occasional committers. This is the main reason I gave up reviewing patches.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Guix Days: Patch flow discussion

[Felix Lechner via]

On Mon, Feb 05 2024, Clément Lassieur wrote:

> On Mon, Feb 05 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:
>
> I see no evidence here.  And I'm unsure which plan you are talking
> about (the plan?).

Two people can look at the same thing and reach different conclusions. I
see no evidence that large numbers of non-committers are eager to review
patches.

> What do you mean with "bottom"?

I'm sorry to put words into your mouth. I meant to quote an executive at
a bank who explained that strategy to me. The word "bottom" was his and
should have been a quote.

The executive referred to people without the authority to act on behalf
of the group.

I believe Guix would be better off to delegate responsibility (rather
than competency) by handing out commit access more generously but
imposing limits as to the type of changes a person may make.

The honor system will work fine.

> Reviewing != Closing

Maybe they should be the same. Two people looking at a patch (submitter
and committer) are more efficient than three people, i.e. a submitter, a
reviewer, and a committer.

It's one of several bottlenecks at Guix. Another is that committers
should commit what they think is right rather than ask for revised
patches.

Please give authorship to the submitter.

Kind regards
Felix

Re: Guix Days: Patch flow discussion

[Suhail]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> This list is missing one point - which has been discussed several
> times already without any result:
>
> The current mail-based workflow is too complicated for new and
> occasional committers.

Could you please share a reference where the key difficulties you
encountered wrt the "current mail-based workflow" are summarized.  Is
the difficulty regd. checking out the code at the right commit and
installing the patches, or something else?

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Mon, Feb 05 2024, Felix Lechner via wrote:

> On Mon, Feb 05 2024, Clément Lassieur wrote:
>
>> On Mon, Feb 05 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:
>>
>> I see no evidence here.  And I'm unsure which plan you are talking
>> about (the plan?).
>
> Two people can look at the same thing and reach different conclusions. I
> see no evidence that large numbers of non-committers are eager to review
> patches.

I believe very few people (commiter or not) are eager to review patches.

>> What do you mean with "bottom"?
>
> I'm sorry to put words into your mouth. I meant to quote an executive at
> a bank who explained that strategy to me. The word "bottom" was his and
> should have been a quote.
>
> The executive referred to people without the authority to act on behalf
> of the group.
>
> I believe Guix would be better off to delegate responsibility (rather
> than competency) by handing out commit access more generously but
> imposing limits as to the type of changes a person may make.
>
> The honor system will work fine.
>
>> Reviewing != Closing
>
> Maybe they should be the same. Two people looking at a patch (submitter
> and committer) are more efficient than three people, i.e. a submitter, a
> reviewer, and a committer.

Committers are trusted, and obviously it takes time to be trusted, so
not everyone can be at once.  Reviewing requires less trust because the
committer can do a quick check that everything is in order before
submitting the patch.

It's less efficient if you compare flows for two patches, but for 1000
patches, it makes it easier to find reviewers, so in the end it's more
efficient.

> It's one of several bottlenecks at Guix. Another is that committers
> should commit what they think is right rather than ask for revised
> patches.

I sometimes update the commit a little bit before pushing, and then
explain what I changed in my email.  But if the changes are big, I often
believe it makes more sense to ask for a v2.

> Please give authorship to the submitter.

I think it's the case?

Cheers,
Clément

Re: Guix Days: Patch flow discussion

[Suhail]

Felix Lechner via <help-guix@gnu.org> writes:

> Another is that committers should commit what they think is right
> rather than ask for revised patches.

I could be mistaken, but I believe this does happen today at least some
of the time.  Is your position that

1. this never happens today and thus, should happen some times when
   warranted.  Or that,

2. it happens far too rarely today, and should happen more often. Or
   that,

3. committers should never ask for revisions?

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Andy Tai]

Hi, this is a sugestion on guix patches:

Other GNU/Linux distributions often have fixed maintainers (or
packagers) for specific packages.
While that model may not apply directly to the Guix project as anyone
can send in patches for anything in the git repo, maybe one thing the
Guix project can do is to recognize if someone has sent in patches for
specific projects many times in the past, then that means the person
has knowledge, interest, familiarity and expertise about this package
and the reviewers can spend minimal time of his/her future patches for
the same package(s).

Thus this creates kind of pseudo package maintainer in Guix, reducing
workload of reviewers and speeding up package update.

Guix can have a text file in the Guix repo recording say someone who
is "responsible" or is a "pseudo packager" for specific packages,
which the committers can refer to to allow quick processing of
patches.  This does not mean the person responsible for certain
package has git commit access but just to allow a quick path for patch
processing by the commiters of certain patches from the familiar
person.

Of course others can still send in patches for these packages already
with pseudo packagers.

Re: Guix Days: Patch flow discussion

[Felix Lechner via]

Hi Suhail,

On Mon, Feb 05 2024, suhail@bayesians.ca wrote:

> Felix Lechner via <help-guix@gnu.org> writes:
>
> Is your position that

First off, I'm sorry I write so much today. For a project the size of
Guix, it's not good for one person to belabor a point repeatedly. I am
responding to your request for a clarification.

> 1. this never happens today and thus, should happen some times when
>    warranted.  Or that,
>
> 2. it happens far too rarely today, and should happen more often. Or
>    that,
>
> 3. committers should never ask for revisions?

None of the above. I believe committers should commit only changes they
are comfortable with. Otherwise, they should leave a bug to folks with
more experience, perhaps tagging it along the way.

I think there should be a hundred people who do nothing but add new
packages. They would free up valuable capacity for the people
experienced with build systems, Guix services and their configuration,
the user interface, the daemon, cross-compilation and other more
difficult speciality areas.

Kind regards
Felix

Re: Guix Days: Patch flow discussion

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

Am 05.02.24 um 19:44 schrieb Suhail:
> Could you please share a reference where the key difficulties you
> encountered wrt the "current mail-based workflow" are summarized.  Is
> the difficulty regd. checking out the code at the right commit and
> installing the patches, or something else?

It's not only installing and testing the patches, but also

  * when has this issue/patch been worked on last - is somebody
    currently working on it
  * what issue/patches I started to review?
  * commenting on code requires to download the patch - strip out parts
    which are okay, comment, then mail the commented code to the correct
    issue number
  * Even when using the debbugs interface in emacs
      o It is is hard to use for occasional users
      o It is an insurmountable obstacle for those not using emacs.
      o It does not tell what issues/patches I've been working on
        already - and waiting for a reply
      o It does not tell which issues are stale

Anyhow, all of this has been discussed several times already. And as 
long as vocal (and active :-) members of the community insist on being 
able to work via e-mail — while also not adopting modern e-mail-capable 
forges — this situation will not change.

Regards
Hartmut Goebel

| Hartmut Goebel          |h.goebel@crazy-compilers.com                |
|www.crazy-compilers.com  | compilers which you thought are impossible |

[-- Attachment #2: Type: text/html, Size: 2090 bytes --]

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Mon, Feb 05 2024, Andy Tai wrote:

> Hi, this is a sugestion on guix patches:
>
> Other GNU/Linux distributions often have fixed maintainers (or
> packagers) for specific packages.
> While that model may not apply directly to the Guix project as anyone
> can send in patches for anything in the git repo, maybe one thing the
> Guix project can do is to recognize if someone has sent in patches for
> specific projects many times in the past, then that means the person
> has knowledge, interest, familiarity and expertise about this package
> and the reviewers can spend minimal time of his/her future patches for
> the same package(s).
>
> Thus this creates kind of pseudo package maintainer in Guix, reducing
> workload of reviewers and speeding up package update.
>
> Guix can have a text file in the Guix repo recording say someone who
> is "responsible" or is a "pseudo packager" for specific packages,
> which the committers can refer to to allow quick processing of
> patches.  This does not mean the person responsible for certain
> package has git commit access but just to allow a quick path for patch
> processing by the commiters of certain patches from the familiar
> person.
>
> Of course others can still send in patches for these packages already
> with pseudo packagers.

We have exactly this:
https://git.savannah.gnu.org/cgit/guix.git/tree/etc/teams.scm

And it's even better: those people automatically receive an email if you
use git-send-email to send a patch about their associated files.

Re: Guix Days: Patch flow discussion

[Suhail]

Hartmut, thank you for elaborating.

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

>  * when has this issue/patch been worked on last - is somebody
>    currently working on it
>  * what issue/patches I started to review?
> ...
>  * Even when using the debbugs interface in emacs
> ...
>      o It does not tell what issues/patches I've been working on
>        already - and waiting for a reply

I believe the state tracking could be improved, and have given some
concrete suggestions in another thread.  However, I believe these are
orthogonal to whether or not the workflow is "mail-based".

For clarity, within Emacs' Debbugs interface I can use a combination of
tags and marks to highlight the issues of interest.

>  * commenting on code requires to download the patch - strip out parts
>    which are okay, comment, then mail the commented code to the correct
>    issue number

I believe the ease or not of this would depend on the email client of
choice.

>  * Even when using the debbugs interface in emacs
> ...
>      o It is an insurmountable obstacle for those not using emacs.

This is probably an important issue if true.  To clarify, in "those not
using [E]macs" are you considering developers using other email clients
such as Alpine, Mutt etc?

>      o It does not tell which issues are stale

What does "stale" mean to you?  Do you mean something other than
Debbugs' notion of staleness?

> And as long as vocal (and active :-) members of the community insist
> on being able to work via e-mail — while also not adopting modern
> e-mail-capable forges — this situation will not change.

In an attempt to focus the discussion on the specific features, it seems
if, in addition to improved state tracking, the below were available it
might help (to varying degrees depending on the user):

1. Some tooling that helps in "checking out" a specific revision of a
   patch series.  I don't know if this already exists, given my relative
   inexperience with Debbugs.

2. An ability to leave comments inline on
   <https://issues.guix.gnu.org/>.

While "forges" provide both of the above in some form, they may not be
the only way to accomplish the above.

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Steve George]

Hi Leo,

On 05/02/2024 14:07, Leo Famulari wrote:
> On Mon, Feb 5, 2024, at 04:39, Steve George wrote:
>> Our goal for the discussion:
>>
>> 	How do we double the number of patches that are *reviewed* and
>> 	*applied* to Guix in the next six months?
>>
>> Patch flow is a pipeline, to change it we could:
>>
>> a. Increase the number of committers - more people to do the
>> work
>> b. Increase the efficiency of existing committers
>> c. Open the gates by decreasing the quality expected from patches
> 
> Hi George,

Just 'Steve' :-)

> It's an important subject and, in my opinion, more important than any technical questions at this stage of the project.
> 
> However, I think the question assumes that all contributions should be accepted, and that the entire problem is that we are not accepting them efficiently enough. We should not unconsciously accept this assumption.
> 
> Guix can reject contributions, either in a general way (we don't want that type of thing in Guix), or due to specific reasons (the code is not idiomatic, the contributor can't work effectively with the rest of the group, etc).
(...)

Today there are 1264 bugs with patches attached to them today [0].

We don't have any stats (that I'm aware of) that show how many patches 
the project reviews and either asks for edits, rejects or applies.

The group - and I hope the minutes reflect this - wanted to maintain the 
current standards. So the discussion of the pipeline focused on how we 
review more patches, and make decisions about them: whether that 
decision is to accept the contribution, ask for more work, or reject it 
as out of scope.

To answer your comment about the 'unconscious' assumptions - my personal 
view is:

* I don't think ALL patches should simply be accepted
* I do think that more patches need to be reviewed and a decision made
* I believe (no evidence) the project is missing out on great 
contributions amongst those 1200+ patches in the tracker
* I believe (no evidence) that potential contributors are put-off by the 
speed of review / lack of clarity

Thanks,

Steve

[0] https://debbugs.gnu.org/rrd/guix-patches.html

Re: Guix Days: Patch flow discussion

[Wilko Meyer]

Hi Guix,

Tomas Volf <~@wolfsden.cz> writes:

> Or, to put it in a different way: The problem is not that too few patches get
> merged.  The problem is that too few patches get reviewed.

I'd say that both things stem from the same premise, a disproportion of
available resources to the work that exists. This is not something
specific to Guix as a project, but can be observed in many other
projects as well (I couldn't name one larger free software or open
source project without this issue, but could easily name some where this
applies). The interests of a contributer sending a patch sometimes may
not align with the interests of the project/sometimes may not align with
the interests of commiters and so on. This happens and is a pretty
common reason for contributions being ignored and I see that as somewhat
a default modus operandi in many projects. Especially if available time
is a rather sparse resource.

I'd like to suggest to explicity refer to pragmatic ways forward in
Guixes Contributing manual section that don't rely on the availability
of other peoples (in this case committers/reviewers) time while
empowering contributors to use their changes in a good way if a patch
doesn't make it in/a bug report gets no reaction? Guix offers ways to
use packages outside of Guix proper in a pretty feasible and
maintainable way (manifests, setting up channels), maybe promoting them
as an alternative to having things in guix proper "as soon as possible"
(as that's not the only option to have things in a usable form) would be
beneficial.

-- 
Kind regards,

Wilko Meyer
w@wmeyer.eu

Re: Guix Days: Patch flow discussion

[Steve George]

On 05/02/2024 15:57, Clément Lassieur wrote:
> Hello,
> 
> On Mon, Feb 05 2024, Steve George wrote:
> 
>> Hi,
>>
>> Our goal for the discussion:
>>
>> 	How do we double the number of patches that are *reviewed* and
>> 	*applied* to Guix in the next six months?
>>
>> Patch flow is a pipeline, to change it we could:
>>
>> a. Increase the number of committers - more people to do the
>> work
> 
> I don't think reviewers have to be committers.  If a patch is marked as
> ‘reviewed’ by a non-commiter, another person can just apply the patch
> and push.  Although the commiter will take some responsibility in doing
> so, and make sure the review was done correctly, it still makes their
> work way easier.
> 
> My point is that not being committer shouldn't discourage people to
> review patches.
(...)

Agreed - I put it a bit lower in the notes - clearly managed to bury it :-)

The discussion focused on increasing the number of Reviewers. A reviewer 
can be anyone - they don't need to be a committer.

The purpose of the proposed 'Patch Review' sessions is for contributors 
to come along and review patches together.

Thanks,

Steve

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Re: Guix Days: Patch flow discussion

[Steve George]

Hi Hartmut,

Apologies, my reply-to went a bit mad and I sent you empty emails :-(

You said:

> The current mail-based workflow is too complicated for new and 
> occasional committers. This is the main reason I gave up reviewing patches.
>

In the case of *reviewing patches* can you give some examples of:

1. Some patches from others you reviewed?
2. The steps you went through to undertake that review?

If for example, you started to review a simple update to a package how do you 
aproach it?

It would be great to hear directly from someone doing 'reviewing' on the 
specifics. Apologies, if this is going over old ground.

Thanks,

Steve

Re: Guix Days: Patch flow discussion

[Andy Tai]

What I proposed is in terms of the scope of a single package. The
teams cover much larger and broader scope, like Gnome or Python.

On Mon, Feb 5, 2024 at 12:50 PM Clément Lassieur <clement@lassieur.org> wrote:
> On Mon, Feb 05 2024, Andy Tai wrote:
> > Thus this creates kind of pseudo package maintainer in Guix, reducing
> > workload of reviewers and speeding up package update.
> We have exactly this:
> https://git.savannah.gnu.org/cgit/guix.git/tree/etc/teams.scm
>
> And it's even better: those people automatically receive an email if you
> use git-send-email to send a patch about their associated files.

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1977 bytes --]

Hi Hartmut,

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Anyhow, all of this has been discussed several times already. And as 
> long as vocal (and active :-) members of the community insist on being 
> able to work via e-mail — while also not adopting modern e-mail-capable 
> forges — this situation will not change.

While I agree with your assessment of what's missing in this workflow,
I'm not sure I know of any e-mail-capable forges that resolve those
particular pain points…  People often think that eg. sr.ht works better
for this, but I don't think the sr.ht MLs have any more structure than
what we have.  For smaller projects, it works okay, but once you get to
the scale of something like Guix, it doesn't offer anything more.

One thing I would like to get rid of though is debbugs.  It causes a lot
of pain for everyone, eg. when sending patchsets, it completely breaks
modern email because it insists on rewriting DMARC-protected headers,
thus needing to also rewrite "From:" to avoid DMARC errors.

I've been following the Linux kernel development a bit closer this past
year, and while there are things that need to improve (like knowing the
status of a patchset in a maintainer's tree), they at least have a lot
of tools that I think we should adopt more broadly:

b4/lei is a nice example (we already have yhetil.org as a back-end, but
maybe a more blessed one would be better) of a tool that lets you
completely automate applying a patchset to a branch.

patchwork is a nice tool to gather up and track patchsets, with status
indicators like "under review", "accepted", etc.  Chris already deploys
one as part of QA, more integration with it would be nice.

One advantage of this is that we would benefit from upstream's bigger
user-base and dev time, and also from the fact that these tools are made
to work together (b4 can automatically mark patchsets as accepted)!

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 534 bytes --]

Hi Sunhail,

Your mailing system is sending out emails that contain an invalid
Message-ID header (missing right part of the msgid as specified in [1]),
which causes erratic behavior from other participants (the gmail smtp
servers for example just rewrite the message id to indicate it is
malformed, breaking the reply chain).  Could you please fix this?

This is quite on-topic in a discussion about the mail-based workflow :)

[1] https://datatracker.ietf.org/doc/html/rfc2822#section-3.6.4

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 940 bytes --]

On 2024-02-05 23:08:29 +0100, Wilko Meyer wrote:
> Guix offers ways to use packages outside of Guix proper in a pretty feasible
> and maintainable way (manifests, setting up channels), maybe promoting them as
> an alternative to having things in guix proper "as soon as possible" (as
> that's not the only option to have things in a usable form) would be
> beneficial.

I would just like to point out that while this applies to packages, it is bit
more complicated for services and other core parts of the Guix.  Combined with
the fact that Guix severely restricts what workflow model can forks use (e.g.
merge not being possible if the fork should be authenticated), maintaining
non-package modifications for longer periods of time can be a hassle.

So it could be beneficial to put some though into that as well.

Tomas Volf

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Tue, Feb 06 2024, Tomas Volf wrote:

> On 2024-02-05 23:08:29 +0100, Wilko Meyer wrote:
>> Guix offers ways to use packages outside of Guix proper in a pretty feasible
>> and maintainable way (manifests, setting up channels), maybe promoting them as
>> an alternative to having things in guix proper "as soon as possible" (as
>> that's not the only option to have things in a usable form) would be
>> beneficial.
>
> I would just like to point out that while this applies to packages, it is bit
> more complicated for services and other core parts of the Guix.  Combined with
> the fact that Guix severely restricts what workflow model can forks use (e.g.
> merge not being possible if the fork should be authenticated), maintaining
> non-package modifications for longer periods of time can be a hassle.

Hi!  Why is it more complicated with services?  You don't need forks at
all to use packages and services outside of Guix proper.

Re: Guix Days: Patch flow discussion

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 1225 bytes --]

On 2024-02-06 13:09:15 +0100, Clément Lassieur wrote:
> Hi!  Why is it more complicated with services?  You don't need forks at
> all to use packages and services outside of Guix proper.

For packages we have transformations, or I can just inherit.  But I am not aware
of such option for services (is there anything?).

Example: For long time the connman-configuration did not support a way to
provide a configuration file (well, it still does not but that is not the
point).  But I needed it.  So, what options I had?

1. Copy&paste the definition (-configuration, -service-type, ...?) into my
   system configuration and modify it.
2. Modify it in-place in my fork.

I tried 1. first, but keeping it up-to-date proved annoying, having zero support
from git.  Since I already made my fork (due to #65002, which just is not
possible to do outside Guix repository), I later switched to 2. and it worked
well, so I prefer that approach now.

I agree there is no issue for new packages nor new services, I should have
clarified that in my original message.

Have a nice day,
Tomas Volf

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Guix Days: Patch flow discussion

[Edouard Klein]

I, for one, would be willing to review patches, hoping that in turn my
patches would be reviewed instead of staying in limbo forever, which is
a drag on me submitting more patches.

Is there a procedure to follow, or do I just start replying "LGTM" to
patch email threads ?

Cheers,

Edouard.
Steve George <steve@futurile.net> writes:

> Hi,
>
> Our goal for the discussion:
>
> 	How do we double the number of patches that are *reviewed* and
> 	*applied* to Guix in the next six months?
>
> Patch flow is a pipeline, to change it we could:
>
> a. Increase the number of committers - more people to do the
> work
> b. Increase the efficiency of existing committers
> c. Open the gates by decreasing the quality expected from patches
>
> We essentially decided to focus our discussion on (b). We looked at
> things that 'hinder' and 'help' patch review:
>
>
> Hinders
> ========
>
> - All our patch reviewers are volunteers doing it in their spare time.
>
> - For a volunteer reviewing someone else's work is not very rewarding, most
>  would prefer to use that precious time to scratch their own itch.
>
> - Can feel like an Sisyphean task: no matter how many patches someone reviews
>  there are more, exacerbated by the number of Guix packages.
>
> - Sense of responsibility: the minute that a reviewer looks at the patch they
>  are now stuck with it
>
> - Repetitive and boring: often patches have minor issues, but it's the same
>  sorts of issues time and time again.
>
> - Risk of negative social interaction: having to tell someone that their patch
>   is incorrect, or that their contribution cannot be used is difficult and
>   draining. Some people felt it was better to say nothing, rather than to
>  respond to a patch.
>
>
> Helps
> ======
>
> This led us to the focus on the fact that **reviewing and applying
> patches can be different people**
>
> We looked for ideas to create more reviewers, make reviewing easier and
> more fun:
>
>
> - Share in the work
> --------------------
>
> 1. encourage new reviewers to step forward - making it more known that reviewing
> patches helps to get them applied. Anyone can review patches.
>
> 2. create directed 'how-to' documentation for reviewing and connect it to QA so
> that 'new reviewers' know what to do
>
> 3. create documentation about 'when' and 'how' it's appropriate to send a 'v2'
> version of a patch so that the QA system builds and accepts it. Sometimes,
> patches rot because non-committers don't want to be seen as 'stealing' someone's
> work with a v2 patch - but making the small changes and resubmitting to QA is
> what is required.
>
> 4. Pay someone else to do it. Noted but out of scope.
> 5. Remove old packages overhead. Old untouched packages create mental overhead,
> and make the task of maintaining the repository in a good state more difficult.
> We could remove old 'untouched' packages and ones that no-longer compile. We
> have methods to hide and notify.
>
>
> - Make it more fun
> -------------------
>
> 1. do online sessions around reviews, some sprints or pairing - both social and
> a way to spread skills
> 2. find ways to recognise and appreciate reviewers - 'reviewer of the month'
> 3. make it a game - we could have a 'Guix London' vs 'Guix Paris' leader board
> for reviews. Make it a group goal 'can we beat januarys reviews number'
> 4. create some graphs / leaderboard so we know how many patches are being
> reviewed and we can recognise the contribution
>
>
> - Automate it away
> -------------------
>
> 1. Chris is continuing to try and automate away the boring work - general
> agreement in the group that QA has made a lot of difference.
>
> 2. general discussion about create a 'guix review' command (Nix has one) which
> would download a branch with the appropriate patch and build it locally. This is
> for instances where some adjustment is needed or to check a build. While this
> can be done today, it's a number of steps and quite involved.
>
>
> Agreed Actions
> ==============
>
> * [Chris]: continuing his work to improve QA automation. Implication was we'll
>  need some reports / graphs - but these were not discussed in detail.
>
> * [Futurile]: organise a **patch review online sessions**. To run every 13 days
>   (so it rotates through the week) - for 3 months to see if it has any traction.
>   Co-ordinate with maintainers so that patches that are reviewed can be
>  committed
>
>
> Actions looking for someone - you?
> ====================================
>
> * Carry forward the 'guix review' command idea
>
> * Write an RFC and discuss the idea of removing older 'bit-rot' packages
>
> * write 'how-to' documentation for reviews and when it's socially
> acceptable to do a v2 patch. A checklist-like approach.
>
>
> If you were in the discussion and I've misrepresented your point, or forgotten
> an important aspect please please reply and correct me.
>
> Also, if you would like to help on any of the tasks please email back to the
> group so we can all co-ordinate.
>
> Finally, thank-you to everyone who came along and put their shared brain power
> to the task - look forward to doing some patch reviews together online in the
> coming weeks!
>
> Thanks,
>
> Steve/Futurile

Re: Guix Days: Patch flow discussion

[Steve George]

Hi Edouard,

On  6 Feb, Edouard Klein wrote:
> I, for one, would be willing to review patches, hoping that in turn my
> patches would be reviewed instead of staying in limbo forever, which is
> a drag on me submitting more patches.
> 
> Is there a procedure to follow, or do I just start replying "LGTM" to
> patch email threads ?

Starting to review patches would be brilliant.

I agreed to organise some 'patch review' online sessions in the next couple of 
weeks. 

Organising a basic process is a good topic for that online session. For 
example, elsewhere in the thread someone mentions some tags we could use 
consistently so maintainers can find patches that have been reviewed easily. It 
would be great to agree those - try them for a bit - and document them in a 
'howto' so that everyone uses the same process.

Steve

Debbugs update (Was: Guix Days: Patch flow discussion)

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Hartmut & Josselin,

On Mon, Feb 05 2024, Hartmut Goebel wrote:

> Am 05.02.24 um 10:39 schrieb Steve George:
>
> [order of quotations reversed]
>
> The current mail-based workflow is too complicated ... which has been
> discussed several times already without any result:

Well, that's not totally true. After hearing the plight last year, I
offered the FSF to assume responsibility for debbugs.gnu.org. [1]

I also packaged and deployed on GNU Guix

  (A) the fifteen-year old Debbugs version deployed at gnu.org [2][3][4]
  (B) the modern Debbugs version deployed at debian.org [5][6][7]
  (C) and a custom version of Mumi for my own bug fixes [8][9]

Together with the official debbugs.gnu.org, which runs on Debian 8, and
issues.guix.gnu.org, I am now working to fold all five deployments into
one.

On Tue, Feb 06 2024, Josselin Poiret wrote:

> One thing I would like to get rid of though is debbugs.

I can do little to appease the hardcore Debbugs haters, but I have a
vision for a bug tracking system that, written in GNU Guile, will
attract less wholesale criticism and more constructive contributions
from from the Emacs and Guix communities, which are the primary users of
debbugs.gnu.org.

Both groups are already in love with the lambda calculus.

Mumi made great steps in that direction but has so far not attracted the
contributions it deserves.

In another piece of the puzzle, Michael Albinus wrote and maintains an
excellent Emacs package called Debbugs.el. It allows bug work to take
place in Gnus and Org Modes [10] rather than in a web browser. With that
package, Emacs could become a favorite way to work on our bugs and
patches, similar to what Magit did for Git.

At Guix, folks also do not use Debbugs to its full potential. Git hooks
are an example. Mumi obscures some features. I know because I worked on
Debian's bugs for many years.

I am committed to Debbugs because I'm not sure interactions between
people can be audited properly on modern development forges. For many
years, I worked with Salsa, Debian's Gitlab instance. While convenient,
it was difficult to find past conversations and code snippets, although
like Josselin I'm watching what the kernel folks are doing.

The databases in forges are also complicated to maintain. Plus, projects
experience immediate vendor lock in.

My primary hurdle with modernizing Debbugs, if anyone cares, is that the
FSF is reluctant to deploy GNU Guix. They insist on Trisquel plus
Ansible [11] which is what they have been using for some time.

Thank you, everyone, for your hard work on GNU Guix!

Kind regards
Felix

[1] https://lists.nongnu.org/archive/html/help-debbugs/2023-10/msg00003.html
[2] https://debbugs.juix.org/cgi/bugreport.cgi?bug=66703
[3] https://codeberg.org/lechner/juix/src/commit/5fffdb0053b18b4b28adadbbebb79a1d9bfe2337/juix/deploy/debbugs.scm#L1046-L1184
[4] https://codeberg.org/lechner/debbugs-gnu
[5] https://better.juix.org/cgi/bugreport.cgi?bug=66703
[6] https://codeberg.org/lechner/juix/src/commit/5fffdb0053b18b4b28adadbbebb79a1d9bfe2337/juix/deploy/debbugs.scm#L1186-L1364
[7] https://salsa.debian.org/debbugs-team/debbugs
[8] https://mumi.juix.org/66703
[9] https://codeberg.org/lechner/mumi
[10] https://elpa.gnu.org/packages/doc/debbugs-ug.html
[11] https://lists.nongnu.org/archive/html/help-debbugs/2024-01/msg00049.html

Re: Guix Days: Patch flow discussion

[Steve George]

Hi Suhail,

On  5 Feb, Suhail wrote:
> Felix Lechner via <help-guix@gnu.org> writes:
> 
> > Another is that committers should commit what they think is right
> > rather than ask for revised patches.
> 
> I could be mistaken, but I believe this does happen today at least some
> of the time.  Is your position that
> 
> 1. this never happens today and thus, should happen some times when
>    warranted.  Or that,
> 
> 2. it happens far too rarely today, and should happen more often. Or
>    that,
> 
> 3. committers should never ask for revisions?
(...)

Just to reply to this part, there were a variety of views on what the correct 
balance was between teaching someone and fixing issues so that a contributors 
patch was applied without them having to make further efforts. The general 
opinion seemed to be, that it was better to fix small issues and commit the 
change for new users, so they had the satisfaction of their contribution making 
it into the repository. One proposal was to do the 'fix', and to then reply 
back to the bug with a diff - showing what was done.

Steve

Re: Guix Days: Patch flow discussion

[Suhail]

Steve George <steve@futurile.net> writes:

> elsewhere in the thread someone mentions some tags we could use
> consistently so maintainers can find patches that have been reviewed
> easily.

It seems on the [dev manual] we already have "reviewed-looks-good"
documented.  Thus, I'd like to propose the below *mutually exclusive*
Debbugs tag set:

- "not-yet-reviewed" :: automatically set for all submissions
- "reviewed-needs-fix" :: set explicitly by the reviewer
- "needs-another-review" :: automatically set if there's a revised
  patch, unless "not-yet-reviewed" (in which case no change)
- "reviewed-looks-good" :: set explicitly by the reviewer

In addition to the above, it might also help for there to be an
additional tag of "might-not-need-review" (or simpler,
"review-not-needed") which gets automatically set, provided we implement
a way to label some changes (for some packages) as being "trivial enough
that they're okay as long as build succeeds".

On a related note, is it possible for a reviewer who isn't a committer
to set debbugs tags?

[dev manual]: <https://guix.gnu.org/en/manual/devel/en/html_node/Debbugs-Usertags.html>

> It would be great to agree those - try them for a bit - and document
> them in a 'howto' so that everyone uses the same process.

In addition to documenting the tags in the "Debbugs Usertags" section of
the manual, it would help for there to be a "howto" which focuses more
on the transition between the tags (i.e., the contribution workflow).

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Suhail]

Steve George <steve@futurile.net> writes:

> The general opinion seemed to be, that it was better to fix small
> issues and commit the change for new users, so they had the
> satisfaction of their contribution making it into the repository. One
> proposal was to do the 'fix', and to then reply back to the bug with a
> diff - showing what was done.

I see.  Thank you for sharing.  I believe being more didactic with "new
users" would be good for the community.

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Suhail]

Josselin Poiret <dev@jpoiret.xyz> writes:

> One thing I would like to get rid of though is debbugs.  It causes a
> lot of pain for everyone, eg. when sending patchsets, it completely
> breaks modern email because it insists on rewriting DMARC-protected
> headers, thus needing to also rewrite "From:" to avoid DMARC errors.

Thank you for sharing (what seems to be) a technical limitation of
Debbugs.  Could you please explain what the consequences of the above
are?  Specifically, how does the rewriting of above headers affect the
contributors' workflow?

> b4/lei is a nice example (we already have yhetil.org as a back-end,
> but maybe a more blessed one would be better) of a tool that lets you
> completely automate applying a patchset to a branch.
>
> patchwork is a nice tool to gather up and track patchsets, with status
> indicators like "under review", "accepted", etc.  Chris already
> deploys one as part of QA, more integration with it would be nice.

It seems (based on above) that "patchwork" can co-exist with debbugs.
Is that also the case with b4/lei?  Specifically, are the
users/reviewers able to benefit from using the above tools at present?
Or are there some reasons (over and above their lack of familiarity with
the above tools) that would prevent them from doing so?

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Suhail]

Josselin Poiret <dev@jpoiret.xyz> writes:

> Your mailing system is sending out emails that contain an invalid
> Message-ID header (missing right part of the msgid as specified in
> [1]),
> ...
> [1] https://datatracker.ietf.org/doc/html/rfc2822#section-3.6.4

My apologies (to everyone)!  Thank you for bringing this to my
attention.

> which causes erratic behavior from other participants (the gmail smtp
> servers for example just rewrite the message id to indicate it is
> malformed, breaking the reply chain).

Ouch.

> Could you please fix this?

I believe this is now fixed.  Please let me know in case the issue
persists, or you notice others (though, perhaps, off list).

-- 
Suhail

Re: Debbugs update (Was: Guix Days: Patch flow discussion)

[Ricardo Wurmus]

Hi Felix,

> I also packaged and deployed on GNU Guix
>
>   (A) the fifteen-year old Debbugs version deployed at gnu.org [2][3][4]
>   (B) the modern Debbugs version deployed at debian.org [5][6][7]
>   (C) and a custom version of Mumi for my own bug fixes [8][9]
>
> Together with the official debbugs.gnu.org, which runs on Debian 8, and
> issues.guix.gnu.org, I am now working to fold all five deployments into
> one.

Let me just pop in to say thank you for that.  I appreciate you taking
the time to talk to Debbugs / GNU folks to work through the obstacles
that the current situation with the GNU fork of Debbugs presents us
with.

I don’t want to add weight to the scales on either side (a future with
or without Debbugs), but I think it is important to identify the actual
limitations and opportunities of the existing platform and chart the
shortest route around them.

It’s how we ended up with mumi in the first place, and I’m hopeful that
this approach can get us to a better place when it comes to tooling.

(The biggest chunk of the problem may well be a social issue, that can
only be overcome if we find a narrative we all can comfortably enact.)

-- 
Ricardo

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 2285 bytes --]

Hi Sunhail,

> Josselin Poiret <dev@jpoiret.xyz> writes:
>
>> One thing I would like to get rid of though is debbugs.  It causes a
>> lot of pain for everyone, eg. when sending patchsets, it completely
>> breaks modern email because it insists on rewriting DMARC-protected
>> headers, thus needing to also rewrite "From:" to avoid DMARC errors.
>
> Thank you for sharing (what seems to be) a technical limitation of
> Debbugs.  Could you please explain what the consequences of the above
> are?  Specifically, how does the rewriting of above headers affect the
> contributors' workflow?

Every reply to one of your mails ends up creating a new thread in my
mail client, because the In-Reply-To chain ends up being broken. 

>> b4/lei is a nice example (we already have yhetil.org as a back-end,
>> but maybe a more blessed one would be better) of a tool that lets you
>> completely automate applying a patchset to a branch.
>>
>> patchwork is a nice tool to gather up and track patchsets, with status
>> indicators like "under review", "accepted", etc.  Chris already
>> deploys one as part of QA, more integration with it would be nice.
>
> It seems (based on above) that "patchwork" can co-exist with debbugs.
> Is that also the case with b4/lei?  Specifically, are the
> users/reviewers able to benefit from using the above tools at present?
> Or are there some reasons (over and above their lack of familiarity with
> the above tools) that would prevent them from doing so?

They both can co-exist with debbugs, and for now the patchwork instance
of QA is not usable for status tracking (because it is not meant to be
used as such for now).  One can already use both of them, but using both
supercedes debbugs, and gets rid of its limitations.  I've been using
b4/lei with the yhetil public-inbox instance, with piem.el as an
interface, and it's really useful.  With a properly configured b4, one
could simply run `b4 shazam some-msg-id` and it would automatically
apply the corresponding patchset.

And before you ask why I'm so intent on getting rid of Debbugs, I
believe that mailing lists should be just that, mailing lists.
Anything that tries to rewrite incoming mail is asking for trouble
nowadays.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1687 bytes --]

Hi again Sunhail,

Josselin Poiret <dev@jpoiret.xyz> writes:

> Hi Sunhail,
>
>> Josselin Poiret <dev@jpoiret.xyz> writes:
>>
>>> One thing I would like to get rid of though is debbugs.  It causes a
>>> lot of pain for everyone, eg. when sending patchsets, it completely
>>> breaks modern email because it insists on rewriting DMARC-protected
>>> headers, thus needing to also rewrite "From:" to avoid DMARC errors.
>>
>> Thank you for sharing (what seems to be) a technical limitation of
>> Debbugs.  Could you please explain what the consequences of the above
>> are?  Specifically, how does the rewriting of above headers affect the
>> contributors' workflow?
>
> Every reply to one of your mails ends up creating a new thread in my
> mail client, because the In-Reply-To chain ends up being broken. 

Ah, whoops, I thought you were asking about the Message-ID problem I
also reported to you, not my above paragraph, sorry.

One big consequence: mails containing patches can often have bogus From:
headers, which are used by git for the commit author.  Thus, we end up
with commits in the repo authored by eg. "Josselin Poiret via
Guix-patches via <guix-patches@gnu.org>".  We added a default gitconfig
entry that also puts the From: in the body of the message so that it is
not corrupted to circumvent this.

The fact that you have to wait for Debbugs's response after the first
mail to get the proper mail to reply to means that we can't automate
sending whole patchsets, and have to resort to hacks like the CLI `mumi`
tool uses.  I can't just send a patchset and be done with it, I have to
wait a couple minutes.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

Hello,

On Wed, Feb 07 2024, Josselin Poiret wrote:

> With a properly configured b4, one
> could simply run `b4 shazam some-msg-id` and it would automatically
> apply the corresponding patchset.

Note that you can do the same with one Emacs command while browsing Gnus
mailing lists:

--8<---------------cut here---------------start------------->8---
(defun my-apply-patch-or-abort ()
  (interactive)
  (my-apply-patch-internal "git am || git am --abort"))

(defun my-apply-patch ()
  (interactive)
  (my-apply-patch-internal "git am -3"))

(defun my-apply-patch-or-abort-attachment (n)
  (interactive "P")
  (my-apply-patch-attachment-internal "git am || git am --abort" n))

(defun my-apply-patch-attachment (n)
  (interactive "P")
  (my-apply-patch-attachment-internal "git am -3" n))

(defun my-apply-patch-attachment-internal (cmd n)
  "C-u <attachment number> M-x my-apply-..."
  (let ((git-dir "~/src/guix"))
    (save-window-excursion
      (gnus-article-part-wrapper
       n
       (lambda (handle)
         (let ((default-directory git-dir))
           (mm-pipe-part handle cmd)))))))

(defun my-apply-patch-internal (cmd)
  "Works with a selection of articles too."
  (let ((git-dir "~/src/guix")
        (articles (gnus-summary-work-articles nil)))
    (save-window-excursion
      (while articles
        (gnus-summary-goto-subject (pop articles))
        (with-current-buffer gnus-summary-buffer
          (let ((default-directory git-dir))
            (gnus-summary-save-in-pipe cmd))
          (gnus-article-hide-headers))))))
--8<---------------cut here---------------end--------------->8---

Cheers,
Clément

Re: Guix Days: Patch flow discussion

[Ricardo Wurmus]

Hi Josselin,

>>> b4/lei is a nice example (we already have yhetil.org as a back-end,
>>> but maybe a more blessed one would be better) of a tool that lets you
>>> completely automate applying a patchset to a branch.
>>>
>>> patchwork is a nice tool to gather up and track patchsets, with status
>>> indicators like "under review", "accepted", etc.  Chris already
>>> deploys one as part of QA, more integration with it would be nice.
>>
>> It seems (based on above) that "patchwork" can co-exist with debbugs.
>> Is that also the case with b4/lei?  Specifically, are the
>> users/reviewers able to benefit from using the above tools at present?
>> Or are there some reasons (over and above their lack of familiarity with
>> the above tools) that would prevent them from doing so?
>
> They both can co-exist with debbugs, and for now the patchwork instance
> of QA is not usable for status tracking (because it is not meant to be
> used as such for now).  One can already use both of them, but using both
> supercedes debbugs, and gets rid of its limitations.  I've been using
> b4/lei with the yhetil public-inbox instance, with piem.el as an
> interface, and it's really useful.  With a properly configured b4, one
> could simply run `b4 shazam some-msg-id` and it would automatically
> apply the corresponding patchset.

I’m interested in adopting this workflow.  Where can I find more
information on how to configure this?

-- 
Ricardo

Re: Guix Days: Patch flow discussion

[Kyle Meyer]

Hi Ricardo,

Ricardo Wurmus writes:

> Hi Josselin,

>> They both can co-exist with debbugs, and for now the patchwork instance
>> of QA is not usable for status tracking (because it is not meant to be
>> used as such for now).  One can already use both of them, but using both
>> supercedes debbugs, and gets rid of its limitations.  I've been using
>> b4/lei with the yhetil public-inbox instance, with piem.el as an
>> interface, and it's really useful.  With a properly configured b4, one
>> could simply run `b4 shazam some-msg-id` and it would automatically
>> apply the corresponding patchset.
>
> I’m interested in adopting this workflow.  Where can I find more
> information on how to configure this?

In 889a6204f8 (doc: Add some guidelines for reviewing, 2023-11-07),
Maxim added some b4 configuration to Guix's etc/git/gitconfig that
points at yhetil.org.  With that setup, running 'b4 shazam MESSAGE-ID'
from a Guix checkout should work.

You may have just be asking about configuring 'b4 shazam', but if you
(or others) are interested in configuring piem, read on :)


piem setup and use
==================

For piem, you need some custom setup in your Emacs config.

  https://docs.kyleam.com/piem/Registering-inboxes.html

piem-inboxes contains entries that provide info to help piem map from a
message (say in Notmuch or Gnus) to a public-inbox URL and to a local
checkout of the repo.

  (setq piem-inboxes
        '(("guix"
           :coderepo "/path/to/your/clone/of/guix/"
           :url "https://yhetil.org/guix/"
           :listid "bug-guix.gnu.org")))

Further setup depends on how (in Emacs) you read your messages.

  https://docs.kyleam.com/piem/Enabling-integration-libraries.html

For example, a Notmuch user would enable piem-notmuch-mode:

  (piem-notmuch-mode 1)

Or a debbugs.el user would enable piem-debbugs-mode (contributed by
Jelle Licht).  You can enable more than one integration library.

Once that's set up, the main entry point to b4 is the piem-b4-am
transient.  That transient in turn has the main command of interest,
piem-b4-am-from-mid.

  https://docs.kyleam.com/piem/Using-b4-to-apply-patches.html

Calling piem-b4-am-from-mid from, say, a Notmuch message with a patch
series will prepare the "am-ready" series, prompt you for the name of a
branch to create, and apply the series with 'git am' to the configured
repo.


piem-b4-am-from-mid vs b4 shazam
================================

piem-b4-am-from-mid (from Emacs) and 'b4 shazam' (from the shell
visiting Guix repo) are similar in spirit: use 'b4 am' to prepare an
am-ready mbox from a thread's full mbox and then use 'git am' apply it
to a repo.  A key difference is that, with piem, you start off in some
non-repository Emacs buffer containing a message and then piem knows how
to get to the repository (using the config above).

As a side note: 'b4 shazam' didn't exist when I wrote piem's b4
integration.  At some point, I plan to add 'b4 shazam' support to piem,
similar to how there is a piem command (piem-b4-am-ready-from-mid)
that's a direct interface to 'b4 am', without the extra handling of
piem-b4-am-from-mid.  This will allow callers that prefer to use shazam
invoke it from Emacs rather than the command line.


lei
===

There's also mention upthread of lei.  I don't think you're asking about
configuring that, but fwiw that's public-inbox's local command-line
client.  I gave a short (and incomplete) description of it here:

  https://yhetil.org/guix-devel/87a6izsoio.fsf@kyleam.com

piem currently has some basic support for lei, focused on searching
public-inbox instances and displaying the results in Emacs.

Re: Guix Days: Patch flow discussion

[Skyler Ferris]

On 2/6/24 05:39, Steve George wrote:
> I agreed to organise some 'patch review' online sessions in the next couple of
> weeks.
>
> Organising a basic process is a good topic for that online session. For
> example, elsewhere in the thread someone mentions some tags we could use
> consistently so maintainers can find patches that have been reviewed easily. It
> would be great to agree those - try them for a bit - and document them in a
> 'howto' so that everyone uses the same process.
Have these been announced somewhere yet (eg, with url & exact time)? If 
not will being subscribed to the help-guix list and/or checking the Guix 
blog be sufficient to receive notification? As someone who has sent 
patches in the past and intends to continue sending more in the future, 
I'd like to do my part to keep the project in a good state. However, I 
am new to interacting with large FLOSS projects so I'm nervous about 
causing more problems than I solve if I just start doing things.

Re: Guix Days: Patch flow discussion

[Edouard Klein]

Skyler Ferris <skyvine@protonmail.com> writes:

> On 2/6/24 05:39, Steve George wrote:
>> I agreed to organise some 'patch review' online sessions in the next couple of
>> weeks.
>>
>> Organising a basic process is a good topic for that online session. For
>> example, elsewhere in the thread someone mentions some tags we could use
>> consistently so maintainers can find patches that have been reviewed easily. It
>> would be great to agree those - try them for a bit - and document them in a
>> 'howto' so that everyone uses the same process.
> Have these been announced somewhere yet (eg, with url & exact time)? If
> not will being subscribed to the help-guix list and/or checking the Guix
> blog be sufficient to receive notification? As someone who has sent
> patches in the past and intends to continue sending more in the future,
> I'd like to do my part to keep the project in a good state. However, I
> am new to interacting with large FLOSS projects so I'm nervous about
> causing more problems than I solve if I just start doing things.

Same here.

Re: Guix Days: Patch flow discussion

[Andreas Enge]

Hello,

Am Fri, Feb 09, 2024 at 05:35:44PM +0100 schrieb Edouard Klein:
> Am Thu, Feb 08, 2024 at 07:56:48PM +0000 schrieb Skyler Ferris:
> > I'd like to do my part to keep the project in a good state. However, I
> > am new to interacting with large FLOSS projects so I'm nervous about
> > causing more problems than I solve if I just start doing things.
> Same here.

no need to be nervous! I would suggest to just start somewhere and learn
by doing. For patches, anyway the committer takes the final responsibility;
so if you can just add, for instance, "I use this package and it works for
me" this will be a useful point.

And do not be afraid to make mistakes; I have been taking part in Guix for
a long time now, and of course make mistakes from time to time. That
happens, and they can be corrected if they occur. What I find important
is to be upfront about them, and to ask for help if needed.

Andreas

Re: Guix Days: Patch flow discussion

[Steve George]

On  9 Feb, Edouard Klein wrote:
> 
> Skyler Ferris <skyvine@protonmail.com> writes:
> 
> > On 2/6/24 05:39, Steve George wrote:
> >> I agreed to organise some 'patch review' online sessions in the next couple of
> >> weeks.
> >>
> >> Organising a basic process is a good topic for that online session. For
> >> example, elsewhere in the thread someone mentions some tags we could use
> >> consistently so maintainers can find patches that have been reviewed easily. It
> >> would be great to agree those - try them for a bit - and document them in a
> >> 'howto' so that everyone uses the same process.
> > Have these been announced somewhere yet (eg, with url & exact time)? If
> > not will being subscribed to the help-guix list and/or checking the Guix
> > blog be sufficient to receive notification? As someone who has sent
> > patches in the past and intends to continue sending more in the future,
> > I'd like to do my part to keep the project in a good state. However, I
> > am new to interacting with large FLOSS projects so I'm nervous about
> > causing more problems than I solve if I just start doing things.
> 
> Same here.
>

Hi Skyler & Edouard - you haven't missed an announcement :-) I'll get it organised next week and will mail out here - plus I'll email you both directly as you've both shown interest.

Thanks,

Steve / Futurile

Re: Guix Days: Patch flow discussion

[Maxim Cournoyer]

Hi,

Kyle Meyer <kyle@kyleam.com> writes:

> Hi Ricardo,
>
> Ricardo Wurmus writes:
>
>> Hi Josselin,
>
>>> They both can co-exist with debbugs, and for now the patchwork instance
>>> of QA is not usable for status tracking (because it is not meant to be
>>> used as such for now).  One can already use both of them, but using both
>>> supercedes debbugs, and gets rid of its limitations.  I've been using
>>> b4/lei with the yhetil public-inbox instance, with piem.el as an
>>> interface, and it's really useful.  With a properly configured b4, one
>>> could simply run `b4 shazam some-msg-id` and it would automatically
>>> apply the corresponding patchset.
>>
>> I’m interested in adopting this workflow.  Where can I find more
>> information on how to configure this?
>
> In 889a6204f8 (doc: Add some guidelines for reviewing, 2023-11-07),
> Maxim added some b4 configuration to Guix's etc/git/gitconfig that
> points at yhetil.org.  With that setup, running 'b4 shazam MESSAGE-ID'
> from a Guix checkout should work.

Indeed!  It should perhaps be more prominently documented, probably as
an new 'Reviewing patch work flows' section in the Guix Cookbook that
could be linked from our already lengthy Contributions section.

'b4 shazam' is probably the most trouble-free way to apply patches; it
even selects the latest revision it finds in the issue thread.  To make
finding a message-id easier, I've also recently added a 'Copy
Message-ID' button to the Mumi interface; try it visiting any issue,
e.g. <https://issues.guix.gnu.org/68990>.  The message-id of any message
can be easily copied to your clipboard via the new button.  From your
Guix checkout, you'd then run:

--8<---------------cut here---------------start------------->8---
$ b4 shazam 'cover.1707383694.git.h.goebel@crazy-compilers.com'
Looking up https://yhetil.org/guix/cover.1707383694.git.h.goebel@crazy-compilers.com
Grabbing thread from yhetil.org/guix/cover.1707383694.git.h.goebel@crazy-compilers.com/t.mbox.gz
Checking for newer revisions
Grabbing search results from lore.kernel.org
Nothing matching that query.
Analyzing 29 messages in the thread
---
  [PATCH 1/28] gnu: Add ruby-test-unit-ruby-core.
  [PATCH 2/28] gnu: Add ruby-excon.
  [PATCH 3/28] gnu: Add ruby-ipaddr.
  [PATCH 4/28] gnu: Add ruby-net-ftp.
  [PATCH 5/28] gnu: Add ruby-fake-ftp.
  [PATCH 6/28] gnu: Add ruby-net-sftp.
  [PATCH 7/28] gnu: Add ruby-net-telnet.
  [PATCH 8/28] gnu: Add ruby-pairing-heap.
  [PATCH 9/28] gnu: Add ruby-stringio.
  [PATCH 10/28] gnu: Add ruby-stream.
  [PATCH 11/28] gnu: Add ruby-rgl.
  [PATCH 12/28] gnu: Add ruby-sfl.
  [PATCH 13/28] gnu: Add ruby-specinfra.
  [PATCH 14/28] gnu: Add ruby-serverspec.
  [PATCH 15/28] gnu: Add ruby-time.
  [PATCH 16/28] gnu: Add ruby-google-protobuf.
  [PATCH 17/28] gnu: Add ruby-googleapis-common-protos-types.
  [PATCH 18/28] gnu: Add ruby-grpc.
  [PATCH 19/28] gnu: Add ruby-vagrant-cloud.
  [PATCH 20/28] gnu: Add ruby-vagrant-spec.
  [PATCH 21/28] gnu: Add ruby-vagrant-spec-helper-basic.
  [PATCH 22/28] gnu: Add ruby-hashicorp-checkpoint.
  [PATCH 23/28] gnu: ruby-childprocess: Update to 4.1.0.
  [PATCH 24/28] gnu: Add ruby-libvirt.
  [PATCH 25/28] gnu: Add ruby-fog-core.
  [PATCH 26/28] gnu: Add ruby-fog-json.
  [PATCH 27/28] gnu: Add ruby-fog-xml.
  [PATCH 28/28] gnu: Add ruby-fog-libvirt.
---
Total patches: 28
---
 Base: using specified base-commit a4464bd0975c811f18af98f69032b29bddda5b81
Application de  gnu: Add ruby-test-unit-ruby-core.
Application de  gnu: Add ruby-excon.
Application de  gnu: Add ruby-ipaddr.
Application de  gnu: Add ruby-net-ftp.
Application de  gnu: Add ruby-fake-ftp.
Application de  gnu: Add ruby-net-sftp.
Application de  gnu: Add ruby-net-telnet.
Application de  gnu: Add ruby-pairing-heap.
Application de  gnu: Add ruby-stringio.
Application de  gnu: Add ruby-stream.
Application de  gnu: Add ruby-rgl.
Application de  gnu: Add ruby-sfl.
Application de  gnu: Add ruby-specinfra.
Application de  gnu: Add ruby-serverspec.
Application de  gnu: Add ruby-time.
Application de  gnu: Add ruby-google-protobuf.
Application de  gnu: Add ruby-googleapis-common-protos-types.
Application de  gnu: Add ruby-grpc.
Application de  gnu: Add ruby-vagrant-cloud.
Application de  gnu: Add ruby-vagrant-spec.
Application de  gnu: Add ruby-vagrant-spec-helper-basic.
Application de  gnu: Add ruby-hashicorp-checkpoint.
Application de  gnu: ruby-childprocess: Update to 4.1.0.
Application de  gnu: Add ruby-libvirt.
Application de  gnu: Add ruby-fog-core.
Application de  gnu: Add ruby-fog-json.
Application de  gnu: Add ruby-fog-xml.
Application de  gnu: Add ruby-fog-libvirt.
--8<---------------cut here---------------end--------------->8---

and done!

-- 
Thanks,
Maxim

Re: Guix Days: Patch flow discussion

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

On 2024-02-07, Josselin Poiret wrote:
> The fact that you have to wait for Debbugs's response after the first
> mail to get the proper mail to reply to means that we can't automate
> sending whole patchsets, and have to resort to hacks like the CLI `mumi`
> tool uses.  I can't just send a patchset and be done with it, I have to
> wait a couple minutes.

Well, that is a conflict between guix policies/practices and
debbugs.

Debbugs handles sending multiple patches in a single email just fine,
which can be done on the initial submission.

That has downsides with regards to threading, but given all the current
limitations, I wonder if the downsides of having to wait for the initial
bug number to land are worth the cost of having to implement a
user-interface tool (mumi CLI) to workaround it.

That said, the tool already exists... but not everyone is aware of it or
for whatever reason is not necesarily using it.

Are there other downsides to allowing a multiple patches in a single
email?

Ideally the technology would fit whatever practicies guix would like to
implement... but then we have what we have right now.

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Guix Days: Patch flow discussion

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Josselin,

On Wed, Feb 07 2024, Josselin Poiret wrote:

> The fact that you have to wait for Debbugs's response after the first
> mail to get the proper mail to reply to means that we can't automate
> sending whole patchsets

Could a modified version of Debbugs group submissions by Message-IDs
instead of bug numbers? Would it allow subsequent emails to be sent
before the initial message was acknowledged?

Kind regards
Felix

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Sun, Feb 11 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:

> Hi Josselin,
>
> On Wed, Feb 07 2024, Josselin Poiret wrote:
>
>> The fact that you have to wait for Debbugs's response after the first
>> mail to get the proper mail to reply to means that we can't automate
>> sending whole patchsets
>
> Could a modified version of Debbugs group submissions by Message-IDs
> instead of bug numbers? Would it allow subsequent emails to be sent
> before the initial message was acknowledged?

Hi Josselin, what do you mean?  Each email has its own message id, so
how would you group them?

Cheers
Clément

Re: Guix Days: Patch flow discussion

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Clément,

On Sun, Feb 11 2024, Clément Lassieur wrote:

> On Sun, Feb 11 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:
>
> Each email has its own message id, so how would you group them?

As author, I'll respond. I was thinking of the In-Reply-To: or the
References: fields from RFC 2822. [1]

Kind regards
Felix

[1] https://datatracker.ietf.org/doc/html/rfc2822#section-3.6.4

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Sun, Feb 11 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:

> Hi Clément,
>
> On Sun, Feb 11 2024, Clément Lassieur wrote:
>
>> On Sun, Feb 11 2024, Felix Lechner via "Development of GNU Guix and the GNU System distribution." wrote:
>>
>> Each email has its own message id, so how would you group them?
>
> As author, I'll respond.

Oh sorry Felix, I'm tired

> I was thinking of the In-Reply-To: or the
> References: fields from RFC 2822. [1]

Ok thx.

> Kind regards
> Felix
>
> [1] https://datatracker.ietf.org/doc/html/rfc2822#section-3.6.4

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

Hi Felix,

Felix Lechner <felix.lechner@lease-up.com> writes:

> Could a modified version of Debbugs group submissions by Message-IDs
> instead of bug numbers? Would it allow subsequent emails to be sent
> before the initial message was acknowledged?

To me, an ideal solution would be a service that

1) Doesn't modify any incoming mail;
2) Provides a way to look-up the bugs related to a thread and their
status, preferably using Message-ID, so that it can be used in
conjunction with other mail-based tools without disturbing them.

Your proposed modification would be a first step in that direction, but
would still not solve 1 and 2 perfectly.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Mon, Feb 12 2024, Josselin Poiret wrote:

> Hi Felix,
>
> Felix Lechner <felix.lechner@lease-up.com> writes:
>
>> Could a modified version of Debbugs group submissions by Message-IDs
>> instead of bug numbers? Would it allow subsequent emails to be sent
>> before the initial message was acknowledged?
>
> To me, an ideal solution would be a service that
>
> 1) Doesn't modify any incoming mail;
> 2) Provides a way to look-up the bugs related to a thread and their
> status, preferably using Message-ID, so that it can be used in
> conjunction with other mail-based tools without disturbing them.
>
> Your proposed modification would be a first step in that direction, but
> would still not solve 1 and 2 perfectly.
>
> Best,

We could use a new email header (say X-GNU-PR-Bug) for the grouping,
whose value would be a unique random id, generated (automatically) by
the patchset sender?

We'd have the guarantee that two different bugs won't have the same
X-GNU-PR-Bug value, and we'd be able to send the patches in a more
automated way.

Clément

Re: Guix Days: Patch flow discussion

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Josselin,

On Mon, Feb 12 2024, Josselin Poiret wrote:

> 1) Doesn't modify any incoming mail;

What if, in addition, Debbugs were to publish bug reports and their
comments via public-inbox?

> 2) Provides a way to look-up the bugs related to a thread and their
> status, preferably using Message-ID

What if those bug reports were available for your consumption via IMAP
folders or NNTP news groups whose idenfier is the initial Message-Id?

Kind regards
Felix

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

Hi Felix,

Felix Lechner <felix.lechner@lease-up.com> writes:

> Hi Josselin,
>
> On Mon, Feb 12 2024, Josselin Poiret wrote:
>
>> 1) Doesn't modify any incoming mail;
>
> What if, in addition, Debbugs were to publish bug reports and their
> comments via public-inbox?

Why “in addition” though?  As long as Debbugs is modifying mails on the
MLs it's being troublesome.

>> 2) Provides a way to look-up the bugs related to a thread and their
>> status, preferably using Message-ID
>
> What if those bug reports were available for your consumption via IMAP
> folders or NNTP news groups whose idenfier is the initial Message-Id?

That might introduce too much complexity: what we probably want is just
a bug tracker that maps bug IDs to Message-IDs, then consumers just use
whatever tool they'd prefer, be it browsing a public-inbox instance
through a browser, their local mail, etc.  Debbugs could include a
public-inbox instance, or just interact with one (preferred so that
efforts are not duplicated).  Just having the header part of
e.g. bugzilla would be nice, maybe with a log of mails that caused a
state change, and a list of relevant threads (there could be multiple,
for example if there are multiple versions of a patchset, or for the bug
report and the ensuing fix).

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Josselin,

On Tue, Feb 13 2024, Josselin Poiret wrote:

> As long as Debbugs is modifying mails on the MLs it's being
> troublesome.

Will you please show an example?  You mentioned it before but I cannot
find your message. Thanks!

Kind regards
Felix

P.S. With a bug tracker, our discussion could be off-list. It would
reduce the reading burden for folks who do not care about the topic.

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

Hi Felix,

Felix Lechner <felix.lechner@lease-up.com> writes:

> Hi Josselin,
>
> On Tue, Feb 13 2024, Josselin Poiret wrote:
>
>> As long as Debbugs is modifying mails on the MLs it's being
>> troublesome.
>
> Will you please show an example?  You mentioned it before but I cannot
> find your message. Thanks!
>
> Kind regards
> Felix
>
> P.S. With a bug tracker, our discussion could be off-list. It would
> reduce the reading burden for folks who do not care about the topic.

See e.g. [1], where my reply should have the subject “Re: bug#nnnn ...”,
but Debbugs doesn't like that and so changes it to just “bug#nnn ...”.
This modification in turn invalidates DMARC because From is protected,
so it *has* to rewrite it to another domain to make it pass DMARC.

[1] mid:87eddk630t.fsf@jpoiret.xyz
    (https://yhetil.org/guix/87eddk630t.fsf@jpoiret.xyz/)

Best,

-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Simon Tournier]

Hi,

On dim., 11 févr. 2024 at 11:38, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

> 'b4 shazam' is probably the most trouble-free way to apply patches;

I agree*!

>                                                                     it
> even selects the latest revision it finds in the issue thread.  To make
> finding a message-id easier, I've also recently added a 'Copy
> Message-ID' button to the Mumi interface; try it visiting any issue,
> e.g. <https://issues.guix.gnu.org/68990>.  The message-id of any message
> can be easily copied to your clipboard via the new button.

I also agree! :-) What appears to me “difficult” is that most of the
tools as Email client are poorly supporting Message-ID.

For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
to get the Message-ID when reading an article (bug/patch) from Debbugs.
There is other means for applying patches.  But still each time appears
to me weird. :-)

So thanks for this Mumi addition!

Cheers,
simon

PS: *agree on B4 although there is tricky bugs as reported here:
<https://github.com/mricon/b4/issues/4>.

Re: Guix Days: Patch flow discussion

[Simon Tournier]

Hi Steve,

  ( On a side note, the triage of old bugs is a similar problem.  They
    are easy to find [2], read, check and send an email to
    12345@debbugs.gnu.org does not appear to me an issue with any tool.
    
    For what it is worth and without any willing of being harsh, I am
    able to count the people doing this boring task.

    What is hard to solve is the incentives for doing the boring, but
    necessary, collective tasks.

    Bah the usual problem of lengthy discussions with roommates in any
    shared apartment: who clean the bathroom? :-) )


On lun., 05 févr. 2024 at 09:39, Steve George <steve@futurile.net> wrote:

> Our goal for the discussion:
>
> 	How do we double the number of patches that are *reviewed* and
> 	*applied* to Guix in the next six months?

Thanks for these notes and leading the session.  On my side, it was a
fruitful discussion.

Well, let me try to quickly summarize my conclusion of the session:

 1. We have a social/organisational problem.

 2. We have some tooling annoyances.



The easy first: #2 about tools.  The email workflow is often cited as
part of the issue.  That’s a false-problem, IMHO.

Projects that use PR/MR workflow have the same problem.  For instance,
Julia [1] has 896 open PR.  On my browser, it means 36 pages so if I go
to – 25 PRs per page – the still open submitted PRs:

   + the 6th page:  around Sept.2023 and Oct. 2023
   + the 12th page: around Apr. 2023 and Mar. 2023
   + the 18th page: around Jul. 2022 and Mar. 2022
   + the 24th page: around Jun. 2021 and May  2021
   + the 30th page: around Mar. 2020 and Oct. 2019
   + the 36th page: around Mar. 2017 and May. 2014

Obviously, an example is not a proof or an evidence.  It is just a
first clue. :-)

I will not speak about the channel ’nonguix’ but it gives another
clue.

That said, for sure, the tools need more love.  Thanks all the people
for all hard work over the years in this area – no name, you know, I
fear to forget someone. ;-)

So, yeah we need to smooth the technical burden for reviewing in order
to focus on the review itself.

To be clear, the email workflow might add burden on submitter side but I
am doubtful it is really part of the bottleneck for reviewing and
pushing submissions.


Although the tools might add some unnecessary friction, the net of the
issue is IMHO #1: reviewing is just boring and time-consuming.

Who feel accountable?  And for what?  That’s the question, IMHO.

If the number of submission is doubled, how do we increase the number of
people that feel enough accountable for doing the boring work?

  ( Maybe accountable is not the correct word.  Obligation neither.
    Well the kind of feeling that is okish if you skip the task but you
    know it will be better if you do it. )


Well, the difficult part is not pressing some buttons for merging and
pushing – whatever the tools or workflow.  The difficult part is to
scrutinize the submission.

I think the bottleneck is not the number of people able to push.
Instead, I think the bottleneck is the number of people confident with
the change for then pushing it.

The question is thus: how to build this confidence?


Look, when a committer has some free-time, most of the time, what is the
process: take first the “easy“ submissions for committing them – from
trivial updates to simple updates.  If free-time remains, then engage
with more “complex” submissions… ah no more free-time. :-)

Why starting by the “easy” submission?  Because it is less boring and
time-consuming; somehow it is easier to feel confident with that sort of
change for pushing it.

As a rule of thumb, about the time it takes – on average –, the order of
magnitude for reviewing is similar as the one for submitting.  Well,
from my experience and although I never did stats. :-)


All in all, I see two paths to move forward:

i) Non-committers can help.  On two fronts:

   + Answer to submitter with the changes for being compliant with Guix
     standards.
   
   + Follow-up on patches already commented but without an updated
     revision: upgrade the re-roll count by sending this revision.

     It eases for merging if I do not have to make many tiny edits
     myself.

ii) Create more teams or at least more people should commit to be part
    of a team and help in reviewing what they know.

    For instance, since Sept. (167 days ago) I have been CC in 108
    patches submissions.  Most of them are from ’core’ team that I would
    qualify as “complex”. :-)

    Many patches assigned to ’core’ team are sent by committers.  The
    issue is not being a committer or not.  Instead, being more eyes
    commenting would increase the confidence.  Thus it would reduce the
    workload.

    That’s the same for any team, IMHO.
    
    And I do not speak about patches that are not assigned to any team.


Somehow, we need to think how people would feel “accountable” for doing
the collective tasks with low, no direct or personal reward.

As with many non-technical topics, it is not easy.  Because it is a
collective journey not clearly identified – and not a kind of
reproducible bug to fix, even the tougher.

Last, the manual says:

    « As a rule of thumb, a contributor should have accumulated
      fifty (50) reviewed commits to be considered as a committer
      and have sustained their activity in the project for at least
      6 months. »

So if people are willing to take more responsibility to help the
project…


Well, while writing that I could have give a look to patches… ;-)

Cheers,
simon


1: https://github.com/JuliaLang/julia
2: https://issues.guix.gnu.org/forgotten

Re: Guix Days: Patch flow discussion

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]

Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> PS: *agree on B4 although there is tricky bugs as reported here:
> <https://github.com/mricon/b4/issues/4>.

I think b4's ML is more active than the GitHub issues, I have already
sent some bug reports and patches there that were picked up quite fast.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

Hey Simon!

On Wed, Feb 14 2024, Simon Tournier wrote:

> Hi,
>
> On dim., 11 févr. 2024 at 11:38, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> 'b4 shazam' is probably the most trouble-free way to apply patches;
>
> I agree*!

I don't agree (both Gwene + Gnus or Emacs Debbugs work perfectly too and
allow to apply a range of n patches at once) but I don't think there is
a need for competition here, it's good that we have several tools.

>>                                                                     it
>> even selects the latest revision it finds in the issue thread.  To make
>> finding a message-id easier, I've also recently added a 'Copy
>> Message-ID' button to the Mumi interface; try it visiting any issue,
>> e.g. <https://issues.guix.gnu.org/68990>.  The message-id of any message
>> can be easily copied to your clipboard via the new button.
>
> I also agree! :-) What appears to me “difficult” is that most of the
> tools as Email client are poorly supporting Message-ID.
>
> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
> to get the Message-ID when reading an article (bug/patch) from Debbugs.
> There is other means for applying patches.  But still each time appears
> to me weird. :-)

It's

--8<---------------cut here---------------start------------->8---
(with-current-buffer gnus-original-article-buffer
  (message-fetch-field "Message-ID"))
--8<---------------cut here---------------end--------------->8---

Cheers
Clément

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Thu, Feb 15 2024, Clément Lassieur wrote:

> Hey Simon!
>
> On Wed, Feb 14 2024, Simon Tournier wrote:
>
>> Hi,
>>
>> On dim., 11 févr. 2024 at 11:38, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>>
>>> 'b4 shazam' is probably the most trouble-free way to apply patches;
>>
>> I agree*!
>
> I don't agree (both Gwene + Gnus or Emacs Debbugs work perfectly too and
> allow to apply a range of n patches at once) but I don't think there is
> a need for competition here, it's good that we have several tools.
>
>>>                                                                     it
>>> even selects the latest revision it finds in the issue thread.  To make
>>> finding a message-id easier, I've also recently added a 'Copy
>>> Message-ID' button to the Mumi interface; try it visiting any issue,
>>> e.g. <https://issues.guix.gnu.org/68990>.  The message-id of any message
>>> can be easily copied to your clipboard via the new button.
>>
>> I also agree! :-) What appears to me “difficult” is that most of the
>> tools as Email client are poorly supporting Message-ID.
>>
>> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
>> to get the Message-ID when reading an article (bug/patch) from Debbugs.
>> There is other means for applying patches.  But still each time appears
>> to me weird. :-)

May I add too, that you can add "Message-ID" in gnus-visible-headers.

>
> It's
>
> (with-current-buffer gnus-original-article-buffer
>   (message-fetch-field "Message-ID"))
>
> Cheers
> Clément

Re: Guix Days: Patch flow discussion

[Simon Tournier]

Hi Josselin,

On jeu., 15 févr. 2024 at 12:07, Josselin Poiret <dev@jpoiret.xyz> wrote:

> I think b4's ML is more active than the GitHub issues, I have already
> sent some bug reports and patches there that were picked up quite fast.

Yeah, Kyle pointed me that out months ago.  Then I have never taken the
time to report there. :-)

Cheers,
simon

Re: Guix Days: Patch flow discussion

[Simon Tournier]

Hi Clément,

On jeu., 15 févr. 2024 at 12:45, Clément Lassieur <clement@lassieur.org> wrote:

>>> 'b4 shazam' is probably the most trouble-free way to apply patches;
>>
>> I agree*!
>
> I don't agree (both Gwene + Gnus or Emacs Debbugs work perfectly too and
> allow to apply a range of n patches at once) but I don't think there is
> a need for competition here, it's good that we have several tools.

Yes for sure it is good to have several tools.  And the ones you like. :-)

No one is advocating to make ’b4 shazam’ THE only one tool.

Instead, I agree with Maxim that exposing Message-ID and relying on ’b4
shazam’ is the most trouble-free and config-less way to apply patches.


>>                  What appears to me “difficult” is that most of the
>> tools as Email client are poorly supporting Message-ID.
>>
>> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
>> to get the Message-ID when reading an article (bug/patch) from Debbugs.
>> There is other means for applying patches.  But still each time appears
>> to me weird. :-)
>
> It's
>
> --8<---------------cut here---------------start------------->8---
> (with-current-buffer gnus-original-article-buffer
>   (message-fetch-field "Message-ID"))
> --8<---------------cut here---------------end--------------->8---

[...]

> May I add too, that you can add "Message-ID" in gnus-visible-headers.

And what about Summary buffer?

Well, it makes my point, no? :-)

For sure the Message-ID is there and for sure it is possible to extract
it.  However, it appears to me weird that it is not built-in.  I mean
Message-ID is one of the heart of Emails, and Debbugs is just Emails,
but debbugs.el does not provide a built-in access to it.


Cheers,
simon

Re: Guix Days: Patch flow discussion

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Simon,

On Thu, Feb 15 2024, Clément Lassieur wrote:

> May I add too, that you can add "Message-ID" in gnus-visible-headers.

The author of Debbugs.el, Michael Albinus, said this was likely the best
solution.

To request a feature in Debbugs.el, please file a bug against the
"debbugs.gnu.org" package on debbugs.gnu.org. Thanks!

Kind regards
Felix

Re: Guix Days: Patch flow discussion

[Simon Tournier]

Hi Felix,

On jeu., 15 févr. 2024 at 07:32, Felix Lechner via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> wrote:

> To request a feature in Debbugs.el, please file a bug against the
> "debbugs.gnu.org" package on debbugs.gnu.org.

To be clear, my message [1] was not to report a “bug” or request for a
“feature” in debbugs.el.  My message was:

                          What appears to me “difficult” is that most of the
        tools as Email client are poorly supporting Message-ID.

        For instance,

And I took as one example the venerable debbugs.el for making my point:
most of the tools that deal with Emails poorly support one key of Email
heart: Message-ID.

Personally, I rely on the cool piem.el and some custom Emacs lisp
helpers, then for dealing with complex patch or bug thread, I inject and
process them with notmuch.el. :-)

Therefore, open a feature request is low on my list of TODO. ;-)

Cheers,
simon

1: Re: Guix Days: Patch flow discussion
Simon Tournier <zimon.toutoune@gmail.com>
Wed, 14 Feb 2024 16:48:07 +0100
id:87mss3kpxk.fsf@gmail.com
https://lists.gnu.org/archive/html/guix-devel/2024-02
https://yhetil.org/guix/87mss3kpxk.fsf@gmail.com

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Thu, Feb 15 2024, Simon Tournier wrote:

> Hi Clément,
>
> On jeu., 15 févr. 2024 at 12:45, Clément Lassieur <clement@lassieur.org> wrote:
>
>>>> 'b4 shazam' is probably the most trouble-free way to apply patches;
>>>
>>> I agree*!
>>
>> I don't agree (both Gwene + Gnus or Emacs Debbugs work perfectly too and
>> allow to apply a range of n patches at once) but I don't think there is
>> a need for competition here, it's good that we have several tools.
>
> Yes for sure it is good to have several tools.  And the ones you like. :-)
>
> No one is advocating to make ’b4 shazam’ THE only one tool.
>
> Instead, I agree with Maxim that exposing Message-ID and relying on ’b4
> shazam’ is the most trouble-free and config-less way to apply patches.

I know ;)

>>>                  What appears to me “difficult” is that most of the
>>> tools as Email client are poorly supporting Message-ID.
>>>
>>> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
>>> to get the Message-ID when reading an article (bug/patch) from Debbugs.
>>> There is other means for applying patches.  But still each time appears
>>> to me weird. :-)
>>
>> It's
>>
>> --8<---------------cut here---------------start------------->8---
>> (with-current-buffer gnus-original-article-buffer
>>   (message-fetch-field "Message-ID"))
>> --8<---------------cut here---------------end--------------->8---
>
> [...]
>
>> May I add too, that you can add "Message-ID" in gnus-visible-headers.
>
> And what about Summary buffer?

You add '%M' in gnus-summary-line-format.

> Well, it makes my point, no? :-)

No, it doesn't :-)

> For sure the Message-ID is there and for sure it is possible to extract
> it.  However, it appears to me weird that it is not built-in.  I mean
> Message-ID is one of the heart of Emails, and Debbugs is just Emails,
> but debbugs.el does not provide a built-in access to it.

Yes it does provide a built-in access, as I showed you.  Just search for
"Message-id" in `C-h v gnus-summary-line-format`.

Cheers,
Clément

Re: Guix Days: Patch flow discussion

[Simon Tournier]

Hi Clément,

If read correctly, you answered about Gnus (debbugs.el):

>>> --8<---------------cut here---------------start------------->8---
>>> (with-current-buffer gnus-original-article-buffer
>>>   (message-fetch-field "Message-ID"))
>>> --8<---------------cut here---------------end--------------->8---

[...]

>>> May I add too, that you can add "Message-ID" in gnus-visible-headers.

[...]

> You add '%M' in gnus-summary-line-format.

[...]

> Yes it does provide a built-in access, as I showed you.  Just search for
> "Message-id" in `C-h v gnus-summary-line-format`.

And my message was:

                      What appears to me “difficult” is that most of the
     tools as Email client are poorly supporting Message-ID.

Somehow, the reader will judge if Message-ID is smoothly supported. :-)

Cheers,
simon

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Thu, Feb 15 2024, Simon Tournier wrote:

> Hi Clément,
>
> If read correctly, you answered about Gnus (debbugs.el):
>
>>>> --8<---------------cut here---------------start------------->8---
>>>> (with-current-buffer gnus-original-article-buffer
>>>>   (message-fetch-field "Message-ID"))
>>>> --8<---------------cut here---------------end--------------->8---
>
> [...]
>
>>>> May I add too, that you can add "Message-ID" in gnus-visible-headers.
>
> [...]
>
>> You add '%M' in gnus-summary-line-format.
>
> [...]
>
>> Yes it does provide a built-in access, as I showed you.  Just search for
>> "Message-id" in `C-h v gnus-summary-line-format`.
>
> And my message was:
>
>                       What appears to me “difficult” is that most of the
>      tools as Email client are poorly supporting Message-ID.

I didn't reply to this, but to your debbugs.el (gnus) example,
explaining how Message-ID was well supported.

> Somehow, the reader will judge if Message-ID is smoothly supported. :-)

Ok, yes.  Good night Simon :)

Clément

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Tue, Feb 06 2024, Suhail wrote:

> Steve George <steve@futurile.net> writes:
>
>> elsewhere in the thread someone mentions some tags we could use
>> consistently so maintainers can find patches that have been reviewed
>> easily.
>
> It seems on the [dev manual] we already have "reviewed-looks-good"
> documented.  Thus, I'd like to propose the below *mutually exclusive*
> Debbugs tag set:
>
> - "not-yet-reviewed" :: automatically set for all submissions
> - "reviewed-needs-fix" :: set explicitly by the reviewer
> - "needs-another-review" :: automatically set if there's a revised
>   patch, unless "not-yet-reviewed" (in which case no change)
> - "reviewed-looks-good" :: set explicitly by the reviewer

Would it makes sense to have a "does-not-apply" tag too?

I believe that would help sorting old those old patches that don't apply
anymore.

> In addition to the above, it might also help for there to be an
> additional tag of "might-not-need-review" (or simpler,
> "review-not-needed") which gets automatically set, provided we implement
> a way to label some changes (for some packages) as being "trivial enough
> that they're okay as long as build succeeds".
>
> On a related note, is it possible for a reviewer who isn't a committer
> to set debbugs tags?
>
> [dev manual]: <https://guix.gnu.org/en/manual/devel/en/html_node/Debbugs-Usertags.html>
>
>> It would be great to agree those - try them for a bit - and document
>> them in a 'howto' so that everyone uses the same process.
>
> In addition to documenting the tags in the "Debbugs Usertags" section of
> the manual, it would help for there to be a "howto" which focuses more
> on the transition between the tags (i.e., the contribution workflow).

Re: Guix Days: Patch flow discussion

[Andreas Enge]

Am Fri, Feb 16, 2024 at 11:56:50AM +0100 schrieb Clément Lassieur:
> Would it makes sense to have a "does-not-apply" tag too?

Should this not appear in the QA page, assuming that once all the new
issues are closed, older ones will bubble to the top and be treated by QA?
(I am not sure if just looking at the n latests issues is how QA works,
but I think so.)

Andreas

Re: Guix Days: Patch flow discussion

[Clément Lassieur]

On Fri, Feb 16 2024, Andreas Enge wrote:

> Am Fri, Feb 16, 2024 at 11:56:50AM +0100 schrieb Clément Lassieur:
>> Would it makes sense to have a "does-not-apply" tag too?
>
> Should this not appear in the QA page, assuming that once all the new
> issues are closed, older ones will bubble to the top and be treated by QA?
> (I am not sure if just looking at the n latests issues is how QA works,
> but I think so.)

I think it would be useful (at least for me) if I browse a list of
patches that I want to review, to document why a review can't be done.

Also, so that other people won't try to apply it.

It would be great that QA does that job, I imagine when it does it it
can also add that tag.

Re: Guix Days: Patch flow discussion

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]


Clément Lassieur <clement@lassieur.org> writes:

> On Fri, Feb 16 2024, Andreas Enge wrote:
>
>> Am Fri, Feb 16, 2024 at 11:56:50AM +0100 schrieb Clément Lassieur:
>>> Would it makes sense to have a "does-not-apply" tag too?
>>
>> Should this not appear in the QA page, assuming that once all the new
>> issues are closed, older ones will bubble to the top and be treated by QA?
>> (I am not sure if just looking at the n latests issues is how QA works,
>> but I think so.)
>
> I think it would be useful (at least for me) if I browse a list of
> patches that I want to review, to document why a review can't be done.
>
> Also, so that other people won't try to apply it.
>
> It would be great that QA does that job, I imagine when it does it it
> can also add that tag.

There's the filter issues form which allows finding patches which can't
be applied:

  https://qa.guix.gnu.org/patches?failed-to-apply-patches=on

I think QA tagging issues might be useful for people to benefit from
that data in other places, and should be possible, it just needs QA to
send emails I think.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Guix Days: Patch flow discussion

[Maxim Cournoyer]

Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On dim., 11 févr. 2024 at 11:38, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> 'b4 shazam' is probably the most trouble-free way to apply patches;
>
> I agree*!
>
>>                                                                     it
>> even selects the latest revision it finds in the issue thread.  To make
>> finding a message-id easier, I've also recently added a 'Copy
>> Message-ID' button to the Mumi interface; try it visiting any issue,
>> e.g. <https://issues.guix.gnu.org/68990>.  The message-id of any message
>> can be easily copied to your clipboard via the new button.
>
> I also agree! :-) What appears to me “difficult” is that most of the
> tools as Email client are poorly supporting Message-ID.
>
> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
> to get the Message-ID when reading an article (bug/patch) from Debbugs.
> There is other means for applying patches.  But still each time appears
> to me weird. :-)

I usually used C-u g to see the message source, then regexp-searched for
^Message-ID, but that's not exactly convenient.

-- 
Thanks,
Maxim

Re: Guix Days: Patch flow discussion

[Maxim Cournoyer]

Hi Clément,

Clément Lassieur <clement@lassieur.org> writes:


[...]

>>> I also agree! :-) What appears to me “difficult” is that most of the
>>> tools as Email client are poorly supporting Message-ID.
>>>
>>> For instance, debbugs.el (Gnus).  To my knowledge, there is not easy way
>>> to get the Message-ID when reading an article (bug/patch) from Debbugs.
>>> There is other means for applying patches.  But still each time appears
>>> to me weird. :-)
>
> May I add too, that you can add "Message-ID" in gnus-visible-headers.

Thanks, that sounds useful!

-- 
Thanks,
Maxim

Re: Guix Days: Patch flow discussion

[Suhail]

Clément Lassieur <clement@lassieur.org> writes:

>> It seems on the [dev manual] we already have "reviewed-looks-good"
>> documented.  Thus, I'd like to propose the below *mutually exclusive*
>> Debbugs tag set:
>>
>> - "not-yet-reviewed" :: automatically set for all submissions
>> - "reviewed-needs-fix" :: set explicitly by the reviewer
>> - "needs-another-review" :: automatically set if there's a revised
>>   patch, unless "not-yet-reviewed" (in which case no change)
>> - "reviewed-looks-good" :: set explicitly by the reviewer
>
> Would it makes sense to have a "does-not-apply" tag too?
>
> I believe that would help sorting old those old patches that don't apply
> anymore.

Agreed.  A "does-not-apply" tag would be helpful.  This tag would be
explicitly applied, either by the QA mechanism or a reviewer.

Assuming there aren't any objections to these tags, what are the next
steps?  If my understanding of Debbugs Usertags is correct it seems we
can simply start using them?  Though noting the above in the manual
would be helpful.  However, that still leaves open the issue of how the
automated setting of tags is accomplished.

Thoughts?

-- 
Suhail

Re: Guix Days: Patch flow discussion

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Vagrant,

On Sun, Feb 11 2024, Vagrant Cascadian wrote:

> Are there other downsides to allowing a multiple patches in a single
> email?

Is it perhaps more difficult to apply those patches?

Kind regards
Felix

Re: Guix Days: Patch flow discussion

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 3922 bytes --]

Hello Simon,

first and foremost: I'd like to say a big thank you to all the people
working in the Guix community...

...and apologise if I still cannot do more to help.

Simon Tournier <zimon.toutoune@gmail.com> writes:

[...]

> Well, let me try to quickly summarize my conclusion of the session:
>
>  1. We have a social/organisational problem.
>
>  2. We have some tooling annoyances.
>
>
> The easy first: #2 about tools.  The email workflow is often cited as
> part of the issue.  That’s a false-problem, IMHO.

yes, we (as a community) already had several discussions around the
false-problem named "email worfkow is too hard", I also dared to send a
*very* lenghty analysis comparing the _so_called_ "pull request model" [1] 

Unfortunately I'm pretty sure that _this_ false issue will be cited
again and again and again when discussing about "how to better help Guix
maintainers"

...unless the (info "(guix) Submitting Patches") one day will finally
(briefly) explain why the project is using an email based workflow and
not a "so called PR workflow" (to understand why PR workflow is "so
called" please read [1]) 

But all this discussion on the "email workflow" issue is more useless
when considering the commit authetication mechanism _embedded_ in Guix
since 2020;  I recently studied this blog post:

https://guix.gnu.org/en/blog/2020/securing-updates/

and it states:

--8<---------------cut here---------------start------------->8---

To implement that, we came up with the following mechanism and rule:

1 The repository contains a .guix-authorizations file that lists the
 OpenPGP key fingerprints of authorized committers.

2 A commit is considered authentic if and only if it is signed by one of
 the keys listed in the .  guix-authorizations file of each of its
 parents. This is the authorization invariant.

[...]

The authorization invariant satisfies our needs for Guix. It has one
downside: it prevents pull-request-style workflows. Indeed, merging the
branch of a contributor not listed in .  guix-authorizations would break
the authorization invariant. It’s a good tradeoff for Guix because our
workflow relies on [patches carved into stone tablets] (patch tracker),
but it’s not suitable for every project out there.

--8<---------------cut here---------------end--------------->8---

[patches carved into stone tablets] is a link to:

https://lwn.net/Articles/702177/
«Why kernel development still uses email»
By Jonathan Corbet, October 1, 2016 

an article with another ton of reasons why "all patch management tools
sucks, email just sucks less.

Anyway, since Guix is using the "authorization invariant" since 2020,
the "email workflow" is embedded in Guix :-D

Am I missing something?

> Projects that use PR/MR workflow have the same problem.  For instance,
> Julia [1] has 896 open PR. 

[...]

> I will not speak about the channel ’nonguix’ but it gives another
> clue.

I will not speak about kubernetes, cited in the above cited LWN article,
I will not speak about Gerrit, also cited there...

[...]

> To be clear, the email workflow might add burden on submitter side but I
> am doubtful it is really part of the bottleneck for reviewing and
> pushing submissions.

Email workflow makes the reviewing workflow _extremely_ easy, provided a
good MUA and a _little_ bit of self-discipline following the /easy/
guidance in (info "(guix) Reviewing the Work of Others")

> Although the tools might add some unnecessary friction, the net of the
> issue is IMHO #1: reviewing is just boring and time-consuming.

This is the one and only reason.

[...]

I don't have anything to add, for now.


Happy hacking! Gio'


[1] id:87y1ha9jj6.fsf@xelera.eu aka
    https://yhetil.org/guix/87y1ha9jj6.fsf@xelera.eu/

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Guix Days: Patch flow discussion

[Matt]

 ---- On Wed, 28 Feb 2024 18:51:16 +0100  Giovanni Biscuolo  wrote --- 
 
 > ...and apologise if I still cannot do more to help.

We do what we can when we can.  And you have done things to help.  Thank you for sharing your thoughts and perspective!  If there's nothing you can do at the moment, that's okay.  I have no doubt you'll step up again when you see an opportunity.  Who knows, someone may even delegate... ;)

on the bug tracker (Re: Guix Days: Patch flow discussion)

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 3048 bytes --]

Hi Josselin,

Josselin Poiret <dev@jpoiret.xyz> writes:

[...]

> One thing I would like to get rid of though is debbugs.

given that a significant part of the Guix infrastructure is provided by
the GNU project, including the bug/issue tracker, and I don't think that
GNU will replace https://debbugs.gnu.org/ (or the forge, Savannah) with
something else, I suggest to concentrate the Guix community efforts in
giving contributors better user interfaces to Debbugs, e.g Mumi (web and
CLI) instead of trying to get rid of it.

In other words: the "problem" it's not the tool, it's the *interface*.

Please also consider that if (I hope not) the Guix would decide to adopt
a different bug/issue tracker system then Someome™ will have to
administrate it, and currently there are other pieces of core
infrastructure that need more resources, e.g. QA.

Speaking of interface features, I simply *love* the email based
interface provided by Debbugs [1]; also the web UI is not bad, and the Mumi
one (https://issues.guix.gnu.org/) it's even better.

But I'm curious: what bug tracker would you suggest instead of Debbugs?

Let's see what some "big players" are using...

> It causes a lot of pain for everyone, eg. when sending patchsets, it
> completely breaks modern email because it insists on rewriting
> DMARC-protected headers, thus needing to also rewrite "From:" to avoid
> DMARC errors.

I don't understand what "completely breaks modern email" means: please
could you point me to a document where this issue/bug is documented?

> I've been following the Linux kernel development a bit closer this past
> year, and while there are things that need to improve (like knowing the
> status of a patchset in a maintainer's tree), they at least have a lot
> of tools that I think we should adopt more broadly:

you mention: b4/lei and patchwork but they are not bug/issue trackers.

the Linux kernel community is using https://bugzilla.kernel.org/;
RedHat, Fedora, OpenSUSE and SUSE are also using Bugzilla

Arch Linux have adopted GitLab issues

Other alternavives:
https://en.wikipedia.org/wiki/Comparison_of_issue-tracking_systems

...or:
https://en.wikipedia.org/wiki/Bug_tracking_system#Distributed_bug_tracking

I personally like the idea that the bug/issue tracker is _embedded_
(integrated?) in the DVCS used by the project, Git in Guix case.

For this reason I find Tissue https://tissue.systemreboot.net/ an
interesting project for *public* issue/bug tracking systems, also
because Tissue is _not_ discussion-oriented: this means that
discussions are managed "somewhere else", because «It's much better to
have a clear succinct actionable issue report. This way, the issue
tracker is a list of clear actionable items rather than a mess of
unreproducible issues.»  [2]

Happy hacking! Gio'

[...]

[1] https://debbugs.gnu.org/server-control.html

[2] https://tissue.systemreboot.net/manual/dev/en/#section11795

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 849 bytes --]

Re: Guix Days: Patch flow discussion

[Daniel Littlewood]

Hi! I'm a complete newbie to Guix, which has the dual effects that I'm much
more excited about it than I otherwise might be, but the disadvantage that I
know nothing and am more likely to cause problems than not at the moment.
Nevertheless I thought I might be able to contribute something to the
discussion.

Something that is not obvious to me when people refer to reviewing patches, is
whether this is purely a matter of adding new packages to the main guix
channel, or of reviewing changes to the system in general, or both. As a
novice, I can imagine becoming comfortable as a package reviewer much more
quickly than as a reviewer of core patches to the system.

It's also not obvious to me whether you mean exactly "reviewing a backlog of
existing patches" or additionally "increasing the amount of patches submitted
and applied". I feel like both are probably good things but I can't tell what
you're focussing on exactly. If lots of gems were imported from other repos
like RubyGems and PyPi, which as I understand it is currently a
partly-automatic partly-manual process, would that be considered a win? What
about increasing version coverage among those packages that are covered?

One point brought up here is about tooling. I wonder whether there is any scope
for fully automatic review. In other words, for some classes of package we
might be able to say that if it builds in CI, and maybe the built package has a
command that's considered enough to say "it's working", then that's all the
review that's needed. I guess I'm mostly thinking here of cases where the
package is imported, partially or entirely automatically, from another repo (as
with `guix import`). So one could argue that as long as it looks like "the
rubygems import is working" then the guix package thus imported can be assumed
to be working too. I guess when I say this I am still mostly thinking of
importing entire histories of a package (lots of versions simultaneously)
without multiplying out the work involved to a huge degree.

One of the points brought up earlier is whether it would be better to have
interactive "review parties" or comprehensive documentation about what
constitutes a good review. Obviously these are not mutually exclusive. But for
my part, I think documentation gives you more "bang for your buck", for a few
reasons:

  1. Documentation is online all the time, whereas interactive sessions are
subject to people's work schedules, time zones, personal commitments, etc.

  2. Documentation is more likely to be complete, and to become more likely to
be complete over time, whereas in meetings (or when doing your first solo
review following the meeting) it is easy to forget some steps or criteria to
check.

  3. Reading docs can be done by anyone, with no commitment to follow up. I
think some people are just scared off socially by the idea of having to join a
meeting in order to learn how to do reviews well. Obviously this is a personal
thing and varies - some people find a welcoming atmosphere in a virtual meeting
much better than a faceless documentation screen. For my part I think I would
prefer to read until I know the basics, and then might prefer to do it
alongside other people. A "patch party" would not convert me from
non-contributor to contributor, but it might convert me from a
seldom-contributor to an active one. Again, there is obviously variation here
among people.

In case it is not obvious, I have only recently joined guix-devel, but would be
keen to help out if I can. If there are things I can read to get "up to speed"
that would be really helpful!

All the best,
Dan

Re: Guix Days: Patch flow discussion

[Andreas Enge]

Hello Dan,

thanks for your thoughts! I think I will restrict my replies to guix-devel
to keep them in one place; the following are just my personal opinions.

Am Thu, Feb 29, 2024 at 03:41:41PM +0000 schrieb Daniel Littlewood:
> Something that is not obvious to me when people refer to reviewing patches, is
> whether this is purely a matter of adding new packages to the main guix
> channel, or of reviewing changes to the system in general, or both. As a
> novice, I can imagine becoming comfortable as a package reviewer much more
> quickly than as a reviewer of core patches to the system.

Both! And indeed what you write is correct, reviewing packages is easier
than services, which is probably easier than other changes. (Personally,
I feel confident only with packages.) Of course then people should only
review things they are comfortable with.

> It's also not obvious to me whether you mean exactly "reviewing a backlog of
> existing patches" or additionally "increasing the amount of patches submitted
> and applied". I feel like both are probably good things but I can't tell what
> you're focussing on exactly. If lots of gems were imported from other repos
> like RubyGems and PyPi, which as I understand it is currently a
> partly-automatic partly-manual process, would that be considered a win? What
> about increasing version coverage among those packages that are covered?

The discussion was about the backlog; in particular also about negative
feelings by contributors of patches that take a long time to be applied.
Of course adding more packages is also a welcome activitiy (but only
makes sense if enough of them are applied in the end...). We concentrated
on "reviewing" to ease the burden of "committers", since reviewing is open
to anybody.

> One point brought up here is about tooling. I wonder whether there is any scope
> for fully automatic review.

I do not think so. Quality is an important aspect of Guix; for instance,
we ask for non-marketing descriptions, which would be difficult to test
automatically. We already have "guix lint", which does some of the work.
And there are fully automated channels such as for CRAN, but which then
are potentially of a lesser quality.

Notice that "easy" packages are also easy to review; most of the time,
there is not much to do about the result of "guix import pypi ...".
Things become more tricky when phases need to be added, to understand
what is going on, and then I usually also look at comments (or criticise
their absence).

> I think some people are just scared off socially by the idea of having to join a
> meeting in order to learn how to do reviews well.

Agreed, there should not be any "having to join a meeting". The idea of
organising one comes from the goal of making the activity more social and
less boring. Apart from that, you can start today and need not wait for
a bug squashing party :)

Andreas

Re: Guix Days: Patch flow discussion

[Attila Lendvai]

> Somehow, the reader will judge if Message-ID is smoothly supported. :-)


i regularly meet this most unfortunate attitude in the GNU circles, where oldtimers dismiss any discussion of friendlier defaults for newcomers with the "argument" that it's configurable, and therefore it's a non-issue.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The wise prince must provide in such a way that the citizens will always be in need of him and of his government.”
	— Niccolo Machiavelli (1469–1527), 'The Prince' (1513)

Re: on the bug tracker (Re: Guix Days: Patch flow discussion)

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 4603 bytes --]


-- 

Giovanni Biscuolo <g@xelera.eu> writes:

> Hi Josselin,
>
> Josselin Poiret <dev@jpoiret.xyz> writes:
>
> [...]
>
>> One thing I would like to get rid of though is debbugs.
>
> given that a significant part of the Guix infrastructure is provided by
> the GNU project, including the bug/issue tracker, and I don't think that
> GNU will replace https://debbugs.gnu.org/ (or the forge, Savannah) with
> something else, I suggest to concentrate the Guix community efforts in
> giving contributors better user interfaces to Debbugs, e.g Mumi (web and
> CLI) instead of trying to get rid of it.

We can re-use some other tool as well, no need to code our own in Guile
though.  Also note that the choice of bug-tracking tool is not benign,
as most other tools (Mumi, but also QA) have to adapt to its
idiosyncrasies.

> In other words: the "problem" it's not the tool, it's the *interface*.

I don't agree: see below for the two examples of where the tool design
itself is problematic.

> Please also consider that if (I hope not) the Guix would decide to adopt
> a different bug/issue tracker system then Someome™ will have to
> administrate it, and currently there are other pieces of core
> infrastructure that need more resources, e.g. QA.
>
> Speaking of interface features, I simply *love* the email based
> interface provided by Debbugs [1]; also the web UI is not bad, and the Mumi
> one (https://issues.guix.gnu.org/) it's even better.

I prefer a public-inbox web frontend for browsing emails personally.

>> It causes a lot of pain for everyone, eg. when sending patchsets, it
>> completely breaks modern email because it insists on rewriting
>> DMARC-protected headers, thus needing to also rewrite "From:" to avoid
>> DMARC errors.
>
> I don't understand what "completely breaks modern email" means: please
> could you point me to a document where this issue/bug is documented?

I don't think there's a "public" discussion of this, but I've conversed
with the mailman maintainers in private about this: because Debbugs
relies on modifying the mail message itself, it clashes with modern
email features that authentify received emails.

DKIM authentifies a subset of mail headers by signing them with a
private key, among which the "From" header is mandatory.  DMARC is a
policy mechanism for domains that can enforce DKIM (and/or SPF) for all
mails with a corresponding "From:".  Together, they prevent email
impersonation.

So Debbugs modifies a protected header (e.g. Subject) -> DKIM fails ->
if DMARC is set to enforce DKIM, DMARC fails -> mail is not received.

To avoid this issue, Mailman (the mailing daemon overseeing the
debbugs-managed MLs) also rewrites the From: header of such mails to a
debbugs-controlled domain, so that DKIM headers can be re-signed and a
different DMARC policy be used.  This is why you often see "via
guix-patches" in From: headers, and if you look at the email it's a
generic one.


I've also mentioned this before, but not being able to simply send a
full patchset without having to go through hoops is, put simply, a
problem.  Even prospective contributors used to email-based workflows
will be turned off by this.


In general, the problem with Debbugs is that it insists on *taking over*
emails, rather than just consuming them.

>> I've been following the Linux kernel development a bit closer this past
>> year, and while there are things that need to improve (like knowing the
>> status of a patchset in a maintainer's tree), they at least have a lot
>> of tools that I think we should adopt more broadly:
>
> you mention: b4/lei and patchwork but they are not bug/issue trackers.

Yes, I was mentioning general tools.

> I personally like the idea that the bug/issue tracker is _embedded_
> (integrated?) in the DVCS used by the project, Git in Guix case.
>
> For this reason I find Tissue https://tissue.systemreboot.net/ an
> interesting project for *public* issue/bug tracking systems, also
> because Tissue is _not_ discussion-oriented: this means that
> discussions are managed "somewhere else", because «It's much better to
> have a clear succinct actionable issue report. This way, the issue
> tracker is a list of clear actionable items rather than a mess of
> unreproducible issues.»  [2]

I like this separation as well, and Tissue seemed interesting the last
time I looked at it.  I liked how it uses standard tools and can be
consulted/modified off-line.  It basically checks all of the boxes :)

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]