gnu: inetutils: Update to 2.4.

gnu: inetutils: Update to 2.4.

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi,

With core-updates possibly being replaced by feature branches, I am
not sure where to place this update to GNU Inetutils 2.4.

The patch series [1] will rebuild 3,416 packages.

Thank you, everyone, for your hard work!

Kind regards,
Felix Lechner

[1] https://issues.guix.gnu.org/62154

Re: gnu: inetutils: Update to 2.4.

[Maxim Cournoyer]

Hi Felix,

Felix Lechner via "Development of GNU Guix and the GNU System
distribution." <guix-devel@gnu.org> writes:

> Hi,
>
> With core-updates possibly being replaced by feature branches, I am
> not sure where to place this update to GNU Inetutils 2.4.
>
> The patch series [1] will rebuild 3,416 packages.

In the future, try to remember using the --subject-prefix='PATCH
core-updates', to avoid committers mistakenly pushing this to master, as
hinted in the manual: info '(guix) Sending a Patch Series' :-).

Thanks for the heads-up!

-- 
Maxim

Re: gnu: inetutils: Update to 2.4.

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Maxim,

On Tue, Mar 14, 2023 at 8:53 AM Maxim Cournoyer
<maxim.cournoyer@gmail.com> wrote:
>
> In the future, try to remember using the --subject-prefix='PATCH
> core-updates', to avoid committers mistakenly pushing this to master

Thank you for your response! I wrote for guidance on that issue. I am
sorry that was unclear from my message.

With the core-updates process now abandoned, I retitled the issue to

    "[PATCH wip-inetutils 0/4] gnu: inetutils: Update to 2.4".

Does that work for you?

I am happy to retitle as "core-updates" as you suggested, if that's
what you want. Of course, you can also retitle it in any way you think
is best.

Kind regards,
Felix Lechner

Re: gnu: inetutils: Update to 2.4.

[Andreas Enge]

Hello Felix,

Am Tue, Mar 14, 2023 at 09:37:23AM -0700 schrieb Felix Lechner via Development of GNU Guix and the GNU System distribution.:
> With the core-updates process now abandoned, I retitled the issue to
>     "[PATCH wip-inetutils 0/4] gnu: inetutils: Update to 2.4".

apologies for not replying earlier - this was because I did not have the
answer... I think this is a place where the feature branch idea is not yet
fully thought out. It is quite clear how branches closer to the leaves
could work (language branches, a qt/kde branch, such things); and probably
also "core things". There I see two kinds of cores: First, things that are
very close to Guile/Guix (switching to gexps, rewriting the daemon in Guile,
these kind of things). And maybe something more like "bootstrap" and "tool-
chain" (changing the Guile version, updating gcc/mpfr/mpc, probably rust,
and so on).

I do not yet have a clear picture for things on a "middle level", but which
cause a lot of rebuilds. Which kind of team would be responsible for
inetutils?

Suggestions would be very welcome!

Something completely different: I am not sure whether inetutils should have
so many dependencies. It only contains binaries and no libraries. For
instance, it is sometimes used as native input for tests, but tests do not
have access to the inet anyway. So maybe it would make sense to identify
the "root" where this package is pulled in, drop the input and modify
the test suite?

Andreas

Re: gnu: inetutils: Update to 2.4.

[Maxim Cournoyer]

Hi Felix,

Felix Lechner <felix.lechner@lease-up.com> writes:

> Hi Maxim,
>
> On Tue, Mar 14, 2023 at 8:53 AM Maxim Cournoyer
> <maxim.cournoyer@gmail.com> wrote:
>>
>> In the future, try to remember using the --subject-prefix='PATCH
>> core-updates', to avoid committers mistakenly pushing this to master
>
> Thank you for your response! I wrote for guidance on that issue. I am
> sorry that was unclear from my message.
>
> With the core-updates process now abandoned, I retitled the issue to

Could you share the reference of that?  I'm not against it, but our
currently documented process still mention the good old staging and
core-updates branches.

>     "[PATCH wip-inetutils 0/4] gnu: inetutils: Update to 2.4".
>
> Does that work for you?
>
> I am happy to retitle as "core-updates" as you suggested, if that's
> what you want. Of course, you can also retitle it in any way you think
> is best.

Until everybody has a good grasp of the new process, I think sticking to
the documented one may be best for now, as it makes it clear that this
cause a mass rebuild and shouldn't land to master.

-- 
Thanks,
Maxim

Re: gnu: inetutils: Update to 2.4.

[Leo Famulari]

On Tue, Mar 14, 2023 at 09:10:33PM -0400, Maxim Cournoyer wrote:
> Felix Lechner <felix.lechner@lease-up.com> writes:
> > With the core-updates process now abandoned, I retitled the issue to
> 
> Could you share the reference of that?  I'm not against it, but our
> currently documented process still mention the good old staging and
> core-updates branches.

At the Guix Days in February, we discussed the branching workflow and
reached a rough consensus that for non-core packages (defined in
%core-packages), we should try to adopt a more targeted "feature branch"
workflow. That's actually what we used to do, before we outgrew our old
build farm, after which we were barely able to build one branch at a
time (IIRC, we would stop building master in order to build core-updates
or staging).

The discussion was summarized by Andreas here:

https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00066.html

Currently we are demo-ing this workflow in the wip-go-updates branch and
go-team Cuirass jobset.

My hope is that we can rewrite the relevant documentation in the coming
months, as we learn from these early efforts.

I don't think the core-updates process is abandoned, but we should
reduce its scope. For the core of Guix, both the packages and Guix
itself, it makes sense to alter things in tandem.

But as we have >22000 packages, there are many packages that would
currently qualify for core-updates but aren't core, and can be
relatively easily tested in smaller themed batches.

I would suggest abandoning the staging branch approach after its current
patches are merged to master. The staging branch was always a kludge
from when our build farm was struggling.

For something like inetutils, I would suggest borrowing from its package
module name (gnu packages admin), and attempt to update the
administrative packages as a group. Those who are interested in system
administration should lead the effort.

Branch and release process (was: gnu: inetutils: Update to 2.4.)

[Maxim Cournoyer]

Hi,

Leo Famulari <leo@famulari.name> writes:

> On Tue, Mar 14, 2023 at 09:10:33PM -0400, Maxim Cournoyer wrote:
>> Felix Lechner <felix.lechner@lease-up.com> writes:
>> > With the core-updates process now abandoned, I retitled the issue to
>> 
>> Could you share the reference of that?  I'm not against it, but our
>> currently documented process still mention the good old staging and
>> core-updates branches.
>
> At the Guix Days in February, we discussed the branching workflow and
> reached a rough consensus that for non-core packages (defined in
> %core-packages), we should try to adopt a more targeted "feature branch"
> workflow. That's actually what we used to do, before we outgrew our old
> build farm, after which we were barely able to build one branch at a
> time (IIRC, we would stop building master in order to build core-updates
> or staging).
>
> The discussion was summarized by Andreas here:
>
> https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00066.html

Thanks!  I had missed it.  It sounds promising!

> Currently we are demo-ing this workflow in the wip-go-updates branch and
> go-team Cuirass jobset.

So the review happens first on the ML, then the changes land to the team
branch, and then finally the feature branch gets merged to master?  If
the review has already happened and the package been tested (and built
by QA), why is a feature branch needed?

> My hope is that we can rewrite the relevant documentation in the coming
> months, as we learn from these early efforts.

OK!  Thanks for allowing me to catch up!

-- 
Thanks,
Maxim

Re: Branch and release process (was: gnu: inetutils: Update to 2.4.)

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2050 bytes --]

On Tue, Mar 14, 2023 at 11:30:52PM -0400, Maxim Cournoyer wrote:
> Hi,
> 
> Leo Famulari <leo@famulari.name> writes:
> 
> > On Tue, Mar 14, 2023 at 09:10:33PM -0400, Maxim Cournoyer wrote:
> >> Felix Lechner <felix.lechner@lease-up.com> writes:
> >> > With the core-updates process now abandoned, I retitled the issue to
> >> 
> >> Could you share the reference of that?  I'm not against it, but our
> >> currently documented process still mention the good old staging and
> >> core-updates branches.
> >
> > At the Guix Days in February, we discussed the branching workflow and
> > reached a rough consensus that for non-core packages (defined in
> > %core-packages), we should try to adopt a more targeted "feature branch"
> > workflow. That's actually what we used to do, before we outgrew our old
> > build farm, after which we were barely able to build one branch at a
> > time (IIRC, we would stop building master in order to build core-updates
> > or staging).
> >
> > The discussion was summarized by Andreas here:
> >
> > https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00066.html
> 
> Thanks!  I had missed it.  It sounds promising!
> 
> > Currently we are demo-ing this workflow in the wip-go-updates branch and
> > go-team Cuirass jobset.
> 
> So the review happens first on the ML, then the changes land to the team
> branch, and then finally the feature branch gets merged to master?  If
> the review has already happened and the package been tested (and built
> by QA), why is a feature branch needed?

So we can group a couple of larger related changes together.

> > My hope is that we can rewrite the relevant documentation in the coming
> > months, as we learn from these early efforts.
> 
> OK!  Thanks for allowing me to catch up!
> 
> -- 
> Thanks,
> Maxim
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Branch and release process

[Maxim Cournoyer]

Hi Efraim,

Efraim Flashner <efraim@flashner.co.il> writes:

> On Tue, Mar 14, 2023 at 11:30:52PM -0400, Maxim Cournoyer wrote:
>> Hi,
>> 
>> Leo Famulari <leo@famulari.name> writes:
>> 
>> > On Tue, Mar 14, 2023 at 09:10:33PM -0400, Maxim Cournoyer wrote:
>> >> Felix Lechner <felix.lechner@lease-up.com> writes:
>> >> > With the core-updates process now abandoned, I retitled the issue to
>> >> 
>> >> Could you share the reference of that?  I'm not against it, but our
>> >> currently documented process still mention the good old staging and
>> >> core-updates branches.
>> >
>> > At the Guix Days in February, we discussed the branching workflow and
>> > reached a rough consensus that for non-core packages (defined in
>> > %core-packages), we should try to adopt a more targeted "feature branch"
>> > workflow. That's actually what we used to do, before we outgrew our old
>> > build farm, after which we were barely able to build one branch at a
>> > time (IIRC, we would stop building master in order to build core-updates
>> > or staging).
>> >
>> > The discussion was summarized by Andreas here:
>> >
>> > https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00066.html
>> 
>> Thanks!  I had missed it.  It sounds promising!
>> 
>> > Currently we are demo-ing this workflow in the wip-go-updates branch and
>> > go-team Cuirass jobset.
>> 
>> So the review happens first on the ML, then the changes land to the team
>> branch, and then finally the feature branch gets merged to master?  If
>> the review has already happened and the package been tested (and built
>> by QA), why is a feature branch needed?
>
> So we can group a couple of larger related changes together.

I see; so it'd be useful for the integration of package changes
impacting multiple others; some kind of staging or core-updates
topic-focused branches.  Simple leaf package updates could still be
merged directly to master without going through the go-team branch,
right?

-- 
Thanks,
Maxim

Re: Branch and release process

[Andreas Enge]

Am Wed, Mar 15, 2023 at 09:32:44AM -0400 schrieb Maxim Cournoyer:
> I see; so it'd be useful for the integration of package changes
> impacting multiple others; some kind of staging or core-updates
> topic-focused branches.  Simple leaf package updates could still be
> merged directly to master without going through the go-team branch,
> right?

Exactly!

Andreas

Re: Branch and release process (was: gnu: inetutils: Update to 2.4.)

[Leo Famulari]

On Tue, Mar 14, 2023, at 23:30, Maxim Cournoyer wrote:
> Hi,
>
> Leo Famulari <leo@famulari.name> writes:
>
>> On Tue, Mar 14, 2023 at 09:10:33PM -0400, Maxim Cournoyer wrote:
>>> Felix Lechner <felix.lechner@lease-up.com> writes:
>>> > With the core-updates process now abandoned, I retitled the issue to
>>> 
>>> Could you share the reference of that?  I'm not against it, but our
>>> currently documented process still mention the good old staging and
>>> core-updates branches.
>>
>> At the Guix Days in February, we discussed the branching workflow and
>> reached a rough consensus that for non-core packages (defined in
>> %core-packages), we should try to adopt a more targeted "feature branch"
>> workflow. That's actually what we used to do, before we outgrew our old
>> build farm, after which we were barely able to build one branch at a
>> time (IIRC, we would stop building master in order to build core-updates
>> or staging).
>>
>> The discussion was summarized by Andreas here:
>>
>> https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00066.html
>
> Thanks!  I had missed it.  It sounds promising!
>
>> Currently we are demo-ing this workflow in the wip-go-updates branch and
>> go-team Cuirass jobset.
>
> So the review happens first on the ML, then the changes land to the team
> branch, and then finally the feature branch gets merged to master?  If
> the review has already happened and the package been tested (and built
> by QA), why is a feature branch needed?

Because QA currently cannot process changes that cause more than 200 rebuilds.

Re: Branch and release process (was: gnu: inetutils: Update to 2.4.)

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2020 bytes --]


"Leo Famulari" <leo@famulari.name> writes:

> On Tue, Mar 14, 2023, at 23:30, Maxim Cournoyer wrote:
>> Hi,
>>
>> Leo Famulari <leo@famulari.name> writes:
>>
>>> On Tue, Mar 14, 2023 at 09:10:33PM -0400, Maxim Cournoyer wrote:
>>>> Felix Lechner <felix.lechner@lease-up.com> writes:
>>>> > With the core-updates process now abandoned, I retitled the issue to
>>>> 
>>>> Could you share the reference of that?  I'm not against it, but our
>>>> currently documented process still mention the good old staging and
>>>> core-updates branches.
>>>
>>> At the Guix Days in February, we discussed the branching workflow and
>>> reached a rough consensus that for non-core packages (defined in
>>> %core-packages), we should try to adopt a more targeted "feature branch"
>>> workflow. That's actually what we used to do, before we outgrew our old
>>> build farm, after which we were barely able to build one branch at a
>>> time (IIRC, we would stop building master in order to build core-updates
>>> or staging).
>>>
>>> The discussion was summarized by Andreas here:
>>>
>>> https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00066.html
>>
>> Thanks!  I had missed it.  It sounds promising!
>>
>>> Currently we are demo-ing this workflow in the wip-go-updates branch and
>>> go-team Cuirass jobset.
>>
>> So the review happens first on the ML, then the changes land to the team
>> branch, and then finally the feature branch gets merged to master?  If
>> the review has already happened and the package been tested (and built
>> by QA), why is a feature branch needed?
>
> Because QA currently cannot process changes that cause more than 200 rebuilds.

Note that this has been changing recently, and it's currently 600 builds
per system [1].

1: https://git.savannah.gnu.org/cgit/guix/qa-frontpage.git/tree/guix-qa-frontpage/manage-builds.scm#n100

We can still increase it, but there's still more work needed on managing
the builds and increasing the hardware available.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: gnu: inetutils: Update to 2.4.

[Andreas Enge]

Am Tue, Mar 14, 2023 at 09:10:33PM -0400 schrieb Maxim Cournoyer:
> Could you share the reference of that?  I'm not against it, but our
> currently documented process still mention the good old staging and
> core-updates branches.

It has not been documented yet, we should do it.
Here is the relevant excerpt from my notes, sent to guix-devel on
February 9 under the title "Discussion notes on releases and branches":
- Create branches with a few patches or patchsets; in any case with
  a "semantic" description of the changes. The branches could be
  shortlived. Feature branches are one incarnation of the concept.
- The numerical criteria for staging and core-updates is outdated:
  Even non-core packages may create an enormous number of rebuilds.
- Negative point: There is a risk of churn, by not regrouping world-
  rebuilding changes - but two non related world rebuilding changes
  might be difficult to review.
...
- There is discussion whether we need a core-updates branch.
  Core updates concern the toolchain, build phase changes, everything
  that has big ramifications all over the system. It would be important
  to not have several "parallel" branches with related (for instance,
  but not exclusively, core-update) changes, which in fact should come
  one after the other. Either they could be collected in one branch,
  or would require coordination of branch creation (inside a team, say).
- "Merge trains" of gitlab are mentioned, as a way of merging several
  branches at the same time.

There was a consensus on advancing in this direction, but details need to
be fleshed out.

The core argument I see against core-updates (and staging) is that they
regroup a mixed bag of unrelated changes, that noone is able to audit
after a while. By matching a focused set of changes with a dedicated team
of people competent in the area, we hope to advance faster and in a more
concentrated fashion. This comes in a context where our compile power in
the berlin build farm is much larger than in the past, and where on the
other hand a growing number of packages lead to non-core packages causing
a number of rebuilds that used to go to core-updates (like we just see
with inetutils).

> Until everybody has a good grasp of the new process, I think sticking to
> the documented one may be best for now, as it makes it clear that this
> cause a mass rebuild and shouldn't land to master.

Definitely in all procedures, old or new, mass rebuilds should not be done
in master!

Andreas

Re: gnu: inetutils: Update to 2.4.

[Maxim Cournoyer]

Hi Andreas,

Andreas Enge <andreas@enge.fr> writes:

> Am Tue, Mar 14, 2023 at 09:10:33PM -0400 schrieb Maxim Cournoyer:
>> Could you share the reference of that?  I'm not against it, but our
>> currently documented process still mention the good old staging and
>> core-updates branches.
>
> It has not been documented yet, we should do it.
> Here is the relevant excerpt from my notes, sent to guix-devel on
> February 9 under the title "Discussion notes on releases and
> branches":

I've now read the full notes, which were well written.  Thank you!
Hopefully I've now caught up with the latest branch process changes
proposals :-).

-- 
Thanks,
Maxim

Re: gnu: inetutils: Update to 2.4.

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Maxim,

On Tue, Mar 14, 2023 at 6:11 PM Maxim Cournoyer
<maxim.cournoyer@gmail.com> wrote:
>
> Until everybody has a good grasp of the new process, I think sticking to
> the documented one may be best for now

The bug was retitled 'core-updates' per your request. [1]

I didn't mean to set a tone in the discussion and merely hoped to
embrace—by mistake it turns out—an existing consensus. My apologies!

May this action please invite further dialog, as needed.

Kind regards
Felix Lechner

[1] https://issues.guix.gnu.org/62154

Re: gnu: inetutils: Update to 2.4.

[Maxim Cournoyer]

Hi Felix,

Felix Lechner <felix.lechner@lease-up.com> writes:

> Hi Maxim,
>
> On Tue, Mar 14, 2023 at 6:11 PM Maxim Cournoyer
> <maxim.cournoyer@gmail.com> wrote:
>>
>> Until everybody has a good grasp of the new process, I think sticking to
>> the documented one may be best for now
>
> The bug was retitled 'core-updates' per your request. [1]
>
> I didn't mean to set a tone in the discussion and merely hoped to
> embrace—by mistake it turns out—an existing consensus. My apologies!

No problem, I was a bit out of touch with the recent achievements from
the Guix Days workshop.  But yes, I think until these experiments are
settled in our reference manual, to ease our day to day collaboration we
can continue labeling our changes with the 'staging' or 'core-updates'
prefixes, as it help guards against mistakes :-).

-- 
Thanks,
Maxim