Discussion notes on releases and branches

Discussion notes on releases and branches

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 126 bytes --]

Attached are the notes of our discussion at the Guix Days concerning
releases, branches, teams and related matters.

Andreas


[-- Attachment #2: Release.txt --]
[-- Type: text/plain, Size: 5024 bytes --]

Releases, branches, teams

Problem: It took us 1.5 years for the last release, and the coreupdates
branch has been open for at least as much.


RELEASES

Why do we make releases?
- How else would people install Guix?
- Quality assurance: Fix the installer before release.
- Regular points for translation.
- Communication and publicity.

- Releases are separate from branches; the release team should pick
  a commit and spin off a release branch from there.
- It is possible to have long term support release, mixed with
  more frequent intermediate releases; example from another project:
  "big" release every year, "smaller" releases every 4 months.
- Stable branches are a frequent user request!
- A release schedule for a release every 4 to 6 months sounds
  reasonable.
- Mathieu, Simon, Julien and Andreas volunteer to be members of a release
  team. The idea would be to change the team for each release, with an
  overlap of about half the people: everyone would have "two terms".
  Such a team could commit on a release date and coordinate contributors,
  keep track of the status, ping people fixing bugs etc. Work should
  start immediately after a release to spot problematic areas (poorly
  supported architectures...) and big goals for the next release.
  It could also coordinate with other teams on planned big changes.
  Communication is key! For instance send a status update once per week.
  At the same time, the team should lower their operational involvement
  to avoid burn-out.


BRANCHES

Suggestion:
- Spin off a stable branch from master with security fixes, maybe important
  backports; after 6 months, branch a new stable branch from master; this
  is almost like a release, but continued into some future.

Counter-suggestion:
- Create branches with a few patches or patchsets; in any case with
  a "semantic" description of the changes. The branches could be
  shortlived. Feature branches are one incarnation of the concept.
- The numerical criteria for staging and core-updates is outdated:
  Even non-core packages may create an enormous number of rebuilds.
- Negative point: There is a risk of churn, by not regrouping world-
  rebuilding changes - but two non related world rebuilding changes
  might be difficult to review.

Before creating new branches, we need to clarify how the old branches
are handled!

- Smaller branches could be taken care of by dedicated persons
  responsible for pushing them forward. For instance by teams.
- Some people already do this for a feature branch on their local
  machine for medium-sized updates (ocaml), or even on ci (haskell,
  kernel updates).
- Branch creators should fix a goal and tentative timeline.
- We need a mapping between branches and maintainers responsible
  for the merge. This could be a team leader, if such a role is created.
  A wiki could be used to keep track of the branches.
- There is discussion whether we need a core-updates branch.
  Core updates concern the toolchain, build phase changes, everything
  that has big ramifications all over the system. It would be important
  to not have several "parallel" branches with related (for instance,
  but not exclusively, core-update) changes, which in fact should come
  one after the other. Either they could be collected in one branch,
  or would require coordination of branch creation (inside a team, say).
- "Merge trains" of gitlab are mentioned, as a way of merging several
  branches at the same time.
- Grafts can also be handled in feature branches: The graft is applied
  to master; the graft is applied to a different branch, directly followed
  by an ungrafting commit and an update of the corresponding package;
  once the branch is built it is merged back to master.
- Minor drawback: If qa has treated a world-rebuilding patch, the
  substitutes will be available on bordeaux, but not on berlin; people
  who have installed a long time ago and not authorised bordeaux might
  be affected. If there are complaints, they can be handled on the
  mailing list.
- Moving fast poses problems for non-x86 architectures, but the build farm
  situation has improved for aarch64 and armhf sufficiently to keep up
  with master. Handling feature branches remains an unsolved problem.
- Currently there is a cap on qa, only patches with at most 300 dependents
  are treated. This cap could be increased. Or it could be weighed with
  the build times of the packages.


TEAMS

- Issues could be tagged more often with responsible teams, or with
  severity information (blocking bug or not).
- Each module should be covered by a team; otherwise it would be
  difficult to get important updates through a feature branches.


ARCHITECTURES

- We distinguish in the manual between available architectures that are
  "supported" or "unsupported". It would make sense to have a category
  in-between, with architectures that are supported in principle, but may
  be low on substitute availability.
- We could use the "supported-systems" fields in packages more liberally.

Moving forward with teams and feature branches (was: Discussion notes on releases and branches)

[Josselin Poiret]

[-- Attachment #1: Type: text/plain, Size: 4245 bytes --]

Hi Andreas and thanks for taking the notes during the discussion!

I think what came out of that discussion was very interesting and that
it would greatly help the scaling problems Guix is facing as well as
maintainer/contributor burnout, but that also means that we need to
create some actionnable plan out of it.  The window before the eventual
core-updates merge would be the best time for it, IMO!

For that plan to materialize, the best would be to agree on 1) what we
want the new workflow to be, and 2) how to get there. To start the
discussion, here's what I got out of the discussion (and is probably
opinionated):


The envisioned goal state would be to stop relying on
staging/core-updates to manage non-trivial changes to Guix, but rather
have specific branches that could be merged way faster, because they are
focused on specific features that people can feel responsible for,
instead of a hodgepodge of unrelated patches that no one wants to
debug. These feature branches would be managed by teams, with one person
from the team being responsible for the feature and it getting merged
into master. Those features could be ephemeral or long-running and each
team would be free to organize around them as they see fit, since e.g. a
Stack upgrade is very different from a Mesa upgrade, or a
gnu-build-system change.

With the improved CI/QA we have now, we shouldn't have to worry too much
since substitutes will be available right away (we should weigh what's
acceptable for people using --no-substitutes vs. keeping software more
up-to-date though, but my opinion is that in favor of the latter).

Teams should thus better document how they work and what they do, either
in the Guix documentation or a specific wiki/manual that would be
maintained by team members themselves. Having all information in one
place would help outsiders find their way around the inner workings of
the project. Maintaining roadmaps for each team, with who's working on
them and where that work can be found would also lure unsuspecting
bystanders into working on Guix features, as well as structure the
future of Guix; it shouldn't be too codified though, since we don't want
to further burden precious team members. One nice thing about
documenting all of this using a manual is that the changes go through
the guix-patches ML, so that they can be discussed and recorded for the
public to see.

Note that this change would also mean that contributors are no longer
responsible for gauging whether a patch belongs on master/staging/c-u,
but rather the reviewer/committer.  Also, as a nice side-effect of the
change, I can see people getting interested in joining teams because
they structure longer-term effort that's put into Guix, not just because
they've been maintaining python-foo-for-bar for 5 years against their
will.

Regarding releases: a release team would have to keep in touch with what
the other teams are doing, make some sort of periodic status report, and
set deadlines such as feature freezes before preparing a new release.


For a potential roadmap (doesn't have to be sequential):
1. Document this workflow in the manual, in a dedicated node, with a
   rationale as well.  One thing worth mentioning would be how to handle
   grafting/ungrafting now.  Also remove the staging/core-updates
   criterion.
2. Teams start organizing around the features they are currently working
   on, and document them accordingly. They can also draft how they work,
   what they do and their roadmap if they wish.
3. CI/QA gets examined and updated to support the new workflow. We test
   it out with a sample feature.
4. staging merge happens, and the branch gets deleted.
5. core-updates merge happens, and the branch gets deleted or
   repurposed (up to the core team).


I bet I forgot a bunch of things, but at least we get the ball rolling!

WDYT?

NB: Just noticed there is no tooling/infrastructure team, this would
probably be a good idea, to publicize the incredible work that is being
put into all of it (mumi/cuirass/qa front page/the data service), as
well as bring in some new souls and document how they all fit together.

Best,
-- 
Josselin Poiret

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 682 bytes --]

Re: Moving forward with teams and feature branches (was: Discussion notes on releases and branches)

[Andreas Enge]

Hello Josselin,

Am Sun, Feb 12, 2023 at 10:13:35PM +0100 schrieb Josselin Poiret:
> For a potential roadmap (doesn't have to be sequential):
> 1. Document this workflow in the manual, in a dedicated node, with a
>    rationale as well.  One thing worth mentioning would be how to handle
>    grafting/ungrafting now.  Also remove the staging/core-updates
>    criterion.
> 2. Teams start organizing around the features they are currently working
>    on, and document them accordingly. They can also draft how they work,
>    what they do and their roadmap if they wish.
> 3. CI/QA gets examined and updated to support the new workflow. We test
>    it out with a sample feature.
> 4. staging merge happens, and the branch gets deleted.
> 5. core-updates merge happens, and the branch gets deleted or
>    repurposed (up to the core team).

This (and the other points you wrote) sound good to me.
I would start with 4. and 5., and in parallel consider 1. to 3.

Andreas

Time for RFC? (was Re: Moving forward with teams and feature branches (was: Discussion notes on releases and branches))

[zimoun]

Hi,

On Sun, 12 Feb 2023 at 22:13, Josselin Poiret <dev@jpoiret.xyz> wrote:

> 1. Document this workflow in the manual, in a dedicated node, with a
>    rationale as well.  One thing worth mentioning would be how to handle
>    grafting/ungrafting now.  Also remove the staging/core-updates
>    criterion.

Maybe it is also time for “RFC” as discussed earlier.  It would fit this
item #1.

        Time for a request-for-comments process?
        id:87cznqb1sl.fsf@inria.fr
        Wed, 27 Oct 2021 23:22:50 +0200
        https://yhetil.org/guix/87cznqb1sl.fsf@inria.fr

Some examples of other projects.

https://github.com/NixOS/rfcs
https://peps.python.org/pep-0000/
https://github.com/rust-lang/rfcs

Cheers,
simon

bug#61475: Staging branch (was: Moving forward with teams and feature branches)

[Andreas Enge]

Am Sun, Feb 12, 2023 at 10:13:35PM +0100 schrieb Josselin Poiret:
> 4. staging merge happens, and the branch gets deleted.

I tried to build staging for my profile on x86_64, but it failed with
a dependency of ffmpeg, rust-rav1e-0.5.1. There is a newer version 0.6.3,
but I did not simply update it, since the package looks particularly
complicated, containing a phase:
         (add-after 'configure 'force-rust-edition-2018
           (lambda* (#:key vendor-dir #:allow-other-keys)
             ;; Force all the dependencies to not be higher than edition 2018.
             (with-fluids ((%default-port-encoding #f))
               (substitute* (find-files vendor-dir "Cargo.toml")
                 (("edition = \\\"2021\\\"") "edition = \"2018\"")))))
and many other changes.

If someone who knows rust could have a look and make a suggestion, that
would be great.

Andreas

bug#61475: Staging branch (was: Moving forward with teams and feature branches)

[Maxim Cournoyer]

Hi,

Andreas Enge <andreas@enge.fr> writes:

> Am Sun, Feb 12, 2023 at 10:13:35PM +0100 schrieb Josselin Poiret:
>> 4. staging merge happens, and the branch gets deleted.
>
> I tried to build staging for my profile on x86_64, but it failed with
> a dependency of ffmpeg, rust-rav1e-0.5.1. There is a newer version 0.6.3,
> but I did not simply update it, since the package looks particularly
> complicated, containing a phase:
>          (add-after 'configure 'force-rust-edition-2018
>            (lambda* (#:key vendor-dir #:allow-other-keys)
>              ;; Force all the dependencies to not be higher than edition 2018.
>              (with-fluids ((%default-port-encoding #f))
>                (substitute* (find-files vendor-dir "Cargo.toml")
>                  (("edition = \\\"2021\\\"") "edition = \"2018\"")))))
> and many other changes.
>
> If someone who knows rust could have a look and make a suggestion, that
> would be great.

Closing, both staging and the rust branch having been merged.

-- 
Thanks,
Maxim

Re: Moving forward with teams and feature branches (was: Discussion notes on releases and branches)

[Andreas Enge]

Am Sun, Feb 12, 2023 at 10:13:35PM +0100 schrieb Josselin Poiret:
> 4. staging merge happens, and the branch gets deleted.

I failed to compile my profile on staging, since rust-rav1e, a dependency
of ffmpeg, failed to build; see bug #61475.

Andreas

Re: Moving forward with teams and feature branches (was: Discussion notes on releases and branches)

[Leo Famulari]

On Mon, Feb 13, 2023 at 03:07:58PM +0100, Andreas Enge wrote:
> Am Sun, Feb 12, 2023 at 10:13:35PM +0100 schrieb Josselin Poiret:
> > 4. staging merge happens, and the branch gets deleted.
> 
> I failed to compile my profile on staging, since rust-rav1e, a dependency
> of ffmpeg, failed to build; see bug #61475.

If this can't be fixed quickly, my recommendation would be to remove the
dependency.

Rust-rav1e is an AV1 video encoder, but FFmpeg also has this
functionality via libaom, which is the reference implementation and
considered to be very high quality.

Release (was Re: Discussion notes on releases and branches)

[Simon Tournier]

Hi,

Thanks Andreas for the notes. :-)

On Thu, 09 Feb 2023 at 13:19, Andreas Enge <andreas@enge.fr> wrote:

> - A release schedule for a release every 4 to 6 months sounds
>   reasonable.
> - Mathieu, Simon, Julien and Andreas volunteer to be members of a release

Ludo was volunteer for helping, if I remember correctly. ;-)

First we should start more or less now for the next release. :-)

Second, it was not clear and is still not clear for me when the branch
is created, if a core-updates branch is merged before, etc.  Well, I do
not remember we discussed the workflow.  Do we?  If yes, what was the
consensus?


Cheers,
simon

Rust team branch (was Re: Discussion notes on releases and branches)

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 4448 bytes --]

On Thu, Feb 09, 2023 at 01:19:28PM +0100, Andreas Enge wrote:
> Attached are the notes of our discussion at the Guix Days concerning
> releases, branches, teams and related matters.
> 
> Andreas
> 

> BRANCHES
> 
> Suggestion:
> - Spin off a stable branch from master with security fixes, maybe important
>   backports; after 6 months, branch a new stable branch from master; this
>   is almost like a release, but continued into some future.
> 
> Counter-suggestion:
> - Create branches with a few patches or patchsets; in any case with
>   a "semantic" description of the changes. The branches could be
>   shortlived. Feature branches are one incarnation of the concept.
> - The numerical criteria for staging and core-updates is outdated:
>   Even non-core packages may create an enormous number of rebuilds.
> - Negative point: There is a risk of churn, by not regrouping world-
>   rebuilding changes - but two non related world rebuilding changes
>   might be difficult to review.
> 
> Before creating new branches, we need to clarify how the old branches
> are handled!
> 
> - Smaller branches could be taken care of by dedicated persons
>   responsible for pushing them forward. For instance by teams.
> - Some people already do this for a feature branch on their local
>   machine for medium-sized updates (ocaml), or even on ci (haskell,
>   kernel updates).
> - Branch creators should fix a goal and tentative timeline.
> - We need a mapping between branches and maintainers responsible
>   for the merge. This could be a team leader, if such a role is created.
>   A wiki could be used to keep track of the branches.
> - There is discussion whether we need a core-updates branch.
>   Core updates concern the toolchain, build phase changes, everything
>   that has big ramifications all over the system. It would be important
>   to not have several "parallel" branches with related (for instance,
>   but not exclusively, core-update) changes, which in fact should come
>   one after the other. Either they could be collected in one branch,
>   or would require coordination of branch creation (inside a team, say).
> - "Merge trains" of gitlab are mentioned, as a way of merging several
>   branches at the same time.
> - Grafts can also be handled in feature branches: The graft is applied
>   to master; the graft is applied to a different branch, directly followed
>   by an ungrafting commit and an update of the corresponding package;
>   once the branch is built it is merged back to master.
> - Minor drawback: If qa has treated a world-rebuilding patch, the
>   substitutes will be available on bordeaux, but not on berlin; people
>   who have installed a long time ago and not authorised bordeaux might
>   be affected. If there are complaints, they can be handled on the
>   mailing list.
> - Moving fast poses problems for non-x86 architectures, but the build farm
>   situation has improved for aarch64 and armhf sufficiently to keep up
>   with master. Handling feature branches remains an unsolved problem.
> - Currently there is a cap on qa, only patches with at most 300 dependents
>   are treated. This cap could be increased. Or it could be weighed with
>   the build times of the packages.
> 
> 
> TEAMS
> 
> - Issues could be tagged more often with responsible teams, or with
>   severity information (blocking bug or not).
> - Each module should be covered by a team; otherwise it would be
>   difficult to get important updates through a feature branches.

On behalf of the Rust Team™¹ we'd like to check our rust source tarballs
for any hidden binaries² and to do a mass upgrade of many of the crates.
Currently there are many crates queued up in the staging branch but we'd
like to pull them out and run a rust-team branch.

As a project we haven't setup anything for starting the team-based
branches and upgrades, and the Rust Team volunteers to go first.

Although the rust team would consider adding librsvg (and
python-cryptography) to our list of packages, we'd like to not touch it
this round, to keep it "small" (as far as rust goes) as a test.


¹ Help wanted
² https://issues.guix.gnu.org/61352


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Rust team branch

[Andreas Enge]

Am Tue, Feb 14, 2023 at 12:14:04PM +0200 schrieb Efraim Flashner:
> On behalf of the Rust Team™¹ we'd like to check our rust source tarballs
> for any hidden binaries² and to do a mass upgrade of many of the crates.
> Currently there are many crates queued up in the staging branch but we'd
> like to pull them out and run a rust-team branch.

By "pull out" you mean revert them in staging and apply them on a separate
branch? That would also delay #61475 and maybe ease merging of the staging
branch.

Would it make sense to wait with the creation of the feature branch
until after the core-updates merge?

> As a project we haven't setup anything for starting the team-based
> branches and upgrades, and the Rust Team volunteers to go first.
> ¹ Help wanted

Splendid! I suppose that increasing the team size beyond 1 would be
a first important goal, call out to volunteers!

Andreas

Re: Rust team branch

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]

On Tue, Feb 14, 2023 at 05:36:13PM +0100, Andreas Enge wrote:
> Am Tue, Feb 14, 2023 at 12:14:04PM +0200 schrieb Efraim Flashner:
> > On behalf of the Rust Team™¹ we'd like to check our rust source tarballs
> > for any hidden binaries² and to do a mass upgrade of many of the crates.
> > Currently there are many crates queued up in the staging branch but we'd
> > like to pull them out and run a rust-team branch.
> 
> By "pull out" you mean revert them in staging and apply them on a separate
> branch? That would also delay #61475 and maybe ease merging of the staging
> branch.

I was thinking more of cherry-picking them into a branch, not
necessarily reverting them on staging.

> Would it make sense to wait with the creation of the feature branch
> until after the core-updates merge?

Normally I'd say yes, but it might be a while until we can get
core-updates merged in.

> > As a project we haven't setup anything for starting the team-based
> > branches and upgrades, and the Rust Team volunteers to go first.
> > ¹ Help wanted
> 
> Splendid! I suppose that increasing the team size beyond 1 would be
> a first important goal, call out to volunteers!
> 
> Andreas
> 
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Rust team branch

[Andreas Enge]

Am Tue, Feb 14, 2023 at 10:07:12PM +0200 schrieb Efraim Flashner:
> > By "pull out" you mean revert them in staging and apply them on a separate
> > branch? That would also delay #61475 and maybe ease merging of the staging
> > branch.
> I was thinking more of cherry-picking them into a branch, not
> necessarily reverting them on staging.

Okay. But it would be nice then to revert at least as much in staging
that the branch builds and can be merged.

Andreas

Re: Rust team branch (was Re: Discussion notes on releases and branches)

[Katherine Cox-Buday]

Efraim Flashner <efraim@flashner.co.il> writes:

> As a project we haven't setup anything for starting the team-based
> branches and upgrades, and the Rust Team volunteers to go first.

Leo has attempted[1] to setup a branch for Go, but we're waiting to hear from
sysadmins on why there's a build error.

[1] https://lists.gnu.org/archive/html/guix-devel/2023-01/msg00131.html

-- 
Katherine

Re: Rust team branch (was Re: Discussion notes on releases and branches)

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 834 bytes --]

On Tue, Feb 14, 2023 at 09:36:46AM -0700, Katherine Cox-Buday wrote:
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > As a project we haven't setup anything for starting the team-based
> > branches and upgrades, and the Rust Team volunteers to go first.
> 
> Leo has attempted[1] to setup a branch for Go, but we're waiting to hear from
> sysadmins on why there's a build error.
> 
> [1] https://lists.gnu.org/archive/html/guix-devel/2023-01/msg00131.html

Second is also good :)

Also I don't think there's a lot of overlap between go and rust so we
can probably have both going at the same time.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Rust team branch (was Re: Discussion notes on releases and branches)

[Katherine Cox-Buday]

On 2/14/23 1:08 PM, Efraim Flashner wrote:

> Also I don't think there's a lot of overlap between go and rust so we
> can probably have both going at the same time.

Oh, yes, I didn't mean to imply that the two were related, or 
conflicted. I just wanted to point it out in case it was of any help.

Best of luck to the rust team!

--
Katherine

Re: Discussion notes on releases and branches

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Andreas,

On Thu, Feb 9, 2023 at 4:20 AM Andreas Enge <andreas@enge.fr> wrote:
>
> [excerpt from the attachment follows]
>
> - Smaller branches could be taken care of by dedicated persons
> [...]
> - Branch creators should fix a goal and tentative timeline.
> - We need a mapping between branches and maintainers responsible
>   for the merge.

Instead of dividing similar tasks among many people with different
priorities, I would appoint a "feature branch manager" (or perhaps,
chief pruner).

That person's job would be to keep an eye on the number of feature
branches on Savannah so they do not grow stale, or out of control.

Kind regards
Felix Lechner

Re: Discussion notes on releases and branches

[Leo Famulari]

On Fri, Mar 17, 2023 at 08:24:44AM -0700, Felix Lechner via Development of GNU Guix and the GNU System distribution. wrote:
> Instead of dividing similar tasks among many people with different
> priorities, I would appoint a "feature branch manager" (or perhaps,
> chief pruner).
> 
> That person's job would be to keep an eye on the number of feature
> branches on Savannah so they do not grow stale, or out of control.

There should be someone like this, although I suspect this particular
role could be informally performed by the people who are paying
attention to loads on the build farm at any given time. Generally, we
don't have success appointing people to jobs; rather, they volunteer or
even emerge non-explicitly.

On the quest for a new release model (was: Discussion notes on releases and branches)

[Cayetano Santos]

[-- Attachment #1: Type: text/plain, Size: 2154 bytes --]


Hi Guix,

Let me spin off last year thread on releases (flavored by an external to
the project point of view), with the goal to relaunch the debate on
releases.

Disclaimer: the rolling release model is great and more than enough for
me, and teams constitute an mportant step forward.

Disclaimer 2: guix-system is probably great, but using guix as a foreign
package manager is absolutely paramount, and would open the door to all
things guix, bringing lots of new users/contributors.

Now, any user in the world may install guix and upgrade to latest
version, sure. But.

First, for the current needs:

- distributions privilege and package guix releases (alpine, debian,
- arch, etc.), as guix itself does
- 1.4.0 dates back from 2 years ago
- in my experience, people tend to give guix a try, check obsolete sw,
  just give up, no matter the remaining advantages
- remote ci/cd images need to refer to a release for traceability
- apt-get install guix (1.4.0) && guix pull is a painful experience

Additionally, releases makes people talk about guix, give an overall
positive impression of the community, and are a good argument (and
publicity !) in favor of using it (unfortunate, but that’s the way it
goes). See emacs devel cycle, for example.

Now, for (a very naif) proposal (from a non contributor).

- simver[1] is the way to go
- devel as the branch for developments, master for releases and
  security/bug fixes
- major should follow core merges to devel
- minor should follow non-core teams merges
- patch fixes are backported to master

Yes, you guess [2] it.

Please, consider all the previous as an excuse to motivate the need for
releases, not as serious guidelines (I’m not the right person for that.)
For what I see around me, guix could easily be used by a larger audience
(foreign first, system then). Releases are a real requirement to that
goal.

Best,

[1] https://semver.org/
[2] https://nvie.com/posts/a-successful-git-branching-model/

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: On the quest for a new release model

[Ricardo Wurmus]

Hi,

Thanks for moving this discussion forward.  I do think we need much more
regular releases.

> - devel as the branch for developments, master for releases and
>   security/bug fixes

Changing the branching model is very difficult to do.  I think it is
better to ignore branches for now and focus on coming to an agreement
about more frequent releases, lest this discussion, too, ends up
reiterating "stable" branches and the finer points of release
maintenance.

> - major should follow core merges to devel
> - minor should follow non-core teams merges

I think this is a good idea to start with.  Releases are made a short
time after the core team branch is merged.  This would give us a new
release whenever e.g. the default GCC and glibc is bumped up.  We could
aim for a release two months after the merge to allow for minor fixes
after the merge.  I'm not sure if these merges should justify a new *major*
release, but I think it is good to have a new release then.

Not all team branch merges may justify a new release.  The r-team
branch, for example, usually contains just a couple hundred patch-level
package upgrades that are restricted to packages from CRAN and
Bioconductor.  It is only sometimes that the R version is increased or
the Bioconductor release version is changed --- only in those cases I
would consider it appropriate to bump up the Guix minor (or patch-level)
version number.

-- 
Ricardo

Re: On the quest for a new release model

[Suhail Singh]

Ricardo Wurmus <rekado@elephly.net> writes:

> Releases are made a short time after the core team branch is merged.
> This would give us a new release whenever e.g. the default GCC and
> glibc is bumped up.  We could aim for a release two months after the
> merge to allow for minor fixes after the merge.

I think agreeing on and following a release schedule is the crucial
part.  Both for how frequently core team branch is merged
(aspirationally) as well as how long (roughly) after the merge the
release is made.

I propose that we have a time-based release model rather than a
feature-based one (similar to the Linux kernel).  However, what's
perhaps even more important is to have a release model that we adhere
to.

-- 
Suhail

Re: On the quest for a new release model

[Greg Hogan]

On Fri, Dec 13, 2024 at 8:02 AM Suhail Singh <suhailsingh247@gmail.com> wrote:
>
> Ricardo Wurmus <rekado@elephly.net> writes:
>
> > Releases are made a short time after the core team branch is merged.
> > This would give us a new release whenever e.g. the default GCC and
> > glibc is bumped up.  We could aim for a release two months after the
> > merge to allow for minor fixes after the merge.
>
> I think agreeing on and following a release schedule is the crucial
> part.  Both for how frequently core team branch is merged
> (aspirationally) as well as how long (roughly) after the merge the
> release is made.
>
> I propose that we have a time-based release model rather than a
> feature-based one (similar to the Linux kernel).  However, what's
> perhaps even more important is to have a release model that we adhere
> to.

Having relatively recently asked [1] about a "1.5.0" release, I think
temver should be preferred over semver. What information has been
conveyed by Guix's version numbers beyond the symbolic "1.0"? Many
user-facing projects have adopted temporal versioning, including
NixOS.

We only need a release team and a documented release process. Releases
should be scheduled rather than depending on other teams. What benefit
is there to the Guix user when glibc or the default gcc are updated?
You're only a "guix pull" away from updated packages.

As I recall, one issue for past releases was having to freeze all
development on the master branch. With the new teams-branches model
the release-team branch is just another branch, moving to the queue
when ready to cut a new release.

Greg

[1] https://lists.gnu.org/archive/html/guix-devel/2024-09/msg00018.html

Re: On the quest for a new release model

[Suhail Singh]

Greg Hogan <code@greghogan.com> writes:

> We only need a release team and a documented release process. Releases
> should be scheduled rather than depending on other teams. What benefit
> is there to the Guix user when glibc or the default gcc are updated?
> You're only a "guix pull" away from updated packages.
>
> As I recall, one issue for past releases was having to freeze all
> development on the master branch. With the new teams-branches model
> the release-team branch is just another branch, moving to the queue
> when ready to cut a new release.

My sentiments precisely.  Thank you, Greg, for describing the situation
clearly.

If the goal is to increase the frequency of releases while maintaining
quality, the only consideration that the teams-branch ought to make is
to ensure that the commit in question (corresponding to the release)
builds correctly (i.e., may, in future, do some automated testing beyond
the test suites included in the individual packages) past some threshold
(i.e., on platforms of interest etc.).  Importantly, instead of making a
release soon after a major merge, such a team may decide to not make a
release too close to major changes.

However, to me, the release cadence is orthogonal to how we do the
versioning.  It's possible for a project to have time-based releases
while still using semver.  I propose that this thread be only about the
release process, and that the discusssion about the versioning happen in
a different thread.

-- 
Suhail

Re: On the quest for a new release model

[Suhail Singh]

Suhail Singh <suhailsingh247@gmail.com> writes:

> If the goal is to increase the frequency of releases while maintaining
> quality, the only consideration that the teams-branch ought to make is
                                           ^^^^^^^^^^^^
> to ensure that the commit in question (corresponding to the release)
> builds correctly (i.e., may, in future, do some automated testing beyond
> the test suites included in the individual packages) past some threshold
> (i.e., on platforms of interest etc.).  Importantly, instead of making a
> release soon after a major merge, such a team may decide to not make a
> release too close to major changes.

I meant release-team, above.

-- 
Suhail

Re: On the quest for a new release model

[Cayetano Santos]

[-- Attachment #1: Type: text/plain, Size: 1556 bytes --]


>ven. 13 déc. 2024 at 10:52, Suhail Singh <suhailsingh247@gmail.com> wrote:

> Greg Hogan <code@greghogan.com> writes:
>
>> We only need a release team and a documented release process. Releases
>> should be scheduled rather than depending on other teams. What benefit
>> is there to the Guix user when glibc or the default gcc are updated?
>> You're only a "guix pull" away from updated packages.
>>
>> As I recall, one issue for past releases was having to freeze all
>> development on the master branch. With the new teams-branches model
>> the release-team branch is just another branch, moving to the queue
>> when ready to cut a new release.
>
> My sentiments precisely.  Thank you, Greg, for describing the situation
> clearly.

Let me just play the dummies advocate for a moment, before leaving the
room to most experienced people: consider the army of regular potential
users not necessarily concerned about monads, variants and derivations
(or even substitutes).

They will be doing `whatever install guix`, and then they’ll happily
start using guix: with a bit of documentation, they’ll love it. This is
all they need to know. Don’t tell them to do a `python pull`, `firefox
pull` or `guix pull`. It comes to the same for them: they won’t
understand.

Once they get used to use guix daily, they’ll care about pulls, channels
or asking support for guix on the cluster.

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: On the quest for a new release model

[Suhail Singh]

Cayetano Santos <csantosb@inventati.org> writes:

> Let me just play the dummies advocate for a moment ...

Thank you.  I believe it is important to consider such scenarios (among
others).

> They will be doing `whatever install guix`, and then they’ll happily
> start using guix: with a bit of documentation, they’ll love it. This
> is all they need to know. Don’t tell them to do a `python pull`,
> `firefox pull` or `guix pull`. It comes to the same for them: they
> won’t understand.

Is my understanding of your scenario as summarized below correct, could
you please confirm?

There are some individuals who would be able to follow the instructions
to install Guix.  However, these individuals wouldn't understand the
instructions to perform a "guix pull".  Further, the number of such
individuals is greater than zero, and you are proposing that we consider
their needs when determining the release process for Guix.

Assuming my understanding above is correct, wouldn't you agree that
(even) for those individuals what's most important is that there is a
_stable_ and _not-very-outdated_ release available?  My (and I believe
Greg's) contention is that following a time-based release process
achieves these objectives more effectively than following a
feature-based release process.

-- 
Suhail

Re: On the quest for a new release model

[Cayetano Santos]

[-- Attachment #1: Type: text/plain, Size: 2040 bytes --]


>ven. 13 déc. 2024 at 12:21, Suhail Singh <suhailsingh247@gmail.com> wrote:

> Is my understanding of your scenario as summarized below correct, could
> you please confirm?
>
> There are some individuals who would be able to follow the instructions
> to install Guix.  However, these individuals wouldn't understand the
> instructions to perform a "guix pull".  Further, the number of such
> individuals is greater than zero, and you are proposing that we consider
> their needs when determining the release process for Guix.

Correct. Except the number of such individuals is huge. Some are doing
research, and they don’t need any more complexity, that they see as an
extra overload : they’ll install guix, and start using it. That’s it.
The more it becomes complex, the less they’ll be prone to try.

> Assuming my understanding above is correct, wouldn't you agree that
> (even) for those individuals what's most important is that there is a
> _stable_ and _not-very-outdated_ release available?

Correct again. At least, on a first contact with guix: one release every
4 months, for example, means using on average a two months out to date
guix, which is fine.

Take julia, for example. The cycle is : install guix, check julia,
outdated, forget guix and use julia installer.

> My (and I believe Greg's) contention is that following a time-based
> release process achieves these objectives more effectively than
> following a feature-based release process.

I’m not against: higher instances decide. Anything, but a 2 years old release.

Once again: sure, anything can be done, but it takes time and skills.
And guix is already good and useful enough for most people to use it
smoothly on a daily basis, solving lots of real life problems. Once they
fall into the rabbit hole, they’ll pull substitutes from channels.

But we need releases, first.

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: On the quest for a new release model

[Ricardo Wurmus]

Suhail Singh <suhailsingh247@gmail.com> writes:

> Assuming my understanding above is correct, wouldn't you agree that
> (even) for those individuals what's most important is that there is a
> _stable_ and _not-very-outdated_ release available?  My (and I believe
> Greg's) contention is that following a time-based release process
> achieves these objectives more effectively than following a
> feature-based release process.

I agree that more frequent releases are necessary, but I don't see much
value in “automatic” time-triggered releases.  After all, that's what
"guix pull" already provides.  Our releases should mean something.

Also note that Guix itself is a library.  I don't think it would be a
good idea to inflate the number of releases.  Our installer script
already offers a way to install the latest untested version of the Guix
package manager.

-- 
Ricardo

Re: On the quest for a new release model

[Suhail Singh]

Ricardo Wurmus <rekado@elephly.net> writes:

> Suhail Singh <suhailsingh247@gmail.com> writes:
>
>> Assuming my understanding above is correct, wouldn't you agree that
>> (even) for those individuals what's most important is that there is a
>> _stable_ and _not-very-outdated_ release available?  My (and I believe
>> Greg's) contention is that following a time-based release process
>> achieves these objectives more effectively than following a
>> feature-based release process.
>
> I agree that more frequent releases are necessary, but I don't see much
> value in “automatic” time-triggered releases.  After all, that's what
> "guix pull" already provides.  Our releases should mean something.

I was not proposing for "automatic" time-triggered releases.  I was
proposing that a periodic cadence start the _process_ of "vetting" the
release candidate.  Notably, the vetting process (particulars yet to be
decided) would be focusing on stability, test and build coverage etc.
I.e., some notions of quality.  This is something "guix pull" does _not_
provide.

I would describe the linux kernel as following a time-based release
process.  The meaningful distinction from a feature-based release
process is that the release process doesn't wait for any feature.

> Also note that Guix itself is a library.  I don't think it would be a
> good idea to inflate the number of releases.

If they are versioned according to an agreed upon versioning scheme, why
would the proliferation of versions _not_ be a good idea?

-- 
Suhail

Re: On the quest for a new release model

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Ricardo,

On Fri, Dec 13 2024, Ricardo Wurmus wrote:

> Our releases should mean something.

What do they mean, please?

Kind regards
Felix

P.S. No flames, please.  It's an innocent question.

Re: On the quest for a new release model

[John Kehayias via Development of GNU Guix and the GNU System distribution.]

Hi all,

On Fri, Dec 13, 2024 at 03:08 PM, Felix Lechner via \"Development of GNU Guix and the GNU System distribution.\" wrote:

> On Fri, Dec 13 2024, Ricardo Wurmus wrote:
>
>> Our releases should mean something.
>
> What do they mean, please?
>

This is actually related to the topic I wanted to bring up before we really get lost in the details of release building and so on.

I've always thought about Guix as a rolling distribution, where our "releases" are essentially installer "snapshots." In other words, the installer has maybe gotten improvements and certainly been thoroughly tested along with good substitute coverage and functionality across all (or as much as we can muster) of Guix. This tagged version is important to make sure there is binary substitute downloads (and maybe a corresponding manual online).

There is no expectation, nor really support, for staying on a "release." One is expected (and warned by guix even) to occasionally at least run "guix pull" if not upgrading, reconfiguring, etc. However, we do have powerful tools for staying at some particular commit, through time-machine, channel pinning, and so on.

I think what I wrote above is where we are currently at, though this perhaps is not documented explicitly as it should be. Many times the question comes up about a Guix "release" and missing that we are really a rolling distribution. Clearly there is room to improve here.

Is this something we agree on? I think being clear what Guix is (in terms of "releases"), and what we want it to be, is essential first.

That is my understanding above, and if that's what we want to continue with, we should say so clearly in the manual, finally resolve the confusing different manual version links, and be clear what a "release" is for Guix. I would move to revise our terminology to something like an "installer" version or "installation snapshot." (I don't see us having the resources to fully support releases as in a versioned distro, at this time.)

Just my 2 insignificant-currency-units, happy to hear this discussed further!

John

Re: On the quest for a new release model

[kiasoc5]

Hi all,

On 12/13/24 20:38, John Kehayias via Development of GNU Guix and the GNU 
System distribution. wrote:
> Hi all,
> 
> On Fri, Dec 13, 2024 at 03:08 PM, Felix Lechner via \"Development of GNU Guix and the GNU System distribution.\" wrote:
> 
>> On Fri, Dec 13 2024, Ricardo Wurmus wrote:
>>
>>> Our releases should mean something.
>>
>> What do they mean, please?
>>
> 
> This is actually related to the topic I wanted to bring up before we really get lost in the details of release building and so on.
> 
> I've always thought about Guix as a rolling distribution, where our "releases" are essentially installer "snapshots." In other words, the installer has maybe gotten improvements and certainly been thoroughly tested along with good substitute coverage and functionality across all (or as much as we can muster) of Guix. This tagged version is important to make sure there is binary substitute downloads (and maybe a corresponding manual online).
 >
> There is no expectation, nor really support, for staying on a "release." One is expected (and warned by guix even) to occasionally at least run "guix pull" if not upgrading, reconfiguring, etc. However, we do have powerful tools for staying at some particular commit, through time-machine, channel pinning, and so on.

I agree with replacing the word "release" with "installer version".

On distros with stable releases (Ubuntu, NixOS, etc), there is an 
expectation for support of packages (usually security updates) for the 
lifetime of the release. Seeing that the Guix versioned branches do not 
get any additional support beyond binary substitutes and a manual for 
the purpose of installation, the word "release" is not justified at this 
time.

But if Guix "releases" just exist for the purpose of versioning the 
installer, then it is the same as how a rolling release like Arch does 
it, where the installer is versioned and users are expected to upgrade 
their system afterwards (but not necessarily the installer).

Even with tools for managing packages at a specific commit, that's 
unique to a rolling release model where each commit is exposed to the 
user. The same cannot be said for versioned releases.

Re: On the quest for a new release model

[Cayetano Santos]

[-- Attachment #1: Type: text/plain, Size: 836 bytes --]


>sam. 14 déc. 2024 at 01:38, John Kehayias <john.kehayias@protonmail.com> wrote:

> Is this something we agree on? I think being clear what Guix is (in terms of "releases"),
> and what we want it to be, is essential first.

Thanks for the summary on current situation. I agree with you, and have
nothing against. But I have a doubt.

In "22.8.3 Version Numbers", we state that "Occasionally, we package
snapshots of upstream’s version control system (VCS) instead of formal
releases. This should remain exceptional, because it is up to upstream
developers to clarify what the stable release is"

So, what exactly is a packager of guix expected to package, from guix
point of view ?

--
Cayetano Santos
GnuPG Key:   https://meta.sr.ht/~csantosb.pgp
FingerPrint: CCB8 1842 F9D7 058E CD67 377A BF5C DF4D F6BF 6682

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: On the quest for a new release model

[Attila Lendvai]

> Our releases should mean something.

one thing i haven't seen mentioned:

AFAIU, not any version of guix can pull and build another guix version. i.e. when the guix command gets a new feature that is then used in the code that pulls and builds itself, then we have a bootstrap/staging problem not unlike with self-hosting compilers.

compilers usually use versioned releases to also mark the stages where stage n-1 is guaranteed to be able to build stage n.

i'm not sure how it is currently handled in guix. actually, i cannot come up with an example right now that requires bootstrapping, nor do i know how guix time-machine handles this. i assume everything around the pulling and building infrastructure is kept backwards compatible, so going back in time is not an issue.

but i do seem to remember a case where pulling from an old enough guix requires manual staging with `guix pull --commit=v1.2.3` to avoid pulling in too fresh commits that the current guix command wouldn't be able to build.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The nation that will insist on drawing a broad line of demarcation between the fighting man and the thinking man is liable to find its fighting done by fools and its thinking done by cowards.”
	— Sir William Francis Butler (1838–1910)

Re: On the quest for a new release model

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1992 bytes --]

On Fri, Dec 13, 2024 at 01:03:05PM +0100, Ricardo Wurmus wrote:
> Hi,
> 
> Thanks for moving this discussion forward.  I do think we need much more
> regular releases.
> 
> > - devel as the branch for developments, master for releases and
> >   security/bug fixes
> 
> Changing the branching model is very difficult to do.  I think it is
> better to ignore branches for now and focus on coming to an agreement
> about more frequent releases, lest this discussion, too, ends up
> reiterating "stable" branches and the finer points of release
> maintenance.
> 
> > - major should follow core merges to devel
> > - minor should follow non-core teams merges
> 
> I think this is a good idea to start with.  Releases are made a short
> time after the core team branch is merged.  This would give us a new
> release whenever e.g. the default GCC and glibc is bumped up.  We could
> aim for a release two months after the merge to allow for minor fixes
> after the merge.  I'm not sure if these merges should justify a new *major*
> release, but I think it is good to have a new release then.
> 
> Not all team branch merges may justify a new release.  The r-team
> branch, for example, usually contains just a couple hundred patch-level
> package upgrades that are restricted to packages from CRAN and
> Bioconductor.  It is only sometimes that the R version is increased or
> the Bioconductor release version is changed --- only in those cases I
> would consider it appropriate to bump up the Guix minor (or patch-level)
> version number.

Since, IMO, the major uses of the actual guix package is for the daemon
and the installer, I think we could tag a minor release just about every
time we bump the guix package.

Lets make releases boring :)

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: On the quest for a new release model

[Ricardo Wurmus]

Efraim Flashner <efraim@flashner.co.il> writes:

> Since, IMO, the major uses of the actual guix package is for the daemon
> and the installer, I think we could tag a minor release just about every
> time we bump the guix package.
>
> Lets make releases boring :)

I'm a very boring person, so this deeply resonates with me.

No matter if people agree with this or not, I think we should get a new
release out very soon.  Do you know of anyone who volunteered to
shepherd the upcoming release?  I've only ever signed off on one release
(0.13.0?) and I'd be happy to help get another one out of the door.

-- 
Ricardo

Re: On the quest for a new release model

[Suhail Singh]

Ricardo Wurmus <rekado@elephly.net> writes:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> Since, IMO, the major uses of the actual guix package is for the daemon
>> and the installer, I think we could tag a minor release just about every
>> time we bump the guix package.

That's a sensible approach.  How should the discussion proceed further?
Do we have a proxy to determine whether everyone who needs to be
involved for consensus-based decision-making has weighed in?

> I've only ever signed off on one release (0.13.0?) and I'd be happy to
> help get another one out of the door.

What does making a release entail?  Is this documented somewhere?

-- 
Suhail

Re: On the quest for a new release model

[Ricardo Wurmus]

Suhail Singh <suhailsingh247@gmail.com> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> I've only ever signed off on one release (0.13.0?) and I'd be happy to
>> help get another one out of the door.
>
> What does making a release entail?  Is this documented somewhere?

It is documented:

https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/release.org

-- 
Ricardo

Re: On the quest for a new release model

[Suhail Singh]

Ricardo Wurmus <rekado@elephly.net> writes:

>> What does making a release entail?  Is this documented somewhere?
>
> It is documented:
>
> https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/release.org

Thank you.  Should this be linked from the Guix manual?  Or is the
omission intentional?

-- 
Suhail

Re: On the quest for a new release model

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]

On Sun, Dec 15, 2024 at 03:21:44PM -0500, Suhail Singh wrote:
> Ricardo Wurmus <rekado@elephly.net> writes:
> 
> > Efraim Flashner <efraim@flashner.co.il> writes:
> >
> >> Since, IMO, the major uses of the actual guix package is for the daemon
> >> and the installer, I think we could tag a minor release just about every
> >> time we bump the guix package.
> 
> That's a sensible approach.  How should the discussion proceed further?
> Do we have a proxy to determine whether everyone who needs to be
> involved for consensus-based decision-making has weighed in?

I'd argue that cutting releases is one of those specifically maintainer
duties but I'd love to hear from other people who disagree.

If we do consider the daemon and installer the main reason to actually
cut a release, then I'd argue the daemon always works (we'd know
otherwise), I think we have an installer team, and we should probably
get some feedback from the desktop teams to make sure everything seems
to be working well enough.

I have some opinions about our aarch64 based images, but that should be
a separate email.

> > I've only ever signed off on one release (0.13.0?) and I'd be happy to
> > help get another one out of the door.
> 
> What does making a release entail?  Is this documented somewhere?
> 
> -- 
> Suhail

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: On the quest for a new release model

[Maxim Cournoyer]

Hi,

Efraim Flashner <efraim@flashner.co.il> writes:

> On Sun, Dec 15, 2024 at 03:21:44PM -0500, Suhail Singh wrote:
>> Ricardo Wurmus <rekado@elephly.net> writes:
>>
>> > Efraim Flashner <efraim@flashner.co.il> writes:
>> >
>> >> Since, IMO, the major uses of the actual guix package is for the daemon
>> >> and the installer, I think we could tag a minor release just about every
>> >> time we bump the guix package.
>>
>> That's a sensible approach.  How should the discussion proceed further?
>> Do we have a proxy to determine whether everyone who needs to be
>> involved for consensus-based decision-making has weighed in?
>
> I'd argue that cutting releases is one of those specifically maintainer
> duties but I'd love to hear from other people who disagree.

I'd tend to agree.  A maintainer who doesn't cut releases or organize to
make them happen is a poor maintainer (hello!).

> If we do consider the daemon and installer the main reason to actually
> cut a release, then I'd argue the daemon always works (we'd know
> otherwise), I think we have an installer team, and we should probably
> get some feedback from the desktop teams to make sure everything seems
> to be working well enough.
>
> I have some opinions about our aarch64 based images, but that should be
> a separate email.

I think one reason I'm so turned off from attempting to produce a
release is mainly the very high standard that Ludovic has set for
releases (hats off!), which is quite a lot of tedious work:

1. Comb thousands of commits (and since it'd been 2 years, more like
tens of thousands of them) to find news worthy items to put in the
NEWS.org file (which goes in the eventual announcement email)

2. Write a new blog post for the release, with the above information,
and often more detailed explanation on new features.

3. The challenging/labor intensive integration work such of attempting
to fix all system tests which often bitrot due to not so many people
(myself included) routinely running them and noticing (it's so costly to
do so, I can't really fault them), though I/we should register to the CI
notifications for the 'tests' job, that'd help keeping track of new
failures.

4. Some arcane Perl tool that needs to patch is used to produce the
final release email/upload artifacts, IIRC.  We should have better tools
than that.

5. I remember it to be very time consuming to try to just produce the
release artifacts for all platforms, building the current Guix then the
nested Guix for all targeted architectures.  If it fails, you're in for
a new lengthy round as the process must be started over ('make
release').

6. And of course, since a Guix release is mostly useful for people
installing it on foreign distributions or as fresh Guix System
installations, testing this is important, but also labor intensive
manual work.

If we can agree that producing a release with lowered expectations is
better than producing none, I don't mind to get the ball going.  I'd
drop the combing of thousands of commits for one thing, focusing just on
what's in our etc/news.scm file, or what is on the top of my head.  The
blog post may be spartan, with little backstory or examples.  The email
announcement email may not match the GNU release format, until we get to
rewrite the tool to match our multi-artifacts requirements (in Scheme,
of course).  Testing may have been mostly left to testers out there that
have foreign systems or VMs to experiment with, or the time to install
Guix from scratch using the installer.  A best effort would be done to
fix as many tests before releasing, without the promise of fixing them
all.

Parties interested in refining the release process can have a look at
the 'release' target of our Makefile.am file at the root of the Guix
repo, or at the doc/release.org file in the guix-maintenance repo.

Long term I think I'd like to see for releases:

1. Simplicity - just a tag, release email, artififact uploads may be
good enough, if we release often.

2. Automation - We already have many things automated, but it could be
polished a bit more (such as a better tool to produce the announcement
email), and perhaps some way to automatically test that a Guix istalled
via guix-install.sh' in various environments work (perhaps Docker could
help with that, having readily minimal images of Debian, Ubuntu, Fedora,
etc. to run our script on).

3. Frequency - we want to release often

-- 
Thanks,
Maxim

Re: On the quest for a new release model

[Suhail Singh]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> If we can agree that producing a release with lowered expectations is
> better than producing none, I don't mind to get the ball going.

I think it would be helpful to consider our approach for the upcoming
release distinctly from what we choose to do for subsequent releases.
It would help to hear Ludo's thoughts regarding the upcoming release.

> Long term I think I'd like to see for releases:
>
> 1. Simplicity - just a tag, release email, artififact uploads may be
> good enough, if we release often.
>
> 2. Automation - We already have many things automated, but it could be
> polished a bit more (such as a better tool to produce the announcement
> email), and perhaps some way to automatically test that a Guix istalled
> via guix-install.sh' in various environments work (perhaps Docker could
> help with that, having readily minimal images of Debian, Ubuntu, Fedora,
> etc. to run our script on).
>
> 3. Frequency - we want to release often

Agreed on all points, but especially frequency (3).  If we aim for that,
feasibility and practicality will encourage 1 and 2.

-- 
Suhail

Re: On the quest for a new release model

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> On Sun, Dec 15, 2024 at 03:21:44PM -0500, Suhail Singh wrote:
>>> Ricardo Wurmus <rekado@elephly.net> writes:
>>>
>>> > Efraim Flashner <efraim@flashner.co.il> writes:
>>> >
>>> >> Since, IMO, the major uses of the actual guix package is for the daemon
>>> >> and the installer, I think we could tag a minor release just about every
>>> >> time we bump the guix package.
>>>
>>> That's a sensible approach.  How should the discussion proceed further?
>>> Do we have a proxy to determine whether everyone who needs to be
>>> involved for consensus-based decision-making has weighed in?
>>
>> I'd argue that cutting releases is one of those specifically maintainer
>> duties but I'd love to hear from other people who disagree.
>
> I'd tend to agree.  A maintainer who doesn't cut releases or organize to
> make them happen is a poor maintainer (hello!).

Heheh.  :-)

As has been discussed multiple times at the Guix Days and on this list
(I think?), I believe what we need is a release team with rotating
duties.  That is, a bunch of 3–5 people commit to doing the work leading
to 1.5.0; then a new team (possibly with overlap) takes over for the
next version, and so on.

This is what NixOS has been doing for some time, for example, and it has
several advantages: it distributes responsibilities and power, and it
ensure everything is properly documented so people can actually carry
out the task.


As mentioned previously, I’m happy to mentor and help whoever steps up.
I know there are people up to the task; at least one person on the team
needs to have commit access, but apart from that, don’t be shy!

Our first exercise will be to make a “fake” release as a way to test
documentation and find out about missing bits and corner cases.

Ludo’.

Re: On the quest for a new release model

[Suhail Singh]

Ludovic Courtès <ludo@gnu.org> writes:

> I believe what we need is a release team with rotating duties.  That
> is, a bunch of 3–5 people commit to doing the work leading to 1.5.0;
> then a new team (possibly with overlap) takes over for the next
> version, and so on.

The issue, as I see it, is the time commitment required from the
release-team.  Per doc/release.org:

#+begin_quote
  The team has to be committed to be available for the duration of the
  whole process, which may typically span several weeks.
#+end_quote

For myself, while I do not have commit access, a several-week commitment
with an unclear effort cap makes me hesitant.

> As mentioned previously, I’m happy to mentor and help whoever steps
> up. I know there are people up to the task; at least one person on the
> team needs to have commit access, but apart from that, don’t be shy!

Unless I am mistaken, at least one person on this list mentioned that
they would be okay volunteering if the effort commitment could be
reduced.

Are you open to adapting the current release process (as documented in
guix/maintenance.git) to reduce the manual effort?

-- 
Suhail

Re: On the quest for a new release model

[Suhail Singh]

Suhail Singh <suhailsingh247@gmail.com> writes:

> The issue, as I see it, is the time commitment required from the
> release-team.

Correction, the issues (IMO) are (in no particular order):
1. the timespan (several weeks)
2. uncertainty around total effort
3. amount of manual effort involved

It's unclear which of the three above is the rate-limiting-step.

-- 
Suhail

Re: On the quest for a new release model

[Greg Hogan]

On Wed, Dec 18, 2024 at 12:43 PM Suhail Singh <suhailsingh247@gmail.com> wrote:
>
> Suhail Singh <suhailsingh247@gmail.com> writes:
>
> > The issue, as I see it, is the time commitment required from the
> > release-team.
>
> Correction, the issues (IMO) are (in no particular order):
> 1. the timespan (several weeks)
> 2. uncertainty around total effort
> 3. amount of manual effort involved
>
> It's unclear which of the three above is the rate-limiting-step.

There is also access to hardware. From doc/release.org:

"Steps #2 and #3 require you to have offloading set up so you can
build for all the supported architectures.  For instance, if you’re
running this on an x86_64 machine, you should have ~armhf-linux~,
~aarch64-linux~ and ~powerpc64le-linux~ machines in your
=/etc/guix/machines.scm=. Transparent emulation via QEMU has shown
limits (such as causing test suite failures); real hardware is a
must."

The project download page no longer lists PowerPC, but powerpc64le is
still included among the 1.4.0 binaries. Can the armhf release
artifacts be built on aarch64? That fails for some packages but might
work for releases.

Greg

Re: On the quest for a new release model

[Maxim Cournoyer]

Hi Greg,

Greg Hogan <code@greghogan.com> writes:

> On Wed, Dec 18, 2024 at 12:43 PM Suhail Singh <suhailsingh247@gmail.com> wrote:
>>
>> Suhail Singh <suhailsingh247@gmail.com> writes:
>>
>> > The issue, as I see it, is the time commitment required from the
>> > release-team.
>>
>> Correction, the issues (IMO) are (in no particular order):
>> 1. the timespan (several weeks)
>> 2. uncertainty around total effort
>> 3. amount of manual effort involved
>>
>> It's unclear which of the three above is the rate-limiting-step.
>
> There is also access to hardware. From doc/release.org:
>
> "Steps #2 and #3 require you to have offloading set up so you can
> build for all the supported architectures.  For instance, if you’re
> running this on an x86_64 machine, you should have ~armhf-linux~,
> ~aarch64-linux~ and ~powerpc64le-linux~ machines in your
> =/etc/guix/machines.scm=. Transparent emulation via QEMU has shown
> limits (such as causing test suite failures); real hardware is a
> must."

Indeed, at least for the person running 'make release'.

> The project download page no longer lists PowerPC, but powerpc64le is
> still included among the 1.4.0 binaries. Can the armhf release
> artifacts be built on aarch64? That fails for some packages but might
> work for releases.

I think the Guix binary release can be built from aarch64; we've never
had true armhf offload machines, as far as I know.

-- 
Thanks,
Maxim

Re: On the quest for a new release model

[Andreas Enge]

Am Fri, Dec 20, 2024 at 09:06:45PM +0900 schrieb Maxim Cournoyer:
> I think the Guix binary release can be built from aarch64; we've never
> had true armhf offload machines, as far as I know.

We did, and it still exists! Guix Foundation owns a Novena board that
was donated to us by Bunnie. I decommissioned it since I think it is
slower by some factor than our aarch64 machines. But if someone is
interested in using it for testing and so on, I could happily take it
out of my closest and bring it to FOSDEM, for instance.

Andreas

Re: On the quest for a new release model

[Thiago Jung Bauermann]

Greg Hogan <code@greghogan.com> writes:

> On Wed, Dec 18, 2024 at 12:43 PM Suhail Singh <suhailsingh247@gmail.com> wrote:
>>
>> Suhail Singh <suhailsingh247@gmail.com> writes:
>>
>> > The issue, as I see it, is the time commitment required from the
>> > release-team.
>>
>> Correction, the issues (IMO) are (in no particular order):
>> 1. the timespan (several weeks)
>> 2. uncertainty around total effort
>> 3. amount of manual effort involved
>>
>> It's unclear which of the three above is the rate-limiting-step.
>
> There is also access to hardware. From doc/release.org:
>
> "Steps #2 and #3 require you to have offloading set up so you can
> build for all the supported architectures.  For instance, if you’re
> running this on an x86_64 machine, you should have ~armhf-linux~,
> ~aarch64-linux~ and ~powerpc64le-linux~ machines in your
> =/etc/guix/machines.scm=. Transparent emulation via QEMU has shown
> limits (such as causing test suite failures); real hardware is a
> must."

In my experience, QEMU system emulation works better than user-mode
emulation so an alternative that I believe would work (though it would
be slow) would be to set up QEMU VMs for armhf, aarch64 and powerpc64le
(virt-manager is a convenient GUI to set these things up) and use them
for offloading.

> Can the armhf release artifacts be built on aarch64? That fails for
> some packages but might work for releases.

Not all aarch64 processors can run armhf code, but many can. This can be
seen in the "CPU op-mode(s)" field of lscpu. E.g.:

$ lscpu
Architecture:                       aarch64
CPU op-mode(s):                     32-bit, 64-bit
Byte Order:                         Little Endian
CPU(s):                             32
On-line CPU(s) list:                0-31
     ⋮
--
Thiago

Re: On the quest for a new release model

[Greg Hogan]

On Wed, Dec 18, 2024 at 11:49 AM Ludovic Courtès <ludo@gnu.org> wrote:
> As has been discussed multiple times at the Guix Days and on this list
> (I think?), I believe what we need is a release team with rotating
> duties.  That is, a bunch of 3–5 people commit to doing the work leading
> to 1.5.0; then a new team (possibly with overlap) takes over for the
> next version, and so on.

So a release team like every other team? An etc/teams.scm team (as
opposed to a mailing list team) also promotes the notion that much of
the needed work is outside of the release process. There were several
ideas for improvement earlier in this thread, but for another I
noticed that NixOS provides AMIs. If someone was to create the
necessary scripts and documentation for publishing Guix release AMIs
it would be handy to have a team to guide that contribution.

> This is what NixOS has been doing for some time, for example, and it has
> several advantages: it distributes responsibilities and power, and it
> ensure everything is properly documented so people can actually carry
> out the task.

Looking through doc/release.org, much of the work should be the
responsibility of other teams. NEWS could be updated when team
branches are merged. "Identifying and addressing bugs considered
release blockers" and "Stabilizing things" are an ever-present issue.
Time-based releases (whether calendar or hybrid versioning) can
coordinate project stability (as with the past unscheduled freezing of
the master branch).

Greg

Re: On the quest for a new release model

[Maxim Cournoyer]

Hi,

Greg Hogan <code@greghogan.com> writes:

> On Wed, Dec 18, 2024 at 11:49 AM Ludovic Courtès <ludo@gnu.org> wrote:
>> As has been discussed multiple times at the Guix Days and on this list
>> (I think?), I believe what we need is a release team with rotating
>> duties.  That is, a bunch of 3–5 people commit to doing the work leading
>> to 1.5.0; then a new team (possibly with overlap) takes over for the
>> next version, and so on.
>
> So a release team like every other team? An etc/teams.scm team (as
> opposed to a mailing list team) also promotes the notion that much of
> the needed work is outside of the release process. There were several
> ideas for improvement earlier in this thread, but for another I
> noticed that NixOS provides AMIs. If someone was to create the
> necessary scripts and documentation for publishing Guix release AMIs
> it would be handy to have a team to guide that contribution.

It could be an etc/teams.scm team, although these have so far always
existed with a scope of files, that works in conjunction with 'git
send-email' to notify people of modified files.  I don't have anything
against the idea though; they'd at least serve as a record of who to
contact for some scope of responsibility.

>> This is what NixOS has been doing for some time, for example, and it has
>> several advantages: it distributes responsibilities and power, and it
>> ensure everything is properly documented so people can actually carry
>> out the task.
>
> Looking through doc/release.org, much of the work should be the
> responsibility of other teams. NEWS could be updated when team
> branches are merged.

That's a low hanging fruit that would greatly help picking to ease the
work of producing a new release: some new contributing guidance that
would mention new significant features should have an entry written in
both the NEWS file and etc/news.scm.

-- 
Thanks,
Maxim

Re: On the quest for a new release model

[pelzflorian (Florian Pelz)]

Ricardo Wurmus <rekado@elephly.net> writes:
> Efraim Flashner <efraim@flashner.co.il> writes:
>> Lets make releases boring :)
>
> I'm a very boring person, so this deeply resonates with me.
>
> No matter if people agree with this or not, I think we should get a new
> release out very soon.  Do you know of anyone who volunteered to
> shepherd the upcoming release?  I've only ever signed off on one release
> (0.13.0?) and I'd be happy to help get another one out of the door.

Ludo asked at
<https://lists.gnu.org/archive/html/guix-devel/2024-09/msg00030.html>
<https://yhetil.org/guix-devel/87o751yvo9.fsf@gnu.org/>
and even though there were responses, there was no drive.

I will *not* be working on it, but one perhaps blocking issue might be
failing tests on Debian:

https://issues.guix.gnu.org/69518

Regards,
Florian

Re: On the quest for a new release model

[Ricardo Wurmus]

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>> Efraim Flashner <efraim@flashner.co.il> writes:
>>> Lets make releases boring :)
>>
>> I'm a very boring person, so this deeply resonates with me.
>>
>> No matter if people agree with this or not, I think we should get a new
>> release out very soon.  Do you know of anyone who volunteered to
>> shepherd the upcoming release?  I've only ever signed off on one release
>> (0.13.0?) and I'd be happy to help get another one out of the door.
>
> Ludo asked at
> <https://lists.gnu.org/archive/html/guix-devel/2024-09/msg00030.html>
> <https://yhetil.org/guix-devel/87o751yvo9.fsf@gnu.org/>
> and even though there were responses, there was no drive.
>
> I will *not* be working on it, but one perhaps blocking issue might be
> failing tests on Debian:
>
> https://issues.guix.gnu.org/69518

With the recent merge of the python-team branch a lot of packages are
now broken, so I suggest scheduling another python-team branch merge
before a release can possibly be made.

I'll be pushing to the python-team branch with some fixes (and likely
some new breakage) soon.  We're at a point with Python packages that we
cannot delay upgrades anymore, so whenever a fix requires an upgrade and
that creates ripples that necessitate more upgrades we should keep
applying them.

-- 
Ricardo

Re: On the quest for a new release model

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2816 bytes --]

On 2024-12-15, Efraim Flashner wrote:
> On Fri, Dec 13, 2024 at 01:03:05PM +0100, Ricardo Wurmus wrote:
>> > - devel as the branch for developments, master for releases and
>> >   security/bug fixes
>> 
>> Changing the branching model is very difficult to do.  I think it is
>> better to ignore branches for now and focus on coming to an agreement
>> about more frequent releases, lest this discussion, too, ends up
>> reiterating "stable" branches and the finer points of release
>> maintenance.
>> 
>> > - major should follow core merges to devel
>> > - minor should follow non-core teams merges
>> 
>> I think this is a good idea to start with.  Releases are made a short
>> time after the core team branch is merged.  This would give us a new
>> release whenever e.g. the default GCC and glibc is bumped up.  We could
>> aim for a release two months after the merge to allow for minor fixes
>> after the merge.  I'm not sure if these merges should justify a new *major*
>> release, but I think it is good to have a new release then.
>> 
>> Not all team branch merges may justify a new release.  The r-team
>> branch, for example, usually contains just a couple hundred patch-level
>> package upgrades that are restricted to packages from CRAN and
>> Bioconductor.  It is only sometimes that the R version is increased or
>> the Bioconductor release version is changed --- only in those cases I
>> would consider it appropriate to bump up the Guix minor (or patch-level)
>> version number.
>
> Since, IMO, the major uses of the actual guix package is for the daemon
> and the installer, I think we could tag a minor release just about every
> time we bump the guix package.

Hear, Hear!

I would like to encourage partial releases, e.g. a release where the
installer is tested and updated, a release where guix-daemon is updated,
and a "full" release closer to what has traditionally been done...

Not every release needs to support everything... it just needs to be
able to bootstrap up to the current guix... and the fewer steps to
bootstrap, the better... being clear about what the focus of each
release is might help?

If there is the occasional "bad" point/mini/micro release, it can be
flagged as such or something. Or even using alpha/beta/release-candidate
snapshots more often. Anything nudging towards something more
incremental...

As the maintainer of the guix package in Debian, regular point releases
with the new guix-daemon would be really, really nice. About to hit a
freeze cycle in Debian again, and no guix release in sight since the
last Debian stable release... I *might* just break down and package a
git snapshot... but tens of thousands of commits to choose from!


> Lets make releases boring :)

Yes!

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: On the quest for a new release model

[Simon Josefsson via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]

How much of the guix git repository content can you remove and still
have a working OS?  In other words, can we strip away almost all of the
packages and still have a minimal bootable system?  If we minimize that
set of really-core packages, maybe there can be one team that works on
this minimal bootable OS and produce stable images that are fast to
update.  If people want to add more packages, there is always the
current "fat" rolling Guix git repository that you can add on top of the
small base OS.  Just an idea that I've come back to while watching the
Guix git repository grow over the years.

/Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: On the quest for a new release model

[Suhail Singh]

Simon Josefsson via "Development of GNU Guix and the GNU System
distribution." <guix-devel@gnu.org> writes:

> can we strip away almost all of the packages and still have a minimal
> bootable system?  If we minimize that set of really-core packages,
> maybe there can be one team that works on this minimal bootable OS and
> produce stable images that are fast to update.  If people want to add
> more packages, there is always the current "fat" rolling Guix git
> repository that you can add on top of the small base OS.

I proposed something similar some weeks ago [1].  Specifically,
splitting Guix's monolithic repository into at least two different
channels (perhaps more).  These channels would be included in
%default-channels, however, the main Guix repository would only include
the packages from Ring-0 and Ring-1 (taking inspiration from OpenSUSE)
[2]:

#+begin_quote
  The core of Factory is divided into two rings (0-Bootstrap,
  1-MinimalX). Ring 0 contains packages that form the most basic,
  minimalist system that can compile itself. On top of that Ring 1 adds
  what's in the default installation of the two primary Desktops. All
  other packages are not part of a ring.
#+end_quote

[1]: <https://yhetil.org/guix/87plnm6cnj.fsf@gmail.com/>
[2]: <https://en.opensuse.org/openSUSE:Factory_development_model#Submit_requests_to_Ring_0_and_Ring_1>

-- 
Suhail

Re: On the quest for a new release model

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 1505 bytes --]

Suhail Singh <suhailsingh247@gmail.com> writes:

> Simon Josefsson via "Development of GNU Guix and the GNU System
> distribution." <guix-devel@gnu.org> writes:
>
>> can we strip away almost all of the packages and still have a minimal
>> bootable system?  If we minimize that set of really-core packages,
>> maybe there can be one team that works on this minimal bootable OS and
>> produce stable images that are fast to update.  If people want to add
>> more packages, there is always the current "fat" rolling Guix git
>> repository that you can add on top of the small base OS.
>
> I proposed something similar some weeks ago [1].  Specifically,
> splitting Guix's monolithic repository into at least two different
> channels (perhaps more).  These channels would be included in
> %default-channels, however, the main Guix repository would only include
> the packages from Ring-0 and Ring-1 (taking inspiration from OpenSUSE)
> [2]:

While I explicitly do not comment on this proposal, I would like to
point out that there is more work involved than just splitting it into
separate channels and adding those into %default-channels.

While multiple channels do work well with guix pull, there are annoying
limitations when used in other places.

Also you would need some way to specify what channel commits are able to
work together version-wise.

Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: On the quest for a new release model

[Suhail Singh]

Tomas Volf <~@wolfsden.cz> writes:

> I would like to point out that there is more work involved than just
> splitting it into separate channels and adding those into
> %default-channels.

Could you please elaborate?  Or were these the two points you noted
below?

> While multiple channels do work well with guix pull, there are annoying
> limitations when used in other places.

Could you please share some references for this?

> Also you would need some way to specify what channel commits are able to
> work together version-wise.

Let's say we have two channels in the future: $guix-slim and
$guix-extras.  Wouldn't it be sufficient for $guix-extras to depend on
$guix-slim ?  If not, could you please elaborate?

-- 
Suhail

Re: On the quest for a new release model

[Ricardo Wurmus]

Suhail Singh <suhailsingh247@gmail.com> writes:

>> Also you would need some way to specify what channel commits are able to
>> work together version-wise.
>
> Let's say we have two channels in the future: $guix-slim and
> $guix-extras.  Wouldn't it be sufficient for $guix-extras to depend on
> $guix-slim ?  If not, could you please elaborate?

You have multiplied the release problem by two.

guix-slim and guix-extras would both need to be versioned somehow, so
that whatever packages and Guix library infrastructure guix-extras needs
are in fact provided by the version of guix-slim.

I strongly advise not to go down this route.  See also my other email on
why it is not practical; feel free to search the archives for prior
discussions.

-- 
Ricardo

Re: On the quest for a new release model

[Suhail Singh]

Ricardo Wurmus <rekado@elephly.net> writes:

>> Let's say we have two channels in the future: $guix-slim and
>> $guix-extras.  Wouldn't it be sufficient for $guix-extras to depend on
>> $guix-slim ?  If not, could you please elaborate?
>
> You have multiplied the release problem by two.

If more releases pose a problem, then yes, we've increased the problem.
The set of challenges imposed by a release are not clear.  As well, it's
unclear whether the challenges are insurmountable, and if not what the
cost of overcoming them is.

However, in order to keep the discussion about releases focused, I
agree/propose that this tangent, about possibly splitting up the
monolithic Guix channel, be discussed in a separate thread (by those
interested).

-- 
Suhail

Re: On the quest for a new release model

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 2324 bytes --]

Suhail Singh <suhailsingh247@gmail.com> writes:

> Tomas Volf <~@wolfsden.cz> writes:
>
>> I would like to point out that there is more work involved than just
>> splitting it into separate channels and adding those into
>> %default-channels.
>
> Could you please elaborate?  Or were these the two points you noted
> below?

Yes, I have meant that the various issues would need to be fixed or
figured out.

>
>> While multiple channels do work well with guix pull, there are annoying
>> limitations when used in other places.
>
> Could you please share some references for this?

Definitely.  I personally have run into these two issues:

#74396
  When you install system-wide guix with multiple channels, it shadows
  channels from your guix-pulled guix (except for 'guix channel).  That
  was unpleasant surprise.

guix-for-channels does not cache
  `guix-for-channels' procedure is a way how to produce a guix package
  for a channel list.  However it does not cache the computation, so it
  is extremely slow when actually used.  I wrote a bit about that here:
  https://wolfsden.cz/blog/post/what-goes-into-guix-shaped-hole.html .

I am not sure if there are more, but my impression is that using
multiple channels (outside of guix-pull) is not well explored territory,
so I would not be surprised.

>
>> Also you would need some way to specify what channel commits are able to
>> work together version-wise.
>
> Let's say we have two channels in the future: $guix-slim and
> $guix-extras.  Wouldn't it be sufficient for $guix-extras to depend on
> $guix-slim ?  If not, could you please elaborate?

You say depend on guix-slim, but on *what* commit from guix-slim?  You
can pin channels.  What happens when you pin guix-slim, but pull
guix-extras?  I believe some human readable error should be given, not a
failure to build something due to, for example, not having the required
version of gcc now (for example because guix-extras increased the
required version).

Also what about substitutes?  Which combinations of commits will have
substitutes built?

I just feel there is a lot to figure out with this proposal.

Have a nice day,
Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: On the quest for a new release model

[Suhail Singh]

Tomas Volf <~@wolfsden.cz> writes:

> I just feel there is a lot to figure out with this proposal.

Thank you for the detailed response.  And I agree that this isn't a
solved problem (yet).  I propose that further discussion on this tangent
happen on a separate thread.

Whether Guix is split into more than one channel doesn't obviate the
need to define and agree upon a release model (which is what this thread
is about): either the current state where installer snapshots come after
several months, or some alternate future state the community sees value
in.

-- 
Suhail

Re: On the quest for a new release model

[Ricardo Wurmus]

Simon Josefsson via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org> writes:

> How much of the guix git repository content can you remove and still
> have a working OS?  In other words, can we strip away almost all of the
> packages and still have a minimal bootable system?  If we minimize that
> set of really-core packages, [...]

This has been discussed in the past and it is far from trivial.
Consider that glibc needs Python at build time and the Linux kernel may
need Rust.  Consider also that it is easy to introduce a package from
the outer ring to the inner ring through transitive dependencies.

Yes, there are many leaf packages, but it seems to me that it's a fool's
errand to attempt to separate packages like this.  (An exception is
package collections like those in CRAN or Bioconductor.)  It's certainly
not work that should be done manually.

We probably don't want to rehash this discussion as part of a discussion
about releases.

-- 
Ricardo

Splitting up Guix channel (was: On the quest for a new release model)

[Suhail Singh]

Ricardo Wurmus <rekado@elephly.net> writes:

> This has been discussed in the past and it is far from trivial.
> Consider that glibc needs Python at build time and the Linux kernel may
> need Rust.  Consider also that it is easy to introduce a package from
> the outer ring to the inner ring through transitive dependencies.

You are pointing out issues which would be necessary to overcome.  Based
on what you've said, it's not clear to me that overcoming these would
not be feasible.

> Yes, there are many leaf packages, but it seems to me that it's a fool's
> errand to attempt to separate packages like this.

Is the fact that distributions like OpenSUSE Tumbleweed accomplish such
a separation not material?  I agree that no definitive conclusion can be
drawn from their success since the set of packages aren't the same.  And
yet, I am not aware of anything that precludes such a separation.

> It's certainly not work that should be done manually.

Agreed.

-- 
Suhail