Proposal: nss updates

Proposal: nss updates

[Ian Eure]

The nss package updates frequently, around once a month.  It's 
also very low in the package graph, so a ton of stuff depends on 
it. The most recent update was a graft for security fixes, so we 
didn't have to rebuild everything, but the new Librewolf version 
once again requires an nss update.  I'm considering options to 
balance update frequency vs. huge rebuilds.

Mozilla has strong compatibility guarantees for nss, so the risk 
of packages breaking is very small.  This is purely about the cost 
in CPU time to build and bandwidth to transfer packages.

Mozilla provides an ESR channel for nss, but Guix doesn't use it — 
we went from 3.88.1 to 3.99, skipping 3.91, which is the current 
ESR.

I think the default nss in Guix should be the ESR, but we should 
also have a package for the latest nss, for stuff which needs it. 
This would let us update to the most recent nss without rebuilding 
so much, and only eat that cost when there’s a new ESR -- this 
happens approximately once a year.

Concretely:

The current nss package should stay how it is.  When the next ESR 
happens, it should update to that (ungrafting nss at the same 
time), and track ESR releases only from that point forward.  I 
don’t think it would make sense to downgrade the current 3.99 
package to the 3.91 ESR, so this will be a little funky until that 
release happens.

The latest version of nss should be added as a second package, 
named "nss-latest", bound to `nss-latest'.  It should track 
updates as frequently as needed.

While I’d prefer having the packages named "nss-esr" and "nss", I 
think the ESR should get the more prominent "nss" name, which 
should make it easy for developers to do the right thing -- if a 
bunch of packages depend on nss-latest, we’re back to the initial 
problem.  Code comments documenting this would also be added.

We might also want to adopt this approach for nspr.

I’m not sure about nss-certs; I think that should probably track 
the nss ESR, and I don’t think there’s a compelling need for a 
package tracking the rapid release channel.  I do want to improve 
this package by having it reuse the origin of nss instead of 
duplicating it.

Does all this seem reasonable to everyone?  If so, I can start 
sending patches.

Thanks,

  — Ian

Re: Proposal: nss updates

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Ian,

On Thu, Jun 27 2024, Ian Eure wrote:

> The nss package updates frequently, around once a month. [...]  I'm
> considering options to balance update frequency vs. huge rebuilds.

Your plan sounds reasonable but my opinion is inconsequential.  Instead,
I'd like to point out that you are not alone:

Wouldn't you like it if two days a month were set aside to allow uploads
that trigger large rebuilds?  The approach would pool intensive uploads
in the time domain rather than how we do it now in space, namely
branches.

The advantages would be (1) as you pointed ou, faster availability of
package updates; (2) gradual acceptance of large, potentially breaking
changes; (3) fewer rebases (4) broader testing of new features; (5) less
bureaucracy because all development happens on the main branch.

Moreover, there is also a better way to provide a "stable" Guix:

High substitute coverage for users---a goal that conflicts in our
current "space" paradigm with your effort to provide needed
updates---could instead be offered by modifying the 'guix pull' command
to accept a manifest of of the software the user needs, or perhaps a
percentage threshold of available substitutes.

Soon we would be able drop point releases, too, in addition to our large
community branches.

Kind regards
Felix

P.S. Ceterum censeo core-updates delendam esse!

Re: Proposal: nss updates

[Ian Eure]

Hi Felix,

Felix Lechner <felix.lechner@lease-up.com> writes:

> Hi Ian,
>
> On Thu, Jun 27 2024, Ian Eure wrote:
>
>> The nss package updates frequently, around once a month. [...] 
>> I'm
>> considering options to balance update frequency vs. huge 
>> rebuilds.
>
> Your plan sounds reasonable but my opinion is inconsequential. 
> Instead,
> I'd like to point out that you are not alone:
>
> Wouldn't you like it if two days a month were set aside to allow 
> uploads
> that trigger large rebuilds?  The approach would pool intensive 
> uploads
> in the time domain rather than how we do it now in space, namely
> branches.
>

I think this is probably a good idea, though the implementation 
might be difficult to manage.  I’m not sure where the patches 
would go if not some short-lived branch, so we’d still likely have 
the space complexity.  A one-month timebox for large changes, 
where they merge or get backed out at the end, seems like it could 
be a reasonable way to break down some of the long-running 
branches.

One thing we should be considerate of is users with limited 
bandwidth.  Even if Guix has the compute to build and bandwidth to 
serve, not all of its users will.

Thanks,

  — Ian

Re: Proposal: nss updates

[Maxim Cournoyer]

Hi Ian,

Ian Eure <ian@retrospec.tv> writes:

[...]

> Concretely:
>
> The current nss package should stay how it is.  When the next ESR
> happens, it should update to that (ungrafting nss at the same time),
> and track ESR releases only from that point forward.  I don’t think it
> would make sense to downgrade the current 3.99 package to the 3.91
> ESR, so this will be a little funky until that release happens.
>
> The latest version of nss should be added as a second package, named
> "nss-latest", bound to `nss-latest'.  It should track updates as
> frequently as needed.

Conventionally in Guix that'd be called nss-next.

> While I’d prefer having the packages named "nss-esr" and "nss", I
> think the ESR should get the more prominent "nss" name, which should
> make it easy for developers to do the right thing -- if a bunch of
> packages depend on nss-latest, we’re back to the initial problem.
> Code comments documenting this would also be added.
>
> We might also want to adopt this approach for nspr.
>
> I’m not sure about nss-certs; I think that should probably track the
> nss ESR, and I don’t think there’s a compelling need for a package
> tracking the rapid release channel.  I do want to improve this package
> by having it reuse the origin of nss instead of duplicating it.
>
> Does all this seem reasonable to everyone?  If so, I can start sending
> patches.

This all sounds reasonable to me.  Thank you for thinking about it and
proposing to improve the situation.

-- 
Thanks,
Maxim

Re: Proposal: nss updates

[Ian Eure]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi Ian,
>
> Ian Eure <ian@retrospec.tv> writes:
>
> [...]
>
>> Concretely:
>>
>> The current nss package should stay how it is.  When the next 
>> ESR
>> happens, it should update to that (ungrafting nss at the same 
>> time),
>> and track ESR releases only from that point forward.  I don’t 
>> think it
>> would make sense to downgrade the current 3.99 package to the 
>> 3.91
>> ESR, so this will be a little funky until that release happens.
>>
>> The latest version of nss should be added as a second package, 
>> named
>> "nss-latest", bound to `nss-latest'.  It should track updates 
>> as
>> frequently as needed.
>
> Conventionally in Guix that'd be called nss-next.
>

Oh boy, naming things!  :)

I’m aware of the -next convention, but I’m not sure it makes sense 
here.  It’s used primarily for newer versions packages in the same 
release channel -- ex. the default Python in Guix is python@3.10, 
but there’s python-next@3.12.  At some point, python will get 
promoted to 3.12.x -- the "next" name is a reflection of this; 
it’s intended to be the next python package in Guix.  Upstream has 
a single release channel, with a sliding support window over those 
releases.

The nss/nspr situation is somewhat different, as there are two 
upstream channels ("rapid release" and "extended support 
release").  ESRs are a subset of rapid releases, and the majority 
of rapid releases never enter the ESR channel.  Naming a rapid 
release nss-next when it will almost never become promoted to the 
nss package feels wrong to me.


>> While I’d prefer having the packages named "nss-esr" and "nss", 
>> I
>> think the ESR should get the more prominent "nss" name, which 
>> should
>> make it easy for developers to do the right thing -- if a bunch 
>> of
>> packages depend on nss-latest, we’re back to the initial 
>> problem.
>> Code comments documenting this would also be added.
>>
>> We might also want to adopt this approach for nspr.
>>
>> I’m not sure about nss-certs; I think that should probably 
>> track the
>> nss ESR, and I don’t think there’s a compelling need for a 
>> package
>> tracking the rapid release channel.  I do want to improve this 
>> package
>> by having it reuse the origin of nss instead of duplicating it.
>>
>> Does all this seem reasonable to everyone?  If so, I can start 
>> sending
>> patches.
>
> This all sounds reasonable to me.  Thank you for thinking about 
> it and
> proposing to improve the situation.
>

Thank you very much for your thoughts.

I opened 71832 which implements the first part of my nss proposal 
and updates Librewolf, so if you have strong feelings about -next 
vs. -latest, that might be the best place to raise them.

Thanks,

  — Ian

Re: Proposal: nss updates

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Ian,

On Mon, Jul 01 2024, Ian Eure wrote:

> if you have strong feelings about -next vs. -latest

How about nss-rapid?  It provides the clue about what was packaged to
someone who knows libnss.

Kind regards
Felix

Re: Proposal: nss updates

[Ian Eure]

Felix Lechner <felix.lechner@lease-up.com> writes:

> Hi Ian,
>
> On Mon, Jul 01 2024, Ian Eure wrote:
>
>> if you have strong feelings about -next vs. -latest
>
> How about nss-rapid?  It provides the clue about what was 
> packaged to
> someone who knows libnss.
>

I like it.  I’ll update the package descriptions to make this 
clear as well.

Thanks,

  — Ian

Re: Proposal: nss updates

[Maxim Cournoyer]

Hi,

Ian Eure <ian@retrospec.tv> writes:

> Felix Lechner <felix.lechner@lease-up.com> writes:
>
>> Hi Ian,
>>
>> On Mon, Jul 01 2024, Ian Eure wrote:
>>
>>> if you have strong feelings about -next vs. -latest
>>
>> How about nss-rapid?  It provides the clue about what was packaged
>> to
>> someone who knows libnss.
>>
>
> I like it.  I’ll update the package descriptions to make this clear as
> well.

Thanks for the explanations regarding the ESR and rapid release channels
of distribution for NSS.  I don't feel strongly about it, but the
'-latest' prefix is a bit easier to grok for someone not acquainted with
libnss.

-- 
Thanks,
Maxim

Re: Proposal: nss updates

[Ian Eure]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi,
>
> Ian Eure <ian@retrospec.tv> writes:
>
>> Felix Lechner <felix.lechner@lease-up.com> writes:
>>
>>> Hi Ian,
>>>
>>> On Mon, Jul 01 2024, Ian Eure wrote:
>>>
>>>> if you have strong feelings about -next vs. -latest
>>>
>>> How about nss-rapid?  It provides the clue about what was 
>>> packaged
>>> to
>>> someone who knows libnss.
>>>
>>
>> I like it.  I’ll update the package descriptions to make this 
>> clear as
>> well.
>
> Thanks for the explanations regarding the ESR and rapid release 
> channels
> of distribution for NSS.  I don't feel strongly about it, but 
> the
> '-latest' prefix is a bit easier to grok for someone not 
> acquainted with
> libnss.

I don’t have a strong preference either way, but lean towards 
calling it -rapid, as it matches the upstream terminology.  The 
package descriptions can disambiguate this, ex. adding "(ESR)" or 
similar to nss.

The recent 3.101.1 NSS release is an ESR, per the relesae 
notes[1].  What’s the process for getting that update into Guix? 
Since it’ll cause many rebuilds, it needs to go into a branch 
first.  core-updates seems like a reasonable place for it -- do I 
just send a patch and use prose to indicate that it should land in 
core-updates instead of master?  Or if I perform the work on the 
core-updates branch, do the patches indicate that when emailed?

Thanks,

  — Ian

[1]: 
https://firefox-source-docs.mozilla.org/security/nss/releases/nss_3_101_1.html

Re: Proposal: nss updates

[Maxim Cournoyer]

Hi Ian,

Ian Eure <ian@retrospec.tv> writes:

[...]

> The recent 3.101.1 NSS release is an ESR, per the relesae notes[1].
> What’s the process for getting that update into Guix? Since it’ll
> cause many rebuilds, it needs to go into a branch first.  core-updates
> seems like a reasonable place for it -- do I just send a patch and use
> prose to indicate that it should land in core-updates instead of
> master?  Or if I perform the work on the core-updates branch, do the
> patches indicate that when emailed?

I'd suggest to create the patches first, then build them on a feature
branch using CI (cuirass). You'll need some help from some committer who
can pushes the patches to a branch and configure the CI to build it.

I have myself little availability at the moment but that's going to
improve in the coming weeks.

-- 
Thanks,
Maxim