[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[pelzflorian (Florian Pelz)]

Hello Ludo.  Having a deprecation policy clarifies things.  Thank you
for writing a good one!

Ludovic Courtès <ludo@gnu.org> writes:
> +@item Package removal
> +Packages that their upstream developers have declared as having reach

typo: reached


> +``end of life'' or being unmaintained may be removed.  There is no
> +formal deprecation mechanism for this case, unless a replacement exists,
> +in which case the @code{deprecated-package} procedure mentioned above
> +can be used.
> +
> +If the package being removed is a ``leaf'' (no other packages depend on
> +it), it may be removed after a one-month review period of the patch
> +removing it.
> +

Could you also reference this one-month period in the Commit Access section,
where policy is one or two weeks?

Thinking of package removals for security reasons, it should be
clearer that this one-month period applies even in this case.  (IMHO
some period should apply.  It is the user’s decision to use software
despite security problems.  We all know web browsers’ security track
record; not everone puts the web to use everywhere, but Guix
thankfully does ship web browsers.)


> […]
> +@cindex deprecation of programming interfaces
> +@item Core interfaces
> +Core programming interfaces, in particular the @code{(guix ...)}
> +modules, may be relied on by a variety of external tools and channels.
> +Any incompatible change must be formally deprecated with

> +@code{define-deprecated}, as shown above, for at least one year before
> +removal.  The manual must clearly document the new interface and, except
> +in obvious cases, explain how to migrate from the old one.
> +
> +As an example, the @code{build-expression->derivation} procedure was
> +superseded by @code{gexp->derivation} and remained available as a
> +deprecated symbol:

This cannot be an absolute truth.  It is not always reasonable to make
changes bacwards-compatible.  For example, the switch from xz
repacking to zstd repacking in recent core-updates did break
guile-manual in doc/build.scm and perhaps did break outside code, but
it was right nonetheless.  Also Guix is in one big guix.git repository
so that we can make changes.

Regards,
Florian

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Thanks for taking the time to write this down.  It'll be useful to many
> I'm sure, including myself.
>
> Apart from the small things I've spotted above, it looks like a pretty
> good starting ground for a deprecation policy.

Coolio.  I’ve sent v3 fixing the typos you reported.

Let’s wait for another week or so to give people a chance to comment.

> One thought I'm having right now is that it would be cool if all these
> deprecation mechanism offered in the (guix deprecation) module were
> fully documented in our manual, and could be referred to from the above
> section/text for extra information; but this can be done in the future.

Right.  At least ‘deprecated-package’, ‘define-deprecated’, etc. will
now have an index entry.

Thanks for taking the time to comment!

Ludo’.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Noé Lopez via Guix-patches via]

Thanks for writing this,

A few things come to mind:

– How do we remember to delete something after one year of deprecation?
Should the deprecation date be noted with the deprecation to easily see?

– There is no policy for updating packages through major versions, IMO
  this should be the same as deleting and the previous version should be
  kept for a while, at least for the time for dependencies to update
  upstream.

>+If the package being removed is a ``leaf'' (no other packages depend on
>+it), it may be removed after a @b{one-month review period} of the patch
>+removing it (this applies even when the removal has additional
>+motivations such as security problems affecting the package).

– Why do « leaves » get removed at all? The dependents could be
  users that installed it in their profiles or manifests, one month
  seems very low.

Overall it makes sense so thanks again for documenting this,
Noé

PS: RFCs don’t get announced to guix-devel? I only found out about this
from mastodon.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Greg Hogan]

On Wed, Sep 11, 2024 at 2:33 PM Noé Lopez via Guix-patches via
<guix-patches@gnu.org> wrote:
>
> – There is no policy for updating packages through major versions, IMO
>   this should be the same as deleting and the previous version should be
>   kept for a while, at least for the time for dependencies to update
>   upstream.

Internal package conflicts result in broken builds. External dependent
projects can simply remain on their current Guix commit and delay
upgrading until compatible with the updated API.

> >+If the package being removed is a ``leaf'' (no other packages depend on
> >+it), it may be removed after a @b{one-month review period} of the patch
> >+removing it (this applies even when the removal has additional
> >+motivations such as security problems affecting the package).
>
> – Why do « leaves » get removed at all? The dependents could be
>   users that installed it in their profiles or manifests, one month
>   seems very low.

If a package has failed to build for and not been updated in a long
time then who would be using it? The package source will be available
in the git history in case someone would like to resurrect it at a
later time.

Greg

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Hi,

Noé Lopez <noe@noé.eu> skribis:

> – How do we remember to delete something after one year of deprecation?
> Should the deprecation date be noted with the deprecation to easily see?

What I and probably others did in the past was to ‘git annotate’ files
to see when a deprecation was added and whether it could “reasonably” be
deleted (though we had no formal rule).

We can always do that, but adding a comment as you suggest is even
better.

> – There is no policy for updating packages through major versions, IMO
>   this should be the same as deleting and the previous version should be
>   kept for a while, at least for the time for dependencies to update
>   upstream.

Interesting point.

For many packages, a major version upgrade goes unnoticed and a
deprecation period of the previous major series wouldn’t be useful.

But for some (interpreters and compilers, “big” libraries/frameworks
like Qt or GTK, and perhaps a few applications), there’s definitely
going to be a need for both the old and new major series for some time.

I’m not sure how to codify that though.  Maybe the best we can do is to
state that different situations exist and that “some” major package
upgrades may require a deprecation period for the older major series?

>>+If the package being removed is a ``leaf'' (no other packages depend on
>>+it), it may be removed after a @b{one-month review period} of the patch
>>+removing it (this applies even when the removal has additional
>>+motivations such as security problems affecting the package).
>
> – Why do « leaves » get removed at all? The dependents could be
>   users that installed it in their profiles or manifests, one month
>   seems very low.

This paragraph talks about packages that are unmaintained or EOL
upstream.  What it says is that such packages could be removed, at the
soonest, one month after they have become umaintained/EOL upstream.

The reasons we’d want to remove such packages is to clean up the package
collection (every package adds to the overall maintenance cost) and to
avoid steering users towards unmaintained and possibly insecure
software.

Is one-month after upstream too short?  I’d say “no”, but we can
discuss.

Two things to keep in mind in this discussion: (1) the policy does not
state an obligation to remove those packages, and (2) packages remain
available “forever” for those who need it via ‘time-machine’.

> PS: RFCs don’t get announced to guix-devel? I only found out about this
> from mastodon.

My bad! I thought I had Cc’d guix-devel, but apparently not? (Did the
‘send-email’ hook override the ‘Cc:’ or ‘X-Debbugs-Cc:’ header I had
put?)

Ludo’.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Hi Konrad,

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> Overall it looks good. I share Noé's concerns about breaking changes in
> packages. If removing a package is subject to the deprecation policy,
> then updating a package to an incompatible version should be handled the
> same way. But it is of course much more difficult to detect, for the
> packager and even more so for the Guix maintainers.

Right; I agree this should be mentioned.

> There's also a use case missing in the list in the beginning: Guix as a
> dependency of some other software, which in the worst case is no longer
> maintained. Users of such software may not even be aware of depending on
> Guix, and thus not follow Guix news at all. The number of such programs
> is probably close to zero right now, but I bet it won't remain
> zero. Every piece of software becomes someone else's dependency one day,
> at the latest during the next metasystem transition (see the last part
> of my talk in Montpellier last year
> (https://hpc.guix.info/events/2023/workshop/program/#caring-for-your-environment-s-)

I think this is covered by the last point:

+development of external tools that use programming interfaces such as
+the @code{(guix ...)} modules.

There are quite a few actually: the CI/QA tools, package browsers like
hpcguix-web, the Guix Workflow Language, Guix-Jupyter, rde, etc.

[...]

> Finally, I wonder about the practicalities. Who will watch out for
> potential violations of this policy, and how? It doesn't look like an
> easy task. In particular detecting "user-visible incompatible changes".

As drafted here, there’s no enforcement and nobody having the duty of
looking for violations and taking action.

I view it as binding though.  If a user complains that their favorite
package as been removed in violation of the policy, then we as a
community should review the claim and reinstate the package, unless it
violates “higher principles” in the project (that would need to be more
clearly defined too, but one of them would be: we mistakenly packaged
non-free software or material that we’re not allowed to distribute for
some reason).

I’ll think about ways to word it, but I’m happy to take other people’s
suggestions.

Thanks for your feedback,
Ludo’.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> …Well, I’m just aware of this only now – thanks mastodon!  Why only
> guix-patches and not guix-devel?  Or do I also missed it?

My bad; as I told Noé, I thought I did advertise it on guix-devel, but
apparently not.

> BTW, that’s the typical subject for a RFC [1], IMHO. :-)

Sure.

> Why not try to push for crossing the final line of the RFC process first
> and make this as the first? ;-)
>
> 1: https://issues.guix.gnu.org/66844

That’s a question for you no?

I like to push things past the finish line in a timely fashion, so I
wouldn’t want this to be blocked by the RFC process definition process
(the process of defining the process…).

I already commented on the proposed RFC process.  I’m happy to further
contribute or even take the lead eventually when time permits, if you’d
like to pass it on.  It’s clearly the missing piece here.  We’ll get
there!

Ludo’.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[indieterminacy]

It may be that Im lacking a little experience,
but Id assume that the categories outlined earlier probably require 
different timeframes:
https://issues.guix.gnu.org/72840#0-lineno49

A year feels a little arbitrary.

For example, Ive found it a little frustrating using randomly scoured 
Guix snippets -
only to then find that at the moment of utilization that I get a WARNING 
without much context.

Am I the only one who feels undermined and demotivated by information 
which hits me over the head with WARNINGS?

Ive had this using elogin-service (cited before):
https://issues.guix.gnu.org/72840#0-lineno155

Perhaps additionally there need to be pointers to real VCS repos,
pinpointing a commit providing the migration inside a living 
repo/infrastructure?

Such examples would be constructively showing how the change is 
achievable,
and could be empowering through assurance and learning.

And if such /Appreciations/ cannot be found in the wild and covering 
enough common adaptations, then perhaps a Depreciation is too soon?

Heck, a REASSURANCE printed following a reconfigure would be gravy!
Even better, Guix promising me a LIMERICK if I adapted off foobar within 
24 hours would work.

Additionally, its worth pointing out that Im slowly adapting my parsing 
approaches to be more commensurate with a form of modern hermeneutics.

I have been intrigued by how the language inside a setup like Guix 
adapts over time.
As such, the topic of how Guix grammars evolve would be worth 
documenting.

Practically, this could one day result in very old discourse from out of 
date mailinglist or Debugs conversations being transformed into more 
recent grammars to solve contemporary issues or suggest precedent when 
evaluating a patch.

Regards,


Jonathan

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Janneke Nieuwenhuizen]

Ludovic Courtès writes:

Hi!

> I realize I did not advertise the “Deprecation Policy” proposal here,
> which is a mistake because it’s relevant to all of us as developers and
> packagers, and it’s also a key element of our relation with the broader
> user base.
>
> So please, consider reading the proposal and joining the discussion:
>
>   https://issues.guix.gnu.org/72840

Thanks, looks good to me!  As a side remark: It would be nice if
upgrading of config.scm / home.scm could be automated.

Anyway, I do have a vaguely related question.  The Dezyne package comes
with a `guix.scm' that uses a package description in
guix/pack/dezyne.scm, which uses `%gnu-build-system-modules'.

Recently, %gnu-build-system-modules was deprecated in

--8<---------------cut here---------------start------------->8---
28dbfdb38f52f5814fb4cba9c02831d2ab0dc079
build-system/gnu: Introduce ‘%gnu-build-system-modules’ deprecated alias.

9e4ce281dbd92e3c52b831824ebb1f77023c960c
build-systems: gnu: Export %default-gnu-imported-modules and %default-gnu-modules.
--8<---------------cut here---------------end--------------->8---

Although the `guix.scm' has a comment like

--8<---------------cut here---------------start------------->8---
;; To use the canonical commit that has everything prebuilt:
;;
;;   guix time-machine --commit=918b7d102c2051c3d6c6ba54c8d265affec5282c -- shell
--8<---------------cut here---------------end--------------->8---

documenting a commit that can be used for building the package and has
substitutes available, usage of the commit is not enforced.  After a
recent `guix pull', we now get this warning

--8<---------------cut here---------------start------------->8---
pack/dezyne.scm:69:20: warning: '%gnu-build-system-modules' is deprecated, use '%default-gnu-imported-modules' instead
--8<---------------cut here---------------end--------------->8---

and I'm wondering what the best moment would be to change the package
description.  Upgrading sooner (i.e., now) means that a future guix that
has this deprecated feature removed will be able to build more
hystorical releases of the package simply by doing `guix shell', so
that's probably the best choice?  It would mean that all developers have
to upgrade now (or use the time machine).

Of course, we can always(?) build hystorical release by doing

    $(grep -o 'guix time.*' guix.scm)

but you'd have to know about that and it probably only works for the
Dezyne package.  Is there a better way or should something like this be
advertised/recommended in the documentation?

Greetings
Janneke

-- 
Janneke Nieuwenhuizen <janneke@gnu.org>  | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Konrad Hinsen]

Hi Ludo,

>> There's also a use case missing in the list in the beginning: Guix as a
>> dependency of some other software, which in the worst case is no longer
>> ...

> I think this is covered by the last point:
>
> +development of external tools that use programming interfaces such as
> +the @code{(guix ...)} modules.

Yes and no. I see external tools as two distinct use cases:

 - their development
 - their application

The missing case is application.

> There are quite a few actually: the CI/QA tools, package browsers like
> hpcguix-web, the Guix Workflow Language, Guix-Jupyter, rde, etc.

All those are add-on tools to the Guix CLI. I doubt these tools have any
user who wouldn't also use the Guix CLI. Meaning that they have a good
chance to learn about deprecations.

I am aware of a single tool that depends on Guix but whose functionality
is unrelated to Guix and could be implemented otherwise:

  https://github.com/khinsen/swh-checkout

It's a Guile script that uses Guix as a library for accessing Software
Heritage. And it's a mere proof-of-concept implementation. I don't
advertise it for general use. But I do expect more such tools to appear
over time, including some with more substantial dependence on Guix.

> As drafted here, there’s no enforcement and nobody having the duty of
> looking for violations and taking action.
>
> I view it as binding though.  If a user complains that their favorite
> package as been removed in violation of the policy, then we as a
> community should review the claim and reinstate the package, unless it

OK, that sounds good enough!

Cheers,
  Konrad.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Hey there!

indieterminacy <indieterminacy@libre.brussels> skribis:

> It may be that Im lacking a little experience,
> but Id assume that the categories outlined earlier probably require
> different timeframes:
> https://issues.guix.gnu.org/72840#0-lineno49
>
> A year feels a little arbitrary.

It’s necessarily arbitrary, but I think we have to set expectations for
users and provide guidelines for contributors; we can’t just say
“eventually”.

[...]

> Am I the only one who feels undermined and demotivated by information
> which hits me over the head with WARNINGS?
>
> Ive had this using elogin-service (cited before):
> https://issues.guix.gnu.org/72840#0-lineno155
>
> Perhaps additionally there need to be pointers to real VCS repos,
> pinpointing a commit providing the migration inside a living
> repo/infrastructure?
>
> Such examples would be constructively showing how the change is
> achievable,
> and could be empowering through assurance and learning.

I agree.  I mean, those warnings could be accompanied with a diff
showing how to change the source (GCC does that these days in some
cases), and/or it could point to examples in the manual.

This is something we should consider, but I think it shouldn’t block
a first version of the policy.

Thanks,
Ludo’.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Hi,

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> Debian has the status of "orphaned" packages for the situations that you
> describe. Maybe Guix should have that as well? The main interest I see
> is keeping a list of "software we had but can't handle any more",
> ideally with a pointer to the last working state in Guix, e.g. the last
> commit for which CI could build the package. I'd even like such packages
> to show up in answers to "guix search", so that I know the difference
> between "not packaged yet" and "tough case we gave up on".

Sounds like a good idea.  This could be achieved with a package
property, maybe.

I’d like this to not block version 1 of the policy though.

>> if during one month nobody steps in to carry out the update, that is
>> telling.
>
> I don't know on which time scales other Guix users live, but for me, one
> month is the average delay between two "guix pull". In other words, one
> month is my notion of "immediately" when it comes to Guix.

Agreed.

Ludo’.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 32 bytes --]

The v3 -> v4 diff for clarity:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2024 bytes --]

diff --git a/doc/contributing.texi b/doc/contributing.texi
index f713765357..d6a684306c 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -3062,8 +3062,8 @@ Deprecation Policy
 @code{home-environment} interfaces together with the service interfaces;
 
 @item
-development of external tools that use programming interfaces such as
-the @code{(guix ...)} modules.
+development or use of external tools that use programming interfaces
+such as the @code{(guix ...)} modules.
 @end itemize
 
 These use cases form a spectrum with varying degrees of coupling---from
@@ -3111,10 +3111,12 @@ Deprecation Policy
 @anchor{package-removal-policy}
 @item Package removal
 Packages whose upstream developers have declared as having reached ``end
-of life'' or being unmaintained may be removed.  There is no formal
-deprecation mechanism for this case, unless a replacement exists, in
-which case the @code{deprecated-package} procedure mentioned above can
-be used.
+of life'' or being unmaintained may be removed; likewise, packages that
+have been @b{failing to build for two months or more} may be removed.
+
+There is no formal deprecation mechanism for this case, unless a
+replacement exists, in which case the @code{deprecated-package}
+procedure mentioned above can be used.
 
 If the package being removed is a ``leaf'' (no other packages depend on
 it), it may be removed after a @b{one-month review period} of the patch
@@ -3133,6 +3135,14 @@ Deprecation Policy
 is a leaf, its deprecation must be announced as an entry in
 @code{etc/news.scm}.
 
+@item Package upgrade
+In the case of packages with many dependents and/or many users, an
+upgrade may be treated like the @emph{removal} of the previous version.
+
+Examples include major version upgrades of programming language
+implementations, as we've seen above with Python, and major upgrades of
+``big'' libraries such as Qt or GTK.
+
 @cindex service deprecation
 @item Services
 Changes to services for Guix Home and Guix System have a direct impact

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Konrad Hinsen]

Hi Ludo,

> Sounds like a good idea.  This could be achieved with a package
> property, maybe.
>
> I’d like this to not block version 1 of the policy though.

Certainly not!

That said, a convention for commit messages related to the cases
described in the deprecation policy would be a good start. Not just
"deprecated" or "removed", but also a keyword stating the reason.

Cheers,
  Konrad.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Hi Janneke,

Janneke Nieuwenhuizen <janneke@gnu.org> skribis:

>>   https://issues.guix.gnu.org/72840
>
> Thanks, looks good to me!  As a side remark: It would be nice if
> upgrading of config.scm / home.scm could be automated.

It would be great, indeed.

> Anyway, I do have a vaguely related question.  The Dezyne package comes
> with a `guix.scm' that uses a package description in
> guix/pack/dezyne.scm, which uses `%gnu-build-system-modules'.
>
> Recently, %gnu-build-system-modules was deprecated in
>
> 28dbfdb38f52f5814fb4cba9c02831d2ab0dc079
> build-system/gnu: Introduce ‘%gnu-build-system-modules’ deprecated alias.
>
> 9e4ce281dbd92e3c52b831824ebb1f77023c960c
> build-systems: gnu: Export %default-gnu-imported-modules and %default-gnu-modules.

I’m not convinced this was a worthwhile change BTW, looking at the
intended clarity improvement vs. cost ratio.

> Although the `guix.scm' has a comment like
>
> ;; To use the canonical commit that has everything prebuilt:
> ;;
> ;;   guix time-machine --commit=918b7d102c2051c3d6c6ba54c8d265affec5282c -- shell
>
>
> documenting a commit that can be used for building the package and has
> substitutes available, usage of the commit is not enforced.  After a
> recent `guix pull', we now get this warning
>
> pack/dezyne.scm:69:20: warning: '%gnu-build-system-modules' is deprecated, use '%default-gnu-imported-modules' instead
>
> and I'm wondering what the best moment would be to change the package
> description.  Upgrading sooner (i.e., now) means that a future guix that
> has this deprecated feature removed will be able to build more
> hystorical releases of the package simply by doing `guix shell', so
> that's probably the best choice?  It would mean that all developers have
> to upgrade now (or use the time machine).

Yeah, that’s always a difficult choice, and I don’t have a good answer.
What’s sure is that the deprecated name will remain available for a
relatively long time, so there’s no urgency at this point.

> but you'd have to know about that and it probably only works for the
> Dezyne package.  Is there a better way or should something like this be
> advertised/recommended in the documentation?

I’m not sure we could recommend one approach that would work for
everyone because it really depends on the use case (for instance whether
building with an older Guix is important for your project.  But at
least, by setting expectations, the deprecation policy lets users making
informed decisions.

Ludo’.

[bug#72840] [PATCH RFC] DRAFT doc: Add “Deprecation Policy” section.

[Ludovic Courtès]

Greg Hogan <code@greghogan.com> skribis:

> On Mon, Sep 23, 2024 at 6:11 PM Ludovic Courtès <ludo@gnu.org> wrote:

[...]

>> +If it has many dependent packages---as is the case for example with
>> +Python version@tie{}2---the relevant team must propose a deprecation
>> +removal agenda and seek consensus with other packagers for @b{at least
>> +one month}.  It may also invite feedback from the broader user
>> +community, for example through a survey.  Removal of all impacted
>> +packages may be gradual, spanning multiple months, to accommodate all
>> +use cases.
>> +
>> +When the package being removed is considered popular, whether or not it
>> +is a leaf, its deprecation must be announced as an entry in
>> +@code{etc/news.scm}.
>
> Hi Ludo',
>
> Is the intent for the news entry to pre-announce the removal of a
> popular package, as specified for other kinds of deprecation and
> removal?

What the paragraph above means is that a news entry is published when
the popular package is deprecated, which could be months before it is
removed.

Does that make sense?

Ludo’.