Re: Stability of core packages (was: Not easy at all to upgrade :core packages like Eglot)

[Eli Zaretskii <eliz@gnu.org>]

> Date: Wed, 19 Apr 2023 01:10:56 +0300
> Cc: joaotavora@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> > This will be one of the serious issues if we ever move to having some
> > packages only in elpa.git, and will then bundle them when preparing an
> > Emacs release tarball.  It will be imperative to know at that time
> > which version/branch of each such package to take as part of preparing
> > a release.  We must have a solution by then, so this is as good time
> > as any to start discussing the issue.
> 
> Sure. Please keep in mind, however, that very few of external packages 
> have separate branches for releases and development. I guess Org does, 
> but I'm not sure if others exist. One of the reasons for that is that 
> the ELPA repositories only package the very latest released version 
> anyway and quickly delete the older ones. And package.el is the foremost 
> distribution method for Elisp code.

Then maybe ELPA will have to change as well.

> So every time a new version if tagged, it's a value judgment on the part 
> of the maintainer: whether enough time has passed since the most recent 
> big feature was added, whether there were bug reports or not.

Similar judgment calls are necessary for Emacs.  So I see nothing here
that brings some fundamentally new issues.

Moreover, I don't see how this affects the need to have good criteria
for stability, and the importance of applying those criteria when
deciding which versions to bundle with Emacs.

> > If some core package is not tested enough before it gets a new
> > version, then why are we okay with telling users of a stable Emacs to
> > update the package willy-nilly as soon as another version is on ELPA?
> > Shouldn't we at least warn them, or, better, somehow indicate that
> > this version is not yet considered stable?
> 
> I have a different answer from all that had been presented here: because 
> the user can uninstall it.

User can also downgrade to a previous version of the package, don't
they?  Or if they cannot, they should be able to do that.  Once that
is possible, what exactly is the difference between these two kinds of
packages?  Downgrading to an older version when a newer one is buggy
or otherwise unsatisfactory should be supported for all packages.

Also, does package.el support "downgrading" to the bundled version?
Did anyone actually try that?  In particular, what happens with the
dependencies the user upgraded together with the package being
"uninstalled", due to the minimum requirements of that package?

> > IOW, shouldn't packages have some "stability gradation" that is
> > visible when users look at the list of packages via package.el?
> > Shouldn't we allow users to tell package.el which stability they want
> > to download, so that unstable packages don't inadvertently get
> > installed and mess up their production environment?
> 
> We have elpa and elpa-devel. The latter is "explicitly unstable" and the 
> former is "checkpoint releases".

So what is the stability measure of "elpa", again?  Is it on par with
the Emacs's release branch?  Could it be?  If not, why not?

> Neither keeps the previous versions around, so it's not like the user at 
> any point could make a choice to install a minor version update instead 
> of going up a full major version.

That in itself is a serious deficiency, IMO, and we should fix it by
providing the "downgrade" option.

> > I don't see what development pace has to do with this issue.  If a
> > package is being developed at a high pace, it might mean that the
> > stable version will lag more.  But what does this change in principle?
> 
> It matters when we're talking not about simple additional, optional 
> features, but about changes that keep pace with the evolution of the LSP 
> standard, with new LSP servers that arrive, new changes in existing 
> protocols. Things that users come to expect to be able to use now, and 
> not in 3 years.

Users who must have these features (presumably because they must use
servers which require that) will have to give up some stability
expectations.  Exactly like users who must have some new enough
feature of Emacs which is only available on the master branch.

So once again: what is fundamentally different or new about packages
which develop at fast pace, in the context of this discussion?  Why do
we need to bring up those details here?

> Taking a conservative stance can work when the ecosystem supports it 
> too. E.g. if every LSP server that we support now had an implicit 
> promise to be properly maintained for 3 years since, at least. I don't 
> think that's the case. Almost every one could be deprecated in that time 
> frame, with community being recommended to switch to a newer one.

How does this help us move forward with the discussion and with
handling this issue?  Conservative doesn't mean stupid; if some
external factor changes outside of our control and forces us to make
potentially destabilizing changes, what else can we do that requires a
discussion?  And how does it impact what we should do about
categorizing package releases according to their stability and about
the decisions which version to bundle with Emacs?

> To clarify: I made that suggestion from the premise that we do expect 
> and encourage a fair fraction of the users to use just the versions of 
> Eglot and Eldoc that come with Emacs 29, and never upgrade to the latest 
> ones.

AFAIU, João is of the opposite opinion: he thinks that most Eglot
users will want the latest version on ELPA, or at least a version that
is newer than the one bundled with Emacs.

> And as per above explanation, Eglot 0.14 depends on Eldoc 0.14.0 not 
> because it's the very latest and spiffiest version, but because it needs 
> a particular feature from it.

And there's no reasonable way of adding to Eglot 1.14 some fallback or
compatibility shim to remove that dependency?