bug#62720: 29.0.60; Not easy at all to upgrade :core packages like Eglot

[Eli Zaretskii <eliz@gnu.org>]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 14 Apr 2023 20:46:02 +0100
> Cc: dmitry@gutov.dev, rpluim@gmail.com, philipk@posteo.net, 
> 	62720@debbugs.gnu.org, larsi@gnus.org, monnier@iro.umontreal.ca
> 
> On Fri, Apr 14, 2023 at 8:28 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: João Távora <joaotavora@gmail.com>
> > > Date: Fri, 14 Apr 2023 20:20:20 +0100
> > > Cc: dmitry@gutov.dev, rpluim@gmail.com, philipk@posteo.net,
> > >       62720@debbugs.gnu.org, larsi@gnus.org, monnier@iro.umontreal.ca
> > >
> > > On Fri, Apr 14, 2023 at 7:51 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > >
> > > The Eglot release 1.14 is not as stable as 1.12.29. It has new features.
> > > Also a recent bugfix in commit a74403adda0 is quite complex and
> > > I'm cautiously waiting for feedback.  Other simple bugfixes have
> > > been backported.
> >
> > So you don't recommend that users who want a stable Eglot upgrade to
> > 1.14?
> 
> Depends on how "stable" they want it and how badly they want new
> features.

You are dodging the question, which is unfortunate.  It is IMO an
important question, much more general and important than this
tempest-in-a-teapot issue we are discussing here.  We will need to
revisit this question seriously if we ever succeed in removing 'core'
packages from emacs.git and bundling them with release tarballs.  At
that time, we will need clear-cut criteria for how to select the
version of a package that is the most suitable one to go into the
tarball.  And that in turn will require the developers of each such
package to seriously address the questions I've asked: what are the
criteria for considering a package version "stable", and how is that
communicated via version numbers and/or repository branches.  For
example, Org has that encoded in its branches, so that the latest
changes from its stable branch are routinely merged into the stable
branch of Emacs, and the latest code from that branch will be in the
Emacs release tarball.

Based on your answer, it sounds like Eglot users are on their own in
this aspect: they have no real guidance from you which version is
currently considered "stable".

> > If so, why is it a problem that package-install by default
> > doesn't update built-in packages?  Users who want the cutting edge of
> > Eglot, and don't mind some instability, can always switch to the
> > master branch of Emacs, where we are free to change package-install to
> > upgrade core packages by default.
> 
> Users don't just switch to the master branch of Emacs.  Many just
> can't because of enterprise complications, or just difficulty of compilation.
> I've worked in companies using Emacs (some of them exclusively!!) for 20
> years.  Most users are two, sometimes, three Emacs releases behind.
> They are not remotely interested in updating.  "IT doesn't like it".
> "It's not the official".  "This one is just fine".
> 
> But if a colleague goes to their workstation and shows them
> M-x package-install RET very-fancy-nice-thing or sends them a
> super-fancy init.el they will take it no problem, and buy you
> coffee and rave about it. I can't be the only one who has
> experienced this :-)

I have no doubt there are such cases.  But I very much doubt they are
the majority.  Whatever we decide, it is clear that some use cases
will want a different behavior.  The issue discussed here is what
should be the default behavior, and whether the default behavior of
Emacs 29.1 would be wrong for most users of Eglot and of other core
packages.

To reiterate: I think each release of Emacs should ship with the
latest stable version of each core package.  If this is the case, the
need to upgrade core packages via package.el is not very important for
the majority of our users who prefer to use a stable Emacs.  Thus, the
arguments you present emphasize the needs of the minority, and
therefore I don't consider them strong enough to invalidate the
compromise solution to which we are converging.

> > > But if someone types M-x install-something they should get what
> > > they ask for.  If they want to be 100% safe, they just shouldn't
> > > invoke commands that download, compile and evaluate code.
> >
> > The logic should be consistent.  Emacs 29 is the stable branch of
> > Emacs, so it should come with the latest stable Eglot.  If that is
> > Eglot 1.12.29, then the fact that package-install won't upgrade it to
> > 1.14 is consistent with the stability of Eglot's versions.  If, OTOH,
> > you think that it's imperative to allow _all_ users of Eglot with
> > Emacs 29 to upgrade to Eglot 1.14 (and 1.15, 1.16, etc., when those
> > become available), then we should release Emacs 29 with 1.14.
> 
> I think it's imperative to _allow_ -- as you say -- and also and
> to _make easy_.

"C-u M-x package-install" is easy enough.

> More importantly, and to the point, to _make it as easy as it was in
> Emacs 26, 27 and 28_.

That's an impractical request, one that most probably prefers Eglot
users (and only part of them at that) to those of other core packages.
We must think about all the users, not just users of Eglot.  The price
of adaptation to the fact that Eglot is now a core package, while it
is not nil, is not high.  So once again, this solution is IMO a
reasonable compromise, given the constraints.

> > Philip presented such a safe modification, and we are in the final
> > stages of discussing its details, before it will be installed.  So
> > yes, it is possible.
> 
> As I've explained to Philip, the big drawback of that -- undoubtedly
> safe -- modification is that it is not compatible to user's
> configurations that have a (use-package 'eglot) or a
> (package-install 'eglot) in them.

That is not a problem serious enough to invalidate the solution.  We
can, for example, mention the change in NEWS, to alleviate some of the
costs.