Re: Reconsider defaults for use-package-vc-prefer-newest

[Philip Kaludercic <philipk@posteo.net>]

"Martin Edström" <meedstrom@runbox.eu> writes:

> Previous discussion: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=67955
>
> Currently, the setting of `use-package-vc-prefer-newest` defaults to nil.
> I want to bring up some problems with that, and suggest fixes.

Sorry for the late response, I'm responsible for this decision and would
like to defend it here.  Feel free to CC me in future discussions
pertaining to package-vc.  I have also added Tony to the CCs, as he has
a better understanding of the use-package side of this.

> PROBLEM 1. MISMATCH WITH PACKAGE-VC
>
> Now use-package :vc is not in line with the behavior of the
> package-vc-install command, which fetches the newest commit.
>
> All other package managers I know of also fetch the newest commit.

The main reference is package.el, which by default uses GNU ELPA and
NonGNU ELPA.  Both distribute explicitly released packages, as indicated
by the version header.  To me this has a much greater weight than what
third-party package managers and archives decide to do.

> It is the new use-package :vc which is the odd one out in the
> ecosystem, and that can cause stability issues, which I go into
> more deeply on this Reddit comment:
>
> https://old.reddit.com/r/emacs/comments/1f8ok7c/do_you_set_packageinstallupgradebuiltin_to_t/llimwe0/

>
> PROBLEM 2. "LAST RELEASE" CANNOT BE ASSUMED TO BE THE LAST RELEASE
>
> I see many packages that have not bumped their Package-Version
> header for years.  Perhaps they bump by git tags, or just stopped
> bumping versions.  I understand them, since it is a lot of manual
> labor to bump it, cut a release commit, then bump again to append a
> "-pre" or a ".0.50-git", and to remember to do this everywhere the
> version string may occur, like in docs.
>
> I'm also guessing there's a segment of developers that have sensed
> that this labor does not end up helping users, so they are not
> motivated to keep doing it.  Versions only work as intended if
> everyone is versioning regularly, and that is possible to enforce
> in a curated repo like GNU Elpa or Debian's repackaged Emacs
> packages, but as soon as you also install packages outside of that
> repo (and well, most packages are outside), there's no longer any
> guarantee and in fact the outdated versions in the curated repo are
> likely to cause breakage.
>
> I have personally had to give up on things like ELPA, Melpa Stable,
> Debian packages etc.  It always caused instability.  By contrast,
> installing 200-400 packages off Melpa, in other words the latest
> dev snapshots, has caused me no issues in ~7 years.  So there are
> two islands of stability:
>
> 1. Stay inside a curated repo entirely
> 2. Leave the curated repo entirely and install only dev snapshots
>
> Now consider what use-package :vc is for.  It is for installing
> things OUTSIDE any curated repo.  I leave it to you to pick a
> default setting for `use-package-vc-prefer-newest`.

No, this is an unjustified assumption.  FWIW my intention of writing
package-vc was to make contributing to packages easier.  The reason the
"install latest release" option exists for package-vc is that it can
make it easier to fix a bug with the current version.

The thing is that with use-package, having a more deterministic
installation is of interest, especially since use-package is used to
automatically set up an Emacs environment.  I do not share your
experience that non-stable MELPA is more stable than explicitly released
ELPA packages.

Regarding the main point of uncurrated packages: Users can circumvent
this by just upgrading the packages by hand, so I don't think it is that
bad.  That being said, I do think that the pressure to maintain packages
and cut versions from time to time is a good thing.  Here again, I don't
think that we should feel pressured by third-party repositories like
MELPA, that have been ignoring guidelines like in (elisp) Simple
Packages, that plainly state

        The version number comes from the ‘Package-Version’ header, if
        it exists, or from the ‘Version’ header otherwise.  One or the
        other _must_ be present.

Package-vc is related to ELPA and follows ELPA's practices.  The reason
that package specifications look the way they do is to encourage
contributing packages to ELPA.

> PROBLEM 3. FORCED REPO REWRITES
>
> I have been forced to use git-filter-repo to rewrite my package's
> commit history, to erase all instances of the Package-Version
> header.
>
> That's because I had been content to leave the header permanently
> at "0.6.0.50-git".  If I kept doing that with the new use-package
> :vc in existence, it would cause users to download a horribly
> outdated version of my package. If I understand correctly.
>
> I foresee one of two things will happen in the future:
>
> 1. More and more package authors carry out the same rewrite I did
> 2. (more likely) Increased instability as package authors do NOT do
>    this rewrite, and users end up with ancient versions all over
>    the place

It is well known that overwriting published git commits is bad practice,
which is why git push --force (or variations) is a thing.

The third option is that package maintainers become aware of the issue,
and start updating the header to mean something.

A feature we can add in the future is to support ELPA's :rolling-release
attribute, so that packages that wish to ensure that the default branch
is stable with every commit can do so without having to do tricks like
custom time-stamp file local variables that update every time the file
is modified.

>
> SUGGESTED FIXES
>
> A few possibilities:
>
> 1. Change `use-package-vc-prefer-newest` to default to t.
>
> 2. Change package-vc so when it walks the commit history to find
>    the "last release" or any specific version, it should target the
>    newest commit that has the given Package-Version, not the oldest
>    commit.
>
>    - That would allow devs to continue having a frozen
>      Package-Version without breaking users' setups.
>
>    - Of course, this is effectively the same as option 1, but the
>      semantic distinction might be usable in the future to
>      e.g. ignore version strings that have a "-pre" inside.

I am afraid I don't understand what you are suggesting here.  The
current approach is to find the newest commit that touches a version
header and use that.

> 3. Roll out a strategy for moving the entire community onto the
>    norm of versioning packages prior to distribution, ideally to
>    the extent that Melpa is no longer necessary and can go offline
>    in favor of Melpa Stable.  (Until then, Melpa Stable is best
>    avoided: https://old.reddit.com/r/emacs/comments/etikbz/)
>
>    Obviously difficult, but as I've said, versioning only works if
>    everyone does it.  Specifically, it gives us stability if every
>    developer is running the "released version" of most packages
>    other than their own.  Currently we have an ecosystem where most
>    developers are running the dev snapshot of most packages, and
>    any half-measures to change this will only destroy the islands
>    of stability that we do currently enjoy.  The topic of this
>    email is one such half-measure.

Everyone on {GNU,NonGNU} ELPA does it and it works fine.  There is
little we can do about third-party projects, so I don't concern myself
about them.  This is Emacs after all, you have the freedom to break
anything you want, and "THERE IS NO WARRANTY FOR THE PROGRAM" as the GPL
says.

> 4. Have Emacs ship an automation tool like
>    https://github.com/magit/sisyphus, recommend it everywhere, then
>    have package.el, package-vc and use-package :vc all throw an
>    error on finding more than one commit with a given
>    Package-Version string and refuse to install due to ambiguity.

Again, I am afraid we are talking past one another.  Are you suggesting
that the version number should be changed from commit to commit?

> 5. Deprecate the Package-Version header and instead maybe let
>    package.el perform an automatic string substitution from the
>    git/hg tag, if it is still desirable to see the version in the
>    source file.  The version could also be substituted into Info
>    docs and other places the version string should occur.
>
>    - This is effectively a convenient alternative to option 4.

IIRC this is not possible as to limitations with git tags that doesn't
allow us to mirror them in elpa.git.

-- 
	Philip Kaludercic on siskin