Re: Reconsider defaults for use-package-vc-prefer-newest

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edström]

> I should also point out that the module it was inspired by, vc-use-package, actually had the opposite default!

At least, I infer that it had the opposite default, going by its README (https://github.com/slotThe/vc-use-package), since it sounds like one had to explicitly pass :last-release to get it to download an old commit.

Re: Reconsider defaults for use-package-vc-prefer-newest

[chad]

[-- Attachment #1: Type: text/plain, Size: 223 bytes --]

Please forgive me if I misunderstand the situation, but I wonder if it
wouldn't be possible to suggest a patch that leaves the conservative
default but detects the potential "catastrophe" and warns the user?

Thanks,
~Chad

[-- Attachment #2: Type: text/html, Size: 308 bytes --]

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edstrom]

> Please forgive me if I misunderstand the situation, but I wonder if it
> wouldn't be possible to suggest a patch that leaves the conservative
> default but detects the potential "catastrophe" and warns the user?
> 
> Thanks,
> ~Chad

The "catastrophe" would be a situation such as:

- In 2020, Developer releases Package for the first time
- In 2021, Developer tires of bumping Package-Version, leaves it at 0.9
- In 2024, Package is now at 2.2 according to the convenient git/hg tag, or maybe it has no official version beyond just "0.9.0.50-git"
- User installs Package using (use-package :vc)
- User gets the version from 2021
- It doesn't seem to work
- 10 such users give up on Package
- The 11th user files the bug reports
- Bug reports are strange and make no sense
- User and Developer finally figure out that it's because the user used (use-package :vc) which fetched the 2021 version

To counteract this would amount to a heuristic that looks up the package repository online and compares the age of recent commits with the commit fetched by (use-package :vc), but what is the threshold that you would set to trigger the warning? I don't think it is realistically doable.

Re: Reconsider defaults for use-package-vc-prefer-newest

[chad]

[-- Attachment #1: Type: text/plain, Size: 2179 bytes --]

On Sun, Sep 15, 2024 at 5:09 PM Martin Edstrom <meedstrom@runbox.eu> wrote:

> The "catastrophe" would be a situation such as:
>
> - In 2020, Developer releases Package for the first time
> - In 2021, Developer tires of bumping Package-Version, leaves it at 0.9
> - In 2024, Package is now at 2.2 according to the convenient git/hg tag,
> or maybe it has no official version beyond just "0.9.0.50-git"
> - User installs Package using (use-package :vc)
> - User gets the version from 2021
> [...]


You're describing the situation in terms of theoretical user results, but
that's not how the code works, obviously. At some point, the code checks
the user's value of use-package-vs-prefer-newest, sees whether it's the
default or has been customized, and decides to fetch the new package
contents or not. My rough understanding is, *in the non-VC case*, it
downloads new archive-contents for the (each) entire archive regardless,
and then I'm guessing that it currently uses the setting to determine
whether to get a new version or not.

I don't know how the code path in the (use-package :vc) works, but
presumably there's a clear point where the two behaviors (that is,
vc-use-package's supposed former update-by-default versus use-package :vc's
...prefer-newer 'nil lack of update) diverge. I'm asking if there's a
reasonable way to change the default behavior from the supposed-old
yes-update and the current default wait-for-Package-version into something
that leaves the package alone but warns the user, perhaps via *Messages*.

Put another way, your concern seems to be that people might not _realize_
that they're "stuck" on an older Package-Version revision. The alternative
package "manager" code for emacs suggests that there is desire both for
version pinning and direct-from-vcs-packages, so it's likely that there are
users who prefer both alternatives, and emacs is often default-conservative
in such matters, but it seems like it might be possible to have use-package
do a little extra work in the default case to address the "not realize"
part, rather than changing the updating behavior itself.

Hope that helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 2670 bytes --]

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edstrom]

On Sun, 15 Sep 2024 18:12:00 -0400, chad <yandros@gmail.com> wrote:

> I don't know how the code path in the (use-package :vc) works, but
> presumably there's a clear point where the two behaviors (that is,
> vc-use-package's supposed former update-by-default versus use-package :vc's
> ...prefer-newer 'nil lack of update) diverge. I'm asking if there's a
> reasonable way to change the default behavior from the supposed-old
> yes-update and the current default wait-for-Package-version into something
> that leaves the package alone but warns the user, perhaps via *Messages*.

If I understand correctly, you suggest that a message be printed when there is any new commit since the last Package-Version?

Something like:

"310 packages installed, 237 have new commits"

But I think people would just learn to ignore this message...

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: "Martin Edstrom" <meedstrom@runbox.eu>
> CC: "emacs-devel" <emacs-devel@gnu.org>
> Date: Sun, 15 Sep 2024 23:09:05 +0200 (CEST)
> 
> The "catastrophe" would be a situation such as:
> 
> - In 2020, Developer releases Package for the first time
> - In 2021, Developer tires of bumping Package-Version, leaves it at 0.9
> - In 2024, Package is now at 2.2 according to the convenient git/hg tag, or maybe it has no official version beyond just "0.9.0.50-git"

If the problem is that there's a tag with the correct version, but the
Package-Version heading was not updated, we could perhaps have
use-package :vc detect that and either display a warning or even
automatically use the commit with the tag.  Crucially, this is NOT the
latest commit in the Git repository, it is the commit which has the
latest release tag -- a far cry from what you suggested originally
(apologies if I misunderstood you back then.)

> - User installs Package using (use-package :vc)
> - User gets the version from 2021
> - It doesn't seem to work
> - 10 such users give up on Package
> - The 11th user files the bug reports
> - Bug reports are strange and make no sense
> - User and Developer finally figure out that it's because the user used (use-package :vc) which fetched the 2021 version

In any case, I wonder why the above mess you describe is not the fault
of the package developer, but of Emacs?  It makes little sense to be,
TBH.

> To counteract this would amount to a heuristic that looks up the package repository online and compares the age of recent commits with the commit fetched by (use-package :vc), but what is the threshold that you would set to trigger the warning? I don't think it is realistically doable.

Why not?  Detecting such suspicious "last versions" could be a good
idea and a good service to users, and is not that hard to implement.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edström]

On Mon, 16 Sep 2024 14:50:51 +0300, Eli Zaretskii <eliz@gnu.org> wrote:
 
> If the problem is that there's a tag with the correct version, but the
> Package-Version heading was not updated, we could perhaps have
> use-package :vc detect that and either display a warning or even
> automatically use the commit with the tag.  Crucially, this is NOT the
> latest commit in the Git repository, it is the commit which has the
> latest release tag -- a far cry from what you suggested originally
> (apologies if I misunderstood you back then.)

I agree it's a good way forward if you are open to look up both the Package-Version and the git tag and basically choose the highest-looking version between the two.  I had somehow ruled out that option.

What I suggested was, technically, to get the latest commit *with a given Package-Version*.

Which need not be the same thing as the latest commit. 

The latest commit could have a pre-release version like "0.6-pre", and so (I assume...? Not sure) Emacs would walk back until finding the commit with "0.6" and check out that one.

It still looks necessary to me to do the change I suggested, since there exists a population of developers who either believe that's how versions work, or who do not really bump versions anyway. Since you can't educate everyone, this is a situation for picking defaults that handle this reality.

At worst, a package can have just ";; Package-Version: 0" since the very first commit, and no tag. I do not think that in this situation, the correct move for use-package :vc is to install literally the first commit, when it would be the very first package manager to behave in such a way.  

It would be a deliberate introduction of breakage in the ecosystem, mainly harming users. It does not matter if the packagers are not doing The Right Thing, you gotta protect the users.

Please consider what use-package :vc is for in the first place. It wouldn't be used for things that exist on [Non]GNU ELPA, but for what exists off it, so its behavior should not be modeled on what you can expect from packages hosted there. On the contrary, its behavior can't expect any norms to be followed. It's not necessarily incompetence nor disagreement about procedure; many novice developers also take their first baby steps in this wilderness.

Of course it is possible to use use-package :vc to install what's on the ELPA, but the crowd that cares about running stable releases are going to be the last ones to choose to do that.

I can see an user switching to use-package :vc globally, but it'd be precisely because they want the bleeding edge version of everything, including the stuff on [Non]GNU ELPA.  If they don't want it, they can simply elect not do so for the stuff on [Non]GNU ELPA.

> > - User installs Package using (use-package :vc)
> > - User gets the version from 2021
> > - It doesn't seem to work
> > - 10 such users give up on Package
> > - The 11th user files the bug reports
> > - Bug reports are strange and make no sense
> > - User and Developer finally figure out that it's because the user used (use-package :vc) which fetched the 2021 version
> 
> In any case, I wonder why the above mess you describe is not the fault
> of the package developer, but of Emacs?  It makes little sense to be,
> TBH.

I don't believe in blaming, but Emacs definitely has some of the responsibility for the simple reason that this didn't use to happen and people do not expect it to happen.

A house of cards falls when someone shakes the table, but it isn't the fault of the one who put it together, it is who shook the table when they could instead have permitted that artwork to stand.

After knowing how this new Emacs builtin behaves, you can consider it to be the fault of the developer who did not take it into account, at least if it is reasonable to expect all developers to be professionals that know how every package manager works instead of only most of them, having missed this one, but even then a crowd will resent this addition to their workload, as I have explained prior. 

My own packaging experience would be unharmed if git tags become a valid version identifier, but that just affects my packages.  I'd like the ecosystem to have guarantees in place, and blaming devs because the fault is theirs will not make them do The Right Thing.  The right thing is that Emacs just decline to shake the table.

> 
> > To counteract this would amount to a heuristic that looks up the package repository online and compares the age of recent commits with the commit fetched by (use-package :vc), but what is the threshold that you would set to trigger the warning? I don't think it is realistically doable.
> 
> Why not?  Detecting such suspicious "last versions" could be a good
> idea and a good service to users, and is not that hard to implement.

But how would you do it? All it takes is a single commit to introduce changes on which another package will rely.  It can be as little as one minute between the commits in question.  Or conversely, a package may be mostly unchanged for years but that does not mean there is a problem. So that brings us to a bunch of UX issues to answer.  Should users "dismiss" each warning individually?  (Bear in mind there could be hundreds.)  Should these dismissals be recorded in `custom-file` or some such location?  It seems like a lot of effort to do well compared to my other suggestions, but that's just my impression.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: "Martin Edström" <meedstrom@runbox.eu>
> CC: "yandros" <yandros@gmail.com>, "emacs-devel" <emacs-devel@gnu.org>
> Date: Mon, 16 Sep 2024 18:46:48 +0200 (CEST)
> 
> On Mon, 16 Sep 2024 14:50:51 +0300, Eli Zaretskii <eliz@gnu.org> wrote:
>  
> > If the problem is that there's a tag with the correct version, but the
> > Package-Version heading was not updated, we could perhaps have
> > use-package :vc detect that and either display a warning or even
> > automatically use the commit with the tag.  Crucially, this is NOT the
> > latest commit in the Git repository, it is the commit which has the
> > latest release tag -- a far cry from what you suggested originally
> > (apologies if I misunderstood you back then.)
> 
> I agree it's a good way forward if you are open to look up both the Package-Version and the git tag and basically choose the highest-looking version between the two.  I had somehow ruled out that option.
> 
> What I suggested was, technically, to get the latest commit *with a given Package-Version*.
> 
> Which need not be the same thing as the latest commit. 

It is also not necessarily a stable commit.  It could have bugs.
Installing the commit with a Package-Version heading means we don't
subject users to the risk of getting a bad version, on the assumption
that bumping a version means the developer announces the code as more
or less stable.  Looking for a commit with a release tag has the same
advantage, whereas what you suggest doesn't.

> The latest commit could have a pre-release version like "0.6-pre", and so (I assume...? Not sure) Emacs would walk back until finding the commit with "0.6" and check out that one.

This is indeed a risk, and this is why we prefer to use
Package-Version.  But we could also try to look at release tags, at
least as an option.

> > > To counteract this would amount to a heuristic that looks up the package repository online and compares the age of recent commits with the commit fetched by (use-package :vc), but what is the threshold that you would set to trigger the warning? I don't think it is realistically doable.
> > 
> > Why not?  Detecting such suspicious "last versions" could be a good
> > idea and a good service to users, and is not that hard to implement.
> 
> But how would you do it?

By looking at commit dates and showing a warning or informational
message.

> All it takes is a single commit to introduce changes on which another package will rely.  It can be as little as one minute between the commits in question.

One minute ago is not outdated in any sense of the word.  In any case,
this is just a warning, meant to draw the user's attention to
something that he/she should look into.  What to do with that is
entirely up to the user.

> Or conversely, a package may be mostly unchanged for years but that does not mean there is a problem. So that brings us to a bunch of UX issues to answer.  Should users "dismiss" each warning individually?  (Bear in mind there could be hundreds.)  Should these dismissals be recorded in `custom-file` or some such location?  It seems like a lot of effort to do well compared to my other suggestions, but that's just my impression.

Showing a message with a 1-sec sit-for is the usual technique.  The
messages are logged in *Messages*, so users could look them up later.
I don't see a significant problem here, sorry.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Suhail Singh]

Eli Zaretskii <eliz@gnu.org> writes:

> If the problem is that there's a tag with the correct version, but the
> Package-Version heading was not updated, we could perhaps have
> use-package :vc detect that and either display a warning or even
> automatically use the commit with the tag.

FWIW, a situation I have encountered is when the commit corresponding to
a release tag is not the same commit that introduced the version in the
package header.  This can come up, for instance, when a package is
available both in melpa-stable and nongnu.  The former uses git tags for
versions whereas the latter uses the package header.  However, while the
version of the packages may nominally be the same, they may correspond
to different commits.  I.e., having two different sources of version
information can result in ambiguity.

When the two notions of version don't correspond to the same commit, it
would be helpful if the user could be notified in some way.

-- 
Suhail

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: Suhail Singh <suhailsingh247@gmail.com>
> Cc: "Martin Edstrom" <meedstrom@runbox.eu>,  yandros@gmail.com,
>   emacs-devel@gnu.org
> Date: Mon, 16 Sep 2024 16:16:03 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If the problem is that there's a tag with the correct version, but the
> > Package-Version heading was not updated, we could perhaps have
> > use-package :vc detect that and either display a warning or even
> > automatically use the commit with the tag.
> 
> FWIW, a situation I have encountered is when the commit corresponding to
> a release tag is not the same commit that introduced the version in the
> package header.  This can come up, for instance, when a package is
> available both in melpa-stable and nongnu.  The former uses git tags for
> versions whereas the latter uses the package header.  However, while the
> version of the packages may nominally be the same, they may correspond
> to different commits.  I.e., having two different sources of version
> information can result in ambiguity.
> 
> When the two notions of version don't correspond to the same commit, it
> would be helpful if the user could be notified in some way.

Yes, that'd be another reason to alert the user.

Patches to that effect are welcome.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Suhail Singh]

Eli Zaretskii <eliz@gnu.org> writes:

>> When the two notions of version don't correspond to the same commit, it
>> would be helpful if the user could be notified in some way.
>
> Yes, that'd be another reason to alert the user.
>
> Patches to that effect are welcome.

I suppose if the output of M-x describe-package mentioned the commit
hash in addition to the version the user would at least be aware when
the same version numbers from different archives differ.  Out of
curiosity, is that what you had in mind, or something else?

Displaying the commit date in addition to the commit hash could also be
helpful.

-- 
Suhail

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: Suhail Singh <suhailsingh247@gmail.com>
> Cc: Suhail Singh <suhailsingh247@gmail.com>,  meedstrom@runbox.eu,
>  yandros@gmail.com,  emacs-devel@gnu.org
> Date: Wed, 18 Sep 2024 23:38:08 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> When the two notions of version don't correspond to the same commit, it
> >> would be helpful if the user could be notified in some way.
> >
> > Yes, that'd be another reason to alert the user.
> >
> > Patches to that effect are welcome.
> 
> I suppose if the output of M-x describe-package mentioned the commit
> hash in addition to the version the user would at least be aware when
> the same version numbers from different archives differ.  Out of
> curiosity, is that what you had in mind, or something else?

No, I meant exactly what you described: to detect the situation where
the commit which introduced the version number is not the same as the
commit with the version tag.  They are supposed to be the same commit.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Suhail Singh]

Eli Zaretskii <eliz@gnu.org> writes:

> No, I meant exactly what you described: to detect the situation where
> the commit which introduced the version number is not the same as the
> commit with the version tag.  They are supposed to be the same commit.

Two questions:

1. When would it make sense for this check to be triggered?

2. Who should be notified (and how) of this when this discrepancy is
   detected?

Ideally, the package authors / maintainers would be notified directly
via the GNU and NonGNU ELPA archives.  Or was your recommendation to
notify the users of the package upon install time, instead?  If former,
are there already some set of checks / tests run by the archives, where
such an additional check could be added?

-- 
Suhail

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: Suhail Singh <suhailsingh247@gmail.com>
> Cc: Suhail Singh <suhailsingh247@gmail.com>,  meedstrom@runbox.eu,
>   yandros@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 19 Sep 2024 08:08:13 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > No, I meant exactly what you described: to detect the situation where
> > the commit which introduced the version number is not the same as the
> > commit with the version tag.  They are supposed to be the same commit.
> 
> Two questions:
> 
> 1. When would it make sense for this check to be triggered?

I guess when we see version tags in the repository?

> 2. Who should be notified (and how) of this when this discrepancy is
>    detected?

The user (for example, by issuing a warning).

> Ideally, the package authors / maintainers would be notified directly
> via the GNU and NonGNU ELPA archives.  Or was your recommendation to
> notify the users of the package upon install time, instead?  If former,
> are there already some set of checks / tests run by the archives, where
> such an additional check could be added?

I meant the users.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edström]

Now that we're on Emacs 30.0.91, I feel the need to call this out as a fairly release-critical bug. Perhaps I should have emailed bug-gnu-emacs, but now that'd split the discussion over two places, so I'll keep it here.

If this is the kind of setting we can flip in a patch release, then there's no rush. Maybe someone can weigh in about that.

Otherwise it would remain as a source of instability for many years, and conscientious devs would have to insert a check for Emacs 30 that reminds the user that the package may be terribly outdated.

At least one other developer agrees the current setting is "fragile" (https://github.com/melpa/melpa/pull/9133#issuecomment-2351653325).

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: "Martin Edström" <meedstrom@runbox.eu>
> CC: "emacs-devel" <emacs-devel@gnu.org>
> Date: Sun, 15 Sep 2024 19:38:55 +0200 (CEST)
> 
> Now that we're on Emacs 30.0.91, I feel the need to call this out as a fairly release-critical bug. Perhaps I should have emailed bug-gnu-emacs, but now that'd split the discussion over two places, so I'll keep it here.
> 
> If this is the kind of setting we can flip in a patch release, then there's no rush. Maybe someone can weigh in about that.
> 
> Otherwise it would remain as a source of instability for many years, and conscientious devs would have to insert a check for Emacs 30 that reminds the user that the package may be terribly outdated.
> 
> At least one other developer agrees the current setting is "fragile" (https://github.com/melpa/melpa/pull/9133#issuecomment-2351653325).

I've read this thread, and I must confess that I'm not convinced.
Besides being waaay too late for such changes in Emacs 30, I also
don't think we have enough experience at this time to make such
decisions.  Both package-vc and use-package are relatively very new in
Emacs, and we have yet to collect enough user experience and
understanding of what are the expectations and how they should work
together.  Moreover, use-package-vc-prefer-newest is a new option in
Emacs 30, and we rarely make new optional behaviors take effect by
default as soon as they are introduced, to avoid behavior changes
which will surprise and perhaps annoy.  We made it an option to make
it easily adaptable to user preferences and needs.

Some of the rationale (like outdated Package-Version) also sounds like
it isn't our problem to solve.  It should be easy for the respective
package developers to get their act together in this matter.  It is
also TRT for them.  When a user uses use-package, he/she doesn't
necessarily want the latest commit, so the change isn't as obvious as
you make it sound, IMO.

Bottom line: I think no catastrophe happens if Emacs 30 is released
with the current default value of use-package-vc-prefer-newest.  It's
a user option, so customizing it is easy for people who want the
latest commit.  Even if we are making a mistake (for reasons we will
learn later), first time you do something there are always some rough
edges, so we can be excused for not realizing something due to lack of
experience.

Thanks.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edstrom]

> > From: "Martin Edström" <meedstrom@runbox.eu>
> > CC: "emacs-devel" <emacs-devel@gnu.org>
> > Date: Sun, 15 Sep 2024 19:38:55 +0200 (CEST)
> > 
> > Now that we're on Emacs 30.0.91, I feel the need to call this out as a fairly release-critical bug. Perhaps I should have emailed bug-gnu-emacs, but now that'd split the discussion over two places, so I'll keep it here.
> > 
> > If this is the kind of setting we can flip in a patch release, then there's no rush. Maybe someone can weigh in about that.
> > 
> > Otherwise it would remain as a source of instability for many years, and conscientious devs would have to insert a check for Emacs 30 that reminds the user that the package may be terribly outdated.
> > 
> > At least one other developer agrees the current setting is "fragile" (https://github.com/melpa/melpa/pull/9133#issuecomment-2351653325).
> 
> I've read this thread, and I must confess that I'm not convinced.
> Besides being waaay too late for such changes in Emacs 30, I also
> don't think we have enough experience at this time to make such
> decisions.  Both package-vc and use-package are relatively very new in
> Emacs, and we have yet to collect enough user experience and
> understanding of what are the expectations and how they should work
> together.  Moreover, use-package-vc-prefer-newest is a new option in
> Emacs 30, and we rarely make new optional behaviors take effect by
> default as soon as they are introduced, to avoid behavior changes
> which will surprise and perhaps annoy.  We made it an option to make
> it easily adaptable to user preferences and needs.
> 
> Some of the rationale (like outdated Package-Version) also sounds like
> it isn't our problem to solve.  It should be easy for the respective
> package developers to get their act together in this matter.  It is
> also TRT for them.  When a user uses use-package, he/she doesn't
> necessarily want the latest commit, so the change isn't as obvious as
> you make it sound, IMO.
> 
> Bottom line: I think no catastrophe happens if Emacs 30 is released
> with the current default value of use-package-vc-prefer-newest.  It's
> a user option, so customizing it is easy for people who want the
> latest commit.  Even if we are making a mistake (for reasons we will
> learn later), first time you do something there are always some rough
> edges, so we can be excused for not realizing something due to lack of
> experience.
> 
> Thanks.

Thanks for engaging!

(What is TRT?)

While the option is a new thing in Emacs 30, use-package :vc itself is a new thing in Emacs 30, so it is not as if there is an old behavior to emulate.  I should also point out that the module it was inspired by, vc-use-package, actually had the opposite default!  So, that setting has already been tested by lots of users on Emacs <29, and it is Emacs 30 that will change things.

Anyway. If that's not good enough, maybe we can test the new setting in Emacs 31, and backport to Emacs 30.2 in the future.

I should also point out that the catastrophe occurs not at release time, but years afterwards, when we're on Emacs 31, 32, 33... but devs still want to support Emacs 30, getting worse with time. So I do hope that it will be possible to change a setting like this with Emacs 30.2 or some other "bugfix" release.

As for making devs get their act together, sure, they could do that. But three problems with that attitude:

1. It makes sense to impose requirements on devs who are submitting packages to NonGNU Elpa, but this setting affects everyone, including those who have not opted in to such requirements.
2. Not every dev will get the memo, naturally, and the ones who get hurt in the meantime are users, who believe that the dev's package is broken when it is not (and the dev should not be punished for being out of the loop).
3. The devs who do get the memo, and were previously content with a frozen Package-Version, will resent GNU for forcing what they perceive as a workaround. I do not think that is worth it.  Like it or not, MELPA has allowed many of us to taste of the convenience of git/hg tags, and it didn't use to be a problem... until Emacs 30. Now they have to adopt some toolchain like sisyphus.el and pollute the commit log with two extra commits for every new version, to solve what used to be a non-issue.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: "Martin Edstrom" <meedstrom@runbox.eu>
> CC: "emacs-devel" <emacs-devel@gnu.org>
> Date: Sun, 15 Sep 2024 21:46:32 +0200 (CEST)
> 
> > Bottom line: I think no catastrophe happens if Emacs 30 is released
> > with the current default value of use-package-vc-prefer-newest.  It's
> > a user option, so customizing it is easy for people who want the
> > latest commit.  Even if we are making a mistake (for reasons we will
> > learn later), first time you do something there are always some rough
> > edges, so we can be excused for not realizing something due to lack of
> > experience.
> > 
> > Thanks.
> 
> Thanks for engaging!
> 
> (What is TRT?)

The Right Thing

> While the option is a new thing in Emacs 30, use-package :vc itself is a new thing in Emacs 30, so it is not as if there is an old behavior to emulate.  I should also point out that the module it was inspired by, vc-use-package, actually had the opposite default!  So, that setting has already been tested by lots of users on Emacs <29, and it is Emacs 30 that will change things.

Yes, you said that in your original message, and I understand that
argument.  But as I responded, I'm not convinced the two options must
always be in sync, as they are used differently for different purposes.

> Anyway. If that's not good enough, maybe we can test the new setting in Emacs 31, and backport to Emacs 30.2 in the future.

Yes, that would be one way forward.  Especially if enough users come
to us asking to flip the default, and explain why.

> I should also point out that the catastrophe occurs not at release time, but years afterwards, when we're on Emacs 31, 32, 33... but devs still want to support Emacs 30, getting worse with time. So I do hope that it will be possible to change a setting like this with Emacs 30.2 or some other "bugfix" release.

There's no reason to do this for user options, since customizing an
option if the user doesn't like its default is easy.

> As for making devs get their act together, sure, they could do that. But three problems with that attitude:
> 
> 1. It makes sense to impose requirements on devs who are submitting packages to NonGNU Elpa, but this setting affects everyone, including those who have not opted in to such requirements.
> 2. Not every dev will get the memo, naturally, and the ones who get hurt in the meantime are users, who believe that the dev's package is broken when it is not (and the dev should not be punished for being out of the loop).
> 3. The devs who do get the memo, and were previously content with a frozen Package-Version, will resent GNU for forcing what they perceive as a workaround. I do not think that is worth it.  Like it or not, MELPA has allowed many of us to taste of the convenience of git/hg tags, and it didn't use to be a problem... until Emacs 30. Now they have to adopt some toolchain like sisyphus.el and pollute the commit log with two extra commits for every new version, to solve what used to be a non-issue.

I consider advancing Package-Version when significant changes are made
a simple and uncontroversial thing to do for any package developer.  I
cannot imagine why someone who wants their package in good shape to
regard this as some annoyance.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edström]

On Mon, 16 Sep 2024 14:34:55 +0300, Eli Zaretskii <eliz@gnu.org> wrote:

> > While the option is a new thing in Emacs 30, use-package :vc itself is a new thing in Emacs 30, so it is not as if there is an old behavior to emulate.  I should also point out that the module it was inspired by, vc-use-package, actually had the opposite default!  So, that setting has already been tested by lots of users on Emacs <29, and it is Emacs 30 that will change things.
> 
> Yes, you said that in your original message, and I understand that
> argument.  But as I responded, I'm not convinced the two options must
> always be in sync, as they are used differently for different purposes.

Okay, I'll accept that for the command package-vc-install as I have not thought much about its use cases.  I want to call attention to the old use-package :vc, shippedwhich had the confusing vc-use-package [1].  The Emacs 30 use-package :vc is a drop-in 1:1 replacement  This thing:    It had the opposite default from the one Emacs 30 is about to have.

> 
> > Anyway. If that's not good enough, maybe we can test the new setting in Emacs 31, and backport to Emacs 30.2 in the future.
> 
> Yes, that would be one way forward.  Especially if enough users come
> to us asking to flip the default, and explain why.
> 
> > I should also point out that the catastrophe occurs not at release time, but years afterwards, when we're on Emacs 31, 32, 33... but devs still want to support Emacs 30, getting worse with time. So I do hope that it will be possible to change a setting like this with Emacs 30.2 or some other "bugfix" release.
> 
> There's no reason to do this for user options, since customizing an
> option if the user doesn't like its default is easy.
> 
> > As for making devs get their act together, sure, they could do that. But three problems with that attitude:
> > 
> > 1. It makes sense to impose requirements on devs who are submitting packages to NonGNU Elpa, but this setting affects everyone, including those who have not opted in to such requirements.
> > 2. Not every dev will get the memo, naturally, and the ones who get hurt in the meantime are users, who believe that the dev's package is broken when it is not (and the dev should not be punished for being out of the loop).
> > 3. The devs who do get the memo, and were previously content with a frozen Package-Version, will resent GNU for forcing what they perceive as a workaround. I do not think that is worth it.  Like it or not, MELPA has allowed many of us to taste of the convenience of git/hg tags, and it didn't use to be a problem... until Emacs 30. Now they have to adopt some toolchain like sisyphus.el and pollute the commit log with two extra commits for every new version, to solve what used to be a non-issue.
> 
> I consider advancing Package-Version when significant changes are made
> a simple and uncontroversial thing to do for any package developer.  I
> cannot imagine why someone who wants their package in good shape to
> regard this as some annoyance.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edström]

Sorry the last email was incomplete, my keyboard has a bug.

On Mon, 16 Sep 2024 14:34:55 +0300, Eli Zaretskii <eliz@gnu.org> wrote:

> > While the option is a new thing in Emacs 30, use-package :vc itself is a new thing in Emacs 30, so it is not as if there is an old behavior to emulate.  I should also point out that the module it was inspired by, vc-use-package, actually had the opposite default!  So, that setting has already been tested by lots of users on Emacs <29, and it is Emacs 30 that will change things.
> 
> Yes, you said that in your original message, and I understand that
> argument.  But as I responded, I'm not convinced the two options must
> always be in sync, as they are used differently for different purposes.

Okay, I'll accept that for the command package-vc-install, as I have not thought much about its use cases.  

I want to call attention to the old use-package :vc, shipped under the confusing name vc-use-package [1], that people could install on Emacs 29 and earlier.  The Emacs 30 use-package :vc is a drop-in almost 1:1 replacement for this thing.  It had the opposite default behavior from the default Emacs 30 is about to have.  That change needs justification, not the other way around!

[1] https://github.com/slotThe/vc-use-package

> > I should also point out that the catastrophe occurs not at release time, but years afterwards, when we're on Emacs 31, 32, 33... but devs still want to support Emacs 30, getting worse with time. So I do hope that it will be possible to change a setting like this with Emacs 30.2 or some other "bugfix" release.
> 
> There's no reason to do this for user options, since customizing an
> option if the user doesn't like its default is easy.

You cannot rely on users to configure this kind of thing for themselves, most of them would not know the reason why things broke. It is not a matter of "liking" the default, since breakage in one direction would be much more severe than is possible to encounter in the other direction (years out of date vs. slightly too new).

It is absolutely Emacs' responsibility, at least as long as stability for users is desired. Isn't it the reason to favor conservative defaults? Avoiding breakage? I get this new default comes from a good place, intending stability, but it is a radical departure from what has been the norm established by Straight/Quelpa/el-get and their use-package integrations, and that will work against stability.  

> 
> > As for making devs get their act together, sure, they could do that. But three problems with that attitude:
> > 
> > 1. It makes sense to impose requirements on devs who are submitting packages to NonGNU Elpa, but this setting affects everyone, including those who have not opted in to such requirements.
> > 2. Not every dev will get the memo, naturally, and the ones who get hurt in the meantime are users, who believe that the dev's package is broken when it is not (and the dev should not be punished for being out of the loop).
> > 3. The devs who do get the memo, and were previously content with a frozen Package-Version, will resent GNU for forcing what they perceive as a workaround. I do not think that is worth it.  Like it or not, MELPA has allowed many of us to taste of the convenience of git/hg tags, and it didn't use to be a problem... until Emacs 30. Now they have to adopt some toolchain like sisyphus.el and pollute the commit log with two extra commits for every new version, to solve what used to be a non-issue.
> 
> I consider advancing Package-Version when significant changes are made
> a simple and uncontroversial thing to do for any package developer.  I
> cannot imagine why someone who wants their package in good shape to
> regard this as some annoyance.

This email discussion risks getting long, but let me know if you want me to describe why I do not find it simple.  For now, I believe the relevant thing from Emacs' perspective is that there does exist developers who will not advance Package-Version, nevermind why.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Eli Zaretskii]

> From: "Martin Edström" <meedstrom@runbox.eu>
> CC: "emacs-devel" <emacs-devel@gnu.org>
> Date: Mon, 16 Sep 2024 18:15:45 +0200 (CEST)
> 
> Okay, I'll accept that for the command package-vc-install, as I have not thought much about its use cases.  
> 
> I want to call attention to the old use-package :vc, shipped under the confusing name vc-use-package [1], that people could install on Emacs 29 and earlier.  The Emacs 30 use-package :vc is a drop-in almost 1:1 replacement for this thing.  It had the opposite default behavior from the default Emacs 30 is about to have.  That change needs justification, not the other way around!
> 
> [1] https://github.com/slotThe/vc-use-package

That's a different package, so I don't see why we should necessarily
follow its decisions.  When users migrate to use-package :vc, they
should expect some changes in the default behavior.

> > > I should also point out that the catastrophe occurs not at release time, but years afterwards, when we're on Emacs 31, 32, 33... but devs still want to support Emacs 30, getting worse with time. So I do hope that it will be possible to change a setting like this with Emacs 30.2 or some other "bugfix" release.
> > 
> > There's no reason to do this for user options, since customizing an
> > option if the user doesn't like its default is easy.
> 
> You cannot rely on users to configure this kind of thing for themselves, most of them would not know the reason why things broke. It is not a matter of "liking" the default, since breakage in one direction would be much more severe than is possible to encounter in the other direction (years out of date vs. slightly too new).

Nowadays an Internet search will bring the answers very quickly and
efficiently.  And if that's not enough, there are enough user-level
help forums where people could ask questions and get useful answers.

IOW, "not a catastrophe".

> It is absolutely Emacs' responsibility, at least as long as stability for users is desired. Isn't it the reason to favor conservative defaults? Avoiding breakage? I get this new default comes from a good place, intending stability, but it is a radical departure from what has been the norm established by Straight/Quelpa/el-get and their use-package integrations, and that will work against stability.  

We can only be responsible for stability when our own implementation
and decisions are concerned.  We cannot be expected to blindly follow
someone else's decision in the name of "stability" for users who
switch from other packages, this kind of expectation is completely
unreasonable.

> > I consider advancing Package-Version when significant changes are made
> > a simple and uncontroversial thing to do for any package developer.  I
> > cannot imagine why someone who wants their package in good shape to
> > regard this as some annoyance.
> 
> This email discussion risks getting long, but let me know if you want me to describe why I do not find it simple.  For now, I believe the relevant thing from Emacs' perspective is that there does exist developers who will not advance Package-Version, nevermind why.

Well, simple or not, it is their act to get together.  Not ours.

Also, if we want to go an extra mile, I suggested a couple more
measures that would delineate the issue to unsuspecting users.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Martin Edström]

On Mon, 16 Sep 2024 20:57:15 +0300, Eli Zaretskii <eliz@gnu.org> wrote:

> > [1] https://github.com/slotThe/vc-use-package
> 
> That's a different package, so I don't see why we should necessarily
> follow its decisions.  When users migrate to use-package :vc, they
> should expect some changes in the default behavior.
> 
> > You cannot rely on users to configure this kind of thing for themselves, most of them would not know the reason why things broke. It is not a matter of "liking" the default, since breakage in one direction would be much more severe than is possible to encounter in the other direction (years out of date vs. slightly too new).
> 
> Nowadays an Internet search will bring the answers very quickly and
> efficiently.  And if that's not enough, there are enough user-level
> help forums where people could ask questions and get useful answers.
> 
> IOW, "not a catastrophe".
> 
> > It is absolutely Emacs' responsibility, at least as long as stability for users is desired. Isn't it the reason to favor conservative defaults? Avoiding breakage? I get this new default comes from a good place, intending stability, but it is a radical departure from what has been the norm established by Straight/Quelpa/el-get and their use-package integrations, and that will work against stability.  
> 
> We can only be responsible for stability when our own implementation
> and decisions are concerned.  We cannot be expected to blindly follow
> someone else's decision in the name of "stability" for users who
> switch from other packages, this kind of expectation is completely
> unreasonable.

I am sorry.  I use the word "responsibility" perhaps different from many people, I think of responsibility as basically good, a nice thing to have, but as words go it's a powder-keg with different interpretations. I did not mean to be entitled nor expect the Emacs volunteers to do any services for me or anyone else.

I also have not contributed any core Emacs code (yet), and I am very grateful to those who have, such as yourself, Eli.

We are currently enjoying a Golden Age of Emacs, and I see that as being a lot about the Emacs Lisp ecosystem wrapped around the Emacs core, and of course, Emacs core doing what it can to be friendly to that ecosystem.  To me, Emacs *is* the ecosystem.

That frames my opinions about this new default setting.  It does not seem to me like the theoretical benefits outweigh the costs to GNU reputation each time a GitHub issue is raised to a developer who preferred to just do rolling releases (e.g. just today I had to warn someone at https://github.com/JuliaEditorSupport/julia-emacs/issues/212).

I believe I've said all I had to say, but to be very clear for the record, I consider this default to be radical, not conservative, at least from the perspective of causing least breakage.  I also do not think it is good stewardship of the ecosystem to say that each user can just Google the solution once they face breakage.  It affects developers too, who may not have learned about this default, and lose users as a consequence.

Anyhoo. The concept of versioned releases is good, and if git/hg tags are checked, that's great. Maybe it's not the end of the world that devs who want pure "rolling releases" are forced to adopt some sort of automated toolchain that pretends to cut releases for every commit, or insert a warning message to their users to upgrade to the master branch. But novice devs and experimental packages frequently fit into this box so I have mixed feelings about that.

Reconsider defaults for use-package-vc-prefer-newest

[Martin Edström]

Previous discussion: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=67955

Currently, the setting of `use-package-vc-prefer-newest` defaults to nil.
I want to bring up some problems with that, and suggest fixes.


PROBLEM 1. MISMATCH WITH PACKAGE-VC

Now use-package :vc is not in line with the behavior of the
package-vc-install command, which fetches the newest commit.

All other package managers I know of also fetch the newest commit.
It is the new use-package :vc which is the odd one out in the
ecosystem, and that can cause stability issues, which I go into
more deeply on this Reddit comment:

https://old.reddit.com/r/emacs/comments/1f8ok7c/do_you_set_packageinstallupgradebuiltin_to_t/llimwe0/


PROBLEM 2. "LAST RELEASE" CANNOT BE ASSUMED TO BE THE LAST RELEASE

I see many packages that have not bumped their Package-Version
header for years.  Perhaps they bump by git tags, or just stopped
bumping versions.  I understand them, since it is a lot of manual
labor to bump it, cut a release commit, then bump again to append a
"-pre" or a ".0.50-git", and to remember to do this everywhere the
version string may occur, like in docs.

I'm also guessing there's a segment of developers that have sensed
that this labor does not end up helping users, so they are not
motivated to keep doing it.  Versions only work as intended if
everyone is versioning regularly, and that is possible to enforce
in a curated repo like GNU Elpa or Debian's repackaged Emacs
packages, but as soon as you also install packages outside of that
repo (and well, most packages are outside), there's no longer any
guarantee and in fact the outdated versions in the curated repo are
likely to cause breakage.

I have personally had to give up on things like ELPA, Melpa Stable,
Debian packages etc.  It always caused instability.  By contrast,
installing 200-400 packages off Melpa, in other words the latest
dev snapshots, has caused me no issues in ~7 years.  So there are
two islands of stability:

1. Stay inside a curated repo entirely
2. Leave the curated repo entirely and install only dev snapshots

Now consider what use-package :vc is for.  It is for installing
things OUTSIDE any curated repo.  I leave it to you to pick a
default setting for `use-package-vc-prefer-newest`.


PROBLEM 3. FORCED REPO REWRITES

I have been forced to use git-filter-repo to rewrite my package's
commit history, to erase all instances of the Package-Version
header.

That's because I had been content to leave the header permanently
at "0.6.0.50-git".  If I kept doing that with the new use-package
:vc in existence, it would cause users to download a horribly
outdated version of my package. If I understand correctly.

I foresee one of two things will happen in the future:

1. More and more package authors carry out the same rewrite I did
2. (more likely) Increased instability as package authors do NOT do
   this rewrite, and users end up with ancient versions all over
   the place


SUGGESTED FIXES

A few possibilities:

1. Change `use-package-vc-prefer-newest` to default to t.

2. Change package-vc so when it walks the commit history to find
   the "last release" or any specific version, it should target the
   newest commit that has the given Package-Version, not the oldest
   commit.

   - That would allow devs to continue having a frozen
     Package-Version without breaking users' setups.

   - Of course, this is effectively the same as option 1, but the
     semantic distinction might be usable in the future to
     e.g. ignore version strings that have a "-pre" inside.

3. Roll out a strategy for moving the entire community onto the
   norm of versioning packages prior to distribution, ideally to
   the extent that Melpa is no longer necessary and can go offline
   in favor of Melpa Stable.  (Until then, Melpa Stable is best
   avoided: https://old.reddit.com/r/emacs/comments/etikbz/)

   Obviously difficult, but as I've said, versioning only works if
   everyone does it.  Specifically, it gives us stability if every
   developer is running the "released version" of most packages
   other than their own.  Currently we have an ecosystem where most
   developers are running the dev snapshot of most packages, and
   any half-measures to change this will only destroy the islands
   of stability that we do currently enjoy.  The topic of this
   email is one such half-measure.

4. Have Emacs ship an automation tool like
   https://github.com/magit/sisyphus, recommend it everywhere, then
   have package.el, package-vc and use-package :vc all throw an
   error on finding more than one commit with a given
   Package-Version string and refuse to install due to ambiguity.

5. Deprecate the Package-Version header and instead maybe let
   package.el perform an automatic string substitution from the
   git/hg tag, if it is still desirable to see the version in the
   source file.  The version could also be substituted into Info
   docs and other places the version string should occur.

   - This is effectively a convenient alternative to option 4.

Re: Reconsider defaults for use-package-vc-prefer-newest

[Philip Kaludercic]

"Martin Edström" <meedstrom@runbox.eu> writes:

> Previous discussion: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=67955
>
> Currently, the setting of `use-package-vc-prefer-newest` defaults to nil.
> I want to bring up some problems with that, and suggest fixes.

Sorry for the late response, I'm responsible for this decision and would
like to defend it here.  Feel free to CC me in future discussions
pertaining to package-vc.  I have also added Tony to the CCs, as he has
a better understanding of the use-package side of this.

> PROBLEM 1. MISMATCH WITH PACKAGE-VC
>
> Now use-package :vc is not in line with the behavior of the
> package-vc-install command, which fetches the newest commit.
>
> All other package managers I know of also fetch the newest commit.

The main reference is package.el, which by default uses GNU ELPA and
NonGNU ELPA.  Both distribute explicitly released packages, as indicated
by the version header.  To me this has a much greater weight than what
third-party package managers and archives decide to do.

> It is the new use-package :vc which is the odd one out in the
> ecosystem, and that can cause stability issues, which I go into
> more deeply on this Reddit comment:
>
> https://old.reddit.com/r/emacs/comments/1f8ok7c/do_you_set_packageinstallupgradebuiltin_to_t/llimwe0/

>
> PROBLEM 2. "LAST RELEASE" CANNOT BE ASSUMED TO BE THE LAST RELEASE
>
> I see many packages that have not bumped their Package-Version
> header for years.  Perhaps they bump by git tags, or just stopped
> bumping versions.  I understand them, since it is a lot of manual
> labor to bump it, cut a release commit, then bump again to append a
> "-pre" or a ".0.50-git", and to remember to do this everywhere the
> version string may occur, like in docs.
>
> I'm also guessing there's a segment of developers that have sensed
> that this labor does not end up helping users, so they are not
> motivated to keep doing it.  Versions only work as intended if
> everyone is versioning regularly, and that is possible to enforce
> in a curated repo like GNU Elpa or Debian's repackaged Emacs
> packages, but as soon as you also install packages outside of that
> repo (and well, most packages are outside), there's no longer any
> guarantee and in fact the outdated versions in the curated repo are
> likely to cause breakage.
>
> I have personally had to give up on things like ELPA, Melpa Stable,
> Debian packages etc.  It always caused instability.  By contrast,
> installing 200-400 packages off Melpa, in other words the latest
> dev snapshots, has caused me no issues in ~7 years.  So there are
> two islands of stability:
>
> 1. Stay inside a curated repo entirely
> 2. Leave the curated repo entirely and install only dev snapshots
>
> Now consider what use-package :vc is for.  It is for installing
> things OUTSIDE any curated repo.  I leave it to you to pick a
> default setting for `use-package-vc-prefer-newest`.

No, this is an unjustified assumption.  FWIW my intention of writing
package-vc was to make contributing to packages easier.  The reason the
"install latest release" option exists for package-vc is that it can
make it easier to fix a bug with the current version.

The thing is that with use-package, having a more deterministic
installation is of interest, especially since use-package is used to
automatically set up an Emacs environment.  I do not share your
experience that non-stable MELPA is more stable than explicitly released
ELPA packages.

Regarding the main point of uncurrated packages: Users can circumvent
this by just upgrading the packages by hand, so I don't think it is that
bad.  That being said, I do think that the pressure to maintain packages
and cut versions from time to time is a good thing.  Here again, I don't
think that we should feel pressured by third-party repositories like
MELPA, that have been ignoring guidelines like in (elisp) Simple
Packages, that plainly state

        The version number comes from the ‘Package-Version’ header, if
        it exists, or from the ‘Version’ header otherwise.  One or the
        other _must_ be present.

Package-vc is related to ELPA and follows ELPA's practices.  The reason
that package specifications look the way they do is to encourage
contributing packages to ELPA.

> PROBLEM 3. FORCED REPO REWRITES
>
> I have been forced to use git-filter-repo to rewrite my package's
> commit history, to erase all instances of the Package-Version
> header.
>
> That's because I had been content to leave the header permanently
> at "0.6.0.50-git".  If I kept doing that with the new use-package
> :vc in existence, it would cause users to download a horribly
> outdated version of my package. If I understand correctly.
>
> I foresee one of two things will happen in the future:
>
> 1. More and more package authors carry out the same rewrite I did
> 2. (more likely) Increased instability as package authors do NOT do
>    this rewrite, and users end up with ancient versions all over
>    the place

It is well known that overwriting published git commits is bad practice,
which is why git push --force (or variations) is a thing.

The third option is that package maintainers become aware of the issue,
and start updating the header to mean something.

A feature we can add in the future is to support ELPA's :rolling-release
attribute, so that packages that wish to ensure that the default branch
is stable with every commit can do so without having to do tricks like
custom time-stamp file local variables that update every time the file
is modified.

>
> SUGGESTED FIXES
>
> A few possibilities:
>
> 1. Change `use-package-vc-prefer-newest` to default to t.
>
> 2. Change package-vc so when it walks the commit history to find
>    the "last release" or any specific version, it should target the
>    newest commit that has the given Package-Version, not the oldest
>    commit.
>
>    - That would allow devs to continue having a frozen
>      Package-Version without breaking users' setups.
>
>    - Of course, this is effectively the same as option 1, but the
>      semantic distinction might be usable in the future to
>      e.g. ignore version strings that have a "-pre" inside.

I am afraid I don't understand what you are suggesting here.  The
current approach is to find the newest commit that touches a version
header and use that.

> 3. Roll out a strategy for moving the entire community onto the
>    norm of versioning packages prior to distribution, ideally to
>    the extent that Melpa is no longer necessary and can go offline
>    in favor of Melpa Stable.  (Until then, Melpa Stable is best
>    avoided: https://old.reddit.com/r/emacs/comments/etikbz/)
>
>    Obviously difficult, but as I've said, versioning only works if
>    everyone does it.  Specifically, it gives us stability if every
>    developer is running the "released version" of most packages
>    other than their own.  Currently we have an ecosystem where most
>    developers are running the dev snapshot of most packages, and
>    any half-measures to change this will only destroy the islands
>    of stability that we do currently enjoy.  The topic of this
>    email is one such half-measure.

Everyone on {GNU,NonGNU} ELPA does it and it works fine.  There is
little we can do about third-party projects, so I don't concern myself
about them.  This is Emacs after all, you have the freedom to break
anything you want, and "THERE IS NO WARRANTY FOR THE PROGRAM" as the GPL
says.

> 4. Have Emacs ship an automation tool like
>    https://github.com/magit/sisyphus, recommend it everywhere, then
>    have package.el, package-vc and use-package :vc all throw an
>    error on finding more than one commit with a given
>    Package-Version string and refuse to install due to ambiguity.

Again, I am afraid we are talking past one another.  Are you suggesting
that the version number should be changed from commit to commit?

> 5. Deprecate the Package-Version header and instead maybe let
>    package.el perform an automatic string substitution from the
>    git/hg tag, if it is still desirable to see the version in the
>    source file.  The version could also be substituted into Info
>    docs and other places the version string should occur.
>
>    - This is effectively a convenient alternative to option 4.

IIRC this is not possible as to limitations with git tags that doesn't
allow us to mirror them in elpa.git.

-- 
	Philip Kaludercic on siskin

Re: Reconsider defaults for use-package-vc-prefer-newest

[Philip Kaludercic]

Suhail Singh <suhailsingh247@gmail.com> writes:

> Philip, I am summarizing below some thoughts / suggestions that I have
> shared elsewhere in this discussion for your benefit.  If you're already
> aware and are caught up with the discussion, please ignore (and excuse
> the noise).

A single message rarely constitutes noise, so don't worry about it :)

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> The current approach is to find the newest commit that touches a
>> version header and use that.
>
> Would it be possible to reflect the commit corresponding to the version
> in the "Other versions:" field in the output of M-x describe-package ?

Why under "Other version:" (which links to other packages).  IIRC the
commit of a VC package should appear under the commit header.  There is
the possibility of using a symbolic name if available.

I now notice that this is not the case as I had assumed... I'll try to
fix it as soon as possible.

> Doing so would make it easier for the user to check when a package that
> is available, say, in both MELPA Stable and NonGNU ELPA have the same
> claimed version, but differ in their corresponding commits.

I know that ELPA annotates packages with the commit used to produce the
package.  When the above issue is fixed, then you should be able to
compare the two commits by following the link under "Other version:".

> Additionally, if it were possible to reflect the commit date as well,
> that would be helpful as well.

I don't think there is a nice vc-generic way to extract the date of a
commit, at least without extending the vc interface any further.  I
might be mistaken though.

The other issues is that if you only have the revision string from the
ELPA server, without any checkout, it isn't easy/cheap to determine the
date even if you just wanted to support Git.

-- 
	Philip Kaludercic on siskin

Re: Reconsider defaults for use-package-vc-prefer-newest

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 639 bytes --]

Philip Kaludercic <philipk@posteo.net> writes:


[...]

>>> The current approach is to find the newest commit that touches a
>>> version header and use that.
>>
>> Would it be possible to reflect the commit corresponding to the version
>> in the "Other versions:" field in the output of M-x describe-package ?
>
> Why under "Other version:" (which links to other packages).  IIRC the
> commit of a VC package should appear under the commit header.  There is
> the possibility of using a symbolic name if available.
>
> I now notice that this is not the case as I had assumed... I'll try to
> fix it as soon as possible.

This is the fix:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Insert-correct-commit-data-into-VC-package-descripti.patch --]
[-- Type: text/x-diff, Size: 2071 bytes --]

From 3a3cd2d4e6915d02bfb66680c2c00f3e179c7e0f Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Thu, 19 Sep 2024 22:27:26 +0200
Subject: [PATCH] Insert correct commit data into VC package descriptions

* lisp/emacs-lisp/package-vc.el (package-vc-commit): Rename
argument from PKG to PKG-DESC.
(package-vc--generate-description-file): Update the "extras"
section of the package description with the revision string at
generation time.
---
 lisp/emacs-lisp/package-vc.el | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/lisp/emacs-lisp/package-vc.el b/lisp/emacs-lisp/package-vc.el
index 34610102aa0..e168096e153 100644
--- a/lisp/emacs-lisp/package-vc.el
+++ b/lisp/emacs-lisp/package-vc.el
@@ -265,13 +265,13 @@ package-vc--download-and-read-archives
 
 (add-hook 'package-read-archive-hook     #'package-vc--read-archive-data 20)
 
-(defun package-vc-commit (pkg)
-  "Return the last commit of a development package PKG."
-  (cl-assert (package-vc-p pkg))
+(defun package-vc-commit (pkg-desc)
+  "Return the last commit of a development package PKG-DESC."
+  (cl-assert (package-vc-p pkg-desc))
   ;; FIXME: vc should be extended to allow querying the commit of a
   ;; directory (as is possible when dealing with git repositories).
   ;; This should be a fallback option.
-  (cl-loop with dir = (package-desc-dir pkg)
+  (cl-loop with dir = (package-desc-dir pkg-desc)
            for file in (directory-files dir t "\\.el\\'" t)
            when (vc-working-revision file) return it
            finally return "unknown"))
@@ -359,7 +359,11 @@ package-vc--generate-description-file
                          requires))))
           (list :kind 'vc)
           (package--alist-to-plist-args
-           (package-desc-extras pkg-desc))))
+           (let ((extras (copy-alist (package-desc-extras pkg-desc))))
+             (setf (alist-get :commit extras)
+                   (package-vc-commit pkg-desc))
+             extras)
+           )))
         "\n")
        nil pkg-file nil 'silent))))
 
-- 
2.45.2


[-- Attachment #3: Type: text/plain, Size: 90 bytes --]


Would anyone mind me pushing this to emacs-30?

[...]


-- 
	Philip Kaludercic on siskin