Request to add Package to GNU ELPA

Request to add Package to GNU ELPA

[Jonas Bernoulli]

Hello,

I would like to request that "package.el" be added to GNU Elpa as a
"core" package.  This would make new features available to users who are
stuck on older Emacs releases and it would even bring us one step closer
to being able to make backward incompatible changes in the future.

Some examples for why that would be desirable:

- A recent addition that could be useful is the new "package-vc.el"
  (which I think should be distributed as part of the `package' package,
  but it could also be distributed as a separate core package).

  That isn't just useful for users who like to live on the edge.  For
  example, if a package drops support for an old Emacs release, then
  a user who is stuck on that release, may wish to keep using the last
  release of the package that still supports that Emacs release, and
  package-vc would allow them to do just that.

- I don't use Package myself, so I am not fully aware of all the quality
  of live enhancements that surely have accumulated over the years.  But
  I am aware of some small missing features, that would be beneficial to
  users of older Emacs releases.

- For example, I think it would be nice if the columns displayed by
  `list-packages' were customizable.  A user might want to increase the
  width of the "Version" column, so that not every GNU-devel ELPA version
  is truncated, or they might want to add an "Author" column.  We could
  even allow third-party packages to add columns such as "Downloads" or
  "Stars".

- I should also mention that my main motivation for pushing for this
  now, is that I would like to improve version handling.  That is a
  whole other can of worms, so I do not wish to discuss it just yet,
  to avoid distracting from the topic at hand, but I thought I should
  at least mention it.  You might very well end up being against my
  suggestions regarding versions strings, once I present them, but that
  should not be cause to oppose the change requested here as well.  Even
  if my suggestions regarding version strings are ultimately rejected, I
  still think it would be a good idea to add `package' to GNU ELPA, for
  the other reasons presented here.

- I should also make an example for a backward incompatible change.
  Yesterday, in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62524 I
  suggested a backward *compatible* change, but doing it in a backward
  compatible fashion is a bit ugly, and I would like to *eventually*
  replace it with a backward *incompatible* change.

  In short, some packages have multiple maintainers and
  `describe-package' should list them all.  Unfortunately `:maintainer'
  (unlike `:authors') in "archive-contents" can only be a single
  maintainer.  If we listed multiple maintainers, then current versions
  of Package would signal an error.  To address that I proposed that we
  *add* a `:maintainers' property.  Older Package don't know about that
  property and would simply ignore it, and newer Package versions could
  start doing all the maintainers justice.

  It isn't exactly horrible to have both `:maintainer' and
  `:maintainers' for a while, but it is a wart that should be remove
  eventually.  We should avoid accumulating more and more such warts
  over time.  One way of doing that is to simply avoid adding features
  that have to be implemented in an ugly fashion to avoid breaking
  backward compatibility.  I feel that approach has been taken too many
  times in the past.  I myself have certainly done it on several
  occasions.

I mentioned that distributing Package on GNU ELPA would bring us one
step closer to being able to make backward incompatible changes, but
unfortunately it is not enough.  Just because `package' is available
on GNU ELPA, that doesn't mean that it gets installed automatically.

There are two additional steps required.  (1) We have to somehow get
users to install `package' from GNU ELPA a first time, and (2) once
users have done that and a new `package' version becomes available,
they have to be informed about the update and be prompted to install
it.

To address (2), `package-refresh-contents' could be taught to check if
a new `package' version is available, and if that is the case, then it
should prompt the user to update that immediately, before any other
packages.

Addressing (1) is harder, and I don't think it possible to do so in a
way that guarantees that not a single user would end up seeing an error
due to an incompatible change.  On the other hand "archive-contents"
comes with a version field, and if we bump that, we at least get a
meaningful warning: "Package archive version 2 is higher than 1".  This
doesn't say that the solution is to install `package' from GNU ELPA, but
it should be enough to feed into a search engine to get to that answer.

Additionally, we could get many users to install `package' from GNU
ELPA, by making a few popular packages explicitly depend on a `package'
version that is available from GNU ELPA for a few months.

     Thanks for considering this proposal,
     Jonas

Re: Request to add Package to GNU ELPA

[Felician Nemeth]

> Some examples for why that would be desirable:
>
> - A recent addition that could be useful is the new "package-vc.el"
>   (which I think should be distributed as part of the `package' package,
>   but it could also be distributed as a separate core package).

Elaborating on this example, I'd like to see the patch in bug#60418 to
be merged.  It adds package-vc support to use-package.el.  But similarly
to Jonas' other examples, it would be useful on an older Emacs only if
package.el/package-vc.el could be updated.

Thanks,
Felicián

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

Felician Nemeth <felician.nemeth@gmail.com> writes:

>> Some examples for why that would be desirable:
>>
>> - A recent addition that could be useful is the new "package-vc.el"
>>   (which I think should be distributed as part of the `package' package,
>>   but it could also be distributed as a separate core package).
>
> Elaborating on this example, I'd like to see the patch in bug#60418 to
> be merged.  It adds package-vc support to use-package.el.  But similarly
> to Jonas' other examples, it would be useful on an older Emacs only if
> package.el/package-vc.el could be updated.

I have commented on the patch in a different report (bug#61937).  Here
is a copy for the sake of convenience:

  I have to admit that I am not a fan of this [adding package-vc support
  to use-package].  Personally I install all my packages using regular
  tarballs, and then manually select packages that I want to track in
  source-form if I am interested in contributing a patch.

  One has to keep in mind that by default you will be checking out the tip
  of the default development branch.  This is not what GNU ELPA and NonGNU
  ELPA do.  I know *some other* archives do this, but IMO this contributes
  to the instability that a lot of Emacs users experience in regards to
  the packages in the Emacs world.  Coincidentally I also believe that the
  popularity of alternative (non-cooperative) package managers like this
  "straight.el" boils down to the promise of being able to pin and revert
  to previous packages states.  Personally I believe that this is an
  up-side-down approach to solve the problem from the wrong angle.  The
  default approach to package management should be reliable and suffice.

> Thanks,
> Felicián
>
>

-- 
Philip Kaludercic

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Hello,

Hi, hope I can add a few useful comments here.

> I would like to request that "package.el" be added to GNU Elpa as a
> "core" package.  This would make new features available to users who are
> stuck on older Emacs releases and it would even bring us one step closer
> to being able to make backward incompatible changes in the future.
>
> Some examples for why that would be desirable:
>
> - A recent addition that could be useful is the new "package-vc.el"
>   (which I think should be distributed as part of the `package' package,
>   but it could also be distributed as a separate core package).

This will be difficult, since package-vc depends on the new "vc-clone"
function, which would also have to be provided on ELPA to work.  This
seems to me to be a general issue with proposing to create new core
packages.  When working in emacs.git it is easy and usually
uncontroversial to make use of whatever is the current state-of-the-art
when it comes to Elisp.  The Compat library on ELPA can help here, as it
already does in a few cases (ERC and Python, and it has been discussed
for project.el).

All of this touches on the general topic of how Emacs should be
developed: Small and to be extended, therefore of little use OOTB or big
and feature-full, but slow to be updated.  IIUC a compromise could be
achieved by having two versions of Emacs that would respectively attempt
to satisfy the two aspirations.  There was some work in that direction a
few years ago[0], but I don't know if anything has happened since.
Perhaps pushing for this more general solution would help solve this
particular problem?

[0] https://git.savannah.gnu.org/cgit/emacs.git/log/?h=elparized-core

>   That isn't just useful for users who like to live on the edge.  For
>   example, if a package drops support for an old Emacs release, then
>   a user who is stuck on that release, may wish to keep using the last
>   release of the package that still supports that Emacs release, and
>   package-vc would allow them to do just that.
>
> - I don't use Package myself, so I am not fully aware of all the quality
>   of live enhancements that surely have accumulated over the years.  But
>   I am aware of some small missing features, that would be beneficial to
>   users of older Emacs releases.
>
> - For example, I think it would be nice if the columns displayed by
>   `list-packages' were customizable.  A user might want to increase the
>   width of the "Version" column, so that not every GNU-devel ELPA version
>   is truncated, 

A more general solution to this problem would be if tabulated-list-mode
could dynamically resize a column, or at least shrink it possible.

Also on another topic, I am still uncertain as to the status of the
-devel archives.  I see them primarily as a means for developers to
check the status of a package pre-release or to make sure that the ELPA
build-system is working the way they want it to (e.g. when building
manuals, news or README files).  They are not advertised anywhere, and I
don't think they are made for general consumption -- or rather I don't
think they should be, especially with package-vc there is a much better
way to track development and make it possible to contribute changes
upstream.

>                 or they might want to add an "Author" column.  We could
>   even allow third-party packages to add columns such as "Downloads" or
>   "Stars".

> - I should also mention that my main motivation for pushing for this
>   now, is that I would like to improve version handling.  That is a
>   whole other can of worms, so I do not wish to discuss it just yet,
>   to avoid distracting from the topic at hand, but I thought I should
>   at least mention it.  You might very well end up being against my
>   suggestions regarding versions strings, once I present them, but that
>   should not be cause to oppose the change requested here as well.  Even
>   if my suggestions regarding version strings are ultimately rejected, I
>   still think it would be a good idea to add `package' to GNU ELPA, for
>   the other reasons presented here.

This implies that packages might themselves depend on newer versions of
package.el -- which I think one should put some thought into before
anything is done.  How can you e.g. change the way versions are handled
in a way that people with older versions of package.el could still
handle the change without confusion?  (This will be an issue for at
least a decade to come, knowing the pace at which users get access to
newer versions of Emacs).  You mentioned updating the ELPA protocol a
few months back, and I believe a few points of interest were collected
in that thread.  It seems to me these issues are all intertwined.

> - I should also make an example for a backward incompatible change.
>   Yesterday, in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62524 I
>   suggested a backward *compatible* change, but doing it in a backward
>   compatible fashion is a bit ugly, and I would like to *eventually*
>   replace it with a backward *incompatible* change.
>
>   In short, some packages have multiple maintainers and
>   `describe-package' should list them all.  Unfortunately `:maintainer'
>   (unlike `:authors') in "archive-contents" can only be a single
>   maintainer.  If we listed multiple maintainers, then current versions
>   of Package would signal an error.  To address that I proposed that we
>   *add* a `:maintainers' property.  Older Package don't know about that
>   property and would simply ignore it, and newer Package versions could
>   start doing all the maintainers justice.
>
>   It isn't exactly horrible to have both `:maintainer' and
>   `:maintainers' for a while, but it is a wart that should be remove
>   eventually.  We should avoid accumulating more and more such warts
>   over time.  One way of doing that is to simply avoid adding features
>   that have to be implemented in an ugly fashion to avoid breaking
>   backward compatibility.  I feel that approach has been taken too many
>   times in the past.  I myself have certainly done it on several
>   occasions.
>
> I mentioned that distributing Package on GNU ELPA would bring us one
> step closer to being able to make backward incompatible changes, but
> unfortunately it is not enough.  Just because `package' is available
> on GNU ELPA, that doesn't mean that it gets installed automatically.
>
> There are two additional steps required.  (1) We have to somehow get
> users to install `package' from GNU ELPA a first time, and (2) once
> users have done that and a new `package' version becomes available,
> they have to be informed about the update and be prompted to install
> it.
>
> To address (2), `package-refresh-contents' could be taught to check if
> a new `package' version is available, and if that is the case, then it
> should prompt the user to update that immediately, before any other
> packages.

With Emacs 29, `package-refresh-contents' now calls a Hook called
`package-refresh-contents-hook'.  This was initially introduced so that
package-vc can download the "elpa-packages.eld" file, but it could also
be of use here as well -- but it wouldn't help anyone with Emacs 28 or
older...  I could imagine a more hacky approach based on carefully
placed advice, but only as a last resort.

> Addressing (1) is harder, and I don't think it possible to do so in a
> way that guarantees that not a single user would end up seeing an error
> due to an incompatible change.  On the other hand "archive-contents"
> comes with a version field, and if we bump that, we at least get a
> meaningful warning: "Package archive version 2 is higher than 1".  This
> doesn't say that the solution is to install `package' from GNU ELPA, but
> it should be enough to feed into a search engine to get to that answer.

The upgrading procedure for archive-contents has never appeared to me to
be very robust.  Perhaps it would be better to introduce a second file
(archive-contents.eld) with a more flexible format?

> Additionally, we could get many users to install `package' from GNU
> ELPA, by making a few popular packages explicitly depend on a `package'
> version that is available from GNU ELPA for a few months.

Again, a few months wouldn't be enough to solve the issue.  But again as
well, Compat could help by adding package as a noop dependency.

>      Thanks for considering this proposal,
>      Jonas

-- 
Philip Kaludercic

Re: Request to add Package to GNU ELPA

[Jonas Bernoulli]

Philip Kaludercic <philipk@posteo.net> writes:

> Jonas Bernoulli <jonas@bernoul.li> writes:
>
>> Hello,
>
> Hi, hope I can add a few useful comments here.
>
>> I would like to request that "package.el" be added to GNU Elpa as a
>> "core" package.  This would make new features available to users who are
>> stuck on older Emacs releases and it would even bring us one step closer
>> to being able to make backward incompatible changes in the future.
>>
>> Some examples for why that would be desirable:
>>
>> - A recent addition that could be useful is the new "package-vc.el"
>>   (which I think should be distributed as part of the `package' package,
>>   but it could also be distributed as a separate core package).
>
> This will be difficult, since package-vc depends on the new "vc-clone"
> function, which would also have to be provided on ELPA to work.

The simplest way to deal with that, while still making the latest
Package available to users of older Emacs releases, would be to *not*
include package-vc.el in the Package package.

Going one step further we could make package-vc available as a separate
package.  That would not be of much use *now*, but future improvements
would then be available to users of Emacs 29.

I had a look at vc-clone and a few vc-*-clone.  They seem trivial
enough to backport, either as part of Compat or in package-vc.el.

> This
> seems to me to be a general issue with proposing to create new core
> packages.  When working in emacs.git it is easy and usually
> uncontroversial to make use of whatever is the current state-of-the-art
> when it comes to Elisp.  The Compat library on ELPA can help here, as it
> already does in a few cases (ERC and Python, and it has been discussed
> for project.el).

Thanks for creating Compat!  It certainly makes things easier for
maintainers of external packages.

> All of this touches on the general topic of how Emacs should be
> developed: Small and to be extended, therefore of little use OOTB or big
> and feature-full, but slow to be updated.  IIUC a compromise could be
> achieved by having two versions of Emacs that would respectively attempt
> to satisfy the two aspirations.

This doesn't sound like a good idea to me because I doubt that it would
have the benefits you seem to expect.  A sizable portion of an external
package would choose to use the slow-moving Emacs.  So the maintainer of
the package would have to continue to support that, similar to how they
now have to support older releases.  So even though there would be a
fast-moving Emacs, they still would not be able to take full advantage
of the new/improved features it provides.  The situation would be
basically the same as now; but not supporting the slow-moving Emacs
would be even harder to justify than dropping support for Emacs 25.1.

> There was some work in that direction a
> few years ago[0], but I don't know if anything has happened since.
> Perhaps pushing for this more general solution would help solve this
> particular problem?
>
> [0] https://git.savannah.gnu.org/cgit/emacs.git/log/?h=elparized-core

I remember that there was some talk about moving in that direction, and
I am in favor and have been patiently waiting for someone to make
another push in that direction, but things indeed seem to have fizzled
out.

Meanwhile making certain packages available on a case by case basis does
not seem like a bad idea to me.  It doesn't mean that we *commit* to
making "most" packages available like that eventually, so even those you
are opposed to that idea might be able to agree to doing it selectively.
Doing it for a select few packages, allows us to gain some experience,
whether that be bad or good.

>>   That isn't just useful for users who like to live on the edge.  For
>>   example, if a package drops support for an old Emacs release, then
>>   a user who is stuck on that release, may wish to keep using the last
>>   release of the package that still supports that Emacs release, and
>>   package-vc would allow them to do just that.
>>
>> - I don't use Package myself, so I am not fully aware of all the quality
>>   of live enhancements that surely have accumulated over the years.  But
>>   I am aware of some small missing features, that would be beneficial to
>>   users of older Emacs releases.
>>
>> - For example, I think it would be nice if the columns displayed by
>>   `list-packages' were customizable.  A user might want to increase the
>>   width of the "Version" column, so that not every GNU-devel ELPA version
>>   is truncated, 
>
> A more general solution to this problem would be if tabulated-list-mode
> could dynamically resize a column, or at least shrink it possible.

That would indeed be nice ... and would make tabulated-list-mode another
candidate for GNU ELPA. ;P

> Also on another topic, I am still uncertain as to the status of the
> -devel archives.  I see them primarily as a means for developers to
> check the status of a package pre-release or to make sure that the ELPA
> build-system is working the way they want it to (e.g. when building
> manuals, news or README files).  They are not advertised anywhere, and I
> don't think they are made for general consumption -- or rather I don't
> think they should be, especially with package-vc there is a much better
> way to track development and make it possible to contribute changes
> upstream.

Most packages still get installed through Melpa these days and that
means installing the latest commit from the development branch.  (The
[Non]GNU-devel ELPA variants probably exist in part to satisfy users who
are used to having access to the development version through "regular"
package i.e., without using package-vc.)

In my experience that seems to work surprisingly well; I can't really
remember any regressions that were not fixed almost instantly, but that
does of course depend on what packages one has installed.  I am sure
little regressions happen all the time, but they get fixed so quickly,
that any one user usually doesn't notice.  In my experience, using the
development versions of some quality packages, is no different from
using the development version of Emacs itself.

But I agree that it would be better to release more often (and yes, I
am guilty of not doing that myself), and tend to agree that it would be
better if, in the long run, the community moved towards relying more on
released versions.

At the same time I don't think that the fact that we are not there yet
is due to "mistakes".  Emacs/Elisp is different from other "languages",
in that everything runs in the same process, and that is at least one
of the causes of the status quo.

We cannot really use multiple versions of the same library in different
parts of Emacs, in the same way one Python or Rust application could
use xyz-v1 and another, related but independent, application could use
xyz-v2.  If one package depends on xyz-v2 but there were incompatible
changes in that release and, another package has not been updated yet to
support xyz-v2, then those two packages cannot be used at the same time.

Again, that does not seem to happen very often, and I think that is
partially the case because the hold out package can be quickly updated
to support v2 on its development branch, *without* also having to do a
release, which the maintainers might not wish to do due to where in its
own release-cycle the package happens to be.

>> - I should also mention that my main motivation for pushing for this
>>   now, is that I would like to improve version handling.  That is a
>>   whole other can of worms, so I do not wish to discuss it just yet,
>>   to avoid distracting from the topic at hand, but I thought I should
>>   at least mention it.  You might very well end up being against my
>>   suggestions regarding versions strings, once I present them, but that
>>   should not be cause to oppose the change requested here as well.  Even
>>   if my suggestions regarding version strings are ultimately rejected, I
>>   still think it would be a good idea to add `package' to GNU ELPA, for
>>   the other reasons presented here.
>
> This implies that packages might themselves depend on newer versions of
> package.el -- which I think one should put some thought into before
> anything is done.

How so? (Further down I suggest making some popular packages temporarily
depend on Package from GNU Elpa as a means to get that installed as
widely as possible, but that doesn't mean that those packages actually
*need* a newer Package.)

> How can you e.g. change the way versions are handled
> in a way that people with older versions of package.el could still
> handle the change without confusion?

You can't, and that is exactly the point I was trying to make.  Certain
things (including, but not limited to how version strings are handled)
are effectively set in stone, unless certain packages/features are made
available to users who, for whatever reason, stick to an old Emacs
release.

> (This will be an issue for at
> least a decade to come, knowing the pace at which users get access to
> newer versions of Emacs).  You mentioned updating the ELPA protocol a
> few months back, and I believe a few points of interest were collected
> in that thread.  It seems to me these issues are all intertwined.

Yes indeed, they are all intertwined, and I am trying to make the point
that we have to stop muddling along once in a while, and make a breaking
change, even if the transition is rough and inconveniences users for a
while.

> With Emacs 29, `package-refresh-contents' now calls a Hook called
> `package-refresh-contents-hook'.  This was initially introduced so that
> package-vc can download the "elpa-packages.eld" file, but it could also
> be of use here as well -- but it wouldn't help anyone with Emacs 28 or
> older...  I could imagine a more hacky approach based on carefully
> placed advice, but only as a last resort.

We cannot travel back in time to fix past mistakes/omissions that are
holding us back in the presence.  All we can do now is to pay the price
and try harder to be forward compatible this time around.  Or live with
the limitations forever.  Or wait some more until they become unbearable
and a fix more urgent; but that would increase the odds of messing it up
again.

>> Addressing (1) is harder, and I don't think it possible to do so in a
>> way that guarantees that not a single user would end up seeing an error
>> due to an incompatible change.  On the other hand "archive-contents"
>> comes with a version field, and if we bump that, we at least get a
>> meaningful warning: "Package archive version 2 is higher than 1".  This
>> doesn't say that the solution is to install `package' from GNU ELPA, but
>> it should be enough to feed into a search engine to get to that answer.
>
> The upgrading procedure for archive-contents has never appeared to me to
> be very robust.  Perhaps it would be better to introduce a second file
> (archive-contents.eld) with a more flexible format?

An effort was made to provide an upgrade procedure -- the
"archive-contents" were prefixed with a version number.  But then
package.el was added to Emacs and we stopped to distribute it
separately.  Apparently we didn't realize at the time that this only
allowed us to tell the user "the archive and tool are not compatible,
deal with it".

We could continue to distribute "archive-contents", which continues to
use version 1, and add "archive-contents.eld" which uses version 2.
But that would do nothing to get users to install a forward-compatible
version of Package (in the sense that once it becomes incompatible, it
will provide a smooth upgrade path to the new version).

>> Additionally, we could get many users to install `package' from GNU
>> ELPA, by making a few popular packages explicitly depend on a `package'
>> version that is available from GNU ELPA for a few months.
>
> Again, a few months wouldn't be enough to solve the issue.  But again as
> well, Compat could help by adding package as a noop dependency.

Much like your "archive-contents.eld" suggestion, doing this would not
*solve* the issue.  Unlike that, it would however significantly reduce
the number of users who would be negatively affected by the upgrade.
(We could do it for a few years instead of months to further decrease
the number of affected users.)

I love Compat, but I don't think it is the right hammer here (except
maybe for the part where it could provide vc-clone et al.).

Maybe it helps to make it explicit what is going to happen to users
if/when all the *ELPA start to expect that package.el support
"archive-contents" v2.

- The user runs "M-x package-refresh-contents" and is told "Package
  archive version 2 is higher than 1".

- They ask a search engine what that means.  We could create a few blog
  and forum posts before hand and give the search engines a few days to
  index them, providing the following instructions:

  (We should invest some time to investigate how to make this as smooth
  as possible, but basically:)

- 1. wget https://...package-2.0.0.tar
  2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/
  3. Restart Emacs and run package-refresh-contents again.

     Cheers, 
     Jonas

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>
>>> Hello,
>>
>> Hi, hope I can add a few useful comments here.
>>
>>> I would like to request that "package.el" be added to GNU Elpa as a
>>> "core" package.  This would make new features available to users who are
>>> stuck on older Emacs releases and it would even bring us one step closer
>>> to being able to make backward incompatible changes in the future.
>>>
>>> Some examples for why that would be desirable:
>>>
>>> - A recent addition that could be useful is the new "package-vc.el"
>>>   (which I think should be distributed as part of the `package' package,
>>>   but it could also be distributed as a separate core package).
>>
>> This will be difficult, since package-vc depends on the new "vc-clone"
>> function, which would also have to be provided on ELPA to work.
>
> The simplest way to deal with that, while still making the latest
> Package available to users of older Emacs releases, would be to *not*
> include package-vc.el in the Package package.
>
> Going one step further we could make package-vc available as a separate
> package.  That would not be of much use *now*, but future improvements
> would then be available to users of Emacs 29.

I would in principle be interested in supporting this.

> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
> enough to backport, either as part of Compat or in package-vc.el.

[...]

>> All of this touches on the general topic of how Emacs should be
>> developed: Small and to be extended, therefore of little use OOTB or big
>> and feature-full, but slow to be updated.  IIUC a compromise could be
>> achieved by having two versions of Emacs that would respectively attempt
>> to satisfy the two aspirations.
>
> This doesn't sound like a good idea to me because I doubt that it would
> have the benefits you seem to expect.  A sizable portion of an external
> package would choose to use the slow-moving Emacs.  So the maintainer of
> the package would have to continue to support that, similar to how they
> now have to support older releases.  So even though there would be a
> fast-moving Emacs, they still would not be able to take full advantage
> of the new/improved features it provides.  The situation would be
> basically the same as now; but not supporting the slow-moving Emacs
> would be even harder to justify than dropping support for Emacs 25.1.

I am not really advocating for one or the other option, just wanted to
bring it up, if for nobody else then just for people lurking the mailing
list.

>> There was some work in that direction a
>> few years ago[0], but I don't know if anything has happened since.
>> Perhaps pushing for this more general solution would help solve this
>> particular problem?
>>
>> [0] https://git.savannah.gnu.org/cgit/emacs.git/log/?h=elparized-core
>
> I remember that there was some talk about moving in that direction, and
> I am in favor and have been patiently waiting for someone to make
> another push in that direction, but things indeed seem to have fizzled
> out.
>
> Meanwhile making certain packages available on a case by case basis does
> not seem like a bad idea to me.  It doesn't mean that we *commit* to
> making "most" packages available like that eventually, so even those you
> are opposed to that idea might be able to agree to doing it selectively.
> Doing it for a select few packages, allows us to gain some experience,
> whether that be bad or good.

True.

>>>   That isn't just useful for users who like to live on the edge.  For
>>>   example, if a package drops support for an old Emacs release, then
>>>   a user who is stuck on that release, may wish to keep using the last
>>>   release of the package that still supports that Emacs release, and
>>>   package-vc would allow them to do just that.
>>>
>>> - I don't use Package myself, so I am not fully aware of all the quality
>>>   of live enhancements that surely have accumulated over the years.  But
>>>   I am aware of some small missing features, that would be beneficial to
>>>   users of older Emacs releases.
>>>
>>> - For example, I think it would be nice if the columns displayed by
>>>   `list-packages' were customizable.  A user might want to increase the
>>>   width of the "Version" column, so that not every GNU-devel ELPA version
>>>   is truncated, 
>>
>> A more general solution to this problem would be if tabulated-list-mode
>> could dynamically resize a column, or at least shrink it possible.
>
> That would indeed be nice ... and would make tabulated-list-mode another
> candidate for GNU ELPA. ;P

It seems like this is possible, or at least I didn't see any difficult
dependencies from a brief skim.

>> Also on another topic, I am still uncertain as to the status of the
>> -devel archives.  I see them primarily as a means for developers to
>> check the status of a package pre-release or to make sure that the ELPA
>> build-system is working the way they want it to (e.g. when building
>> manuals, news or README files).  They are not advertised anywhere, and I
>> don't think they are made for general consumption -- or rather I don't
>> think they should be, especially with package-vc there is a much better
>> way to track development and make it possible to contribute changes
>> upstream.
>
> Most packages still get installed through Melpa these days and that
> means installing the latest commit from the development branch.  (The
> [Non]GNU-devel ELPA variants probably exist in part to satisfy users who
> are used to having access to the development version through "regular"
> package i.e., without using package-vc.)
>
> In my experience that seems to work surprisingly well; I can't really
> remember any regressions that were not fixed almost instantly, but that
> does of course depend on what packages one has installed.  I am sure
> little regressions happen all the time, but they get fixed so quickly,
> that any one user usually doesn't notice.  In my experience, using the
> development versions of some quality packages, is no different from
> using the development version of Emacs itself.

My experience was not that good, especially back before I got into Emacs
development (or understood what was going on at all) I constantly ran
into issues when updating packages from MELPA.  The result was that I'd
usually just not update packages at all for long periods of time.  The
next best thing for me was MELPA stable until NonGNU ELPA came around.

> But I agree that it would be better to release more often (and yes, I
> am guilty of not doing that myself), and tend to agree that it would be
> better if, in the long run, the community moved towards relying more on
> released versions.
>
> At the same time I don't think that the fact that we are not there yet
> is due to "mistakes".  Emacs/Elisp is different from other "languages",
> in that everything runs in the same process, and that is at least one
> of the causes of the status quo.
>
> We cannot really use multiple versions of the same library in different
> parts of Emacs, in the same way one Python or Rust application could
> use xyz-v1 and another, related but independent, application could use
> xyz-v2.  If one package depends on xyz-v2 but there were incompatible
> changes in that release and, another package has not been updated yet to
> support xyz-v2, then those two packages cannot be used at the same time.
>
> Again, that does not seem to happen very often, and I think that is
> partially the case because the hold out package can be quickly updated
> to support v2 on its development branch, *without* also having to do a
> release, which the maintainers might not wish to do due to where in its
> own release-cycle the package happens to be.
>
>>> - I should also mention that my main motivation for pushing for this
>>>   now, is that I would like to improve version handling.  That is a
>>>   whole other can of worms, so I do not wish to discuss it just yet,
>>>   to avoid distracting from the topic at hand, but I thought I should
>>>   at least mention it.  You might very well end up being against my
>>>   suggestions regarding versions strings, once I present them, but that
>>>   should not be cause to oppose the change requested here as well.  Even
>>>   if my suggestions regarding version strings are ultimately rejected, I
>>>   still think it would be a good idea to add `package' to GNU ELPA, for
>>>   the other reasons presented here.
>>
>> This implies that packages might themselves depend on newer versions of
>> package.el -- which I think one should put some thought into before
>> anything is done.
>
> How so? (Further down I suggest making some popular packages temporarily
> depend on Package from GNU Elpa as a means to get that installed as
> widely as possible, but that doesn't mean that those packages actually
> *need* a newer Package.)
>
>> How can you e.g. change the way versions are handled
>> in a way that people with older versions of package.el could still
>> handle the change without confusion?
>
> You can't, and that is exactly the point I was trying to make.  Certain
> things (including, but not limited to how version strings are handled)
> are effectively set in stone, unless certain packages/features are made
> available to users who, for whatever reason, stick to an old Emacs
> release.

Other than the version, it seems the other big one is the dependency
list.  Is there anything else that could cause issues?

>> (This will be an issue for at
>> least a decade to come, knowing the pace at which users get access to
>> newer versions of Emacs).  You mentioned updating the ELPA protocol a
>> few months back, and I believe a few points of interest were collected
>> in that thread.  It seems to me these issues are all intertwined.
>
> Yes indeed, they are all intertwined, and I am trying to make the point
> that we have to stop muddling along once in a while, and make a breaking
> change, even if the transition is rough and inconveniences users for a
> while.
>
>> With Emacs 29, `package-refresh-contents' now calls a Hook called
>> `package-refresh-contents-hook'.  This was initially introduced so that
>> package-vc can download the "elpa-packages.eld" file, but it could also
>> be of use here as well -- but it wouldn't help anyone with Emacs 28 or
>> older...  I could imagine a more hacky approach based on carefully
>> placed advice, but only as a last resort.
>
> We cannot travel back in time to fix past mistakes/omissions that are
> holding us back in the presence.  All we can do now is to pay the price
> and try harder to be forward compatible this time around.  Or live with
> the limitations forever.  Or wait some more until they become unbearable
> and a fix more urgent; but that would increase the odds of messing it up
> again.

2+

>>> Addressing (1) is harder, and I don't think it possible to do so in a
>>> way that guarantees that not a single user would end up seeing an error
>>> due to an incompatible change.  On the other hand "archive-contents"
>>> comes with a version field, and if we bump that, we at least get a
>>> meaningful warning: "Package archive version 2 is higher than 1".  This
>>> doesn't say that the solution is to install `package' from GNU ELPA, but
>>> it should be enough to feed into a search engine to get to that answer.
>>
>> The upgrading procedure for archive-contents has never appeared to me to
>> be very robust.  Perhaps it would be better to introduce a second file
>> (archive-contents.eld) with a more flexible format?
>
> An effort was made to provide an upgrade procedure -- the
> "archive-contents" were prefixed with a version number.  But then
> package.el was added to Emacs and we stopped to distribute it
> separately.  Apparently we didn't realize at the time that this only
> allowed us to tell the user "the archive and tool are not compatible,
> deal with it".

What could be done instead?  Should the updated version of package.el
try to install a newer version of itself in case the version string is
updated?

> We could continue to distribute "archive-contents", which continues to
> use version 1, and add "archive-contents.eld" which uses version 2.
> But that would do nothing to get users to install a forward-compatible
> version of Package (in the sense that once it becomes incompatible, it
> will provide a smooth upgrade path to the new version).

This seems like the better option to me.  Most people don't /need/ the
newest version right away, and even if they did there would be no
straightforward way of making sure all users would have it installed
within some fixed time-frame.

>>> Additionally, we could get many users to install `package' from GNU
>>> ELPA, by making a few popular packages explicitly depend on a `package'
>>> version that is available from GNU ELPA for a few months.
>>
>> Again, a few months wouldn't be enough to solve the issue.  But again as
>> well, Compat could help by adding package as a noop dependency.
>
> Much like your "archive-contents.eld" suggestion, doing this would not
> *solve* the issue.  Unlike that, it would however significantly reduce
> the number of users who would be negatively affected by the upgrade.
> (We could do it for a few years instead of months to further decrease
> the number of affected users.)

You are probably right.

> I love Compat, but I don't think it is the right hammer here (except
> maybe for the part where it could provide vc-clone et al.).
>
> Maybe it helps to make it explicit what is going to happen to users
> if/when all the *ELPA start to expect that package.el support
> "archive-contents" v2.
>
> - The user runs "M-x package-refresh-contents" and is told "Package
>   archive version 2 is higher than 1".
>
> - They ask a search engine what that means.  We could create a few blog
>   and forum posts before hand and give the search engines a few days to
>   index them, providing the following instructions:
>
>   (We should invest some time to investigate how to make this as smooth
>   as possible, but basically:)
>
> - 1. wget https://...package-2.0.0.tar
>   2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/
>   3. Restart Emacs and run package-refresh-contents again.

This is exactly what I would like to avoid.  What happens when the .tar
disappears and some user tries to fix this issue in a few years with
these outdated instructions?  What if they use .emacs.d or some other
directory?  For while there will be plenty of users that will figure
this out, I know many others will be confused (this also makes me think
that a v2 should have support for some kind of MOTD system to explain
issues like these).

So again, what would be the issue with an opt-in system to upgrading
package.el, that could also be pushed forward by a few popular packages,
without the need to break anything.

>      Cheers, 
>      Jonas

Re: Request to add Package to GNU ELPA

[Jonas Bernoulli]

Philip Kaludercic <philipk@posteo.net> writes:

> Jonas Bernoulli <jonas@bernoul.li> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>>

>> Going one step further we could make package-vc available as a separate
>> package.  That would not be of much use *now*, but future improvements
>> would then be available to users of Emacs 29.
>
> I would in principle be interested in supporting this.

I am hoping that someone would take charge of Package on Emacs' side,
and bring some new momentum to its development.  You would be a good
candidate for that. ;D

Your package-vc is welcome addition (I haven't really tried it, since
my Borg serves me well) and Stefan certainly put a lot of work into his
elpa-admin, but Package itself has been a bit stagnant.

[I myself contribute to a lot of elisp packaging related projects, and
 do not plan to take on any new responsibilities; in fact I am trying
 to reduce them.]

>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>> enough to backport, either as part of Compat or in package-vc.el.
>
> [...]

I assume that means "well it may *seem* that way, but if you are the one
actually doing the work...".  I know the feeling.  oO

>> [using a bleeding edge *ELPA]
>
> My experience was not that good, especially back before I got into Emacs
> development (or understood what was going on at all) I constantly ran
> into issues when updating packages from MELPA.  The result was that I'd
> usually just not update packages at all for long periods of time.  The
> next best thing for me was MELPA stable until NonGNU ELPA came around.

Using MELPA stable comes with its own problems.  MELPA's maintainers
advice users to stick to the bleeding-edge variant and use that
themselves.  And many users take that advice to heart, and many
developers are aware of that and act accordingly.

I don't think anyone is against moving towards relying more on released,
stable versions, but there seem to be some chicken and eggs involved.
We can all do a little bit to get closer to a more stable ecosystem, but
it will take time.

[I expect that once MELPA starts using sane version strings for snapshot
 packages, that will be a big step in that direction, because that makes
 it much simpler to properly specify the minimal required versions of
 dependencies, that work for both the bleeding-edge and stable channels.

 I am in fact working on that right now.  I was about to start rebuild
 all packages for testing purposes, but briefly got distracted by your
 reply to this. :P

 But there are other areas where improvements are needed, and I would
 like to leave it to others to tackle those.]

>>>> - I should also mention that my main motivation for pushing for this
>>>>   now, is that I would like to improve version handling.  That is a
>>>>   whole other can of worms, so I do not wish to discuss it just yet,
>>>>   to avoid distracting from the topic at hand, but I thought I should
>>>>   at least mention it.  You might very well end up being against my
>>>>   suggestions regarding versions strings, once I present them, but that
>>>>   should not be cause to oppose the change requested here as well.  Even
>>>>   if my suggestions regarding version strings are ultimately rejected, I
>>>>   still think it would be a good idea to add `package' to GNU ELPA, for
>>>>   the other reasons presented here.
>>>
>>> This implies that packages might themselves depend on newer versions of
>>> package.el -- which I think one should put some thought into before
>>> anything is done.
>>
>> How so? (Further down I suggest making some popular packages temporarily
>> depend on Package from GNU Elpa as a means to get that installed as
>> widely as possible, but that doesn't mean that those packages actually
>> *need* a newer Package.)
>>
>>> How can you e.g. change the way versions are handled
>>> in a way that people with older versions of package.el could still
>>> handle the change without confusion?
>>
>> You can't, and that is exactly the point I was trying to make.  Certain
>> things (including, but not limited to how version strings are handled)
>> are effectively set in stone, unless certain packages/features are made
>> available to users who, for whatever reason, stick to an old Emacs
>> release.
>
> Other than the version, it seems the other big one is the dependency
> list.  Is there anything else that could cause issues?

I can't think of any right now, but that of course doesn't mean we won't
discover any later on.

>>> The upgrading procedure for archive-contents has never appeared to me to
>>> be very robust.  Perhaps it would be better to introduce a second file
>>> (archive-contents.eld) with a more flexible format?
>>
>> An effort was made to provide an upgrade procedure -- the
>> "archive-contents" were prefixed with a version number.  But then
>> package.el was added to Emacs and we stopped to distribute it
>> separately.  Apparently we didn't realize at the time that this only
>> allowed us to tell the user "the archive and tool are not compatible,
>> deal with it".
>
> What could be done instead?  Should the updated version of package.el
> try to install a newer version of itself in case the version string is
> updated?

Yes, that's what I had in mind.

>> We could continue to distribute "archive-contents", which continues to
>> use version 1, and add "archive-contents.eld" which uses version 2.
>> But that would do nothing to get users to install a forward-compatible
>> version of Package (in the sense that once it becomes incompatible, it
>> will provide a smooth upgrade path to the new version).
>
> This seems like the better option to me.  Most people don't /need/ the
> newest version right away, and even if they did there would be no
> straightforward way of making sure all users would have it installed
> within some fixed time-frame.

We keep "archive-contents" going for as long as possible.  For the final
push to get users to upgrade, all packages except Package and its
dependencies could be removed from "archive-contents".  A pseudo package
could be added, whose summary (displayed in the list) and commentary
(displayed by describe-package) could be used to instruct users to
upgrade Package, and how.

>>   (We should invest some time to investigate how to make this as smooth
>>   as possible, but basically:)
>>
>> - 1. wget https://...package-2.0.0.tar
>>   2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/
>>   3. Restart Emacs and run package-refresh-contents again.
>
> This is exactly what I would like to avoid.

I am certainly interested in hearing better ideas.

> What happens when the .tar disappears and some user tries to fix this
> issue in a few years with these outdated instructions?

We should prevent that.  The tar file mentioned in the upgrade
instructions doesn't have to be hosted in the usual location only.
It could additionally be hosted somewhere else on gnu.org, where it
is safe from being discarded by elpa-admin or some cleanup script
that isn't aware of this special case.

> What if they use .emacs.d or some other directory?  For while there
> will be plenty of users that will figure this out, I know many others
> will be confused

Well, these steps were the executive summary, the instructions actually
shown to users should of course be more detailed and account for such
differences.

> (this also makes me think that a v2 should have support for some kind
> of MOTD system to explain issues like these).

+1

> So again, what would be the issue with an opt-in system to upgrading
> package.el, that could also be pushed forward by a few popular packages,
> without the need to break anything.

I am not against a long transitional phase, but I think that eventually
we will want to make some breaking change.

One example of an incompatible change is a change in how version strings
are handled.  As I said, I would like to advocate that in a separate
thread.  Here, it is just serves as an example, other incompatible
changes may become desirable.  Just the gist of it:

For a while [Non]GNU-devel ELPA used this version string format:

  VERSION.0-snapshotTIMESTAMP

"snapshot" has to be used because "archive-contents" contains the
version string in list format, e.g., (1 2 3 0 -4 20230405 111).  All of
"snapshot", "cvs", "git", "bzr", "svn", "hg", "darcs" and "unknown" are
encoded as -4 in the list format, but package-version-join maps it to
"snapshot".

That leads to very long version strings, which I assume is why Stefan
switched to just

  VERSION.0.TIMESTAMP

The ".0" serves as a pseudo separator.  Currently "1.0-snapshot" is
*smaller* than "1.0".  If we inject an additional ".0" then it is
smaller than "N.0", but not smaller than "N".

I would like to use a format that not only supported "pre-releases" but
also "post-releases".  Debian uses "post-releases" to implement
"debian-revision" [1]; we could use it to separate the "timestamp" part
from "latest released upstream version" part, in snapshot release
version strings.

[1]: https://manpages.debian.org/wheezy/dpkg-dev/deb-version.5.en.html

But again, maybe I am the only who feels a need to do something about
that; in this context it is just an example of a change that would break
backward compatibility; to demonstrate that some potential changes could
not be made merely opt-in, but instead would have to be either rolled
out to everyone or nobody.

     Jonas

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>>>
>
>>> Going one step further we could make package-vc available as a separate
>>> package.  That would not be of much use *now*, but future improvements
>>> would then be available to users of Emacs 29.
>>
>> I would in principle be interested in supporting this.
>
> I am hoping that someone would take charge of Package on Emacs' side,
> and bring some new momentum to its development.  You would be a good
> candidate for that. ;D
>
> Your package-vc is welcome addition (I haven't really tried it, since
> my Borg serves me well) and Stefan certainly put a lot of work into his
> elpa-admin, but Package itself has been a bit stagnant.
>
> [I myself contribute to a lot of elisp packaging related projects, and
>  do not plan to take on any new responsibilities; in fact I am trying
>  to reduce them.]
>
>>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>>> enough to backport, either as part of Compat or in package-vc.el.
>>
>> [...]
>
> I assume that means "well it may *seem* that way, but if you are the one
> actually doing the work...".  I know the feeling.  oO
>
>>> [using a bleeding edge *ELPA]
>>
>> My experience was not that good, especially back before I got into Emacs
>> development (or understood what was going on at all) I constantly ran
>> into issues when updating packages from MELPA.  The result was that I'd
>> usually just not update packages at all for long periods of time.  The
>> next best thing for me was MELPA stable until NonGNU ELPA came around.
>
> Using MELPA stable comes with its own problems.  MELPA's maintainers
> advice users to stick to the bleeding-edge variant and use that
> themselves.  And many users take that advice to heart, and many
> developers are aware of that and act accordingly.
>
> I don't think anyone is against moving towards relying more on released,
> stable versions, but there seem to be some chicken and eggs involved.
> We can all do a little bit to get closer to a more stable ecosystem, but
> it will take time.
>
> [I expect that once MELPA starts using sane version strings for snapshot
>  packages, that will be a big step in that direction, because that makes
>  it much simpler to properly specify the minimal required versions of
>  dependencies, that work for both the bleeding-edge and stable channels.
>
>  I am in fact working on that right now.  I was about to start rebuild
>  all packages for testing purposes, but briefly got distracted by your
>  reply to this. :P
>
>  But there are other areas where improvements are needed, and I would
>  like to leave it to others to tackle those.]
>
>>>>> - I should also mention that my main motivation for pushing for this
>>>>>   now, is that I would like to improve version handling.  That is a
>>>>>   whole other can of worms, so I do not wish to discuss it just yet,
>>>>>   to avoid distracting from the topic at hand, but I thought I should
>>>>>   at least mention it.  You might very well end up being against my
>>>>>   suggestions regarding versions strings, once I present them, but that
>>>>>   should not be cause to oppose the change requested here as well.  Even
>>>>>   if my suggestions regarding version strings are ultimately rejected, I
>>>>>   still think it would be a good idea to add `package' to GNU ELPA, for
>>>>>   the other reasons presented here.
>>>>
>>>> This implies that packages might themselves depend on newer versions of
>>>> package.el -- which I think one should put some thought into before
>>>> anything is done.
>>>
>>> How so? (Further down I suggest making some popular packages temporarily
>>> depend on Package from GNU Elpa as a means to get that installed as
>>> widely as possible, but that doesn't mean that those packages actually
>>> *need* a newer Package.)
>>>
>>>> How can you e.g. change the way versions are handled
>>>> in a way that people with older versions of package.el could still
>>>> handle the change without confusion?
>>>
>>> You can't, and that is exactly the point I was trying to make.  Certain
>>> things (including, but not limited to how version strings are handled)
>>> are effectively set in stone, unless certain packages/features are made
>>> available to users who, for whatever reason, stick to an old Emacs
>>> release.
>>
>> Other than the version, it seems the other big one is the dependency
>> list.  Is there anything else that could cause issues?
>
> I can't think of any right now, but that of course doesn't mean we won't
> discover any later on.
>
>>>> The upgrading procedure for archive-contents has never appeared to me to
>>>> be very robust.  Perhaps it would be better to introduce a second file
>>>> (archive-contents.eld) with a more flexible format?
>>>
>>> An effort was made to provide an upgrade procedure -- the
>>> "archive-contents" were prefixed with a version number.  But then
>>> package.el was added to Emacs and we stopped to distribute it
>>> separately.  Apparently we didn't realize at the time that this only
>>> allowed us to tell the user "the archive and tool are not compatible,
>>> deal with it".
>>
>> What could be done instead?  Should the updated version of package.el
>> try to install a newer version of itself in case the version string is
>> updated?
>
> Yes, that's what I had in mind.
>
>>> We could continue to distribute "archive-contents", which continues to
>>> use version 1, and add "archive-contents.eld" which uses version 2.
>>> But that would do nothing to get users to install a forward-compatible
>>> version of Package (in the sense that once it becomes incompatible, it
>>> will provide a smooth upgrade path to the new version).
>>
>> This seems like the better option to me.  Most people don't /need/ the
>> newest version right away, and even if they did there would be no
>> straightforward way of making sure all users would have it installed
>> within some fixed time-frame.
>
> We keep "archive-contents" going for as long as possible.  For the final
> push to get users to upgrade, all packages except Package and its
> dependencies could be removed from "archive-contents".  A pseudo package
> could be added, whose summary (displayed in the list) and commentary
> (displayed by describe-package) could be used to instruct users to
> upgrade Package, and how.
>
>>>   (We should invest some time to investigate how to make this as smooth
>>>   as possible, but basically:)
>>>
>>> - 1. wget https://...package-2.0.0.tar
>>>   2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/
>>>   3. Restart Emacs and run package-refresh-contents again.
>>
>> This is exactly what I would like to avoid.
>
> I am certainly interested in hearing better ideas.
>
>> What happens when the .tar disappears and some user tries to fix this
>> issue in a few years with these outdated instructions?
>
> We should prevent that.  The tar file mentioned in the upgrade
> instructions doesn't have to be hosted in the usual location only.
> It could additionally be hosted somewhere else on gnu.org, where it
> is safe from being discarded by elpa-admin or some cleanup script
> that isn't aware of this special case.
>
>> What if they use .emacs.d or some other directory?  For while there
>> will be plenty of users that will figure this out, I know many others
>> will be confused
>
> Well, these steps were the executive summary, the instructions actually
> shown to users should of course be more detailed and account for such
> differences.
>
>> (this also makes me think that a v2 should have support for some kind
>> of MOTD system to explain issues like these).
>
> +1
>
>> So again, what would be the issue with an opt-in system to upgrading
>> package.el, that could also be pushed forward by a few popular packages,
>> without the need to break anything.
>
> I am not against a long transitional phase, but I think that eventually
> we will want to make some breaking change.
>
> One example of an incompatible change is a change in how version strings
> are handled.  As I said, I would like to advocate that in a separate
> thread.  Here, it is just serves as an example, other incompatible
> changes may become desirable.  Just the gist of it:
>
> For a while [Non]GNU-devel ELPA used this version string format:
>
>   VERSION.0-snapshotTIMESTAMP
>
> "snapshot" has to be used because "archive-contents" contains the
> version string in list format, e.g., (1 2 3 0 -4 20230405 111).  All of
> "snapshot", "cvs", "git", "bzr", "svn", "hg", "darcs" and "unknown" are
> encoded as -4 in the list format, but package-version-join maps it to
> "snapshot".
>
> That leads to very long version strings, which I assume is why Stefan
> switched to just
>
>   VERSION.0.TIMESTAMP
>
> The ".0" serves as a pseudo separator.  Currently "1.0-snapshot" is
> *smaller* than "1.0".  If we inject an additional ".0" then it is
> smaller than "N.0", but not smaller than "N".
>
> I would like to use a format that not only supported "pre-releases" but
> also "post-releases".  Debian uses "post-releases" to implement
> "debian-revision" [1]; we could use it to separate the "timestamp" part
> from "latest released upstream version" part, in snapshot release
> version strings.
>
> [1]: https://manpages.debian.org/wheezy/dpkg-dev/deb-version.5.en.html
>
> But again, maybe I am the only who feels a need to do something about
> that; in this context it is just an example of a change that would break
> backward compatibility; to demonstrate that some potential changes could
> not be made merely opt-in, but instead would have to be either rolled
> out to everyone or nobody.
>
>      Jonas

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

(I apologise for the last message, accidentally sent it out before
writing anything (somehow?))

Jonas Bernoulli <jonas@bernoul.li> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>>>
>
>>> Going one step further we could make package-vc available as a separate
>>> package.  That would not be of much use *now*, but future improvements
>>> would then be available to users of Emacs 29.
>>
>> I would in principle be interested in supporting this.
>
> I am hoping that someone would take charge of Package on Emacs' side,
> and bring some new momentum to its development.  You would be a good
> candidate for that. ;D

I am glad to help where I can (e.g. making sure that package.el can be
distributed on ELPA), but I don't know exactly what you have in mind
with the other changes so I would rather let you tackle those issues.

> Your package-vc is welcome addition (I haven't really tried it, since
> my Borg serves me well) and Stefan certainly put a lot of work into his
> elpa-admin, but Package itself has been a bit stagnant.
>
> [I myself contribute to a lot of elisp packaging related projects, and
>  do not plan to take on any new responsibilities; in fact I am trying
>  to reduce them.]

>>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>>> enough to backport, either as part of Compat or in package-vc.el.
>>
>> [...]
>
> I assume that means "well it may *seem* that way, but if you are the one
> actually doing the work...".  I know the feeling.  oO

I am not sure what you are talking about here?


[...]

>>>>> - I should also mention that my main motivation for pushing for this
>>>>>   now, is that I would like to improve version handling.  That is a
>>>>>   whole other can of worms, so I do not wish to discuss it just yet,
>>>>>   to avoid distracting from the topic at hand, but I thought I should
>>>>>   at least mention it.  You might very well end up being against my
>>>>>   suggestions regarding versions strings, once I present them, but that
>>>>>   should not be cause to oppose the change requested here as well.  Even
>>>>>   if my suggestions regarding version strings are ultimately rejected, I
>>>>>   still think it would be a good idea to add `package' to GNU ELPA, for
>>>>>   the other reasons presented here.
>>>>
>>>> This implies that packages might themselves depend on newer versions of
>>>> package.el -- which I think one should put some thought into before
>>>> anything is done.
>>>
>>> How so? (Further down I suggest making some popular packages temporarily
>>> depend on Package from GNU Elpa as a means to get that installed as
>>> widely as possible, but that doesn't mean that those packages actually
>>> *need* a newer Package.)
>>>
>>>> How can you e.g. change the way versions are handled
>>>> in a way that people with older versions of package.el could still
>>>> handle the change without confusion?
>>>
>>> You can't, and that is exactly the point I was trying to make.  Certain
>>> things (including, but not limited to how version strings are handled)
>>> are effectively set in stone, unless certain packages/features are made
>>> available to users who, for whatever reason, stick to an old Emacs
>>> release.
>>
>> Other than the version, it seems the other big one is the dependency
>> list.  Is there anything else that could cause issues?
>
> I can't think of any right now, but that of course doesn't mean we won't
> discover any later on.

Right, I was just thinking that it would make sense to have some list.

>>>> The upgrading procedure for archive-contents has never appeared to me to
>>>> be very robust.  Perhaps it would be better to introduce a second file
>>>> (archive-contents.eld) with a more flexible format?
>>>
>>> An effort was made to provide an upgrade procedure -- the
>>> "archive-contents" were prefixed with a version number.  But then
>>> package.el was added to Emacs and we stopped to distribute it
>>> separately.  Apparently we didn't realize at the time that this only
>>> allowed us to tell the user "the archive and tool are not compatible,
>>> deal with it".
>>
>> What could be done instead?  Should the updated version of package.el
>> try to install a newer version of itself in case the version string is
>> updated?
>
> Yes, that's what I had in mind.

Sounds good.

>>> We could continue to distribute "archive-contents", which continues to
>>> use version 1, and add "archive-contents.eld" which uses version 2.
>>> But that would do nothing to get users to install a forward-compatible
>>> version of Package (in the sense that once it becomes incompatible, it
>>> will provide a smooth upgrade path to the new version).
>>
>> This seems like the better option to me.  Most people don't /need/ the
>> newest version right away, and even if they did there would be no
>> straightforward way of making sure all users would have it installed
>> within some fixed time-frame.
>
> We keep "archive-contents" going for as long as possible.  For the final
> push to get users to upgrade, all packages except Package and its
> dependencies could be removed from "archive-contents".  A pseudo package
> could be added, whose summary (displayed in the list) and commentary
> (displayed by describe-package) could be used to instruct users to
> upgrade Package, and how.

OK, but this is something I don't expect to take place soon.

>>>   (We should invest some time to investigate how to make this as smooth
>>>   as possible, but basically:)
>>>
>>> - 1. wget https://...package-2.0.0.tar
>>>   2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/
>>>   3. Restart Emacs and run package-refresh-contents again.
>>
>> This is exactly what I would like to avoid.
>
> I am certainly interested in hearing better ideas.

For one thing an Elisp script would be preferable to avoid issues with
people who might not have wget or a specific version of tar installed.

But otherwise I would just try and avoid the need to force users to
upgrade.  Upgrading should be incentivized with the advantages of doing
so, not the issues of (perhaps unconsciously) holding back.

>> What happens when the .tar disappears and some user tries to fix this
>> issue in a few years with these outdated instructions?
>
> We should prevent that.  The tar file mentioned in the upgrade
> instructions doesn't have to be hosted in the usual location only.
> It could additionally be hosted somewhere else on gnu.org, where it
> is safe from being discarded by elpa-admin or some cleanup script
> that isn't aware of this special case.
>
>> What if they use .emacs.d or some other directory?  For while there
>> will be plenty of users that will figure this out, I know many others
>> will be confused
>
> Well, these steps were the executive summary, the instructions actually
> shown to users should of course be more detailed and account for such
> differences.
>
>> (this also makes me think that a v2 should have support for some kind
>> of MOTD system to explain issues like these).
>
> +1
>
>> So again, what would be the issue with an opt-in system to upgrading
>> package.el, that could also be pushed forward by a few popular packages,
>> without the need to break anything.
>
> I am not against a long transitional phase, but I think that eventually
> we will want to make some breaking change.
>
> One example of an incompatible change is a change in how version strings
> are handled.  As I said, I would like to advocate that in a separate
> thread.  Here, it is just serves as an example, other incompatible
> changes may become desirable.  Just the gist of it:
>
> For a while [Non]GNU-devel ELPA used this version string format:
>
>   VERSION.0-snapshotTIMESTAMP
>
> "snapshot" has to be used because "archive-contents" contains the
> version string in list format, e.g., (1 2 3 0 -4 20230405 111).  All of
> "snapshot", "cvs", "git", "bzr", "svn", "hg", "darcs" and "unknown" are
> encoded as -4 in the list format, but package-version-join maps it to
> "snapshot".
>
> That leads to very long version strings, which I assume is why Stefan
> switched to just
>
>   VERSION.0.TIMESTAMP
>
> The ".0" serves as a pseudo separator.  Currently "1.0-snapshot" is
> *smaller* than "1.0".  If we inject an additional ".0" then it is
> smaller than "N.0", but not smaller than "N".
>
> I would like to use a format that not only supported "pre-releases" but
> also "post-releases".  Debian uses "post-releases" to implement
> "debian-revision" [1]; we could use it to separate the "timestamp" part
> from "latest released upstream version" part, in snapshot release
> version strings.
>
> [1]: https://manpages.debian.org/wheezy/dpkg-dev/deb-version.5.en.html
>
> But again, maybe I am the only who feels a need to do something about
> that; in this context it is just an example of a change that would break
> backward compatibility; to demonstrate that some potential changes could
> not be made merely opt-in, but instead would have to be either rolled
> out to everyone or nobody.

I cannot comment on this, but "post-releases" (as opposed to
"pre-releases") sounds like just another entry in `version-regexp-alist'?

>      Jonas

So to summarise, I think an approximate plan would be useful:

1. Update package.el to be published on ELPA
2. (Optional) Update package-vc.el to be published on ELPA
3. Specify and implement a new package-archives format
4. Update package archives to generate the new archive format

Anything I am forgetting?

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 209 bytes --]

Philip Kaludercic <philipk@posteo.net> writes:

> 1. Update package.el to be published on ELPA

It might be that this would be enough to publish package.el with support
back to 27.1, with the help of Compat:


[-- Attachment #2: Type: text/plain, Size: 4484 bytes --]

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 0258ed52bee..520e02f4c33 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -5,9 +5,12 @@
 ;; Author: Tom Tromey <tromey@redhat.com>
 ;;         Daniel Hackney <dan@haxney.org>
 ;; Created: 10 Mar 2007
-;; Version: 1.1.0
+;; Version: 1.1.1
 ;; Keywords: tools
-;; Package-Requires: ((tabulated-list "1.0"))
+;; Package-Requires: ((emacs "27.1") (compat "29.1.0.0"))
+
+;; This is a GNU ELPA :core package.  Avoid functionality that is not
+;; compatible with the version of Emacs recorded above.
 
 ;; This file is part of GNU Emacs.
 
@@ -147,6 +150,7 @@
 (eval-when-compile (require 'subr-x))
 (eval-when-compile (require 'epg))      ;For setf accessors.
 (eval-when-compile (require 'inline))   ;For `define-inline'
+(require 'compat nil 'noerror)
 (require 'seq)
 
 (require 'tabulated-list)
@@ -1124,7 +1128,8 @@ package--native-compile-async
   "Native compile installed package PKG-DESC asynchronously.
 This assumes that `pkg-desc' has already been activated with
 `package-activate-1'."
-  (when (native-comp-available-p)
+  (when (and (fboundp 'native-comp-available-p)
+             (native-comp-available-p))
     (let ((warning-minimum-level :error))
       (native-compile-async (package-desc-dir pkg-desc) t))))
 
@@ -1160,9 +1165,9 @@ package--prepare-dependencies
 
 (declare-function lm-header "lisp-mnt" (header))
 (declare-function lm-header-multiline "lisp-mnt" (header))
-(declare-function lm-website "lisp-mnt" (&optional file))
+(declare-function lm-homepage "lisp-mnt" (&optional file))
 (declare-function lm-keywords-list "lisp-mnt" (&optional file))
-(declare-function lm-maintainers "lisp-mnt" (&optional file))
+(declare-function lm-maintainer "lisp-mnt" (&optional file))
 (declare-function lm-authors "lisp-mnt" (&optional file))
 
 (defun package-buffer-info ()
@@ -1195,7 +1200,7 @@ package-buffer-info
             (or (lm-header "package-version") (lm-header "version")))
            (pkg-version (package-strip-rcs-id version-info))
            (keywords (lm-keywords-list))
-           (website (lm-website)))
+           (website (lm-homepage)))
       (unless pkg-version
          (if version-info
              (error "Unrecognized package version: %s" version-info)
@@ -1208,10 +1213,13 @@ package-buffer-info
        :kind 'single
        :url website
        :keywords keywords
-       :maintainer
+       :maintainer (lm-maintainer)
        ;; For backward compatibility, use a single string if there's only
        ;; one maintainer (the most common case).
-       (let ((maints (lm-maintainers))) (if (cdr maints) maints (car maints)))
+       (if (fboundp 'lm-maintainers)
+           (let ((maints (lm-maintainers)))
+             (if (cdr maints) maints (car maints)))
+         (lm-maintainer))
        :authors (lm-authors)))))
 
 (defun package--read-pkg-desc (kind)
@@ -2302,8 +2310,6 @@ package-strip-rcs-id
       ;; to make sure we use a "canonical name"!
       (if l (package-version-join l)))))
 
-(declare-function lm-website "lisp-mnt" (&optional file))
-
 ;;;###autoload
 (defun package-install-from-buffer ()
   "Install a package from the current buffer.
@@ -2902,8 +2908,11 @@ describe-package-1
 
       ;; Insert news if available.
       (when news
-        (insert "\n" (make-separator-line) "\n"
-                (propertize "* News" 'face 'package-help-section-name)
+        (newline)
+        (when (fboundp 'make-separator-line)
+          (insert (make-separator-line))
+          (newline))
+        (insert (propertize "* News" 'face 'package-help-section-name)
                 "\n\n")
         (insert-file-contents news))
 
@@ -4571,8 +4580,11 @@ package-report-bug
       (dolist (ent (get (cdr group) 'custom-group))
         (when (and (custom-variable-p (car ent))
                    (boundp (car ent))
-                   (not (eq (custom--standard-value (car ent))
-                            (default-toplevel-value (car ent))))
+                   (not (eq
+                         ;; We are not using `custom--standard-value'
+                         ;; to retain compatibility for Emacs 27.
+                         (eval (car (get (car ent) 'standard-value)) t)
+                         (default-toplevel-value (car ent))))
                    (file-in-directory-p (car group) (package-desc-dir desc)))
           (push (car ent) vars))))
     (dlet ((reporter-prompt-for-summary-p t))

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 411 bytes --]

Philip Kaludercic <philipk@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> 1. Update package.el to be published on ELPA
>
> It might be that this would be enough to publish package.el with support
> back to 27.1, with the help of Compat:

Stefan fell out of the CCs, so I've added him again since I am most
interested in what he thinks of these changes, added here again as a
patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Prepare-package.el-to-be-published-on-GNU-ELPA.patch --]
[-- Type: text/x-diff, Size: 5385 bytes --]

From 1fcf56725848984ef6da50ef2b7a130e1ffd3282 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 5 Apr 2023 23:16:39 +0200
Subject: [PATCH] Prepare package.el to be published on GNU ELPA

* lisp/emacs-lisp/package.el: Add Compat as a dependency.
(package--native-compile-async): Check if 'native-comp-available-p' is
bound.
(lm-homepage, lm-website): Use backwards-compatible alias
'lm-homepage'.
(package-buffer-info): Call 'lm-maintainer' if 'lm-maintainers' is not
defined.
(describe-package-1): Avoid using 'make-separator-line' if not bound.
(package-report-bug): Expand 'custom--standard-value' definition.

Check out these discussions for more context:
https://lists.gnu.org/archive/html/emacs-devel/2023-03/msg00995.html.
---
 lisp/emacs-lisp/package.el | 39 +++++++++++++++++++++++++-------------
 1 file changed, 26 insertions(+), 13 deletions(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 0258ed52bee..6b68c84a211 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -5,9 +5,12 @@
 ;; Author: Tom Tromey <tromey@redhat.com>
 ;;         Daniel Hackney <dan@haxney.org>
 ;; Created: 10 Mar 2007
-;; Version: 1.1.0
+;; Version: 1.1.1
 ;; Keywords: tools
-;; Package-Requires: ((tabulated-list "1.0"))
+;; Package-Requires: ((emacs "26.1") (compat "29.1.0.0"))
+
+;; This is a GNU ELPA :core package.  Avoid functionality that is not
+;; compatible with the version of Emacs recorded above.
 
 ;; This file is part of GNU Emacs.
 
@@ -147,6 +150,7 @@
 (eval-when-compile (require 'subr-x))
 (eval-when-compile (require 'epg))      ;For setf accessors.
 (eval-when-compile (require 'inline))   ;For `define-inline'
+(require 'compat nil 'noerror)
 (require 'seq)
 
 (require 'tabulated-list)
@@ -1124,7 +1128,8 @@ package--native-compile-async
   "Native compile installed package PKG-DESC asynchronously.
 This assumes that `pkg-desc' has already been activated with
 `package-activate-1'."
-  (when (native-comp-available-p)
+  (when (and (fboundp 'native-comp-available-p)
+             (native-comp-available-p))
     (let ((warning-minimum-level :error))
       (native-compile-async (package-desc-dir pkg-desc) t))))
 
@@ -1160,9 +1165,10 @@ package--prepare-dependencies
 
 (declare-function lm-header "lisp-mnt" (header))
 (declare-function lm-header-multiline "lisp-mnt" (header))
-(declare-function lm-website "lisp-mnt" (&optional file))
+(declare-function lm-homepage "lisp-mnt" (&optional file))
 (declare-function lm-keywords-list "lisp-mnt" (&optional file))
 (declare-function lm-maintainers "lisp-mnt" (&optional file))
+(declare-function lm-maintainer "lisp-mnt" (&optional file))
 (declare-function lm-authors "lisp-mnt" (&optional file))
 
 (defun package-buffer-info ()
@@ -1195,7 +1201,7 @@ package-buffer-info
             (or (lm-header "package-version") (lm-header "version")))
            (pkg-version (package-strip-rcs-id version-info))
            (keywords (lm-keywords-list))
-           (website (lm-website)))
+           (website (lm-homepage)))
       (unless pkg-version
          (if version-info
              (error "Unrecognized package version: %s" version-info)
@@ -1208,10 +1214,13 @@ package-buffer-info
        :kind 'single
        :url website
        :keywords keywords
-       :maintainer
+       :maintainer (lm-maintainer)
        ;; For backward compatibility, use a single string if there's only
        ;; one maintainer (the most common case).
-       (let ((maints (lm-maintainers))) (if (cdr maints) maints (car maints)))
+       (if (fboundp 'lm-maintainers)
+           (let ((maints (lm-maintainers)))
+             (if (cdr maints) maints (car maints)))
+         (lm-maintainer))
        :authors (lm-authors)))))
 
 (defun package--read-pkg-desc (kind)
@@ -2302,8 +2311,6 @@ package-strip-rcs-id
       ;; to make sure we use a "canonical name"!
       (if l (package-version-join l)))))
 
-(declare-function lm-website "lisp-mnt" (&optional file))
-
 ;;;###autoload
 (defun package-install-from-buffer ()
   "Install a package from the current buffer.
@@ -2902,8 +2909,11 @@ describe-package-1
 
       ;; Insert news if available.
       (when news
-        (insert "\n" (make-separator-line) "\n"
-                (propertize "* News" 'face 'package-help-section-name)
+        (newline)
+        (when (fboundp 'make-separator-line)
+          (insert (make-separator-line))
+          (newline))
+        (insert (propertize "* News" 'face 'package-help-section-name)
                 "\n\n")
         (insert-file-contents news))
 
@@ -4571,8 +4581,11 @@ package-report-bug
       (dolist (ent (get (cdr group) 'custom-group))
         (when (and (custom-variable-p (car ent))
                    (boundp (car ent))
-                   (not (eq (custom--standard-value (car ent))
-                            (default-toplevel-value (car ent))))
+                   (not (eq
+                         ;; We are not using `custom--standard-value'
+                         ;; to retain compatibility for Emacs 27.
+                         (eval (car (get (car ent) 'standard-value)) t)
+                         (default-toplevel-value (car ent))))
                    (file-in-directory-p (car group) (package-desc-dir desc)))
           (push (car ent) vars))))
     (dlet ((reporter-prompt-for-summary-p t))
-- 
2.39.2


[-- Attachment #3: Type: text/plain, Size: 152 bytes --]


elpa-admin.el appears to be handling the changes well, I'll still have
to try and install it on an older Emacs to see if everything works as
expected.

Re: Request to add Package to GNU ELPA

[Stefan Monnier]

> +;; Package-Requires: ((emacs "26.1") (compat "29.1.0.0"))

[ I'm curious which definitions of `compat` this uses.  ]

> @@ -1208,10 +1214,13 @@ package-buffer-info
>         :kind 'single
>         :url website
>         :keywords keywords
> -       :maintainer
> +       :maintainer (lm-maintainer)
>         ;; For backward compatibility, use a single string if there's only
>         ;; one maintainer (the most common case).
> -       (let ((maints (lm-maintainers))) (if (cdr maints) maints (car maints)))
> +       (if (fboundp 'lm-maintainers)
> +           (let ((maints (lm-maintainers)))
> +             (if (cdr maints) maints (car maints)))
> +         (lm-maintainer))
>         :authors (lm-authors)))))

Hmm... isn't the `(lm-maintainer)` right after `:maintainer` erroneous?
[ BTW, I think Jonas has another change pending for this code.  ]

>        (when news
> -        (insert "\n" (make-separator-line) "\n"
> -                (propertize "* News" 'face 'package-help-section-name)
> +        (newline)
> +        (when (fboundp 'make-separator-line)
> +          (insert (make-separator-line))
> +          (newline))
> +        (insert (propertize "* News" 'face 'package-help-section-name)
>                  "\n\n")
>          (insert-file-contents news))

Why not just replace (make-separator-line) with
(if (fboundp 'make-separator-line) (make-separator-line) "")?
`newline` is a command and I think it's preferable not to use it
from ELisp.

> @@ -4571,8 +4581,11 @@ package-report-bug
>        (dolist (ent (get (cdr group) 'custom-group))
>          (when (and (custom-variable-p (car ent))
>                     (boundp (car ent))
> -                   (not (eq (custom--standard-value (car ent))
> -                            (default-toplevel-value (car ent))))
> +                   (not (eq
> +                         ;; We are not using `custom--standard-value'
> +                         ;; to retain compatibility for Emacs 27.
> +                         (eval (car (get (car ent) 'standard-value)) t)
> +                         (default-toplevel-value (car ent))))

I'd use

    (if (fboundp 'custom--standard-value) ;; Emacs≥27
        (custom--standard-value (car ent))
      (eval (car (get (car ent) 'standard-value)) t))

so the code "speaks for itself".


        Stefan

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> +;; Package-Requires: ((emacs "26.1") (compat "29.1.0.0"))
>
> [ I'm curious which definitions of `compat` this uses.  ]

Those should be

- file-name-concat
- length>
- with-suppressed-warnings
- named-let
- length=
- ensure-list
- macroexp-file-name
- dlet

Do you think it is worth mentioning them?

>> @@ -1208,10 +1214,13 @@ package-buffer-info
>>         :kind 'single
>>         :url website
>>         :keywords keywords
>> -       :maintainer
>> +       :maintainer (lm-maintainer)
>>         ;; For backward compatibility, use a single string if there's only
>>         ;; one maintainer (the most common case).
>> -       (let ((maints (lm-maintainers))) (if (cdr maints) maints (car maints)))
>> +       (if (fboundp 'lm-maintainers)
>> +           (let ((maints (lm-maintainers)))
>> +             (if (cdr maints) maints (car maints)))
>> +         (lm-maintainer))
>>         :authors (lm-authors)))))
>
> Hmm... isn't the `(lm-maintainer)` right after `:maintainer` erroneous?

Yes, that was a typo.

> [ BTW, I think Jonas has another change pending for this code.  ]
>
>>        (when news
>> -        (insert "\n" (make-separator-line) "\n"
>> -                (propertize "* News" 'face 'package-help-section-name)
>> +        (newline)
>> +        (when (fboundp 'make-separator-line)
>> +          (insert (make-separator-line))
>> +          (newline))
>> +        (insert (propertize "* News" 'face 'package-help-section-name)
>>                  "\n\n")
>>          (insert-file-contents news))
>
> Why not just replace (make-separator-line) with
> (if (fboundp 'make-separator-line) (make-separator-line) "")?

I believe what I had in mind was to avoid formatting issues when
`make-separator-line' was not available.  I'll check again.

> `newline` is a command and I think it's preferable not to use it
> from ELisp.

Oops, right.

>> @@ -4571,8 +4581,11 @@ package-report-bug
>>        (dolist (ent (get (cdr group) 'custom-group))
>>          (when (and (custom-variable-p (car ent))
>>                     (boundp (car ent))
>> -                   (not (eq (custom--standard-value (car ent))
>> -                            (default-toplevel-value (car ent))))
>> +                   (not (eq
>> +                         ;; We are not using `custom--standard-value'
>> +                         ;; to retain compatibility for Emacs 27.
>> +                         (eval (car (get (car ent) 'standard-value)) t)
>> +                         (default-toplevel-value (car ent))))
>
> I'd use
>
>     (if (fboundp 'custom--standard-value) ;; Emacs≥27
>         (custom--standard-value (car ent))
>       (eval (car (get (car ent) 'standard-value)) t))
>
> so the code "speaks for itself".

Good point.

-- 
Philip Kaludercic

Re: Request to add Package to GNU ELPA

[Stefan Monnier]

>> [ I'm curious which definitions of `compat` this uses.  ]
>
> Those should be
>
> - file-name-concat
> - length>
> - with-suppressed-warnings
> - named-let
> - length=
> - ensure-list
> - macroexp-file-name
> - dlet

Thanks.  Nice.

> Do you think it is worth mentioning them?

No, I don't think so.

The patch LGTM now,


        Stefan

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 618 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> [ I'm curious which definitions of `compat` this uses.  ]
>>
>> Those should be
>>
>> - file-name-concat
>> - length>
>> - with-suppressed-warnings
>> - named-let
>> - length=
>> - ensure-list
>> - macroexp-file-name
>> - dlet
>
> Thanks.  Nice.
>
>> Do you think it is worth mentioning them?
>
> No, I don't think so.
>
> The patch LGTM now,

As mentioned in my other message, we are also missing
`loaddefs-generate'.  I am not exactly sure what motivated the change,
so I don't know if checking the version and doing what was done prior to
29 would be enough:


[-- Attachment #2: Type: text/plain, Size: 1059 bytes --]

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index b340848a6f9..48551f59b43 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -1003,6 +1003,7 @@ package-generate-description-file
 
 (defun package-autoload-ensure-default-file (file)
   "Make sure that the autoload file FILE exists and if not create it."
+  (declare (obsolete nil "29.1"))
   (unless (file-exists-p file)
     (require 'autoload)
     (let ((coding-system-for-write 'utf-8-emacs-unix))
@@ -1021,8 +1022,11 @@ package-generate-autoloads
          (autoload-timestamps nil)
          (backup-inhibited t)
          (version-control 'never))
-    (package-autoload-ensure-default-file output-file)
-    (make-directory-autoloads pkg-dir output-file)
+    (loaddefs-generate
+     pkg-dir output-file
+     nil
+     "(add-to-list 'load-path (directory-file-name
+                         (or (file-name-directory #$) (car load-path))))")
     (let ((buf (find-buffer-visiting output-file)))
       (when buf (kill-buffer buf)))
     auto-name))

[-- Attachment #3: Type: text/plain, Size: 23 bytes --]


-- 
Philip Kaludercic

Re: Request to add Package to GNU ELPA

[Stefan Monnier]

> As mentioned in my other message, we are also missing
> `loaddefs-generate'.  I am not exactly sure what motivated the change,
> so I don't know if checking the version and doing what was done prior to
> 29 would be enough:

I'd check (fboundp 'loaddefs-generate) to select between the old and the
new code.


        Stefan

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 403 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As mentioned in my other message, we are also missing
>> `loaddefs-generate'.  I am not exactly sure what motivated the change,
>> so I don't know if checking the version and doing what was done prior to
>> 29 would be enough:
>
> I'd check (fboundp 'loaddefs-generate) to select between the old and the
> new code.

Ok, I have modified the patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Prepare-package.el-to-be-published-on-GNU-ELPA.patch --]
[-- Type: text/x-diff, Size: 6070 bytes --]

From e63ac132cc8d220e642836f8f246d82147793f57 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 5 Apr 2023 23:16:39 +0200
Subject: [PATCH] Prepare package.el to be published on GNU ELPA

* lisp/emacs-lisp/package.el: Add Compat as a dependency.
(package--native-compile-async): Check if 'native-comp-available-p' is
bound.
(lm-homepage, lm-website): Use backwards-compatible alias
'lm-homepage'.
(package-buffer-info): Call 'lm-maintainer' if 'lm-maintainers' is not
defined.
(describe-package-1): Avoid using 'make-separator-line' if not bound.
(package-report-bug): Expand 'custom--standard-value' definition.

For the background and motivation behind these changes, please consult
this thread:
https://lists.gnu.org/archive/html/emacs-devel/2023-03/msg00995.html.
---
 lisp/emacs-lisp/package.el | 51 ++++++++++++++++++++++++--------------
 1 file changed, 32 insertions(+), 19 deletions(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 0258ed52bee..2657a4f98b4 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -5,9 +5,12 @@
 ;; Author: Tom Tromey <tromey@redhat.com>
 ;;         Daniel Hackney <dan@haxney.org>
 ;; Created: 10 Mar 2007
-;; Version: 1.1.0
+;; Version: 1.1.1
 ;; Keywords: tools
-;; Package-Requires: ((tabulated-list "1.0"))
+;; Package-Requires: ((emacs "27.1") (compat "29.1.0.0"))
+
+;; This is a GNU ELPA :core package.  Avoid functionality that is not
+;; compatible with the version of Emacs recorded above.
 
 ;; This file is part of GNU Emacs.
 
@@ -147,6 +150,7 @@
 (eval-when-compile (require 'subr-x))
 (eval-when-compile (require 'epg))      ;For setf accessors.
 (eval-when-compile (require 'inline))   ;For `define-inline'
+(require 'compat nil 'noerror)
 (require 'seq)
 
 (require 'tabulated-list)
@@ -1084,16 +1088,20 @@ package-generate-autoloads
          (autoload-timestamps nil)
          (backup-inhibited t)
          (version-control 'never))
-    (loaddefs-generate
-     pkg-dir output-file nil
-     (prin1-to-string
-      '(add-to-list
-        'load-path
-        ;; Add the directory that will contain the autoload file to
-        ;; the load path.  We don't hard-code `pkg-dir', to avoid
-        ;; issues if the package directory is moved around.
-        (or (and load-file-name (file-name-directory load-file-name))
-            (car load-path)))))
+    (if (fboundp 'loaddefs-generate)
+        (loaddefs-generate
+         pkg-dir output-file nil
+         (prin1-to-string
+          '(add-to-list
+            'load-path
+            ;; Add the directory that will contain the autoload file to
+            ;; the load path.  We don't hard-code `pkg-dir', to avoid
+            ;; issues if the package directory is moved around.
+            (or (and load-file-name (file-name-directory load-file-name))
+                (car load-path)))))
+      (with-suppressed-warnings ((obsolete package-autoload-ensure-default-file))
+        (package-autoload-ensure-default-file output-file))
+      (make-directory-autoloads pkg-dir output-file))
     (let ((buf (find-buffer-visiting output-file)))
       (when buf (kill-buffer buf)))
     auto-name))
@@ -1124,7 +1132,8 @@ package--native-compile-async
   "Native compile installed package PKG-DESC asynchronously.
 This assumes that `pkg-desc' has already been activated with
 `package-activate-1'."
-  (when (native-comp-available-p)
+  (when (and (fboundp 'native-comp-available-p)
+             (native-comp-available-p))
     (let ((warning-minimum-level :error))
       (native-compile-async (package-desc-dir pkg-desc) t))))
 
@@ -1160,9 +1169,10 @@ package--prepare-dependencies
 
 (declare-function lm-header "lisp-mnt" (header))
 (declare-function lm-header-multiline "lisp-mnt" (header))
-(declare-function lm-website "lisp-mnt" (&optional file))
+(declare-function lm-homepage "lisp-mnt" (&optional file))
 (declare-function lm-keywords-list "lisp-mnt" (&optional file))
 (declare-function lm-maintainers "lisp-mnt" (&optional file))
+(declare-function lm-maintainer "lisp-mnt" (&optional file))
 (declare-function lm-authors "lisp-mnt" (&optional file))
 
 (defun package-buffer-info ()
@@ -1195,7 +1205,7 @@ package-buffer-info
             (or (lm-header "package-version") (lm-header "version")))
            (pkg-version (package-strip-rcs-id version-info))
            (keywords (lm-keywords-list))
-           (website (lm-website)))
+           (website (lm-homepage)))
       (unless pkg-version
          (if version-info
              (error "Unrecognized package version: %s" version-info)
@@ -1211,7 +1221,10 @@ package-buffer-info
        :maintainer
        ;; For backward compatibility, use a single string if there's only
        ;; one maintainer (the most common case).
-       (let ((maints (lm-maintainers))) (if (cdr maints) maints (car maints)))
+       (if (fboundp 'lm-maintainers)
+           (let ((maints (lm-maintainers)))
+             (if (cdr maints) maints (car maints)))
+         (lm-maintainer))
        :authors (lm-authors)))))
 
 (defun package--read-pkg-desc (kind)
@@ -2302,8 +2315,6 @@ package-strip-rcs-id
       ;; to make sure we use a "canonical name"!
       (if l (package-version-join l)))))
 
-(declare-function lm-website "lisp-mnt" (&optional file))
-
 ;;;###autoload
 (defun package-install-from-buffer ()
   "Install a package from the current buffer.
@@ -4571,7 +4582,9 @@ package-report-bug
       (dolist (ent (get (cdr group) 'custom-group))
         (when (and (custom-variable-p (car ent))
                    (boundp (car ent))
-                   (not (eq (custom--standard-value (car ent))
+                   (not (eq (if (fboundp 'custom--standard-value)
+                                (custom--standard-value (car ent))
+                              (eval (car (get (car ent) 'standard-value)) t))
                             (default-toplevel-value (car ent))))
                    (file-in-directory-p (car group) (package-desc-dir desc)))
           (push (car ent) vars))))
-- 
2.30.2


[-- Attachment #3: Type: text/plain, Size: 62 bytes --]


Eli, are you OK with this being done?

-- 
Philip Kaludercic

Re: Request to add Package to GNU ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 341 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Why not just replace (make-separator-line) with
> (if (fboundp 'make-separator-line) (make-separator-line) "")?
> `newline` is a command and I think it's preferable not to use it
> from ELisp.

Turns out that Compat supports this, so we don't need the check.

Here is the revised patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Prepare-package.el-to-be-published-on-GNU-ELPA.patch --]
[-- Type: text/x-diff, Size: 4682 bytes --]

From e778cff125cf48f933256c488adbcd362a31f715 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 5 Apr 2023 23:16:39 +0200
Subject: [PATCH] Prepare package.el to be published on GNU ELPA

* lisp/emacs-lisp/package.el: Add Compat as a dependency.
(package--native-compile-async): Check if 'native-comp-available-p' is
bound.
(lm-homepage, lm-website): Use backwards-compatible alias
'lm-homepage'.
(package-buffer-info): Call 'lm-maintainer' if 'lm-maintainers' is not
defined.
(describe-package-1): Avoid using 'make-separator-line' if not bound.
(package-report-bug): Expand 'custom--standard-value' definition.

For the background and motivation behind these changes, please consult
this thread:
https://lists.gnu.org/archive/html/emacs-devel/2023-03/msg00995.html.
---
 lisp/emacs-lisp/package.el | 27 ++++++++++++++++++---------
 1 file changed, 18 insertions(+), 9 deletions(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 0258ed52bee..8f5da5af1ea 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -5,9 +5,12 @@
 ;; Author: Tom Tromey <tromey@redhat.com>
 ;;         Daniel Hackney <dan@haxney.org>
 ;; Created: 10 Mar 2007
-;; Version: 1.1.0
+;; Version: 1.1.1
 ;; Keywords: tools
-;; Package-Requires: ((tabulated-list "1.0"))
+;; Package-Requires: ((emacs "27.1") (compat "29.1.0.0"))
+
+;; This is a GNU ELPA :core package.  Avoid functionality that is not
+;; compatible with the version of Emacs recorded above.
 
 ;; This file is part of GNU Emacs.
 
@@ -147,6 +150,7 @@
 (eval-when-compile (require 'subr-x))
 (eval-when-compile (require 'epg))      ;For setf accessors.
 (eval-when-compile (require 'inline))   ;For `define-inline'
+(require 'compat nil 'noerror)
 (require 'seq)
 
 (require 'tabulated-list)
@@ -1124,7 +1128,8 @@ package--native-compile-async
   "Native compile installed package PKG-DESC asynchronously.
 This assumes that `pkg-desc' has already been activated with
 `package-activate-1'."
-  (when (native-comp-available-p)
+  (when (and (fboundp 'native-comp-available-p)
+             (native-comp-available-p))
     (let ((warning-minimum-level :error))
       (native-compile-async (package-desc-dir pkg-desc) t))))
 
@@ -1160,9 +1165,10 @@ package--prepare-dependencies
 
 (declare-function lm-header "lisp-mnt" (header))
 (declare-function lm-header-multiline "lisp-mnt" (header))
-(declare-function lm-website "lisp-mnt" (&optional file))
+(declare-function lm-homepage "lisp-mnt" (&optional file))
 (declare-function lm-keywords-list "lisp-mnt" (&optional file))
 (declare-function lm-maintainers "lisp-mnt" (&optional file))
+(declare-function lm-maintainer "lisp-mnt" (&optional file))
 (declare-function lm-authors "lisp-mnt" (&optional file))
 
 (defun package-buffer-info ()
@@ -1195,7 +1201,7 @@ package-buffer-info
             (or (lm-header "package-version") (lm-header "version")))
            (pkg-version (package-strip-rcs-id version-info))
            (keywords (lm-keywords-list))
-           (website (lm-website)))
+           (website (lm-homepage)))
       (unless pkg-version
          (if version-info
              (error "Unrecognized package version: %s" version-info)
@@ -1211,7 +1217,10 @@ package-buffer-info
        :maintainer
        ;; For backward compatibility, use a single string if there's only
        ;; one maintainer (the most common case).
-       (let ((maints (lm-maintainers))) (if (cdr maints) maints (car maints)))
+       (if (fboundp 'lm-maintainers)
+           (let ((maints (lm-maintainers)))
+             (if (cdr maints) maints (car maints)))
+         (lm-maintainer))
        :authors (lm-authors)))))
 
 (defun package--read-pkg-desc (kind)
@@ -2302,8 +2311,6 @@ package-strip-rcs-id
       ;; to make sure we use a "canonical name"!
       (if l (package-version-join l)))))
 
-(declare-function lm-website "lisp-mnt" (&optional file))
-
 ;;;###autoload
 (defun package-install-from-buffer ()
   "Install a package from the current buffer.
@@ -4571,7 +4578,9 @@ package-report-bug
       (dolist (ent (get (cdr group) 'custom-group))
         (when (and (custom-variable-p (car ent))
                    (boundp (car ent))
-                   (not (eq (custom--standard-value (car ent))
+                   (not (eq (if (fboundp 'custom--standard-value)
+                                (custom--standard-value (car ent))
+                              (eval (car (get (car ent) 'standard-value)) t))
                             (default-toplevel-value (car ent))))
                    (file-in-directory-p (car group) (package-desc-dir desc)))
           (push (car ent) vars))))
-- 
2.30.2


[-- Attachment #3: Type: text/plain, Size: 5604 bytes --]


And here is the compile-log I got after installing the package on Emacs
27.1:

--8<---------------cut here---------------start------------->8---
Compiling file /home/philip/.config/emacs/elpa/package-1.1.1/package.el at Fri Apr  7 11:58:06 2023
Entering directory ‘/home/philip/.config/emacs/elpa/package-1.1.1/’

In package-tar-file-info:
package.el:1243:8:Warning: ‘(filename (tar-header-name (car tar-parse-info)))’
    is a malformed function
package.el:1247:31:Warning: reference to free variable ‘loop’
package.el:1249:58:Warning: reference to free variable ‘filename’

In package-menu-toggle-hiding:
package.el:3208:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-hide-package:
package.el:3556:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-describe-package:
package.el:3580:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-mark-delete:
package.el:3590:4:Warning: malformed interactive spec: (interactive "p"
    package-menu-mode)

In package-menu-mark-install:
package.el:3599:4:Warning: malformed interactive spec: (interactive "p"
    package-menu-mode)

In package-menu-mark-unmark:
package.el:3607:4:Warning: malformed interactive spec: (interactive "p"
    package-menu-mode)

In package-menu-backup-unmark:
package.el:3613:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-mark-obsolete-for-deletion:
package.el:3620:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-quick-help:
package.el:3652:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-mark-upgrades:
package.el:3745:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-execute:
package.el:3870:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-filter-by-archive:
package.el:4127:4:Warning: malformed interactive spec: (interactive (list
    (completing-read-multiple "Filter by archive (comma separated): " (mapcar
    #'car package-archives))) package-menu-mode)

In package-menu-filter-by-description:
package.el:4149:4:Warning: malformed interactive spec: (interactive (list
    (read-regexp "Filter by description (regexp)")) package-menu-mode)

In package-menu-filter-by-keyword:
package.el:4169:4:Warning: malformed interactive spec: (interactive (list
    (completing-read-multiple "Keywords (comma separated): "
    (package-all-keywords))) package-menu-mode)

In package-menu-filter-by-name-or-description:
package.el:4194:4:Warning: malformed interactive spec: (interactive (list
    (read-regexp "Filter by name or description (regexp)")) package-menu-mode)

In package-menu-filter-by-name:
package.el:4214:4:Warning: malformed interactive spec: (interactive (list
    (read-regexp "Filter by name (regexp)")) package-menu-mode)

In package-menu-filter-by-status:
package.el:4234:4:Warning: malformed interactive spec: (interactive (list
    (completing-read "Filter by status: " '("avail-obso" "available"
    "built-in" "dependency" "disabled" "external" "held" "incompat"
    "installed" "source" "new" "unsigned"))) package-menu-mode)

In package-menu-filter-by-version:
package.el:4273:4:Warning: malformed interactive spec: (interactive (let
    ((choice (intern (char-to-string (read-char-choice "Filter by version?
    [Type =, <, > or q] " '(60 62 61 113)))))) (if (eq choice 'q) '(quit nil)
    (list (read-from-minibuffer (concat "Filter by version (" (cond ((eq
    choice '=) "= equal to") ((eq choice '<) "< less than") ((eq choice '>) ">
    greater than") (t nil)) "): ")) choice))) package-menu-mode)

In package-menu-filter-marked:
package.el:4307:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-filter-upgradable:
package.el:4335:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-menu-clear-filter:
package.el:4344:4:Warning: malformed interactive spec: (interactive nil
    package-menu-mode)

In package-browse-url:
package.el:4528:4:Warning: malformed interactive spec: (interactive (list
    (package--query-desc) current-prefix-arg) package-menu-mode)

In package-report-bug:
package.el:4571:4:Warning: malformed interactive spec: (interactive (list
    (package--query-desc package-alist)) package-menu-mode)
package.el:4571:43:Warning: ‘(reporter-prompt-for-summary-p t)’ is a malformed
    function

In end of data:
package.el:4602:1:Warning: the following functions are not known to be defined:
    file-name-concat, length>, loaddefs-generate,
    native-compile-async, named-let, loop, length=,
    comp-clean-up-stale-eln, format-prompt, substitute-quotes,
    make-separator-line, defvar-keymap, ensure-list,
    macroexp-file-name, dlet
--8<---------------cut here---------------end--------------->8---

The most common issue is that the interactive spec has to be revised to
use only one argument.  IIUC this can be replaced by a `declare' form
with a `modes' entry?

This also gives us a better picture of the unknown functions:

- loaddefs-generate (not provided by Compat)
- substitute-quotes (provided by Compat)
- defvar-keymap (provided by Compat)

I will take a look at what is necessary to back-port `loaddefs-generate'
via Compat.

-- 
Philip Kaludercic

Re: Request to add Package to GNU ELPA

[Jonas Bernoulli]

Philip Kaludercic <philipk@posteo.net> writes:

>> I am hoping that someone would take charge of Package on Emacs' side,
>> and bring some new momentum to its development.  You would be a good
>> candidate for that. ;D
>
> I am glad to help where I can (e.g. making sure that package.el can be
> distributed on ELPA), but I don't know exactly what you have in mind
> with the other changes so I would rather let you tackle those issues.

Thanks.  Once it is on ELPA, I can start propose backward compatible
patches, and so can others.

>>>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>>>> enough to backport, either as part of Compat or in package-vc.el.
>>>
>>> [...]
>>
>> I assume that means "well it may *seem* that way, but if you are the one
>> actually doing the work...".  I know the feeling.  oO
>
> I am not sure what you are talking about here?
>
>
> [...]

I was speculating on the meaning of the initial "[...]".

Make of that what you want [...] ;P

> But otherwise I would just try and avoid the need to force users to
> upgrade.  Upgrading should be incentivized with the advantages of doing
> so, not the issues of (perhaps unconsciously) holding back.

Sounds good.

>> I would like to use a format that not only supported "pre-releases" but
>> also "post-releases".  Debian uses "post-releases" to implement
>> "debian-revision" [1]; we could use it to separate the "timestamp" part
>> from "latest released upstream version" part, in snapshot release
>> version strings. [...]
>
> I cannot comment on this, but "post-releases" (as opposed to
> "pre-releases") sounds like just another entry in `version-regexp-alist'?

Unfortunately no.  You cannot encode both pre- and post-releases using a
flat list of integers only.  (Unless you make one of them magical, but
then you would better use a magical non-integer; which is exactly what I
plan to do).  I'll propose a patch to add nearly backward compatible
support.

> So to summarise, I think an approximate plan would be useful:
>
> 1. Update package.el to be published on ELPA
> 2. (Optional) Update package-vc.el to be published on ELPA
> 3. Specify and implement a new package-archives format
> 4. Update package archives to generate the new archive format
>
> Anything I am forgetting?

I don't think so.

Well, it's "to ALSO generate the new archive format".

Re: Adding package-vc to ELPA

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Jonas Bernoulli <jonas@bernoul.li> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>>
>>>> Hello,
>>>
>>> Hi, hope I can add a few useful comments here.
>>>
>>>> I would like to request that "package.el" be added to GNU Elpa as a
>>>> "core" package.  This would make new features available to users who are
>>>> stuck on older Emacs releases and it would even bring us one step closer
>>>> to being able to make backward incompatible changes in the future.
>>>>
>>>> Some examples for why that would be desirable:
>>>>
>>>> - A recent addition that could be useful is the new "package-vc.el"
>>>>   (which I think should be distributed as part of the `package' package,
>>>>   but it could also be distributed as a separate core package).
>>>
>>> This will be difficult, since package-vc depends on the new "vc-clone"
>>> function, which would also have to be provided on ELPA to work.
>>
>> The simplest way to deal with that, while still making the latest
>> Package available to users of older Emacs releases, would be to *not*
>> include package-vc.el in the Package package.
>>
>> Going one step further we could make package-vc available as a separate
>> package.  That would not be of much use *now*, but future improvements
>> would then be available to users of Emacs 29.
>
> I would in principle be interested in supporting this.
>
>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>> enough to backport, either as part of Compat or in package-vc.el.

I have put some more though into this, and I don't think adding the new
VC functionality to Compat or package-vc.el would be a good idea.
Beside vc-clone we would also need `vc-prepare-patch' and the
"last-change" method.  What would be imaginable would be to add the
package to ELPA and make it depend on Emacs 29?  Would that sound
useful?

Re: Adding package-vc to ELPA

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

>>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>>> enough to backport, either as part of Compat or in package-vc.el.
>
> I have put some more though into this, and I don't think adding the new
> VC functionality to Compat or package-vc.el would be a good idea.
> Beside vc-clone we would also need `vc-prepare-patch' and the
> "last-change" method.  What would be imaginable would be to add the
> package to ELPA and make it depend on Emacs 29?  Would that sound
> useful?

What happened to this?  package-vc is still not on ELPA, is it?

I can't see any replies to the question you asked above, but FWIW I
think requiring Emacs 29 is fine if backporting is too much of a hassle.

Note that Debian stable is still on Emacs 28, so presumably that is what
will be available in many Debian derivative distros for the next year or
two.  That may or may not be a factor in your decision here.

Re: Adding package-vc to ELPA

[Philip Kaludercic]

Stefan Kangas <stefankangas@gmail.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>>>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>>>> enough to backport, either as part of Compat or in package-vc.el.
>>
>> I have put some more though into this, and I don't think adding the new
>> VC functionality to Compat or package-vc.el would be a good idea.
>> Beside vc-clone we would also need `vc-prepare-patch' and the
>> "last-change" method.  What would be imaginable would be to add the
>> package to ELPA and make it depend on Emacs 29?  Would that sound
>> useful?
>
> What happened to this?  package-vc is still not on ELPA, is it?

I have posted an update in another thread:
https://mail.gnu.org/archive/html/emacs-devel/2023-08/msg00667.html.  To
summarise, if you take a look at the feature/elpa-package branch you'll
find the changes necessary to add both package.el and package-vc.el to
ELPA.  I have been wanting for confirmation from above™ before
progressing with the idea.

There were some objections to adding package.el to ELPA, in case
package.el breaks itself when updating, but I hope that that issue
should be handled by adding a verbatim autoloaded function that can be
used to remove the local package.el installation.  Of course if the
complains remain, we can also just add package-vc.el to ELPA.

> I can't see any replies to the question you asked above, but FWIW I
> think requiring Emacs 29 is fine if backporting is too much of a hassle.

Agree.

> Note that Debian stable is still on Emacs 28, so presumably that is what
> will be available in many Debian derivative distros for the next year or
> two.  That may or may not be a factor in your decision here.

-- 
Philip Kaludercic

Re: Request to add Package to GNU ELPA

[chad]

[-- Attachment #1: Type: text/plain, Size: 1307 bytes --]

Tangential to your request:

On Thu, Mar 30, 2023 at 10:23 AM Jonas Bernoulli <jonas@bernoul.li> wrote:

> - For example, I think it would be nice if the columns displayed by
>   `list-packages' were customizable.  A user might want to increase the
>   width of the "Version" column, so that not every GNU-devel ELPA version
>   is truncated, or they might want to add an "Author" column.  We could
>   even allow third-party packages to add columns such as "Downloads" or
>   "Stars".
>

FWIW, I've been using these for a while now:

(add-to-list 'package-archives '("NGD" . "
> https://elpa.nongnu.org/nongnu-devel/"))
> (add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
> (add-to-list 'package-archives '("EGD" . "https://elpa.gnu.org/devel/"))
>
(custom-set-variables

 '(package-name-column-width 22)
>  '(package-archive-column-width 2)
>  '(package-status-column-width 2)
>  '(package-version-column-width 2))


The result isn't perfect, but it lets me see the parts I care about in an
~80 column window with 5 different archives. The main motivation I had was
that the names of _many_ packages were non-uniquely truncated by default.
This would be hard to pull off without severely truncating the version
column.

Hope that helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 2398 bytes --]