Re: Request to add Package to GNU ELPA

[Philip Kaludercic <philipk@posteo.net>]

Jonas Bernoulli <jonas@bernoul.li> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>>>
>
>>> Going one step further we could make package-vc available as a separate
>>> package.  That would not be of much use *now*, but future improvements
>>> would then be available to users of Emacs 29.
>>
>> I would in principle be interested in supporting this.
>
> I am hoping that someone would take charge of Package on Emacs' side,
> and bring some new momentum to its development.  You would be a good
> candidate for that. ;D
>
> Your package-vc is welcome addition (I haven't really tried it, since
> my Borg serves me well) and Stefan certainly put a lot of work into his
> elpa-admin, but Package itself has been a bit stagnant.
>
> [I myself contribute to a lot of elisp packaging related projects, and
>  do not plan to take on any new responsibilities; in fact I am trying
>  to reduce them.]
>
>>> I had a look at vc-clone and a few vc-*-clone.  They seem trivial
>>> enough to backport, either as part of Compat or in package-vc.el.
>>
>> [...]
>
> I assume that means "well it may *seem* that way, but if you are the one
> actually doing the work...".  I know the feeling.  oO
>
>>> [using a bleeding edge *ELPA]
>>
>> My experience was not that good, especially back before I got into Emacs
>> development (or understood what was going on at all) I constantly ran
>> into issues when updating packages from MELPA.  The result was that I'd
>> usually just not update packages at all for long periods of time.  The
>> next best thing for me was MELPA stable until NonGNU ELPA came around.
>
> Using MELPA stable comes with its own problems.  MELPA's maintainers
> advice users to stick to the bleeding-edge variant and use that
> themselves.  And many users take that advice to heart, and many
> developers are aware of that and act accordingly.
>
> I don't think anyone is against moving towards relying more on released,
> stable versions, but there seem to be some chicken and eggs involved.
> We can all do a little bit to get closer to a more stable ecosystem, but
> it will take time.
>
> [I expect that once MELPA starts using sane version strings for snapshot
>  packages, that will be a big step in that direction, because that makes
>  it much simpler to properly specify the minimal required versions of
>  dependencies, that work for both the bleeding-edge and stable channels.
>
>  I am in fact working on that right now.  I was about to start rebuild
>  all packages for testing purposes, but briefly got distracted by your
>  reply to this. :P
>
>  But there are other areas where improvements are needed, and I would
>  like to leave it to others to tackle those.]
>
>>>>> - I should also mention that my main motivation for pushing for this
>>>>>   now, is that I would like to improve version handling.  That is a
>>>>>   whole other can of worms, so I do not wish to discuss it just yet,
>>>>>   to avoid distracting from the topic at hand, but I thought I should
>>>>>   at least mention it.  You might very well end up being against my
>>>>>   suggestions regarding versions strings, once I present them, but that
>>>>>   should not be cause to oppose the change requested here as well.  Even
>>>>>   if my suggestions regarding version strings are ultimately rejected, I
>>>>>   still think it would be a good idea to add `package' to GNU ELPA, for
>>>>>   the other reasons presented here.
>>>>
>>>> This implies that packages might themselves depend on newer versions of
>>>> package.el -- which I think one should put some thought into before
>>>> anything is done.
>>>
>>> How so? (Further down I suggest making some popular packages temporarily
>>> depend on Package from GNU Elpa as a means to get that installed as
>>> widely as possible, but that doesn't mean that those packages actually
>>> *need* a newer Package.)
>>>
>>>> How can you e.g. change the way versions are handled
>>>> in a way that people with older versions of package.el could still
>>>> handle the change without confusion?
>>>
>>> You can't, and that is exactly the point I was trying to make.  Certain
>>> things (including, but not limited to how version strings are handled)
>>> are effectively set in stone, unless certain packages/features are made
>>> available to users who, for whatever reason, stick to an old Emacs
>>> release.
>>
>> Other than the version, it seems the other big one is the dependency
>> list.  Is there anything else that could cause issues?
>
> I can't think of any right now, but that of course doesn't mean we won't
> discover any later on.
>
>>>> The upgrading procedure for archive-contents has never appeared to me to
>>>> be very robust.  Perhaps it would be better to introduce a second file
>>>> (archive-contents.eld) with a more flexible format?
>>>
>>> An effort was made to provide an upgrade procedure -- the
>>> "archive-contents" were prefixed with a version number.  But then
>>> package.el was added to Emacs and we stopped to distribute it
>>> separately.  Apparently we didn't realize at the time that this only
>>> allowed us to tell the user "the archive and tool are not compatible,
>>> deal with it".
>>
>> What could be done instead?  Should the updated version of package.el
>> try to install a newer version of itself in case the version string is
>> updated?
>
> Yes, that's what I had in mind.
>
>>> We could continue to distribute "archive-contents", which continues to
>>> use version 1, and add "archive-contents.eld" which uses version 2.
>>> But that would do nothing to get users to install a forward-compatible
>>> version of Package (in the sense that once it becomes incompatible, it
>>> will provide a smooth upgrade path to the new version).
>>
>> This seems like the better option to me.  Most people don't /need/ the
>> newest version right away, and even if they did there would be no
>> straightforward way of making sure all users would have it installed
>> within some fixed time-frame.
>
> We keep "archive-contents" going for as long as possible.  For the final
> push to get users to upgrade, all packages except Package and its
> dependencies could be removed from "archive-contents".  A pseudo package
> could be added, whose summary (displayed in the list) and commentary
> (displayed by describe-package) could be used to instruct users to
> upgrade Package, and how.
>
>>>   (We should invest some time to investigate how to make this as smooth
>>>   as possible, but basically:)
>>>
>>> - 1. wget https://...package-2.0.0.tar
>>>   2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/
>>>   3. Restart Emacs and run package-refresh-contents again.
>>
>> This is exactly what I would like to avoid.
>
> I am certainly interested in hearing better ideas.
>
>> What happens when the .tar disappears and some user tries to fix this
>> issue in a few years with these outdated instructions?
>
> We should prevent that.  The tar file mentioned in the upgrade
> instructions doesn't have to be hosted in the usual location only.
> It could additionally be hosted somewhere else on gnu.org, where it
> is safe from being discarded by elpa-admin or some cleanup script
> that isn't aware of this special case.
>
>> What if they use .emacs.d or some other directory?  For while there
>> will be plenty of users that will figure this out, I know many others
>> will be confused
>
> Well, these steps were the executive summary, the instructions actually
> shown to users should of course be more detailed and account for such
> differences.
>
>> (this also makes me think that a v2 should have support for some kind
>> of MOTD system to explain issues like these).
>
> +1
>
>> So again, what would be the issue with an opt-in system to upgrading
>> package.el, that could also be pushed forward by a few popular packages,
>> without the need to break anything.
>
> I am not against a long transitional phase, but I think that eventually
> we will want to make some breaking change.
>
> One example of an incompatible change is a change in how version strings
> are handled.  As I said, I would like to advocate that in a separate
> thread.  Here, it is just serves as an example, other incompatible
> changes may become desirable.  Just the gist of it:
>
> For a while [Non]GNU-devel ELPA used this version string format:
>
>   VERSION.0-snapshotTIMESTAMP
>
> "snapshot" has to be used because "archive-contents" contains the
> version string in list format, e.g., (1 2 3 0 -4 20230405 111).  All of
> "snapshot", "cvs", "git", "bzr", "svn", "hg", "darcs" and "unknown" are
> encoded as -4 in the list format, but package-version-join maps it to
> "snapshot".
>
> That leads to very long version strings, which I assume is why Stefan
> switched to just
>
>   VERSION.0.TIMESTAMP
>
> The ".0" serves as a pseudo separator.  Currently "1.0-snapshot" is
> *smaller* than "1.0".  If we inject an additional ".0" then it is
> smaller than "N.0", but not smaller than "N".
>
> I would like to use a format that not only supported "pre-releases" but
> also "post-releases".  Debian uses "post-releases" to implement
> "debian-revision" [1]; we could use it to separate the "timestamp" part
> from "latest released upstream version" part, in snapshot release
> version strings.
>
> [1]: https://manpages.debian.org/wheezy/dpkg-dev/deb-version.5.en.html
>
> But again, maybe I am the only who feels a need to do something about
> that; in this context it is just an example of a change that would break
> backward compatibility; to demonstrate that some potential changes could
> not be made merely opt-in, but instead would have to be either rolled
> out to everyone or nobody.
>
>      Jonas