From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jonas Bernoulli Newsgroups: gmane.emacs.devel Subject: Re: Request to add Package to GNU ELPA Date: Wed, 05 Apr 2023 16:13:57 +0200 Message-ID: <87lej6ju1m.fsf@bernoul.li> References: <87tty24ap2.fsf@bernoul.li> <87wn2y5em5.fsf@posteo.net> <87r0t44zbx.fsf@bernoul.li> <87sfde7lh4.fsf@posteo.net> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="34917"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Philip Kaludercic Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Apr 05 16:26:55 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pk46B-0008r7-0p for ged-emacs-devel@m.gmane-mx.org; Wed, 05 Apr 2023 16:26:55 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pk45X-0000p9-MR; Wed, 05 Apr 2023 10:26:16 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk3tl-0004Zt-PD for emacs-devel@gnu.org; Wed, 05 Apr 2023 10:14:05 -0400 Original-Received: from mail.hostpark.net ([212.243.197.30]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pk3ti-0007AQ-Ea for emacs-devel@gnu.org; Wed, 05 Apr 2023 10:14:05 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by mail.hostpark.net (Postfix) with ESMTP id E85BF16509; Wed, 5 Apr 2023 16:13:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bernoul.li; h= content-type:content-type:mime-version:message-id:date:date :references:in-reply-to:subject:subject:from:from:received :received; s=sel2011a; t=1680704037; bh=A4eJkJ2opWvi7ClBRv690NBi ewxMojYgV5Gz5CwKi2c=; b=ztTRl9MYvXSrLN8PV1mF16G6tRdyJpo6dlWXu5nd 5PS1WyQCvVlrx2MCPhz07F9RV+669BJl6mzw3WKSk9Uu25l4DZWQgBeD2BEl5p+a rHheZ6Q3ppxsob9S6KKZsMXm4SWUXKcDGn4+jB52KkKjqmdQzSCuzxt+981OD4li V58= X-Virus-Scanned: by Hostpark/NetZone Mailprotection at hostpark.net Original-Received: from mail.hostpark.net ([127.0.0.1]) by localhost (mail0.hostpark.net [127.0.0.1]) (amavisd-new, port 10224) with ESMTP id 8SLUw4F1gb1u; Wed, 5 Apr 2023 16:13:57 +0200 (CEST) Original-Received: from customer (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.hostpark.net (Postfix) with ESMTPSA id 97066164C2; Wed, 5 Apr 2023 16:13:57 +0200 (CEST) In-Reply-To: <87sfde7lh4.fsf@posteo.net> Received-SPF: none client-ip=212.243.197.30; envelope-from=jonas@bernoul.li; helo=mail.hostpark.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 05 Apr 2023 10:26:10 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:305134 Archived-At: Philip Kaludercic writes: > Jonas Bernoulli writes: > >> Philip Kaludercic writes: >> >>> Jonas Bernoulli writes: >>> >> Going one step further we could make package-vc available as a separate >> package. That would not be of much use *now*, but future improvements >> would then be available to users of Emacs 29. > > I would in principle be interested in supporting this. I am hoping that someone would take charge of Package on Emacs' side, and bring some new momentum to its development. You would be a good candidate for that. ;D Your package-vc is welcome addition (I haven't really tried it, since my Borg serves me well) and Stefan certainly put a lot of work into his elpa-admin, but Package itself has been a bit stagnant. [I myself contribute to a lot of elisp packaging related projects, and do not plan to take on any new responsibilities; in fact I am trying to reduce them.] >> I had a look at vc-clone and a few vc-*-clone. They seem trivial >> enough to backport, either as part of Compat or in package-vc.el. > > [...] I assume that means "well it may *seem* that way, but if you are the one actually doing the work...". I know the feeling. oO >> [using a bleeding edge *ELPA] > > My experience was not that good, especially back before I got into Emacs > development (or understood what was going on at all) I constantly ran > into issues when updating packages from MELPA. The result was that I'd > usually just not update packages at all for long periods of time. The > next best thing for me was MELPA stable until NonGNU ELPA came around. Using MELPA stable comes with its own problems. MELPA's maintainers advice users to stick to the bleeding-edge variant and use that themselves. And many users take that advice to heart, and many developers are aware of that and act accordingly. I don't think anyone is against moving towards relying more on released, stable versions, but there seem to be some chicken and eggs involved. We can all do a little bit to get closer to a more stable ecosystem, but it will take time. [I expect that once MELPA starts using sane version strings for snapshot packages, that will be a big step in that direction, because that makes it much simpler to properly specify the minimal required versions of dependencies, that work for both the bleeding-edge and stable channels. I am in fact working on that right now. I was about to start rebuild all packages for testing purposes, but briefly got distracted by your reply to this. :P But there are other areas where improvements are needed, and I would like to leave it to others to tackle those.] >>>> - I should also mention that my main motivation for pushing for this >>>> now, is that I would like to improve version handling. That is a >>>> whole other can of worms, so I do not wish to discuss it just yet, >>>> to avoid distracting from the topic at hand, but I thought I should >>>> at least mention it. You might very well end up being against my >>>> suggestions regarding versions strings, once I present them, but that >>>> should not be cause to oppose the change requested here as well. Even >>>> if my suggestions regarding version strings are ultimately rejected, I >>>> still think it would be a good idea to add `package' to GNU ELPA, for >>>> the other reasons presented here. >>> >>> This implies that packages might themselves depend on newer versions of >>> package.el -- which I think one should put some thought into before >>> anything is done. >> >> How so? (Further down I suggest making some popular packages temporarily >> depend on Package from GNU Elpa as a means to get that installed as >> widely as possible, but that doesn't mean that those packages actually >> *need* a newer Package.) >> >>> How can you e.g. change the way versions are handled >>> in a way that people with older versions of package.el could still >>> handle the change without confusion? >> >> You can't, and that is exactly the point I was trying to make. Certain >> things (including, but not limited to how version strings are handled) >> are effectively set in stone, unless certain packages/features are made >> available to users who, for whatever reason, stick to an old Emacs >> release. > > Other than the version, it seems the other big one is the dependency > list. Is there anything else that could cause issues? I can't think of any right now, but that of course doesn't mean we won't discover any later on. >>> The upgrading procedure for archive-contents has never appeared to me to >>> be very robust. Perhaps it would be better to introduce a second file >>> (archive-contents.eld) with a more flexible format? >> >> An effort was made to provide an upgrade procedure -- the >> "archive-contents" were prefixed with a version number. But then >> package.el was added to Emacs and we stopped to distribute it >> separately. Apparently we didn't realize at the time that this only >> allowed us to tell the user "the archive and tool are not compatible, >> deal with it". > > What could be done instead? Should the updated version of package.el > try to install a newer version of itself in case the version string is > updated? Yes, that's what I had in mind. >> We could continue to distribute "archive-contents", which continues to >> use version 1, and add "archive-contents.eld" which uses version 2. >> But that would do nothing to get users to install a forward-compatible >> version of Package (in the sense that once it becomes incompatible, it >> will provide a smooth upgrade path to the new version). > > This seems like the better option to me. Most people don't /need/ the > newest version right away, and even if they did there would be no > straightforward way of making sure all users would have it installed > within some fixed time-frame. We keep "archive-contents" going for as long as possible. For the final push to get users to upgrade, all packages except Package and its dependencies could be removed from "archive-contents". A pseudo package could be added, whose summary (displayed in the list) and commentary (displayed by describe-package) could be used to instruct users to upgrade Package, and how. >> (We should invest some time to investigate how to make this as smooth >> as possible, but basically:) >> >> - 1. wget https://...package-2.0.0.tar >> 2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/ >> 3. Restart Emacs and run package-refresh-contents again. > > This is exactly what I would like to avoid. I am certainly interested in hearing better ideas. > What happens when the .tar disappears and some user tries to fix this > issue in a few years with these outdated instructions? We should prevent that. The tar file mentioned in the upgrade instructions doesn't have to be hosted in the usual location only. It could additionally be hosted somewhere else on gnu.org, where it is safe from being discarded by elpa-admin or some cleanup script that isn't aware of this special case. > What if they use .emacs.d or some other directory? For while there > will be plenty of users that will figure this out, I know many others > will be confused Well, these steps were the executive summary, the instructions actually shown to users should of course be more detailed and account for such differences. > (this also makes me think that a v2 should have support for some kind > of MOTD system to explain issues like these). +1 > So again, what would be the issue with an opt-in system to upgrading > package.el, that could also be pushed forward by a few popular packages, > without the need to break anything. I am not against a long transitional phase, but I think that eventually we will want to make some breaking change. One example of an incompatible change is a change in how version strings are handled. As I said, I would like to advocate that in a separate thread. Here, it is just serves as an example, other incompatible changes may become desirable. Just the gist of it: For a while [Non]GNU-devel ELPA used this version string format: VERSION.0-snapshotTIMESTAMP "snapshot" has to be used because "archive-contents" contains the version string in list format, e.g., (1 2 3 0 -4 20230405 111). All of "snapshot", "cvs", "git", "bzr", "svn", "hg", "darcs" and "unknown" are encoded as -4 in the list format, but package-version-join maps it to "snapshot". That leads to very long version strings, which I assume is why Stefan switched to just VERSION.0.TIMESTAMP The ".0" serves as a pseudo separator. Currently "1.0-snapshot" is *smaller* than "1.0". If we inject an additional ".0" then it is smaller than "N.0", but not smaller than "N". I would like to use a format that not only supported "pre-releases" but also "post-releases". Debian uses "post-releases" to implement "debian-revision" [1]; we could use it to separate the "timestamp" part from "latest released upstream version" part, in snapshot release version strings. [1]: https://manpages.debian.org/wheezy/dpkg-dev/deb-version.5.en.html But again, maybe I am the only who feels a need to do something about that; in this context it is just an example of a change that would break backward compatibility; to demonstrate that some potential changes could not be made merely opt-in, but instead would have to be either rolled out to everyone or nobody. Jonas