From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jonas Bernoulli Newsgroups: gmane.emacs.devel Subject: Re: Request to add Package to GNU ELPA Date: Fri, 31 Mar 2023 19:15:14 +0200 Message-ID: <87r0t44zbx.fsf@bernoul.li> References: <87tty24ap2.fsf@bernoul.li> <87wn2y5em5.fsf@posteo.net> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="3279"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Philip Kaludercic Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Mar 31 20:00:49 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1piJ3O-0000b5-2Q for ged-emacs-devel@m.gmane-mx.org; Fri, 31 Mar 2023 20:00:46 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1piJ2s-0007E7-5G; Fri, 31 Mar 2023 14:00:14 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1piILX-0002Pe-5f for emacs-devel@gnu.org; Fri, 31 Mar 2023 13:15:27 -0400 Original-Received: from mail.hostpark.net ([212.243.197.30]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1piILQ-0006e7-Mt for emacs-devel@gnu.org; Fri, 31 Mar 2023 13:15:23 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by mail.hostpark.net (Postfix) with ESMTP id 054A0162AB; Fri, 31 Mar 2023 19:15:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bernoul.li; h= content-type:content-type:mime-version:message-id:date:date :references:in-reply-to:subject:subject:from:from:received :received; s=sel2011a; t=1680282914; bh=PAR1YZxFDC/rEHB1r9gtzSQK NWDYIhDXOz7rpRDpZjY=; b=QE77JlhXlyIdHOZsyNM6YcA1JzHLopzRtUxxJNwx mKXp2lBC0x/m+ZWQbGR8f3gph/66duY1noyX+b+wkf3JVTKyFCkcOY/DxQuRK2yD rH5mSFY5bRO5+ynzhQd/3xSSogA1kiDPLUAzvz0d3i0iKUftC0rCxRsXR9/ZO+AN 6Ms= X-Virus-Scanned: by Hostpark/NetZone Mailprotection at hostpark.net Original-Received: from mail.hostpark.net ([127.0.0.1]) by localhost (mail1.hostpark.net [127.0.0.1]) (amavisd-new, port 10224) with ESMTP id bEbzBhsvxn6u; Fri, 31 Mar 2023 19:15:14 +0200 (CEST) Original-Received: from customer (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.hostpark.net (Postfix) with ESMTPSA id A418B1629D; Fri, 31 Mar 2023 19:15:14 +0200 (CEST) In-Reply-To: <87wn2y5em5.fsf@posteo.net> Received-SPF: none client-ip=212.243.197.30; envelope-from=jonas@bernoul.li; helo=mail.hostpark.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 31 Mar 2023 14:00:11 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:304944 Archived-At: Philip Kaludercic writes: > Jonas Bernoulli writes: > >> Hello, > > Hi, hope I can add a few useful comments here. > >> I would like to request that "package.el" be added to GNU Elpa as a >> "core" package. This would make new features available to users who are >> stuck on older Emacs releases and it would even bring us one step closer >> to being able to make backward incompatible changes in the future. >> >> Some examples for why that would be desirable: >> >> - A recent addition that could be useful is the new "package-vc.el" >> (which I think should be distributed as part of the `package' package, >> but it could also be distributed as a separate core package). > > This will be difficult, since package-vc depends on the new "vc-clone" > function, which would also have to be provided on ELPA to work. The simplest way to deal with that, while still making the latest Package available to users of older Emacs releases, would be to *not* include package-vc.el in the Package package. Going one step further we could make package-vc available as a separate package. That would not be of much use *now*, but future improvements would then be available to users of Emacs 29. I had a look at vc-clone and a few vc-*-clone. They seem trivial enough to backport, either as part of Compat or in package-vc.el. > This > seems to me to be a general issue with proposing to create new core > packages. When working in emacs.git it is easy and usually > uncontroversial to make use of whatever is the current state-of-the-art > when it comes to Elisp. The Compat library on ELPA can help here, as it > already does in a few cases (ERC and Python, and it has been discussed > for project.el). Thanks for creating Compat! It certainly makes things easier for maintainers of external packages. > All of this touches on the general topic of how Emacs should be > developed: Small and to be extended, therefore of little use OOTB or big > and feature-full, but slow to be updated. IIUC a compromise could be > achieved by having two versions of Emacs that would respectively attempt > to satisfy the two aspirations. This doesn't sound like a good idea to me because I doubt that it would have the benefits you seem to expect. A sizable portion of an external package would choose to use the slow-moving Emacs. So the maintainer of the package would have to continue to support that, similar to how they now have to support older releases. So even though there would be a fast-moving Emacs, they still would not be able to take full advantage of the new/improved features it provides. The situation would be basically the same as now; but not supporting the slow-moving Emacs would be even harder to justify than dropping support for Emacs 25.1. > There was some work in that direction a > few years ago[0], but I don't know if anything has happened since. > Perhaps pushing for this more general solution would help solve this > particular problem? > > [0] https://git.savannah.gnu.org/cgit/emacs.git/log/?h=elparized-core I remember that there was some talk about moving in that direction, and I am in favor and have been patiently waiting for someone to make another push in that direction, but things indeed seem to have fizzled out. Meanwhile making certain packages available on a case by case basis does not seem like a bad idea to me. It doesn't mean that we *commit* to making "most" packages available like that eventually, so even those you are opposed to that idea might be able to agree to doing it selectively. Doing it for a select few packages, allows us to gain some experience, whether that be bad or good. >> That isn't just useful for users who like to live on the edge. For >> example, if a package drops support for an old Emacs release, then >> a user who is stuck on that release, may wish to keep using the last >> release of the package that still supports that Emacs release, and >> package-vc would allow them to do just that. >> >> - I don't use Package myself, so I am not fully aware of all the quality >> of live enhancements that surely have accumulated over the years. But >> I am aware of some small missing features, that would be beneficial to >> users of older Emacs releases. >> >> - For example, I think it would be nice if the columns displayed by >> `list-packages' were customizable. A user might want to increase the >> width of the "Version" column, so that not every GNU-devel ELPA version >> is truncated, > > A more general solution to this problem would be if tabulated-list-mode > could dynamically resize a column, or at least shrink it possible. That would indeed be nice ... and would make tabulated-list-mode another candidate for GNU ELPA. ;P > Also on another topic, I am still uncertain as to the status of the > -devel archives. I see them primarily as a means for developers to > check the status of a package pre-release or to make sure that the ELPA > build-system is working the way they want it to (e.g. when building > manuals, news or README files). They are not advertised anywhere, and I > don't think they are made for general consumption -- or rather I don't > think they should be, especially with package-vc there is a much better > way to track development and make it possible to contribute changes > upstream. Most packages still get installed through Melpa these days and that means installing the latest commit from the development branch. (The [Non]GNU-devel ELPA variants probably exist in part to satisfy users who are used to having access to the development version through "regular" package i.e., without using package-vc.) In my experience that seems to work surprisingly well; I can't really remember any regressions that were not fixed almost instantly, but that does of course depend on what packages one has installed. I am sure little regressions happen all the time, but they get fixed so quickly, that any one user usually doesn't notice. In my experience, using the development versions of some quality packages, is no different from using the development version of Emacs itself. But I agree that it would be better to release more often (and yes, I am guilty of not doing that myself), and tend to agree that it would be better if, in the long run, the community moved towards relying more on released versions. At the same time I don't think that the fact that we are not there yet is due to "mistakes". Emacs/Elisp is different from other "languages", in that everything runs in the same process, and that is at least one of the causes of the status quo. We cannot really use multiple versions of the same library in different parts of Emacs, in the same way one Python or Rust application could use xyz-v1 and another, related but independent, application could use xyz-v2. If one package depends on xyz-v2 but there were incompatible changes in that release and, another package has not been updated yet to support xyz-v2, then those two packages cannot be used at the same time. Again, that does not seem to happen very often, and I think that is partially the case because the hold out package can be quickly updated to support v2 on its development branch, *without* also having to do a release, which the maintainers might not wish to do due to where in its own release-cycle the package happens to be. >> - I should also mention that my main motivation for pushing for this >> now, is that I would like to improve version handling. That is a >> whole other can of worms, so I do not wish to discuss it just yet, >> to avoid distracting from the topic at hand, but I thought I should >> at least mention it. You might very well end up being against my >> suggestions regarding versions strings, once I present them, but that >> should not be cause to oppose the change requested here as well. Even >> if my suggestions regarding version strings are ultimately rejected, I >> still think it would be a good idea to add `package' to GNU ELPA, for >> the other reasons presented here. > > This implies that packages might themselves depend on newer versions of > package.el -- which I think one should put some thought into before > anything is done. How so? (Further down I suggest making some popular packages temporarily depend on Package from GNU Elpa as a means to get that installed as widely as possible, but that doesn't mean that those packages actually *need* a newer Package.) > How can you e.g. change the way versions are handled > in a way that people with older versions of package.el could still > handle the change without confusion? You can't, and that is exactly the point I was trying to make. Certain things (including, but not limited to how version strings are handled) are effectively set in stone, unless certain packages/features are made available to users who, for whatever reason, stick to an old Emacs release. > (This will be an issue for at > least a decade to come, knowing the pace at which users get access to > newer versions of Emacs). You mentioned updating the ELPA protocol a > few months back, and I believe a few points of interest were collected > in that thread. It seems to me these issues are all intertwined. Yes indeed, they are all intertwined, and I am trying to make the point that we have to stop muddling along once in a while, and make a breaking change, even if the transition is rough and inconveniences users for a while. > With Emacs 29, `package-refresh-contents' now calls a Hook called > `package-refresh-contents-hook'. This was initially introduced so that > package-vc can download the "elpa-packages.eld" file, but it could also > be of use here as well -- but it wouldn't help anyone with Emacs 28 or > older... I could imagine a more hacky approach based on carefully > placed advice, but only as a last resort. We cannot travel back in time to fix past mistakes/omissions that are holding us back in the presence. All we can do now is to pay the price and try harder to be forward compatible this time around. Or live with the limitations forever. Or wait some more until they become unbearable and a fix more urgent; but that would increase the odds of messing it up again. >> Addressing (1) is harder, and I don't think it possible to do so in a >> way that guarantees that not a single user would end up seeing an error >> due to an incompatible change. On the other hand "archive-contents" >> comes with a version field, and if we bump that, we at least get a >> meaningful warning: "Package archive version 2 is higher than 1". This >> doesn't say that the solution is to install `package' from GNU ELPA, but >> it should be enough to feed into a search engine to get to that answer. > > The upgrading procedure for archive-contents has never appeared to me to > be very robust. Perhaps it would be better to introduce a second file > (archive-contents.eld) with a more flexible format? An effort was made to provide an upgrade procedure -- the "archive-contents" were prefixed with a version number. But then package.el was added to Emacs and we stopped to distribute it separately. Apparently we didn't realize at the time that this only allowed us to tell the user "the archive and tool are not compatible, deal with it". We could continue to distribute "archive-contents", which continues to use version 1, and add "archive-contents.eld" which uses version 2. But that would do nothing to get users to install a forward-compatible version of Package (in the sense that once it becomes incompatible, it will provide a smooth upgrade path to the new version). >> Additionally, we could get many users to install `package' from GNU >> ELPA, by making a few popular packages explicitly depend on a `package' >> version that is available from GNU ELPA for a few months. > > Again, a few months wouldn't be enough to solve the issue. But again as > well, Compat could help by adding package as a noop dependency. Much like your "archive-contents.eld" suggestion, doing this would not *solve* the issue. Unlike that, it would however significantly reduce the number of users who would be negatively affected by the upgrade. (We could do it for a few years instead of months to further decrease the number of affected users.) I love Compat, but I don't think it is the right hammer here (except maybe for the part where it could provide vc-clone et al.). Maybe it helps to make it explicit what is going to happen to users if/when all the *ELPA start to expect that package.el support "archive-contents" v2. - The user runs "M-x package-refresh-contents" and is told "Package archive version 2 is higher than 1". - They ask a search engine what that means. We could create a few blog and forum posts before hand and give the search engines a few days to index them, providing the following instructions: (We should invest some time to investigate how to make this as smooth as possible, but basically:) - 1. wget https://...package-2.0.0.tar 2. tar -xf package-2.2.0.tar -C ~/.config/emacs/elpa/ 3. Restart Emacs and run package-refresh-contents again. Cheers, Jonas