From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?B?Sm/Do28gVMOhdm9yYQ==?= Newsgroups: gmane.emacs.devel Subject: Re: Stability of core packages (was: Not easy at all to upgrade :core packages like Eglot) Date: Tue, 18 Apr 2023 18:49:59 +0100 Message-ID: References: <87a5zj2vfo.fsf@gmail.com> <83wn2h5825.fsf@gnu.org> <87wn2gkhzr.fsf@posteo.net> <83cz485oxi.fsf@gnu.org> <87leiwdyff.fsf@posteo.net> <834jpk5hih.fsf@gnu.org> <871qkom3fj.fsf@posteo.net> <83mt3b4yfc.fsf@gnu.org> <87edonlsxi.fsf@posteo.net> <83jzyf4vzb.fsf@gnu.org> <871qknllkj.fsf@posteo.net> <83fs934pjf.fsf@gnu.org> <87wn2fk47y.fsf@posteo.net> <83sfd2g2ek.fsf@gnu.org> <875y9yfxrr.fsf@gmail.com> <87y1muefks.fsf@gmail.com> <834jpifizy.fsf@gnu.org> <83y1mue1qi.fsf@gnu.org> <83sfd2e01f.fsf@gnu.org> <1a5e5837-513b-84d8-3260-cdbf42b71267@gutov.dev> <83sfcz9rf2.fsf@gnu.org> <09a49ab9-ac72-36a9-3e68-9c633710eba7@gutov.dev> <83r0sh8i1q.fsf@gnu.org> <83pm818cx2.fsf@gnu.org> <83mt3588or.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="17965"; mail-complaints-to="usenet@ciao.gmane.io" Cc: dmitry@gutov.dev, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Apr 18 19:48:47 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1popRf-0004PN-0B for ged-emacs-devel@m.gmane-mx.org; Tue, 18 Apr 2023 19:48:47 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1popR4-0008Sd-1m; Tue, 18 Apr 2023 13:48:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1popR1-0008SK-Hp for emacs-devel@gnu.org; Tue, 18 Apr 2023 13:48:07 -0400 Original-Received: from mail-oa1-x2f.google.com ([2001:4860:4864:20::2f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1popQy-0000t5-Qa; Tue, 18 Apr 2023 13:48:07 -0400 Original-Received: by mail-oa1-x2f.google.com with SMTP id 586e51a60fabf-187b70ab997so5162885fac.0; Tue, 18 Apr 2023 10:48:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681840083; x=1684432083; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=RP63+8SDghcL0H4a5PwvAnUkXLmlWGtCtPHcZNfKsuY=; b=sKoHVvrKjr4ZGN7pX+hSlasAceeNakN005hYKjB9hjOPMwMpTlqY293/NM9sGZTNMJ VXfe/jV68XzEJDa0tq7DK5dNuxrZvOqqMjAbTB1FyEGfYmKEzlLQL6oS3YY5asQL+kwZ Z1jkxnQP4gB654+AxAGV//EU0rAFmGGGiqUfxquGeiXRpHzfdBDipS9wTpLPVknGlY1z pC3LpCKQ6i1Kk2kiY5ypcmrHZ4D5k/+65yiHI8eMfCAUoH8BFLOV9yTGA92zzYsl5mBm kThIKTlODw9nKkpVAb5z90e87DIfpXkxCCO6THUwLL+CYqUusnoHJjwvQlIFocJtUnyc PC8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681840083; x=1684432083; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RP63+8SDghcL0H4a5PwvAnUkXLmlWGtCtPHcZNfKsuY=; b=eOR4EvPcD0JwF57kpWXxb/LrqxiTBZEVLZeKf7BH+ZhV0vW6N544koGzYVHu1nSh+U wjA1C2iHUmdlieit7ApGVNCeA4ui5mEE2SvVW1F+FFoW1vk2sM+fLLmx76fuasPkc/5G XOpTx18cyrJXs15hJsatE6rVsKNlHatB6cmLWSVPQp3mF2sVvSsvWObXi0KQ4FewtGkP IfkldtFhw8pkWg5lwY8cr9z2sTC7MGZH/8DHtE5al9kuCLZjh8xjngqfPBb+oTU/6x5f pAeTQPoJXIChbF8LLKkeOQpZ+mixnqTFB2lr69MhpXCqW+tAYwlc3koVKdjEeCUk3JLE f4WQ== X-Gm-Message-State: AAQBX9fXyqrU1a+Zv4eXPskQsU8V3T7aJN+yIiyUUmdasSegwyTR/ilE o2Mcx/d/D4mr371aOp/10zKRUGq5OvsnZNd/RBxHkssDlpc= X-Google-Smtp-Source: AKy350bMuBW89z7QQ3m+CeKxPPbCDrNtW6n3VR0EK/BZYwhyQP4UaMCj0837/YWJvhXzdi2dVV9FT8/D17/lkCq71tc= X-Received: by 2002:a05:6830:565:b0:6a5:d909:4851 with SMTP id f5-20020a056830056500b006a5d9094851mr1025718otc.1.1681840082941; Tue, 18 Apr 2023 10:48:02 -0700 (PDT) In-Reply-To: <83mt3588or.fsf@gnu.org> Received-SPF: pass client-ip=2001:4860:4864:20::2f; envelope-from=joaotavora@gmail.com; helo=mail-oa1-x2f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:305416 Archived-At: On Tue, Apr 18, 2023 at 5:19=E2=80=AFPM Eli Zaretskii wrote: > > > From: Jo=C3=A3o T=C3=A1vora > > Date: Tue, 18 Apr 2023 16:45:33 +0100 > > Cc: dmitry@gutov.dev, emacs-devel@gnu.org > > > > > > It's not a "must". Eglot can work without it. The problem happens= in > > > > ElDoc and doesn't affect its interface. > > > > > > Then what Dmitry said about Eglot 1.15 being dependent on that change > > > in ElDoc is not relevant to the issue at hand, which is whether Eglot > > > 1.15 could be bundled with Emacs 29.1. > > > > It is quite relevant. > > > > Eglot 1.15 depends on many other things in ElDoc. > > It isn't like ElDoc is not available in Emacs 29. It is. But in Emacs 29, it's not the bleeding edge anymore. It's 1.13.0 which doesn't have an important feature of 1.14.0 commit e19994fe8c000b0ed2dbc667cdec26cf54356907 Author: Jo=C3=A3o T=C3=A1vora Date: Thu Mar 23 09:02:18 2023 +0000 ElDoc: rework rendering of echo area (bug#62029) And it doesn't have the fix for bug#62816, which is in master and will appear in ElDoc 1.14.1. Unless you want it to, of course. In the case of that single backport, ElDoc bundled with Emacs 29 (and with Emacs 29 _only_) should become known as 1.13.29 (imperfect versioning scheme, but not terrible). > > That particular bugfix might not -- or might indeed -- included, > > depending on what other non-bugfix things Eglot will require of > > ElDoc at the time. ElDoc is now 1.14 but it could be 1.15 at the > > time motivated by Eglot 1.15/16/17. > > My point is that Eglot and any other core package will do its users a > favor if it deliberately and consistently makes a point to depend on > as few _new_ features of other core packages as possible. If you > disagree with that, then we will have to agree to disagree, because > for me this is a very basic issue with core packages and the future of > Emacs. Yes, as few _new_ features as possible, but not fewer than that. Yes, I agree good design is when there is nothing more to take out. But sometimes you have to add to make things appear. > > And even in ElDoc 1.14 there are already things (the :echo display > > option) that are not in Emacs 29. And Eglot 1.14 directly depends > > on those things. It relies on them to do a good job. > > Which IMO is not a good thing, not unless Eglot 1.14 has fallbacks for > when these features are not available. It has. It doesn't fail. I try to make everything forward and backward compatible. But certainly your at-point documentation experience in Eglot if you merge if you backport it to Emacs 29 without also backporting ElDoc will _not_ be the same. Users will notice this and if we tell them "why yes, Eglot 1.15 is in Emacs 29!" they will be triply befuddled, and rightfully so, because Eglot 1.15 running on Emacs 28 will have much better behaviour and fewer bugs and run smoother. > But if that is impossible or impractical for some reason in this > particular case, then it simply means that users of Emacs 29 will be > unable to upgrade Eglot without also risking less stability due to > upgrading the dependencies. Again, if you don't think this > "dependencies hell" is a Bad Thing in general, we will have to agree > to disagree. No, it doesn't mean that, at all. First, dependencies exist. And here it's not hell at all, not IME. Secondly, stability is a matter of expectations. Eglot users _expect_ :core dependencies to be upgraded when they request Eglot to be upgraded. That's the way it has always worked for ~5 years. And non-Eglot users _also_ expect that, because installing any non-:core ELPA package that depends on :core packages has _also_ always produced that behaviour for dependencies. Thirdly, in practice, I have been following this for some time, I've not seen many (any?) bugs related to these kinds of "furtive" dependency hell upgrade by package-install. Maybe Dmitry has? Or you have? If anything, I've seen bugs related to upgrades of dependencies _not_ happening (because of package.el's inability to understand transitive dependencies -- Basil spotted that, I think). > > I would think that if you oppose a bugfix backport you would also > > oppose a _feature_ backport, which usually is (and is indeed in > > this case) a much more complicated, "scary", bug-prone development. > > I consider each case of a request to backport on its own merit. So > conclusions such as the above, which don't consider the specifics of > each change, but instead go by some abstract principles, are not what > I usually make. That makes full sense, btw. > > > Understood. My point is that if you want Eglot users to be able to > > > upgrade to a newer versions, you need to have compatibility layers, t= o > > > avoid the need to upgrade too many other packages, which might hamper > > > stability. Otherwise, we cannot in good faith recommend that users o= f > > > stable Emacs update their core packages without a second thought. > > > > Yes, and this is why each released version of Eglot specifies exactly > > the _released_ versions of its dependencies that it depends on. > > You are missing my point. I'm not talking about dependencies with > other packages. My point is not about other core packages, it's about > Emacs itself and its stability as a whole. Users of a stable release > of Emacs can, and usually do, expect their entire Emacs configurations > to be stable, and telling them to upgrade packages without any clear > indication of their stability goes against that. Yes, requiring > specific versions of the other N packages will minimize breakage due > to incompatibilities between those packages, but what about unintended > consequences, regressions, etc.? Suppose Eglot 1.14 brings with it > some package whose updated version has a bug -- why should a user who > wants to have a stable Emacs environment to risk this? She shouldn't. She should never M-x package-install RET eglot or have it in her configuration. This is not a new problem. Things that download code bear risks, at least when compared to the stock Emacs 29 that went through a lenghty pretest period. This is _precisely_ why I think backporting Eglot 1.15/16/whatever to Emacs 29 is not a good idea. This is why that user should use Eglot bundled with Emacs 29. It's a good, stable, well-tested Eglot that you can do lots of cool stuff in. > > > Updating a package P1 should require update of as few packages P2...P= n > > > as possible. Ideally, none at all. > > > > And very often that does happen, I suppose. Not every Eglot release > > _requires_ installation of new versions of its dependencies. But some > > do. > > I'm asking whether you make an extra effort to avoid such requirements > whenever possible, even if that would call for extra work on your > part? I hope you and other package developers do, because otherwise > proliferation of core packages would be a very bad deal for the future > of Emacs, which traditionally is perceived as an extremely stable > platform. Yep, I do those efforts, but I wouldn't go so far as to call them "extra". I think things should go where they belong. That's, again, the main design philosophy of Eglot. Don't invent (too much) UI in Eglot. Put UI and other functionality in client-agnostic libraries and use Eglot to link up LSP interfaces to those libraries. Consequently, that requires new libraries and updates to the libraries. For example, for the "breadcrumb headerline" feature of bug#58431 I'll be proposing the introduction of a new lisp/progmodes/breadcrumb.el :core library that depends solely on Emacs's imenu.el. It works with Eglot and without Eglot (and it's turning out quite nice, should work with the imenu info produced by the tree-sitter modes, too). So is that "proliferation of :core libraries"? Yes maybe, but it's also what we want. We don't want a from-scratch LSP-specific breadcrumb implementation in Eglot, I think. So a library is the way to go. If you don't want it in core, I can very well put it in GNU ELPA, or even MELPA. Then Eglot will "optionally" depend on it, much in the same way it already depends "optionally" on Markdown.el, Company and Yasnippet. > > > Users should be able to decide > > > whether they want or don't want to update any single package without > > > also needing to decide whether they are okay with updating half a > > > dozen of others. This should be our goal, because otherwise updating > > > a package will be unsafe if you use any Emacs except master (and thus > > > don't care much about stability anyway). > > > > I don't know how you can meet that goal in general. > > If you at least think we should try, then we have something to work > with. Even if the ideal of having to upgrade no package is > unattainable, bringing the number close to zero is a worthy goal. ... > I'm saying that we should make an extra effort to avoid that, not > accept dependencies without a "fight". OK, you have a foot soldier to "fight" against dependency hell :-). But sometimes I will enter talks with these dependencies who may not be from hell, rather from some kind of more well behaved purgatory. > > This isn't the end of the analysis, of course. > > I'm not sure history is the aspect that distinguishes them. An > important aspect is how "low' is the functionality provided by the > package. Some packages provide infrastructure -- those affect many > other places by definition. Others are applications on which nothing > else depends. Perhaps these are the important factors, I don't know. Yes, these are other important factors. But history is important as a factor because the _past_ dictates user expectations, which are by definition based on it. And stability is about expectations (as someone wisely wrote here recently, wisely, apropos a security bug). > > - That members of set 1 shouldn't be upgradable "willy nilly" to > > maintain exact backward-compatibility. > > > > - That members of set 2 should be upgraded in much easier fashion > > because that's what guarantees that people's configs already > > doing so won't break. > > That is completely irrelevant for this discussion, IMO. It is up to > the users whether to upgrade and how aggressively. We don't know > enough about the configurations, the environment, and the usage > patterns of the users, we can only give them information -- in this > case regarding stability of each available version -- and let them > make the conclusions as they see fit. We will never be able to second > guess them well enough. It is not irrelevant. We can know _exactly_ how certain things worked in Emacs 26, 27 and 28 and how much of a backward-incompatible change we're making. Thus we can act accordingly to minimize that change. Or even eliminate it for the specific cases where we want to eliminate it. Minimizing change is the bread-and-butter of stability, which seems to be something you're also interested in, but also at the same time, contradictingly, aren't. That part I still don't understand. In this case, the "certain thing" is the behaviour of the very popular and extremely simple (use-package eglot :ensure t) or the presence or a (package-install eglot) in a configuration. It does one thing in Emacs 26/7/8 it will do another different thing in Emacs 29. How "popular" is this (use-package eglot :ensure t), you ask? Just check it out for yourself: https://github.com/joaotavora/eglot/search?q=3D%22use-package%22&type=3Diss= ues it pops up every other issue. Jo=C3=A3o