bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic <philipk@posteo.net>]

[-- Attachment #1: Type: text/plain, Size: 7068 bytes --]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Philip Kaludercic <philipk@posteo.net>
>>> Cc: Daniel Mendler <mail@daniel-mendler.de>,  66554@debbugs.gnu.org,
>>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>>> Date: Fri, 12 Jan 2024 22:27:37 +0000
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> >> > I guess I'm misunderstanding something.  The scenario that I have in
>>> >> > mind is this:
>>> >> >
>>> >> >   . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
>>> >> >   . as part of the pretest, some function changes that requires
>>> >> >     addition or change in compat.el
>>> >> >   . compat.el still claims version NN.1, although it includes changes
>>> >> >     not present in Emacs NN.1
>>> >> >
>>> >> > Did I succeed in explaining my worries?
>>> >> 
>>> >> I hope I understood your reasoning correctly. Shortly after you bump
>>> >> NN.1.90 we can release compat-NN.2.0 which will include the necessary
>>> >> change.
>>> >
>>> > Release where? on ELPA or as part of the Emacs tarball?
>>> >
>>> > And how do we make sure we will not forget to release this new version
>>> > of compat.el?
>>> 
>>> We have to distinguish the ELPA package Compat and the compat.el file
>>> being added here.
>>
>> Yes, we should.  I'm asking exactly that: when you say "we can release
>> compat-NN.2.0", do you mean ELPA or do you mean compat.el in Emacs?
>
> In this case we mean compat-NN.2.0 on ELPA.
>
>>> The ELPA package Compat is manually released on our
>>> behalf, usually after a release of Emacs has been announced.  The plan
>>> is that the compat.el file does not have to be touched at all, since it
>>> registers itself to use the right version, while Emacs is scraping the
>>> for autoloads.  So there shouldn't be any additional effort from the
>>> side of Emacs maintenance, and nothing one can "forget".
>>
>> But that's exactly the problem I'm struggling with: compat.el in Emacs
>> registers itself with inaccurate version, which lacks the last part
>> after emacs-minor.
>
> The internal compat.el in Emacs does not register itself with an
> inaccurate version. The version of compat.el in Emacs for a version NN.x
> is always the newest version one can get by definition.
>
> If the Emacs version is 30.1, the version of the internal compat.el is
> 30.1.most-positive-fixnum. This means the internal compat.el is
> considered newer than any compat-30.1.x on ELPA. The reasoning is that
> compat-30.1.x on ELPA cannot provide anything newer which is not already
> in Emacs 30.1, and as such it is not necessary to install it.
>
> However if there exists a compat-30.2.x on ELPA, then this package will
> take precendence over the internal compat.el with version
> 30.1.most-positive-fixnum. Then the ELPA package compat-30.2.x can get
> installed on an Emacs 30.1, if depended on by another ELPA package,
> e.g., Magit.
>
> For example let's assume that Magit depends on compat-30.2. Note that
> this dependency can either be satisfied by an ELPA compat-30.2.x or by
> an internal compat.el with version 30.2.most-positive-fixnum. Magit
> wants to take advantage of new APIs or API changes introduced in Emacs
> 30.2, which are also made available by ELPA compat-30.2. This means in
> this case ELPA compat-30.2.x must be installed if the internal compat.el
> has version 30.1.most-positive-fixnum.
>
>> As for "forgetting", I do mean whoever should remember to release a
>> new version of Compat on ELPA -- if this is a manual operation, it can
>> be forgotten, especially if it has to be related to pretest releases
>> as well as the official releases.
>
> Currently I am responsible of releasing new versions of Compat to ELPA.
> But there is no risk of forgetting, or rather, it is okay to not release
> new ELPA Compat versions immediately. There can be a time window.
>
> In the scenario described above, Magit wants to depend on compat-30.2.
> Since Magit itself is released on ELPA, the package can only be
> installed when compat-30.2 is available on ELPA. This means that Jonas
> will only start to depend on compat-30.2 after I've released it to ELPA.
> Before that has happened, Magit cannot take advantage of features which
> have been introduced in Emacs 30.2.
>
>> I guess I lack an overall picture of how this is supposed to work, as
>> part of our pretest and release flow.  Could you or someone else post
>> such a complete description?  Without this, I'm not sure I agree with
>> the changes being considered, or at least don't quite understand their
>> impact on the routine maintenance.
>
> The inclusion of compat.el into Emacs won't have an impact on the Emacs
> pretest or release flow. The Compat ELPA package can be released
> independently as I described above. There are these reasons why we are
> proposing the small compat.el file for inclusion:
>
> 1. It will be easier for :core packages which are available on ELPA to
> take advantage of Compat. They can (require 'compat) and they don't have
> to use (require 'compat nil 'noerror).
>
> 2. Core packages which use Compat do not have to replicate the
> `compat-function' and `compat-call' macros in their own code. See
> erc-compat.el for a core package, where these macros are already
> duplicated with the names `erc-compat-function' and `erc-compat-call'.
> Instead of this replication, erc-compat.el can just (require 'compat)
> and use `compat-function' and `compat-call'.
>
> 3. Compat should not be installed automatically as dependency if Emacs
> itself already provides the required API. For example on Emacs 30.1 I do
> not want to install compat-30.1 from ELPA, since Emacs itself already
> provides everything compat-30.1.x from ELPA ever could provide. Emacs is
> in charge of the API definition and Compat won't add anything itself.
>
> Now to describe the pretest or release flows - the Emacs release flow
> can proceeds as usual, as it has happened before with 28.1, 28.2, 29.1
> etc. The compat.el file as part of Emacs will not have to be updated or
> changed.
>
> The ELPA package Compat will be released independently. After a release
> of pretest Emacs-30.0 we may release Compat-30.0.x on ELPA. This does
> not have to happen immediately. Only after Compat-30.0.x has landed on
> ELPA, other ELPA packages (which for example depend on Emacs 26.1 as
> base line) may start to depend on compat-30.0 and start using new Emacs
> 30.0 functions or macros, e.g., `static-if'. Before the release of
> Compat, an external package cannot take advantage of `static-if', since
> the ELPA Compat does not provide it yet, and the base line Emacs 26.1
> does not provide it.
>
> Note that Compat on ELPA is already used by many packages exactly in the
> way I've described. The inclusion of compat.el in Emacs will smoothen
> the situation for core packages (which are also released to ELPA) and
> will ensure that no unnecessary Compat package is installed, the points
> 1, 2 and 3 I listed above.
>
> Daniel

Pinging this thread with an updated version of the patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 7227 bytes --]

From 09ae06230f7ca541ba97dafc1d8b2388fc604d6a Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
* doc/lispref/package.texi (Forwards-Compatibility): Mention Compat
and link to the manual.
* etc/NEWS: Document change.  (Bug#66554)
---
 doc/lispref/package.texi  | 13 ++++++
 etc/NEWS                  | 11 ++++++
 lisp/emacs-lisp/compat.el | 83 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 4 files changed, 108 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
index 6f52a33d194..9e652ab2515 100644
--- a/doc/lispref/package.texi
+++ b/doc/lispref/package.texi
@@ -28,6 +28,7 @@ Packaging
 * Multi-file Packages::     How to package multiple files.
 * Package Archives::        Maintaining package archives.
 * Archive Web Server::      Interfacing to an archive web server.
+* Forwards-Compatibility::  Supporting older versions of Emacs.
 @end menu
 
 @node Packaging Basics
@@ -390,3 +391,15 @@ Archive Web Server
 package, or the single file for a simple package.
 
 @end table
+
+@node Forwards-Compatibility
+@section Supporting older versions of Emacs
+@cindex compatibility compat
+
+Packages that wish to support older releases of Emacs, without giving
+up on newer functionality from recent Emacs releases, one can make use
+of the Compat package on GNU ELPA.  For details on how to make use of
+the package, @xref{Usage,, Usage, compat, "Compat" Manual}. In case
+you don't have the package installed, you can also read the
+@url{https://elpa.gnu.org/packages/doc/compat.html#Usage, Online
+Compat manual}.
diff --git a/etc/NEWS b/etc/NEWS
index 735a05f6579..f599e54b168 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1309,6 +1309,17 @@ This minor mode generates the tags table automatically based on the
 current project configuration, and later updates it as you edit the
 files and save the changes.
 
++++
+** New package Compat
+The Compat package on GNU ELPA provides forwards-compatibility
+support, so that packages that still provide support for older
+versions of Emacs can still make use of newer definitions that can be
+reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
+part of Emacs, that doesn't provide any compatibility support, but
+only implements the public-facing API of Compat so that core packages
+can use Compat, while also preventing the installation of Compat on
+the most recent version of Emacs.
+
 \f
 * Incompatible Lisp Changes in Emacs 30.1
 
diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..726b3fd298b
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,83 @@
+;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; You can find out more about Compat and how to use it from the Info
+;; node `(compat) Top' (installed along with the Compat package) or
+;; read the same manual online:
+;; https.gnu.org/packages/doc/compat.html.
+
+;; Note that Compat is NOT a core package and this file is NOT
+;; available on GNU ELPA.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+;;;; Hack to avoid installing Compat if not necessary
+
+;; The versioning scheme of the Compat package follows that of Emacs,
+;; to indicate what version of Emacs is being supported.  For example,
+;; the Compat version number 29.2.3.9 would attempt to provide
+;; compatibility definitions up to Emacs 29.2, while also designating
+;; that this is the third major release and ninth minor release of
+;; Compat, for the specific Emacs release.
+
+;; To ensure that if the user is using Emacs X.Y installed, the ELPA
+;; package Compat X.Y.Z* (for any values of Z*) does not get
+;; unnecessarily installed, as there are no missing features that
+;; Compat could provide, we programmatically specify the version of
+;; the package to be that of the current Emacs version plus a high
+;; "major release" to exceed the major version of Compat.
+
+;;;###autoload (push (list 'compat emacs-major-version emacs-minor-version most-positive-fixnum) package--builtin-versions)
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..1f4a8e01294 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2