bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic <philipk@posteo.net>]

[-- Attachment #1: Type: text/plain, Size: 6365 bytes --]

(Sorry for the late response.)

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
> [...]
>
>>> Philip, do you plan to submit a new version of the patch or do you
>>> want
>>> me to update the patch with a more extensive explanation? We should
>>> keep
>>> in mind that the information we add to the Emacs compat.el cannot be
>>> self sufficient. Emacs developers who want to use Compat must consult
>>> the Compat manual, since that's the place where we document the
>>> available compatibility definitions. Therefore referring to the manual
>>> for further details should be okay, as long as the general mechanism
>>> (and the versioning) is explained sufficiently well in the
>>> commentary of
>>> the compat.el file in Emacs.
>>
>> I have tried to update the patch to clarify some of the points in the
>> discussion, but feel free to change anything you think ought to be
>> changed:
>
> Thank you, Philip. I added a few comments below, mostly about some
> details of the wording. Eli, Stefans, do you think the level of
> information provided in the patch is sufficient?
>
>> diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
>> index 6f52a33d194..b9239521d33 100644
>> --- a/doc/lispref/package.texi
>> +++ b/doc/lispref/package.texi
>
> [...]
>
>> +The versioning of Compat follows that of Emacs, so one can implicitly
>> +declare what range of Emacs versions a package supports like so:
>> +
>> +@example
>> +;; Package-Requires: ((emacs "27.2") (compat "29.1"))
>> +@end example
>
> The word "range" is misleading. It sounds as if the package supports
> 27.2 to 29.1, while in it actually supports 27.2 and newer and relies on
> some 29.1 APIs.

Changed this.

> [...]
>
>> diff --git a/etc/NEWS b/etc/NEWS
>> index a1874313502..46859d75aac 100644
>> --- a/etc/NEWS
>> +++ b/etc/NEWS
>> @@ -1321,6 +1321,17 @@ This minor mode generates the tags table
>> automatically based on the
>>  current project configuration, and later updates it as you edit the
>>  files and save the changes.
>>  
>> ++++
>> +** New package Compat
>> +The Compat package on GNU ELPA provides forwards-compatibility
>> +support, so that packages that still provide support for older
>> +versions of Emacs can still make use of newer definitions that can be
>> +reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
>> +part of Emacs, that doesn't provide any compatibility support, but
>> +only implements the public-facing API of Compat so that core packages
>> +can use Compat, while also preventing the installation of Compat on
>> +the most recent version of Emacs.
>
> This NEWS entry explains the addition well, but it is a bit verbose
> compared to other entries. I am not fond of the quoted word "pseudo".
> Maybe say stub of Compat?

Abbreviated it.

>>  * Incompatible Lisp Changes in Emacs 30.1
>>  
>> diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
>> new file mode 100644
>> index 00000000000..2882974cf2d
>> --- /dev/null
>> +++ b/lisp/emacs-lisp/compat.el
>> @@ -0,0 +1,94 @@
>> +;;; compat.el --- Pseudo-Compatibility for Elisp -*-
>> lexical-binding: t; -*-
>
> Instead of "Pseudo" maybe write "Stub of the Emacs Lisp Compatibility
> Library"? We could also use "Emacs Lisp Compatibility Library" like in
> ELPA Compat package, since for packages using the Compat library it
> should not make a difference if the builtin compat.el stub or the ELPA
> package is used.

I changed it to "Stub of the Compatibility Library", to avoid an
overlong line and also because Elisp should be implicit.

>> +;;; Commentary:
>
> [...]
>
>> +;; Note that Compat is NOT a core package and this file is NOT
>> +;; available on GNU ELPA.
>
> I find this sentence a bit confusing. What you want to tell here is that
> the compat.el file in Emacs differs from the compat.el file in the ELPA
> package. This is maybe already clear from the other comments so we can
> as well remove this sentence?

Done.

> [...]
>
>> +;;;; Clever trick to avoid installing Compat if not necessary
>> +
>> +;; The versioning scheme of the Compat package follows that of Emacs,
>> +;; to indicate what version of Emacs is being supported.  For example,
>                                                  ^^^^^^^^^
>
> Should we say "supported" here? Compat supports other version than the
> one specified. Compat "provides" functionality from that Emacs version.

Rephrased this, but sounds a bit clunky.

>> +;; the Compat version number 29.2.3.9 would attempt to provide
>> +;; compatibility definitions up to Emacs 29.2, while also designating
>> +;; that this is the third major release and ninth minor release of
>> +;; Compat, for the specific Emacs release.
>> +
>> +;; The package version of this file is specified programmatically,
>> +;; instead of giving a fixed version in the header of this file.  This
>> +;; is done to ensure that the version of compat.el provided by Emacs
>> +;; is always corresponds to the current version of Emacs.  In addition
>       ^^
>
> The "is" should be removed.

Right.

>> +;; to the major-minor version, a large "major release" makes sure that
>> +;; the built-in version of Compat is always preferred over an external
>> +;; installation.  This means that if a package specifies a dependency
>> +;; on Compat which matches the current version of Emacs that is being
>                               ^^^^^^^^^^^^^^^^^^^
>                               
> ...which matches the current *or an older version* of Emacs...

Ok.

>> +;; used, no additional dependencies have to be downloaded.
>> +;;
>> +;; Further details and background on this file can be found in the
>> +;; bug#66554 discussion.
>> +
>> +;;;###autoload (push (list 'compat
>> +;;;###autoload             emacs-major-version
>> +;;;###autoload             emacs-minor-version
>> +;;;###autoload             1.0e+INF)
>> +;;;###autoload       package--builtin-versions)
>
> I prefer if we use 9999 here instead of 1.0e+INF. While infinity is
> semantically correct, the float may lead to problems and hurt
> readability in the package list.

It seems the main issue is that 30.0.1.0e+INF can be displayed without
any problems, but version-to-list rejects it.  Sad, because I think this
is a neat idea but I've changed it to 9999.

>> +(provide 'compat)
>> +;;; compat.el ends here
>
> Daniel


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 8829 bytes --]

From bc1aaa2e5983d77b3ac0c3a95d73197bc9127dac Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
* doc/lispref/package.texi (Forwards-Compatibility): Mention Compat
and link to the manual.
* etc/NEWS: Document change.  (Bug#66554)
---
 doc/lispref/package.texi  | 46 ++++++++++++++++++++
 etc/NEWS                  |  7 +++
 lisp/emacs-lisp/compat.el | 92 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 4 files changed, 146 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
index 6f52a33d194..071c87170f5 100644
--- a/doc/lispref/package.texi
+++ b/doc/lispref/package.texi
@@ -28,6 +28,7 @@ Packaging
 * Multi-file Packages::     How to package multiple files.
 * Package Archives::        Maintaining package archives.
 * Archive Web Server::      Interfacing to an archive web server.
+* Forwards-Compatibility::  Supporting older versions of Emacs.
 @end menu
 
 @node Packaging Basics
@@ -390,3 +391,48 @@ Archive Web Server
 package, or the single file for a simple package.
 
 @end table
+
+@node Forwards-Compatibility
+@section Supporting older versions of Emacs
+@cindex compatibility compat
+
+Packages that wish to support older releases of Emacs, without giving
+up on newer functionality from recent Emacs releases, one can make use
+of the Compat package on GNU ELPA.  By depending on the package, Emacs
+can provide compatibility definitions for missing functionality.
+
+The versioning of Compat follows that of Emacs, so next to the oldest
+version that a package relies on (via the @code{emacs}-package), one
+can also indicate what the newest version of Emacs is, that a package
+wishes to use definitions from:
+
+@example
+;; Package-Requires: ((emacs "27.2") (compat "29.1"))
+@end example
+
+By default, one can refer to compatibility definitions by their given
+names from future versions.  For example, one can use the function
+@code{take} on before Emacs 29 as such.  Due to changes of function
+and macro calling conventions over time, this is not always possible
+and it is occasionally necessary to explicitly refer to compatibility
+code.  To this end one can use the Compat API:
+
+@defmac compat-call fun &rest args
+This macro calls the compatibility function @var{fun} with @var{args}.
+Many functions provided by Compat can be called directly without this
+macro.  However in the case where Compat provides an alternative
+version of an existing function, the function call has to go through
+@code{compat-call}.
+@end defmac
+
+@defmac compat-function fun
+This macro returns the compatibility function symbol for @var{fun}.
+See @code{compat-call} for a more convenient macro to directly call
+compatibility functions.
+@end defmac
+
+For further details on how to make use of the package, see
+@ref{Usage,, Usage, compat, "Compat" Manual}.  In case you don't have
+the package installed, you can also read the
+@url{https://elpa.gnu.org/packages/doc/compat.html#Usage, Online
+Compat manual}.
diff --git a/etc/NEWS b/etc/NEWS
index a1874313502..7d0a202a787 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1321,6 +1321,13 @@ This minor mode generates the tags table automatically based on the
 current project configuration, and later updates it as you edit the
 files and save the changes.
 
++++
+** New package Compat
+Emacs now comes with a stub implementation of the
+forwards-compatibility Compat package from GNU ELPA.  This allows
+built-in packages to use the library more effectively, and helps
+preventing the installation of Compat if unnecessary.
+
 \f
 * Incompatible Lisp Changes in Emacs 30.1
 
diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..f7037dc4101
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,92 @@
+;;; compat.el --- Stub of the Compatibility Library -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; A basic introduction to Compat is given in the Info node `(elisp)
+;; Forwards Compatibility'.  Further details on Compat are documented
+;; in the Info node `(compat) Top' (installed along with the Compat
+;; package) or read the same manual online:
+;; https://elpa.gnu.org/packages/doc/compat.html.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+;;;; Clever trick to avoid installing Compat if not necessary
+
+;; The versioning scheme of the Compat package follows that of Emacs,
+;; to indicate the version of Emacs, that functionality is being
+;; provided for.  For example, the Compat version number 29.2.3.9
+;; would attempt to provide compatibility definitions up to Emacs
+;; 29.2, while also designating that this is the third major release
+;; and ninth minor release of Compat, for the specific Emacs release.
+
+;; The package version of this file is specified programmatically,
+;; instead of giving a fixed version in the header of this file.  This
+;; is done to ensure that the version of compat.el provided by Emacs
+;; always corresponds to the current version of Emacs.  In addition to
+;; the major-minor version, a large "major release" makes sure that
+;; the built-in version of Compat is always preferred over an external
+;; installation.  This means that if a package specifies a dependency
+;; on Compat which matches the current or an older version of Emacs
+;; that is being used, no additional dependencies have to be
+;; downloaded.
+;;
+;; Further details and background on this file can be found in the
+;; bug#66554 discussion.
+
+;;;###autoload (push (list 'compat
+;;;###autoload             emacs-major-version
+;;;###autoload             emacs-minor-version
+;;;###autoload             9999)
+;;;###autoload       package--builtin-versions)
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..1f4a8e01294 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2