bug#66554: [PATCH] Add the public API of Compat to the core

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 480 bytes --]

Tags: patch


See https://lists.gnu.org/archive/html/emacs-devel/2023-10/msg00260.html
for the background behind this proposal.


In GNU Emacs 30.0.50 (build 5, x86_64-pc-linux-gnu, GTK+ Version
 3.24.37, cairo version 1.16.0) of 2023-10-13 built on quetzal
Repository revision: 97959349651a8fb170c1c754e73a4d86ed24d018
Repository branch: master
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --with-pgtk --with-imagemagick --with-tree-sitter'


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/patch, Size: 3834 bytes --]

From 9ab5748d0e93c84c36c5559a4cda2087b3a24560 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
---
 lisp/emacs-lisp/compat.el | 61 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 2 files changed, 62 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..4c60093b6be
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,61 @@
+;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2023 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Version: 1.0
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; Note that Compat is not a core package and this file is not
+;; available on GNU ELPA.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 4b940b3f13b..bf0e27bc786 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -264,7 +264,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2

bug#66554: [PATCH] Add the public API of Compat to the core

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

> See https://lists.gnu.org/archive/html/emacs-devel/2023-10/msg00260.html
> for the background behind this proposal.

I'm fine with this, and see the same benefits as described above.

I'd like to hear if Eli and Stefan M has anything to add, though, so I'm
copying them in here.

> From 9ab5748d0e93c84c36c5559a4cda2087b3a24560 Mon Sep 17 00:00:00 2001
> From: Philip Kaludercic <philipk@posteo.net>
> Date: Wed, 13 Sep 2023 12:26:22 +0200
> Subject: [PATCH] Add the public API of Compat to the core
>
> * lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
> so that core packages, that haven't been installed from ELPA, can make
> use of the public API and use more recent function signatures.
> * lisp/progmodes/python.el (compat): Remove 'noerror flag, because
> Compat can now be required without the real package being available.
> ---
>  lisp/emacs-lisp/compat.el | 61 +++++++++++++++++++++++++++++++++++++++
>  lisp/progmodes/python.el  |  2 +-
>  2 files changed, 62 insertions(+), 1 deletion(-)
>  create mode 100644 lisp/emacs-lisp/compat.el
>
> diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
> new file mode 100644
> index 00000000000..4c60093b6be
> --- /dev/null
> +++ b/lisp/emacs-lisp/compat.el
> @@ -0,0 +1,61 @@
> +;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
> +
> +;; Copyright (C) 2021-2023 Free Software Foundation, Inc.
> +
> +;; Author:								\
> +;;   Philip Kaludercic <philipk@posteo.net>,				\
> +;;   Daniel Mendler <mail@daniel-mendler.de>
> +;; Maintainer:								\
> +;;   Daniel Mendler <mail@daniel-mendler.de>,				\
> +;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
> +;;   emacs-devel@gnu.org
> +;; URL: https://github.com/emacs-compat/compat
> +;; Version: 1.0
> +;; Keywords: lisp, maint
> +
> +;; This program is free software; you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation, either version 3 of the License, or
> +;; (at your option) any later version.
> +
> +;; This program is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
> +
> +;;; Commentary:
> +
> +;; The Compat package on ELPA provides forward-compatibility
> +;; definitions for other packages.  While mostly transparent, a
> +;; minimal API is necessary whenever core definitions change calling
> +;; conventions (e.g. `plist-get' can be invoked with a predicate from
> +;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
> +;; advantage of this functionality, the macros `compat-function' and
> +;; `compat-call' have to be available in the core, usable even if
> +;; users do not have the Compat package installed, which this file
> +;; ensures.
> +
> +;; Note that Compat is not a core package and this file is not
> +;; available on GNU ELPA.
> +
> +;;; Code:
> +
> +(defmacro compat-function (fun)
> +  "Return compatibility function symbol for FUN.
> +This is a pseudo-compatibility stub for core packages on ELPA,
> +that depend on the Compat package, whenever the user doesn't have
> +the package installed on their current system."
> +  `#',fun)
> +
> +(defmacro compat-call (fun &rest args)
> +  "Call compatibility function or macro FUN with ARGS.
> +This is a pseudo-compatibility stub for core packages on ELPA,
> +that depend on the Compat package, whenever the user doesn't have
> +the package installed on their current system."
> +  (cons fun args))
> +
> +(provide 'compat)
> +;;; compat.el ends here
> diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
> index 4b940b3f13b..bf0e27bc786 100644
> --- a/lisp/progmodes/python.el
> +++ b/lisp/progmodes/python.el
> @@ -264,7 +264,7 @@
>  (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
>  (require 'treesit)
>  (require 'pcase)
> -(require 'compat nil 'noerror)
> +(require 'compat)
>  (require 'project nil 'noerror)
>  (require 'seq)

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas <stefankangas@gmail.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> See https://lists.gnu.org/archive/html/emacs-devel/2023-10/msg00260.html
>> for the background behind this proposal.
>
> I'm fine with this, and see the same benefits as described above.
>
> I'd like to hear if Eli and Stefan M has anything to add, though, so I'm
> copying them in here.

Thanks. I also want to make sure that we specify the Compat version
somewhere in the package builtins, such that Compat won't be installed
if it is not needed, e.g., when a package requires (compat "30.1.x") and
is installed on Emacs 30. Philip, what is needed to achieve this? It
seems to me that this is not yet handled in the proposed patch.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> See https://lists.gnu.org/archive/html/emacs-devel/2023-10/msg00260.html
>>> for the background behind this proposal.
>>
>> I'm fine with this, and see the same benefits as described above.
>>
>> I'd like to hear if Eli and Stefan M has anything to add, though, so I'm
>> copying them in here.
>
> Thanks. I also want to make sure that we specify the Compat version
> somewhere in the package builtins, such that Compat won't be installed
> if it is not needed, e.g., when a package requires (compat "30.1.x") and
> is installed on Emacs 30. Philip, what is needed to achieve this? It
> seems to me that this is not yet handled in the proposed patch.

If I am not mistaken, all we would have to do is to adjust the version
number in the patch to the current Emacs version -- and update it
whenever Emacs is updated.  I have a slight concern that this could be
easily forgotten, if it isn't hooked into some update scripts.

> Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Philip Kaludercic <philipk@posteo.net> writes:
>> Thanks. I also want to make sure that we specify the Compat version
>> somewhere in the package builtins, such that Compat won't be installed
>> if it is not needed, e.g., when a package requires (compat "30.1.x") and
>> is installed on Emacs 30. Philip, what is needed to achieve this? It
>> seems to me that this is not yet handled in the proposed patch.
>
> If I am not mistaken, all we would have to do is to adjust the version
> number in the patch to the current Emacs version -- and update it
> whenever Emacs is updated.  I have a slight concern that this could be
> easily forgotten, if it isn't hooked into some update scripts.

Okay, I see. The specified version should probably be something like
30.1.999 to make sure that packages requiring any Compat 30.1.x version
will not download the ELPA package?

Is there a place where we could add a warning to make sure that bumping
the version won't be forgotten?

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>>> Thanks. I also want to make sure that we specify the Compat version
>>> somewhere in the package builtins, such that Compat won't be installed
>>> if it is not needed, e.g., when a package requires (compat "30.1.x") and
>>> is installed on Emacs 30. Philip, what is needed to achieve this? It
>>> seems to me that this is not yet handled in the proposed patch.
>>
>> If I am not mistaken, all we would have to do is to adjust the version
>> number in the patch to the current Emacs version -- and update it
>> whenever Emacs is updated.  I have a slight concern that this could be
>> easily forgotten, if it isn't hooked into some update scripts.
>
> Okay, I see. The specified version should probably be something like
> 30.1.999 to make sure that packages requiring any Compat 30.1.x version
> will not download the ELPA package?

Because of 

  (version< "30.1" "30.1.1")

we can just set it to the actual Emacs version.

> Is there a place where we could add a warning to make sure that bumping
> the version won't be forgotten?

I think that admin/admin.el's `set-version' could be used like this:


[-- Attachment #2: Type: text/plain, Size: 629 bytes --]

diff --git a/admin/admin.el b/admin/admin.el
index 7fa2727aeb7..b8a9b41350a 100644
--- a/admin/admin.el
+++ b/admin/admin.el
@@ -115,6 +115,10 @@ set-version
   (set-version-in-file root "nt/README.W32" version
 		       (rx (and "version" (1+ space)
 				(submatch (1+ (in "0-9."))))))
+  (set-version-in-file root "lisp/emacs-lisp/compat.el" version
+		       (rx (and bol ";; Version : " (1+ space)
+				(submatch (1+ (in "0-9."))))))
+
   ;; TODO: msdos could easily extract the version number from
   ;; configure.ac with sed, rather than duplicating the information.
   (set-version-in-file root "msdos/sed2v2.inp" version

[-- Attachment #3: Type: text/plain, Size: 1017 bytes --]


> Daniel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Kangas <stefankangas@gmail.com>
>> Date: Wed, 10 Jan 2024 14:02:02 -0800
>> Cc: 66554@debbugs.gnu.org, eliz@gnu.org, 
>> 	Stefan Monnier <monnier@iro.umontreal.ca>
>> 
>> Philip Kaludercic <philipk@posteo.net> writes:
>> 
>> > See https://lists.gnu.org/archive/html/emacs-devel/2023-10/msg00260.html
>> > for the background behind this proposal.
>> 
>> I'm fine with this, and see the same benefits as described above.
>> 
>> I'd like to hear if Eli and Stefan M has anything to add, though, so I'm
>> copying them in here.
>
> I don't mind, but it sounds like the details of adding compat.el to
> core are not yet clear to all, and are still being discussed.  Also,
> it sounds like we would need to modify compat.el for every release or
> something? if so, this should be in make-tarball.txt.

As mentioned above, I think this could be automated.

> It might be also a good idea to mention this in the ELisp manual
> somewhere.

This being "Compat"?

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Philip Kaludercic <philipk@posteo.net> writes:
>>> If I am not mistaken, all we would have to do is to adjust the version
>>> number in the patch to the current Emacs version -- and update it
>>> whenever Emacs is updated.  I have a slight concern that this could be
>>> easily forgotten, if it isn't hooked into some update scripts.
>>
>> Okay, I see. The specified version should probably be something like
>> 30.1.999 to make sure that packages requiring any Compat 30.1.x version
>> will not download the ELPA package?
>
> Because of 
>
>   (version< "30.1" "30.1.1")
>
> we can just set it to the actual Emacs version.

No. We want a version larger than any Compat version corresponding to
the current Emacs version. For example when there is compat-31.1.1.7 on
ELPA and Magit depends on compat>=31.1 and Magit is installed on an
Emacs 31.1, Compat should not be installed, since the builtin Compat
already satisfies the dependency. However if Magit is installed on an
Emacs 30.2, compat-31.1.1.7 should be installed. This means the internal
Compat version on Emacs 30.1 should be like 30.1.9999, or another large
enough version.

>> Is there a place where we could add a warning to make sure that
>> bumping the version won't be forgotten?
>
> I think that admin/admin.el's `set-version' could be used like this:
>
>> I don't mind, but it sounds like the details of adding compat.el to
>> core are not yet clear to all, and are still being discussed.  Also,
>> it sounds like we would need to modify compat.el for every release or
>> something? if so, this should be in make-tarball.txt.
>
> As mentioned above, I think this could be automated.

I agree, it would be good to automate the version bump, such that it
cannot be forgotten.

>> It might be also a good idea to mention this in the ELisp manual
>> somewhere.
>
> This being "Compat"?

It makes sense to also mention Compat in the Elisp manual and refer to
the Compat manual on ELPA. There we already document Compat usage for
Emacs core packages, which should be updated accordingly
(https://elpa.gnu.org/packages/doc/compat.html#Usage).

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 1337 bytes --]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>>>> If I am not mistaken, all we would have to do is to adjust the version
>>>> number in the patch to the current Emacs version -- and update it
>>>> whenever Emacs is updated.  I have a slight concern that this could be
>>>> easily forgotten, if it isn't hooked into some update scripts.
>>>
>>> Okay, I see. The specified version should probably be something like
>>> 30.1.999 to make sure that packages requiring any Compat 30.1.x version
>>> will not download the ELPA package?
>>
>> Because of 
>>
>>   (version< "30.1" "30.1.1")
>>
>> we can just set it to the actual Emacs version.
>
> No. We want a version larger than any Compat version corresponding to
> the current Emacs version. For example when there is compat-31.1.1.7 on
> ELPA and Magit depends on compat>=31.1 and Magit is installed on an
> Emacs 31.1, Compat should not be installed, since the builtin Compat
> already satisfies the dependency. However if Magit is installed on an
> Emacs 30.2, compat-31.1.1.7 should be installed. This means the internal
> Compat version on Emacs 30.1 should be like 30.1.9999, or another large
> enough version.

My bad I mixed up what we wanted to achieve, in that case we could make
use of `version-regexp-alist' might make this


[-- Attachment #2: Type: text/plain, Size: 651 bytes --]

diff --git a/lisp/subr.el b/lisp/subr.el
index df28989b399..f5de5c3becf 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -6873,9 +6873,9 @@ version-separator
 
 Usually the separator is \".\", but it can be any other string.")
 
-
 (defconst version-regexp-alist
-  '(("^[-._+ ]?snapshot$"                                 . -4)
+  `(("^[-._+ ]?hyper$"                                    . ,most-positive-fixnum)
+    ("^[-._+ ]?snapshot$"                                 . -4)
     ;; treat "1.2.3-20050920" and "1.2-3" as snapshot releases
     ("^[-._+]$"                                           . -4)
     ;; treat "1.2.3-CVS" as snapshot release

[-- Attachment #3: Type: text/plain, Size: 1124 bytes --]


so that

  (version< "30.1.hyper" "30.1.0.1") => nil
  (version< "30.1.0.1" "30.1.hyper") => t

>
>>> Is there a place where we could add a warning to make sure that
>>> bumping the version won't be forgotten?
>>
>> I think that admin/admin.el's `set-version' could be used like this:
>>
>>> I don't mind, but it sounds like the details of adding compat.el to
>>> core are not yet clear to all, and are still being discussed.  Also,
>>> it sounds like we would need to modify compat.el for every release or
>>> something? if so, this should be in make-tarball.txt.
>>
>> As mentioned above, I think this could be automated.
>
> I agree, it would be good to automate the version bump, such that it
> cannot be forgotten.
>
>>> It might be also a good idea to mention this in the ELisp manual
>>> somewhere.
>>
>> This being "Compat"?
>
> It makes sense to also mention Compat in the Elisp manual and refer to
> the Compat manual on ELPA. There we already document Compat usage for
> Emacs core packages, which should be updated accordingly
> (https://elpa.gnu.org/packages/doc/compat.html#Usage).

Right, I agree.

> Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> --- a/lisp/subr.el
> +++ b/lisp/subr.el
> @@ -6873,9 +6873,9 @@ version-separator
>  
>  Usually the separator is \".\", but it can be any other string.")
>  
> -
>  (defconst version-regexp-alist
> -  '(("^[-._+ ]?snapshot$"                                 . -4)
> +  `(("^[-._+ ]?hyper$"                                    . ,most-positive-fixnum)
> +    ("^[-._+ ]?snapshot$"                                 . -4)
>      ;; treat "1.2.3-20050920" and "1.2-3" as snapshot releases
>      ("^[-._+]$"                                           . -4)
>      ;; treat "1.2.3-CVS" as snapshot release

Relying on a change in `version-regexp-alist` open a can of worms.
It's much simpler to make sure the GNU ELPA `compat` package uses
versions of the form `30.0.NN` (or `30.1.NN` if it means it
offers functionality from Emacs-30.2, tho hopefully we'll never get
there).

>>>> Is there a place where we could add a warning to make sure that
>>>> bumping the version won't be forgotten?

The addition cab done procedurally, as in

    ;;;;####autoload (push (list 'compat emacs-major-version emacs-minor-version) package--builtin-versions)


-- Stefan

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> --- a/lisp/subr.el
>> +++ b/lisp/subr.el
>> @@ -6873,9 +6873,9 @@ version-separator
>>  
>>  Usually the separator is \".\", but it can be any other string.")
>>  
>> -
>>  (defconst version-regexp-alist
>> -  '(("^[-._+ ]?snapshot$"                                 . -4)
>> +  `(("^[-._+ ]?hyper$"                                    . ,most-positive-fixnum)
>> +    ("^[-._+ ]?snapshot$"                                 . -4)
>>      ;; treat "1.2.3-20050920" and "1.2-3" as snapshot releases
>>      ("^[-._+]$"                                           . -4)
>>      ;; treat "1.2.3-CVS" as snapshot release
>
> Relying on a change in `version-regexp-alist` open a can of worms.
> It's much simpler to make sure the GNU ELPA `compat` package uses
> versions of the form `30.0.NN` (or `30.1.NN` if it means it
> offers functionality from Emacs-30.2, tho hopefully we'll never get
> there).

I had also considered to use versions like 29.0, 30.0, etc. for Compat.
However this is unfortunately not what we have done for Compat so far. I
argue that such versions are more confusing for package developers. If
developers want to depend on Emacs 30.1 features, it is more natural to
require (compat "30.1") in the package header.

If you don't want to use `version-regexp-alist' what about my initial
proposal of specifying a version like 30.1.9999 in the header of
compat.el?

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 766 bytes --]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>> The addition cab done procedurally, as in
>>>
>>>     ;;;;####autoload (push (list 'compat emacs-major-version
>>> emacs-minor-version) package--builtin-versions)
>>
>> If this work, I like this idea a lot!  I'll test it out to see if this
>> could simplify things.
>
> Philip, could we push (list 'compat emacs-major-version
> emacs-minor-version most-positive-fixnum)? Wouldn't this solve the two
> problems we have - no manual version bump and Compat could keep its
> current version scheme?

Yes, I just tried it out and it works (and it doesn't work the way we
want it to without `most-positive-fixnum'):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 4138 bytes --]

From 303a8be8c3cbc9810517a76baf4fc806da3d5aee Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being
available.  (Bug#66554)
---
 lisp/emacs-lisp/compat.el | 66 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 2 files changed, 67 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..b2c1125cb8d
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,66 @@
+;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; Note that Compat is not a core package and this file is not
+;; available on GNU ELPA.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+;; To ensure that if the user has installed Compat X.Y.Z.A on Emacs
+;; X.Y, that the this file is preferred by package.el, we
+;; programmatically set the version of this file:
+
+;;;###autoload (push (list 'compat emacs-major-version emacs-minor-version most-positive-fixnum) package--builtin-versions)
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 1148da11a06..c3f18692f61 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Philip Kaludercic <philipk@posteo.net> writes:

> Daniel Mendler <mail@daniel-mendler.de> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>>> The addition cab done procedurally, as in
>>>>
>>>>     ;;;;####autoload (push (list 'compat emacs-major-version
>>>> emacs-minor-version) package--builtin-versions)
>>>
>>> If this work, I like this idea a lot!  I'll test it out to see if this
>>> could simplify things.
>>
>> Philip, could we push (list 'compat emacs-major-version
>> emacs-minor-version most-positive-fixnum)? Wouldn't this solve the two
>> problems we have - no manual version bump and Compat could keep its
>> current version scheme?
>
> Yes, I just tried it out and it works (and it doesn't work the way we
> want it to without `most-positive-fixnum'):

Thanks, your patch looks good. Stefan's idea of using an autoload to
specify the version is nice. As Eli mentioned the Elisp manual should
probably refer to the Compat manual, and maybe the change also needs a
NEWS entry? Anything else?

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Daniel Mendler <mail@daniel-mendler.de> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>>>> The addition cab done procedurally, as in
>>>>>
>>>>>     ;;;;####autoload (push (list 'compat emacs-major-version
>>>>> emacs-minor-version) package--builtin-versions)
>>>>
>>>> If this work, I like this idea a lot!  I'll test it out to see if this
>>>> could simplify things.
>>>
>>> Philip, could we push (list 'compat emacs-major-version
>>> emacs-minor-version most-positive-fixnum)? Wouldn't this solve the two
>>> problems we have - no manual version bump and Compat could keep its
>>> current version scheme?
>>
>> Yes, I just tried it out and it works (and it doesn't work the way we
>> want it to without `most-positive-fixnum'):
>
> Thanks, your patch looks good. Stefan's idea of using an autoload to
> specify the version is nice. As Eli mentioned the Elisp manual should
> probably refer to the Compat manual, and maybe the change also needs a
> NEWS entry? Anything else?

I don't think we need anything else, so here is the updated patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 6345 bytes --]

From a93f008f1fc58fdfbe4e832624e3ae7b28a6fda3 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
* doc/lispref/package.texi (Forwards-Compatibility): Mention Compat
and link to the manual.
* etc/NEWS: Document change.  (Bug#66554)
---
 doc/lispref/package.texi  | 13 ++++++++
 etc/NEWS                  | 11 +++++++
 lisp/emacs-lisp/compat.el | 66 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 4 files changed, 91 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
index 6f52a33d194..b2b6e6b9671 100644
--- a/doc/lispref/package.texi
+++ b/doc/lispref/package.texi
@@ -28,6 +28,7 @@ Packaging
 * Multi-file Packages::     How to package multiple files.
 * Package Archives::        Maintaining package archives.
 * Archive Web Server::      Interfacing to an archive web server.
+* Forwards-Compatibility::  Supporting older versions of Emacs.
 @end menu
 
 @node Packaging Basics
@@ -390,3 +391,15 @@ Archive Web Server
 package, or the single file for a simple package.
 
 @end table
+
+@node Forwards-Compatibility
+@section Supporting older versions of Emacs
+@cindex compatibility compat
+
+Packages that wish to support older releases of Emacs, without giving
+up on newer functionality from recent Emacs releases, one can make use
+of the Compat package on GNU ELPA.  @xref{Usage,, Usage, compat,
+"Compat" Manual} For details on how to make use of the package (in
+case you don't have the package installed, you can also read the
+@url{https://elpa.gnu.org/packages/doc/compat.html#Usage, Online
+Compat manual}).
diff --git a/etc/NEWS b/etc/NEWS
index bce33f96aee..11dc90d5067 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1298,6 +1298,17 @@ This minor mode generates the tags table automatically based on the
 current project configuration, and later updates it as you edit the
 files and save the changes.
 
++++
+** New package Compat
+The Compat package on GNU ELPA provides forwards-compatibility
+support, so that packages that still provide support for older
+versions of Emacs can still make use of newer definitions that can be
+reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
+part of Emacs, that doesn't provide any compatibility support, but
+only implements the public-facing API of Compat so that core packages
+can use Compat, while also preventing the installation of Compat on
+the most recent version of Emacs.
+
 \f
 * Incompatible Lisp Changes in Emacs 30.1
 
diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..b2c1125cb8d
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,66 @@
+;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; Note that Compat is not a core package and this file is not
+;; available on GNU ELPA.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+;; To ensure that if the user has installed Compat X.Y.Z.A on Emacs
+;; X.Y, that the this file is preferred by package.el, we
+;; programmatically set the version of this file:
+
+;;;###autoload (push (list 'compat emacs-major-version emacs-minor-version most-positive-fixnum) package--builtin-versions)
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 1148da11a06..c3f18692f61 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2


[-- Attachment #3: Type: text/plain, Size: 1362 bytes --]


> Daniel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  66554@debbugs.gnu.org,
>>   eliz@gnu.org,  Stefan Kangas <stefankangas@gmail.com>
>> Date: Thu, 11 Jan 2024 20:43:11 +0000
>> 
>> +;; To ensure that if the user has installed Compat X.Y.Z.A on Emacs
>> +;; X.Y, that the this file is preferred by package.el, we
>                 ^^^^^^^^
> Typo.

Thanks; I also rewrote the comment to explain what is going on.

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: monnier@iro.umontreal.ca,  mail@daniel-mendler.de,
>>   66554@debbugs.gnu.org,  stefankangas@gmail.com
>> Date: Fri, 12 Jan 2024 07:38:20 +0000
>> 
>> The consequence is that we don't have to update any files, and I would
>> assume we don't have to document anything in make-tarball.txt, because
>> nothing has to be done when bumping the version of Emacs itself.
>
> Thanks.
>
> What happens when we bump Emacs version from NN.x to NN.x.90 or from
> NN.x.90 to NN.x.91?  emacs-major-version and emacs-minor-version stay
> the same, but the Emacs version is bumped, and there could be changes
> in compat.el specific to the next release, NN.x+1 or NN+1.1, no?

Generally speaking, Compat only provides support for "proper" releases,
and not for pre-releases.

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: 66554@debbugs.gnu.org,  eliz@gnu.org,  Stefan Monnier
>  <monnier@iro.umontreal.ca>,  Stefan Kangas <stefankangas@gmail.com>
> Date: Fri, 12 Jan 2024 16:29:15 +0000
> 
> > What happens when we bump Emacs version from NN.x to NN.x.90 or from
> > NN.x.90 to NN.x.91?  emacs-major-version and emacs-minor-version stay
> > the same, but the Emacs version is bumped, and there could be changes
> > in compat.el specific to the next release, NN.x+1 or NN+1.1, no?
> 
> Generally speaking, Compat only provides support for "proper" releases,
> and not for pre-releases.

Is that wise?

Can't we support pretest versions as well?

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: 66554@debbugs.gnu.org,  eliz@gnu.org,  Stefan Monnier
>>  <monnier@iro.umontreal.ca>,  Stefan Kangas <stefankangas@gmail.com>
>> Date: Fri, 12 Jan 2024 16:29:15 +0000
>> 
>> > What happens when we bump Emacs version from NN.x to NN.x.90 or from
>> > NN.x.90 to NN.x.91?  emacs-major-version and emacs-minor-version stay
>> > the same, but the Emacs version is bumped, and there could be changes
>> > in compat.el specific to the next release, NN.x+1 or NN+1.1, no?
>> 
>> Generally speaking, Compat only provides support for "proper" releases,
>> and not for pre-releases.
>
> Is that wise?
>
> Can't we support pretest versions as well?

I hope I am not misunderstanding something, but from the perspective of
Compat and Emacs development as a whole, I think it would be preferable
to not start propagating additions and changes to functions that might
change before a proper release, so as to not unnecessarily burden
development with sticking to decisions that one realises were mistakes
before the final release is cut.

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Fri, 12 Jan 2024 18:17:21 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Philip Kaludercic <philipk@posteo.net>
> >> Cc: 66554@debbugs.gnu.org,  eliz@gnu.org,  Stefan Monnier
> >>  <monnier@iro.umontreal.ca>,  Stefan Kangas <stefankangas@gmail.com>
> >> Date: Fri, 12 Jan 2024 16:29:15 +0000
> >> 
> >> > What happens when we bump Emacs version from NN.x to NN.x.90 or from
> >> > NN.x.90 to NN.x.91?  emacs-major-version and emacs-minor-version stay
> >> > the same, but the Emacs version is bumped, and there could be changes
> >> > in compat.el specific to the next release, NN.x+1 or NN+1.1, no?
> >> 
> >> Generally speaking, Compat only provides support for "proper" releases,
> >> and not for pre-releases.
> >
> > Is that wise?
> >
> > Can't we support pretest versions as well?
> 
> I hope I am not misunderstanding something, but from the perspective of
> Compat and Emacs development as a whole, I think it would be preferable
> to not start propagating additions and changes to functions that might
> change before a proper release, so as to not unnecessarily burden
> development with sticking to decisions that one realises were mistakes
> before the final release is cut.

I guess I'm misunderstanding something.  The scenario that I have in
mind is this:

  . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
  . as part of the pretest, some function changes that requires
    addition or change in compat.el
  . compat.el still claims version NN.1, although it includes changes
    not present in Emacs NN.1

Did I succeed in explaining my worries?

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,
>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>> Date: Fri, 12 Jan 2024 18:17:21 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Philip Kaludercic <philipk@posteo.net>
>> >> Cc: 66554@debbugs.gnu.org,  eliz@gnu.org,  Stefan Monnier
>> >>  <monnier@iro.umontreal.ca>,  Stefan Kangas <stefankangas@gmail.com>
>> >> Date: Fri, 12 Jan 2024 16:29:15 +0000
>> >> 
>> >> > What happens when we bump Emacs version from NN.x to NN.x.90 or from
>> >> > NN.x.90 to NN.x.91?  emacs-major-version and emacs-minor-version stay
>> >> > the same, but the Emacs version is bumped, and there could be changes
>> >> > in compat.el specific to the next release, NN.x+1 or NN+1.1, no?
>> >> 
>> >> Generally speaking, Compat only provides support for "proper" releases,
>> >> and not for pre-releases.
>> >
>> > Is that wise?
>> >
>> > Can't we support pretest versions as well?
>> 
>> I hope I am not misunderstanding something, but from the perspective of
>> Compat and Emacs development as a whole, I think it would be preferable
>> to not start propagating additions and changes to functions that might
>> change before a proper release, so as to not unnecessarily burden
>> development with sticking to decisions that one realises were mistakes
>> before the final release is cut.
>
> I guess I'm misunderstanding something.  The scenario that I have in
> mind is this:
>
>   . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
>   . as part of the pretest, some function changes that requires
>     addition or change in compat.el
>   . compat.el still claims version NN.1, although it includes changes
>     not present in Emacs NN.1
>
> Did I succeed in explaining my worries?

I hope I understood your reasoning correctly. Shortly after you bump
NN.1.90 we can release compat-NN.2.0 which will include the necessary
change. This Compat version will be installed on Emacs with version NN.1
but not on the upcoming Emacs NN.2.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Daniel Mendler <mail@daniel-mendler.de>
> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Fri, 12 Jan 2024 19:40:38 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I guess I'm misunderstanding something.  The scenario that I have in
> > mind is this:
> >
> >   . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
> >   . as part of the pretest, some function changes that requires
> >     addition or change in compat.el
> >   . compat.el still claims version NN.1, although it includes changes
> >     not present in Emacs NN.1
> >
> > Did I succeed in explaining my worries?
> 
> I hope I understood your reasoning correctly. Shortly after you bump
> NN.1.90 we can release compat-NN.2.0 which will include the necessary
> change.

Release where? on ELPA or as part of the Emacs tarball?

And how do we make sure we will not forget to release this new version
of compat.el?

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Mendler <mail@daniel-mendler.de>
>> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>> Date: Fri, 12 Jan 2024 19:40:38 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I guess I'm misunderstanding something.  The scenario that I have in
>> > mind is this:
>> >
>> >   . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
>> >   . as part of the pretest, some function changes that requires
>> >     addition or change in compat.el
>> >   . compat.el still claims version NN.1, although it includes changes
>> >     not present in Emacs NN.1
>> >
>> > Did I succeed in explaining my worries?
>> 
>> I hope I understood your reasoning correctly. Shortly after you bump
>> NN.1.90 we can release compat-NN.2.0 which will include the necessary
>> change.
>
> Release where? on ELPA or as part of the Emacs tarball?
>
> And how do we make sure we will not forget to release this new version
> of compat.el?

We have to distinguish the ELPA package Compat and the compat.el file
being added here.  The ELPA package Compat is manually released on our
behalf, usually after a release of Emacs has been announced.  The plan
is that the compat.el file does not have to be touched at all, since it
registers itself to use the right version, while Emacs is scraping the
for autoloads.  So there shouldn't be any additional effort from the
side of Emacs maintenance, and nothing one can "forget".

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Daniel Mendler <mail@daniel-mendler.de>,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Fri, 12 Jan 2024 22:27:37 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > I guess I'm misunderstanding something.  The scenario that I have in
> >> > mind is this:
> >> >
> >> >   . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
> >> >   . as part of the pretest, some function changes that requires
> >> >     addition or change in compat.el
> >> >   . compat.el still claims version NN.1, although it includes changes
> >> >     not present in Emacs NN.1
> >> >
> >> > Did I succeed in explaining my worries?
> >> 
> >> I hope I understood your reasoning correctly. Shortly after you bump
> >> NN.1.90 we can release compat-NN.2.0 which will include the necessary
> >> change.
> >
> > Release where? on ELPA or as part of the Emacs tarball?
> >
> > And how do we make sure we will not forget to release this new version
> > of compat.el?
> 
> We have to distinguish the ELPA package Compat and the compat.el file
> being added here.

Yes, we should.  I'm asking exactly that: when you say "we can release
compat-NN.2.0", do you mean ELPA or do you mean compat.el in Emacs?

> The ELPA package Compat is manually released on our
> behalf, usually after a release of Emacs has been announced.  The plan
> is that the compat.el file does not have to be touched at all, since it
> registers itself to use the right version, while Emacs is scraping the
> for autoloads.  So there shouldn't be any additional effort from the
> side of Emacs maintenance, and nothing one can "forget".

But that's exactly the problem I'm struggling with: compat.el in Emacs
registers itself with inaccurate version, which lacks the last part
after emacs-minor.

As for "forgetting", I do mean whoever should remember to release a
new version of Compat on ELPA -- if this is a manual operation, it can
be forgotten, especially if it has to be related to pretest releases
as well as the official releases.

I guess I lack an overall picture of how this is supposed to work, as
part of our pretest and release flow.  Could you or someone else post
such a complete description?  Without this, I'm not sure I agree with
the changes being considered, or at least don't quite understand their
impact on the routine maintenance.

Thanks.

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Daniel Mendler <mail@daniel-mendler.de>,  66554@debbugs.gnu.org,
>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>> Date: Fri, 12 Jan 2024 22:27:37 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> > I guess I'm misunderstanding something.  The scenario that I have in
>> >> > mind is this:
>> >> >
>> >> >   . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
>> >> >   . as part of the pretest, some function changes that requires
>> >> >     addition or change in compat.el
>> >> >   . compat.el still claims version NN.1, although it includes changes
>> >> >     not present in Emacs NN.1
>> >> >
>> >> > Did I succeed in explaining my worries?
>> >> 
>> >> I hope I understood your reasoning correctly. Shortly after you bump
>> >> NN.1.90 we can release compat-NN.2.0 which will include the necessary
>> >> change.
>> >
>> > Release where? on ELPA or as part of the Emacs tarball?
>> >
>> > And how do we make sure we will not forget to release this new version
>> > of compat.el?
>> 
>> We have to distinguish the ELPA package Compat and the compat.el file
>> being added here.
>
> Yes, we should.  I'm asking exactly that: when you say "we can release
> compat-NN.2.0", do you mean ELPA or do you mean compat.el in Emacs?

In this case we mean compat-NN.2.0 on ELPA.

>> The ELPA package Compat is manually released on our
>> behalf, usually after a release of Emacs has been announced.  The plan
>> is that the compat.el file does not have to be touched at all, since it
>> registers itself to use the right version, while Emacs is scraping the
>> for autoloads.  So there shouldn't be any additional effort from the
>> side of Emacs maintenance, and nothing one can "forget".
>
> But that's exactly the problem I'm struggling with: compat.el in Emacs
> registers itself with inaccurate version, which lacks the last part
> after emacs-minor.

The internal compat.el in Emacs does not register itself with an
inaccurate version. The version of compat.el in Emacs for a version NN.x
is always the newest version one can get by definition.

If the Emacs version is 30.1, the version of the internal compat.el is
30.1.most-positive-fixnum. This means the internal compat.el is
considered newer than any compat-30.1.x on ELPA. The reasoning is that
compat-30.1.x on ELPA cannot provide anything newer which is not already
in Emacs 30.1, and as such it is not necessary to install it.

However if there exists a compat-30.2.x on ELPA, then this package will
take precendence over the internal compat.el with version
30.1.most-positive-fixnum. Then the ELPA package compat-30.2.x can get
installed on an Emacs 30.1, if depended on by another ELPA package,
e.g., Magit.

For example let's assume that Magit depends on compat-30.2. Note that
this dependency can either be satisfied by an ELPA compat-30.2.x or by
an internal compat.el with version 30.2.most-positive-fixnum. Magit
wants to take advantage of new APIs or API changes introduced in Emacs
30.2, which are also made available by ELPA compat-30.2. This means in
this case ELPA compat-30.2.x must be installed if the internal compat.el
has version 30.1.most-positive-fixnum.

> As for "forgetting", I do mean whoever should remember to release a
> new version of Compat on ELPA -- if this is a manual operation, it can
> be forgotten, especially if it has to be related to pretest releases
> as well as the official releases.

Currently I am responsible of releasing new versions of Compat to ELPA.
But there is no risk of forgetting, or rather, it is okay to not release
new ELPA Compat versions immediately. There can be a time window.

In the scenario described above, Magit wants to depend on compat-30.2.
Since Magit itself is released on ELPA, the package can only be
installed when compat-30.2 is available on ELPA. This means that Jonas
will only start to depend on compat-30.2 after I've released it to ELPA.
Before that has happened, Magit cannot take advantage of features which
have been introduced in Emacs 30.2.

> I guess I lack an overall picture of how this is supposed to work, as
> part of our pretest and release flow.  Could you or someone else post
> such a complete description?  Without this, I'm not sure I agree with
> the changes being considered, or at least don't quite understand their
> impact on the routine maintenance.

The inclusion of compat.el into Emacs won't have an impact on the Emacs
pretest or release flow. The Compat ELPA package can be released
independently as I described above. There are these reasons why we are
proposing the small compat.el file for inclusion:

1. It will be easier for :core packages which are available on ELPA to
take advantage of Compat. They can (require 'compat) and they don't have
to use (require 'compat nil 'noerror).

2. Core packages which use Compat do not have to replicate the
`compat-function' and `compat-call' macros in their own code. See
erc-compat.el for a core package, where these macros are already
duplicated with the names `erc-compat-function' and `erc-compat-call'.
Instead of this replication, erc-compat.el can just (require 'compat)
and use `compat-function' and `compat-call'.

3. Compat should not be installed automatically as dependency if Emacs
itself already provides the required API. For example on Emacs 30.1 I do
not want to install compat-30.1 from ELPA, since Emacs itself already
provides everything compat-30.1.x from ELPA ever could provide. Emacs is
in charge of the API definition and Compat won't add anything itself.

Now to describe the pretest or release flows - the Emacs release flow
can proceeds as usual, as it has happened before with 28.1, 28.2, 29.1
etc. The compat.el file as part of Emacs will not have to be updated or
changed.

The ELPA package Compat will be released independently. After a release
of pretest Emacs-30.0 we may release Compat-30.0.x on ELPA. This does
not have to happen immediately. Only after Compat-30.0.x has landed on
ELPA, other ELPA packages (which for example depend on Emacs 26.1 as
base line) may start to depend on compat-30.0 and start using new Emacs
30.0 functions or macros, e.g., `static-if'. Before the release of
Compat, an external package cannot take advantage of `static-if', since
the ELPA Compat does not provide it yet, and the base line Emacs 26.1
does not provide it.

Note that Compat on ELPA is already used by many packages exactly in the
way I've described. The inclusion of compat.el in Emacs will smoothen
the situation for core packages (which are also released to ELPA) and
will ensure that no unnecessary Compat package is installed, the points
1, 2 and 3 I listed above.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 7068 bytes --]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Philip Kaludercic <philipk@posteo.net>
>>> Cc: Daniel Mendler <mail@daniel-mendler.de>,  66554@debbugs.gnu.org,
>>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>>> Date: Fri, 12 Jan 2024 22:27:37 +0000
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> >> > I guess I'm misunderstanding something.  The scenario that I have in
>>> >> > mind is this:
>>> >> >
>>> >> >   . we bump Emacs version to NN.1.90 as part of pretesting version NN.2
>>> >> >   . as part of the pretest, some function changes that requires
>>> >> >     addition or change in compat.el
>>> >> >   . compat.el still claims version NN.1, although it includes changes
>>> >> >     not present in Emacs NN.1
>>> >> >
>>> >> > Did I succeed in explaining my worries?
>>> >> 
>>> >> I hope I understood your reasoning correctly. Shortly after you bump
>>> >> NN.1.90 we can release compat-NN.2.0 which will include the necessary
>>> >> change.
>>> >
>>> > Release where? on ELPA or as part of the Emacs tarball?
>>> >
>>> > And how do we make sure we will not forget to release this new version
>>> > of compat.el?
>>> 
>>> We have to distinguish the ELPA package Compat and the compat.el file
>>> being added here.
>>
>> Yes, we should.  I'm asking exactly that: when you say "we can release
>> compat-NN.2.0", do you mean ELPA or do you mean compat.el in Emacs?
>
> In this case we mean compat-NN.2.0 on ELPA.
>
>>> The ELPA package Compat is manually released on our
>>> behalf, usually after a release of Emacs has been announced.  The plan
>>> is that the compat.el file does not have to be touched at all, since it
>>> registers itself to use the right version, while Emacs is scraping the
>>> for autoloads.  So there shouldn't be any additional effort from the
>>> side of Emacs maintenance, and nothing one can "forget".
>>
>> But that's exactly the problem I'm struggling with: compat.el in Emacs
>> registers itself with inaccurate version, which lacks the last part
>> after emacs-minor.
>
> The internal compat.el in Emacs does not register itself with an
> inaccurate version. The version of compat.el in Emacs for a version NN.x
> is always the newest version one can get by definition.
>
> If the Emacs version is 30.1, the version of the internal compat.el is
> 30.1.most-positive-fixnum. This means the internal compat.el is
> considered newer than any compat-30.1.x on ELPA. The reasoning is that
> compat-30.1.x on ELPA cannot provide anything newer which is not already
> in Emacs 30.1, and as such it is not necessary to install it.
>
> However if there exists a compat-30.2.x on ELPA, then this package will
> take precendence over the internal compat.el with version
> 30.1.most-positive-fixnum. Then the ELPA package compat-30.2.x can get
> installed on an Emacs 30.1, if depended on by another ELPA package,
> e.g., Magit.
>
> For example let's assume that Magit depends on compat-30.2. Note that
> this dependency can either be satisfied by an ELPA compat-30.2.x or by
> an internal compat.el with version 30.2.most-positive-fixnum. Magit
> wants to take advantage of new APIs or API changes introduced in Emacs
> 30.2, which are also made available by ELPA compat-30.2. This means in
> this case ELPA compat-30.2.x must be installed if the internal compat.el
> has version 30.1.most-positive-fixnum.
>
>> As for "forgetting", I do mean whoever should remember to release a
>> new version of Compat on ELPA -- if this is a manual operation, it can
>> be forgotten, especially if it has to be related to pretest releases
>> as well as the official releases.
>
> Currently I am responsible of releasing new versions of Compat to ELPA.
> But there is no risk of forgetting, or rather, it is okay to not release
> new ELPA Compat versions immediately. There can be a time window.
>
> In the scenario described above, Magit wants to depend on compat-30.2.
> Since Magit itself is released on ELPA, the package can only be
> installed when compat-30.2 is available on ELPA. This means that Jonas
> will only start to depend on compat-30.2 after I've released it to ELPA.
> Before that has happened, Magit cannot take advantage of features which
> have been introduced in Emacs 30.2.
>
>> I guess I lack an overall picture of how this is supposed to work, as
>> part of our pretest and release flow.  Could you or someone else post
>> such a complete description?  Without this, I'm not sure I agree with
>> the changes being considered, or at least don't quite understand their
>> impact on the routine maintenance.
>
> The inclusion of compat.el into Emacs won't have an impact on the Emacs
> pretest or release flow. The Compat ELPA package can be released
> independently as I described above. There are these reasons why we are
> proposing the small compat.el file for inclusion:
>
> 1. It will be easier for :core packages which are available on ELPA to
> take advantage of Compat. They can (require 'compat) and they don't have
> to use (require 'compat nil 'noerror).
>
> 2. Core packages which use Compat do not have to replicate the
> `compat-function' and `compat-call' macros in their own code. See
> erc-compat.el for a core package, where these macros are already
> duplicated with the names `erc-compat-function' and `erc-compat-call'.
> Instead of this replication, erc-compat.el can just (require 'compat)
> and use `compat-function' and `compat-call'.
>
> 3. Compat should not be installed automatically as dependency if Emacs
> itself already provides the required API. For example on Emacs 30.1 I do
> not want to install compat-30.1 from ELPA, since Emacs itself already
> provides everything compat-30.1.x from ELPA ever could provide. Emacs is
> in charge of the API definition and Compat won't add anything itself.
>
> Now to describe the pretest or release flows - the Emacs release flow
> can proceeds as usual, as it has happened before with 28.1, 28.2, 29.1
> etc. The compat.el file as part of Emacs will not have to be updated or
> changed.
>
> The ELPA package Compat will be released independently. After a release
> of pretest Emacs-30.0 we may release Compat-30.0.x on ELPA. This does
> not have to happen immediately. Only after Compat-30.0.x has landed on
> ELPA, other ELPA packages (which for example depend on Emacs 26.1 as
> base line) may start to depend on compat-30.0 and start using new Emacs
> 30.0 functions or macros, e.g., `static-if'. Before the release of
> Compat, an external package cannot take advantage of `static-if', since
> the ELPA Compat does not provide it yet, and the base line Emacs 26.1
> does not provide it.
>
> Note that Compat on ELPA is already used by many packages exactly in the
> way I've described. The inclusion of compat.el in Emacs will smoothen
> the situation for core packages (which are also released to ELPA) and
> will ensure that no unnecessary Compat package is installed, the points
> 1, 2 and 3 I listed above.
>
> Daniel

Pinging this thread with an updated version of the patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 7227 bytes --]

From 09ae06230f7ca541ba97dafc1d8b2388fc604d6a Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
* doc/lispref/package.texi (Forwards-Compatibility): Mention Compat
and link to the manual.
* etc/NEWS: Document change.  (Bug#66554)
---
 doc/lispref/package.texi  | 13 ++++++
 etc/NEWS                  | 11 ++++++
 lisp/emacs-lisp/compat.el | 83 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 4 files changed, 108 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
index 6f52a33d194..9e652ab2515 100644
--- a/doc/lispref/package.texi
+++ b/doc/lispref/package.texi
@@ -28,6 +28,7 @@ Packaging
 * Multi-file Packages::     How to package multiple files.
 * Package Archives::        Maintaining package archives.
 * Archive Web Server::      Interfacing to an archive web server.
+* Forwards-Compatibility::  Supporting older versions of Emacs.
 @end menu
 
 @node Packaging Basics
@@ -390,3 +391,15 @@ Archive Web Server
 package, or the single file for a simple package.
 
 @end table
+
+@node Forwards-Compatibility
+@section Supporting older versions of Emacs
+@cindex compatibility compat
+
+Packages that wish to support older releases of Emacs, without giving
+up on newer functionality from recent Emacs releases, one can make use
+of the Compat package on GNU ELPA.  For details on how to make use of
+the package, @xref{Usage,, Usage, compat, "Compat" Manual}. In case
+you don't have the package installed, you can also read the
+@url{https://elpa.gnu.org/packages/doc/compat.html#Usage, Online
+Compat manual}.
diff --git a/etc/NEWS b/etc/NEWS
index 735a05f6579..f599e54b168 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1309,6 +1309,17 @@ This minor mode generates the tags table automatically based on the
 current project configuration, and later updates it as you edit the
 files and save the changes.
 
++++
+** New package Compat
+The Compat package on GNU ELPA provides forwards-compatibility
+support, so that packages that still provide support for older
+versions of Emacs can still make use of newer definitions that can be
+reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
+part of Emacs, that doesn't provide any compatibility support, but
+only implements the public-facing API of Compat so that core packages
+can use Compat, while also preventing the installation of Compat on
+the most recent version of Emacs.
+
 \f
 * Incompatible Lisp Changes in Emacs 30.1
 
diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..726b3fd298b
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,83 @@
+;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; You can find out more about Compat and how to use it from the Info
+;; node `(compat) Top' (installed along with the Compat package) or
+;; read the same manual online:
+;; https.gnu.org/packages/doc/compat.html.
+
+;; Note that Compat is NOT a core package and this file is NOT
+;; available on GNU ELPA.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+;;;; Hack to avoid installing Compat if not necessary
+
+;; The versioning scheme of the Compat package follows that of Emacs,
+;; to indicate what version of Emacs is being supported.  For example,
+;; the Compat version number 29.2.3.9 would attempt to provide
+;; compatibility definitions up to Emacs 29.2, while also designating
+;; that this is the third major release and ninth minor release of
+;; Compat, for the specific Emacs release.
+
+;; To ensure that if the user is using Emacs X.Y installed, the ELPA
+;; package Compat X.Y.Z* (for any values of Z*) does not get
+;; unnecessarily installed, as there are no missing features that
+;; Compat could provide, we programmatically specify the version of
+;; the package to be that of the current Emacs version plus a high
+;; "major release" to exceed the major version of Compat.
+
+;;;###autoload (push (list 'compat emacs-major-version emacs-minor-version most-positive-fixnum) package--builtin-versions)
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..1f4a8e01294 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Eli Zaretskii <eliz@gnu.org>,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Thu, 18 Jan 2024 19:51:27 +0000
> 
> Pinging this thread with an updated version of the patch:

I find the documentation of this arrangement still insufficient.  The
way this stuff works (which required Daniel to write 150 lines of
explanation) is mostly kept out of the written docs, so we'll have to
rely on people's memory.  Can we document this machinery better?

> +Packages that wish to support older releases of Emacs, without giving
> +up on newer functionality from recent Emacs releases, one can make use
> +of the Compat package on GNU ELPA.  For details on how to make use of
> +the package, @xref{Usage,, Usage, compat, "Compat" Manual}. In case
                ^^^^^                                        ^^
This should be "see @ref" or "@pxref".  @xref is only suitable at the
beginning of a statement.  And please leave 2 spaces between sentences
there.

> +** New package Compat
> +The Compat package on GNU ELPA provides forwards-compatibility
> +support, so that packages that still provide support for older

I think this is known as "backward compatibility".

> +versions of Emacs can still make use of newer definitions that can be
> +reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
> +part of Emacs, that doesn't provide any compatibility support, but
> +only implements the public-facing API of Compat so that core packages
> +can use Compat, while also preventing the installation of Compat on
> +the most recent version of Emacs.

Not sure this detailed description is useful.  Why not just say that
Compat is now also available with Emacs?

bug#66554: [PATCH] Add the public API of Compat to the core

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> +** New package Compat
>> +The Compat package on GNU ELPA provides forwards-compatibility
>> +support, so that packages that still provide support for older
>
> I think this is known as "backward compatibility".

AFAIU, backwards-compatibility is about making code written for Emacs 28
work without changes on Emacs 29 (N -> N+1).

But forwards-compatibility is about making code written for Emacs 29
work without changes on Emacs 28 (N -> N-1).

Compat is doing the latter.

>> +versions of Emacs can still make use of newer definitions that can be
>> +reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
>> +part of Emacs, that doesn't provide any compatibility support, but
>> +only implements the public-facing API of Compat so that core packages
>> +can use Compat, while also preventing the installation of Compat on
>> +the most recent version of Emacs.
>
> Not sure this detailed description is useful.  Why not just say that
> Compat is now also available with Emacs?

Maybe this could be moved to the git commit message, and we could focus
in the documentation on what the consequences are for users and authors
of third-party packages (not authors of :core packages, who are
basically in the know already).

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 18 Jan 2024 12:33:31 -0800
> Cc: mail@daniel-mendler.de, 66554@debbugs.gnu.org, monnier@iro.umontreal.ca
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> +** New package Compat
> >> +The Compat package on GNU ELPA provides forwards-compatibility
> >> +support, so that packages that still provide support for older
> >
> > I think this is known as "backward compatibility".
> 
> AFAIU, backwards-compatibility is about making code written for Emacs 28
> work without changes on Emacs 29 (N -> N+1).
> 
> But forwards-compatibility is about making code written for Emacs 29
> work without changes on Emacs 28 (N -> N-1).
> 
> Compat is doing the latter.

AFAIU, Compat allows Lisp programs written for newer Emacs version to
work with older Emacs versions.  This makes those Lisp programs
compatible with old versions of Emacs, and in my book this is
backwards-compatibility.  But if I'm the only one who thinks that, so
be it.

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Kangas <stefankangas@gmail.com>
>> Date: Thu, 18 Jan 2024 12:33:31 -0800
>> Cc: mail@daniel-mendler.de, 66554@debbugs.gnu.org, monnier@iro.umontreal.ca
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> +** New package Compat
>> >> +The Compat package on GNU ELPA provides forwards-compatibility
>> >> +support, so that packages that still provide support for older
>> >
>> > I think this is known as "backward compatibility".
>> 
>> AFAIU, backwards-compatibility is about making code written for Emacs 28
>> work without changes on Emacs 29 (N -> N+1).
>> 
>> But forwards-compatibility is about making code written for Emacs 29
>> work without changes on Emacs 28 (N -> N-1).
>> 
>> Compat is doing the latter.
>
> AFAIU, Compat allows Lisp programs written for newer Emacs version to
> work with older Emacs versions.  This makes those Lisp programs
> compatible with old versions of Emacs, and in my book this is
> backwards-compatibility.  But if I'm the only one who thinks that, so
> be it.

That's right. To formulate this differently, Compat allows Lisp programs
written for old Emacs versions to use newer Emacs features. This makes
Compat a forwards-compatibility library for the newer Emacs features.

The difference is what you consider the base line Emacs version. A
package could for example specify `Package-Requires: ((emacs "27.1")
(compat "29.1"))' in its package header. The header means that the
package was written for an old Emacs (27.1) but takes advantage of newer
features (29.1).

However one may also say that the package is actually for 29.1 and
was back-ported via Compat to 27.1. This could indeed have happened if
one looks at the history of the package.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  66554@debbugs.gnu.org,
>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>> Date: Thu, 18 Jan 2024 19:51:27 +0000
>> 
>> Pinging this thread with an updated version of the patch:
>
> I find the documentation of this arrangement still insufficient.  The
> way this stuff works (which required Daniel to write 150 lines of
> explanation) is mostly kept out of the written docs, so we'll have to
> rely on people's memory.  Can we document this machinery better?

I agree. Where do you suggest to add the documentation? My intention is
to update the Compat manual (of the Compat ELPA package) with a more
detailed explanation of the mechanism as soon as the compat.el file gets
added to the Emacs core. We may want to avoid to duplicate the
information, by keeping the documentation in the Emacs compat.el file
concise, referring mainly to the Compat manual.

As of now, the Compat manual already mentions the arrangement with the
compat.el file in core as a possible future development, but does not
expand on it in detail. This part of the manual will be updated and
greatly expanded.

>> +** New package Compat
>> +The Compat package on GNU ELPA provides forwards-compatibility
>> +support, so that packages that still provide support for older
>
> I think this is known as "backward compatibility".

Compat allows packages to use APIs which are newer than the Emacs they
depend on. A package depending on ((emacs "27.1") (compat "29.1"))
relies on Emacs 27.1 as base line, but can use some APIs from 29.1,
specifically the ones provided by Compat. Is there a better and more
specific term for this than "forward compatibility"?

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Daniel Mendler <mail@daniel-mendler.de>
> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Thu, 18 Jan 2024 21:35:59 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I find the documentation of this arrangement still insufficient.  The
> > way this stuff works (which required Daniel to write 150 lines of
> > explanation) is mostly kept out of the written docs, so we'll have to
> > rely on people's memory.  Can we document this machinery better?
> 
> I agree. Where do you suggest to add the documentation? My intention is
> to update the Compat manual (of the Compat ELPA package) with a more
> detailed explanation of the mechanism as soon as the compat.el file gets
> added to the Emacs core. We may want to avoid to duplicate the
> information, by keeping the documentation in the Emacs compat.el file
> concise, referring mainly to the Compat manual.

The Compat manual cannot be the only place, because the information
I'm talking about should be aimed at the Emacs developers, so it must
be part of Emacs.  I suggest to have it in the commentary in
compat.el.

> >> +** New package Compat
> >> +The Compat package on GNU ELPA provides forwards-compatibility
> >> +support, so that packages that still provide support for older
> >
> > I think this is known as "backward compatibility".
> 
> Compat allows packages to use APIs which are newer than the Emacs they
> depend on. A package depending on ((emacs "27.1") (compat "29.1"))
> relies on Emacs 27.1 as base line, but can use some APIs from 29.1,
> specifically the ones provided by Compat. Is there a better and more
> specific term for this than "forward compatibility"?

How about

  Compat allows packages written for newer Emacs version to still work
  with older Emacs versions.

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Mendler <mail@daniel-mendler.de>
>> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>> Date: Thu, 18 Jan 2024 21:35:59 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I find the documentation of this arrangement still insufficient.  The
>> > way this stuff works (which required Daniel to write 150 lines of
>> > explanation) is mostly kept out of the written docs, so we'll have to
>> > rely on people's memory.  Can we document this machinery better?
>> 
>> I agree. Where do you suggest to add the documentation? My intention is
>> to update the Compat manual (of the Compat ELPA package) with a more
>> detailed explanation of the mechanism as soon as the compat.el file gets
>> added to the Emacs core. We may want to avoid to duplicate the
>> information, by keeping the documentation in the Emacs compat.el file
>> concise, referring mainly to the Compat manual.
>
> The Compat manual cannot be the only place, because the information
> I'm talking about should be aimed at the Emacs developers, so it must
> be part of Emacs.  I suggest to have it in the commentary in
> compat.el.

Okay, let's add the information to the commentary. Then we also copy the
information to the Compat manual, to a section where we describe how the
versioning works.

>> >> +** New package Compat
>> >> +The Compat package on GNU ELPA provides forwards-compatibility
>> >> +support, so that packages that still provide support for older
>> >
>> > I think this is known as "backward compatibility".
>> 
>> Compat allows packages to use APIs which are newer than the Emacs they
>> depend on. A package depending on ((emacs "27.1") (compat "29.1"))
>> relies on Emacs 27.1 as base line, but can use some APIs from 29.1,
>> specifically the ones provided by Compat. Is there a better and more
>> specific term for this than "forward compatibility"?
>
> How about
>
>   Compat allows packages written for newer Emacs version to still work
>   with older Emacs versions.

I think this formulation gets it backwards. See my other mail regarding
backward- vs forwards-compatibility. One could also write:

   Compat allows packages written for older Emacs versions to use
   features introduced in newer Emacs versions.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Mendler <mail@daniel-mendler.de>
>> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>> Date: Thu, 18 Jan 2024 21:35:59 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I find the documentation of this arrangement still insufficient.  The
>> > way this stuff works (which required Daniel to write 150 lines of
>> > explanation) is mostly kept out of the written docs, so we'll have to
>> > rely on people's memory.  Can we document this machinery better?
>> 
>> I agree. Where do you suggest to add the documentation? My intention is
>> to update the Compat manual (of the Compat ELPA package) with a more
>> detailed explanation of the mechanism as soon as the compat.el file gets
>> added to the Emacs core. We may want to avoid to duplicate the
>> information, by keeping the documentation in the Emacs compat.el file
>> concise, referring mainly to the Compat manual.
>
> The Compat manual cannot be the only place, because the information
> I'm talking about should be aimed at the Emacs developers, so it must
> be part of Emacs.  I suggest to have it in the commentary in
> compat.el.

OK.

>> >> +** New package Compat
>> >> +The Compat package on GNU ELPA provides forwards-compatibility
>> >> +support, so that packages that still provide support for older
>> >
>> > I think this is known as "backward compatibility".
>> 
>> Compat allows packages to use APIs which are newer than the Emacs they
>> depend on. A package depending on ((emacs "27.1") (compat "29.1"))
>> relies on Emacs 27.1 as base line, but can use some APIs from 29.1,
>> specifically the ones provided by Compat. Is there a better and more
>> specific term for this than "forward compatibility"?
>
> How about
>
>   Compat allows packages written for newer Emacs version to still work
>   with older Emacs versions.

I think the danger here is that Compat cannot provide full compatibility
support (some features either too complex to maintain, too slow to
reproduce or requiring changes in the core), so the way you phrase it
sounds like more than is being delivered.

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Daniel Mendler <mail@daniel-mendler.de>,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Fri, 19 Jan 2024 16:44:06 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Compat allows packages to use APIs which are newer than the Emacs they
> >> depend on. A package depending on ((emacs "27.1") (compat "29.1"))
> >> relies on Emacs 27.1 as base line, but can use some APIs from 29.1,
> >> specifically the ones provided by Compat. Is there a better and more
> >> specific term for this than "forward compatibility"?
> >
> > How about
> >
> >   Compat allows packages written for newer Emacs version to still work
> >   with older Emacs versions.
> 
> I think the danger here is that Compat cannot provide full compatibility
> support (some features either too complex to maintain, too slow to
> reproduce or requiring changes in the core), so the way you phrase it
> sounds like more than is being delivered.

I fail to see how what I wrote is different from the text quoted above
that.  If my text promises more than is being delivered, so is the
original text.

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Philip Kaludercic <philipk@posteo.net> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Daniel Mendler <mail@daniel-mendler.de>
>>> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>>> Date: Thu, 18 Jan 2024 21:35:59 +0100
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> > I find the documentation of this arrangement still insufficient.  The
>>> > way this stuff works (which required Daniel to write 150 lines of
>>> > explanation) is mostly kept out of the written docs, so we'll have to
>>> > rely on people's memory.  Can we document this machinery better?
>>> 
>>> I agree. Where do you suggest to add the documentation? My intention is
>>> to update the Compat manual (of the Compat ELPA package) with a more
>>> detailed explanation of the mechanism as soon as the compat.el file gets
>>> added to the Emacs core. We may want to avoid to duplicate the
>>> information, by keeping the documentation in the Emacs compat.el file
>>> concise, referring mainly to the Compat manual.
>>
>> The Compat manual cannot be the only place, because the information
>> I'm talking about should be aimed at the Emacs developers, so it must
>> be part of Emacs.  I suggest to have it in the commentary in
>> compat.el.
>
> OK.

Philip, do you plan to submit a new version of the patch or do you want
me to update the patch with a more extensive explanation? We should keep
in mind that the information we add to the Emacs compat.el cannot be
self sufficient. Emacs developers who want to use Compat must consult
the Compat manual, since that's the place where we document the
available compatibility definitions. Therefore referring to the manual
for further details should be okay, as long as the general mechanism
(and the versioning) is explained sufficiently well in the commentary of
the compat.el file in Emacs.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 2149 bytes --]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Daniel Mendler <mail@daniel-mendler.de>
>>>> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>>>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>>>> Date: Thu, 18 Jan 2024 21:35:59 +0100
>>>> 
>>>> Eli Zaretskii <eliz@gnu.org> writes:
>>>> 
>>>> > I find the documentation of this arrangement still insufficient.  The
>>>> > way this stuff works (which required Daniel to write 150 lines of
>>>> > explanation) is mostly kept out of the written docs, so we'll have to
>>>> > rely on people's memory.  Can we document this machinery better?
>>>> 
>>>> I agree. Where do you suggest to add the documentation? My intention is
>>>> to update the Compat manual (of the Compat ELPA package) with a more
>>>> detailed explanation of the mechanism as soon as the compat.el file gets
>>>> added to the Emacs core. We may want to avoid to duplicate the
>>>> information, by keeping the documentation in the Emacs compat.el file
>>>> concise, referring mainly to the Compat manual.
>>>
>>> The Compat manual cannot be the only place, because the information
>>> I'm talking about should be aimed at the Emacs developers, so it must
>>> be part of Emacs.  I suggest to have it in the commentary in
>>> compat.el.
>>
>> OK.
>
> Philip, do you plan to submit a new version of the patch or do you want
> me to update the patch with a more extensive explanation? We should keep
> in mind that the information we add to the Emacs compat.el cannot be
> self sufficient. Emacs developers who want to use Compat must consult
> the Compat manual, since that's the place where we document the
> available compatibility definitions. Therefore referring to the manual
> for further details should be okay, as long as the general mechanism
> (and the versioning) is explained sufficiently well in the commentary of
> the compat.el file in Emacs.

I have tried to update the patch to clarify some of the points in the
discussion, but feel free to change anything you think ought to be changed:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 9054 bytes --]

From 83d84c625800215a6582320e3ed139efc43761d0 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
* doc/lispref/package.texi (Forwards-Compatibility): Mention Compat
and link to the manual.
* etc/NEWS: Document change.  (Bug#66554)
---
 doc/lispref/package.texi  | 44 ++++++++++++++++++
 etc/NEWS                  | 11 +++++
 lisp/emacs-lisp/compat.el | 94 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 4 files changed, 150 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
index 6f52a33d194..b9239521d33 100644
--- a/doc/lispref/package.texi
+++ b/doc/lispref/package.texi
@@ -28,6 +28,7 @@ Packaging
 * Multi-file Packages::     How to package multiple files.
 * Package Archives::        Maintaining package archives.
 * Archive Web Server::      Interfacing to an archive web server.
+* Forwards-Compatibility::  Supporting older versions of Emacs.
 @end menu
 
 @node Packaging Basics
@@ -390,3 +391,46 @@ Archive Web Server
 package, or the single file for a simple package.
 
 @end table
+
+@node Forwards-Compatibility
+@section Supporting older versions of Emacs
+@cindex compatibility compat
+
+Packages that wish to support older releases of Emacs, without giving
+up on newer functionality from recent Emacs releases, one can make use
+of the Compat package on GNU ELPA.  By depending on the package, Emacs
+can provide compatibility definitions for missing functionality.
+
+The versioning of Compat follows that of Emacs, so one can implicitly
+declare what range of Emacs versions a package supports like so:
+
+@example
+;; Package-Requires: ((emacs "27.2") (compat "29.1"))
+@end example
+
+By default, one can refer to compatibility definitions by their given
+names from future versions.  For example, one can use the function
+@code{take} on before Emacs 29 as such.  Due to changes of function
+and macro calling conventions over time, this is not always possible
+and it is occasionally necessary to explicitly refer to compatibility
+code.  To this end one can use the Compat API:
+
+@defmac compat-call fun &rest args
+This macro calls the compatibility function @var{fun} with @var{args}.
+Many functions provided by Compat can be called directly without this
+macro.  However in the case where Compat provides an alternative
+version of an existing function, the function call has to go through
+@code{compat-call}.
+@end defmac
+
+@defmac compat-function fun
+This macro returns the compatibility function symbol for @var{fun}.
+See @code{compat-call} for a more convenient macro to directly call
+compatibility functions.
+@end defmac
+
+For further details on how to make use of the package, see
+@ref{Usage,, Usage, compat, "Compat" Manual}.  In case you don't have
+the package installed, you can also read the
+@url{https://elpa.gnu.org/packages/doc/compat.html#Usage, Online
+Compat manual}.
diff --git a/etc/NEWS b/etc/NEWS
index a1874313502..46859d75aac 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1321,6 +1321,17 @@ This minor mode generates the tags table automatically based on the
 current project configuration, and later updates it as you edit the
 files and save the changes.
 
++++
+** New package Compat
+The Compat package on GNU ELPA provides forwards-compatibility
+support, so that packages that still provide support for older
+versions of Emacs can still make use of newer definitions that can be
+reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
+part of Emacs, that doesn't provide any compatibility support, but
+only implements the public-facing API of Compat so that core packages
+can use Compat, while also preventing the installation of Compat on
+the most recent version of Emacs.
+
 \f
 * Incompatible Lisp Changes in Emacs 30.1
 
diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..2882974cf2d
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,94 @@
+;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; A basic introduction to Compat is given in the Info node `(elisp)
+;; Forwards Compatibility'.  Further details on Compat are documented
+;; in the Info node `(compat) Top' (installed along with the Compat
+;; package) or read the same manual online:
+;; https://elpa.gnu.org/packages/doc/compat.html.
+
+;; Note that Compat is NOT a core package and this file is NOT
+;; available on GNU ELPA.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+;;;; Clever trick to avoid installing Compat if not necessary
+
+;; The versioning scheme of the Compat package follows that of Emacs,
+;; to indicate what version of Emacs is being supported.  For example,
+;; the Compat version number 29.2.3.9 would attempt to provide
+;; compatibility definitions up to Emacs 29.2, while also designating
+;; that this is the third major release and ninth minor release of
+;; Compat, for the specific Emacs release.
+
+;; The package version of this file is specified programmatically,
+;; instead of giving a fixed version in the header of this file.  This
+;; is done to ensure that the version of compat.el provided by Emacs
+;; is always corresponds to the current version of Emacs.  In addition
+;; to the major-minor version, a large "major release" makes sure that
+;; the built-in version of Compat is always preferred over an external
+;; installation.  This means that if a package specifies a dependency
+;; on Compat which matches the current version of Emacs that is being
+;; used, no additional dependencies have to be downloaded.
+;;
+;; Further details and background on this file can be found in the
+;; bug#66554 discussion.
+
+;;;###autoload (push (list 'compat
+;;;###autoload             emacs-major-version
+;;;###autoload             emacs-minor-version
+;;;###autoload             1.0e+INF)
+;;;###autoload       package--builtin-versions)
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..1f4a8e01294 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Philip Kaludercic <philipk@posteo.net> writes:

[...]

>> Philip, do you plan to submit a new version of the patch or do you want
>> me to update the patch with a more extensive explanation? We should keep
>> in mind that the information we add to the Emacs compat.el cannot be
>> self sufficient. Emacs developers who want to use Compat must consult
>> the Compat manual, since that's the place where we document the
>> available compatibility definitions. Therefore referring to the manual
>> for further details should be okay, as long as the general mechanism
>> (and the versioning) is explained sufficiently well in the commentary of
>> the compat.el file in Emacs.
>
> I have tried to update the patch to clarify some of the points in the
> discussion, but feel free to change anything you think ought to be changed:

Thank you, Philip. I added a few comments below, mostly about some
details of the wording. Eli, Stefans, do you think the level of
information provided in the patch is sufficient?

> diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
> index 6f52a33d194..b9239521d33 100644
> --- a/doc/lispref/package.texi
> +++ b/doc/lispref/package.texi

[...]

> +The versioning of Compat follows that of Emacs, so one can implicitly
> +declare what range of Emacs versions a package supports like so:
> +
> +@example
> +;; Package-Requires: ((emacs "27.2") (compat "29.1"))
> +@end example

The word "range" is misleading. It sounds as if the package supports
27.2 to 29.1, while in it actually supports 27.2 and newer and relies on
some 29.1 APIs.

[...]

> diff --git a/etc/NEWS b/etc/NEWS
> index a1874313502..46859d75aac 100644
> --- a/etc/NEWS
> +++ b/etc/NEWS
> @@ -1321,6 +1321,17 @@ This minor mode generates the tags table automatically based on the
>  current project configuration, and later updates it as you edit the
>  files and save the changes.
>  
> ++++
> +** New package Compat
> +The Compat package on GNU ELPA provides forwards-compatibility
> +support, so that packages that still provide support for older
> +versions of Emacs can still make use of newer definitions that can be
> +reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
> +part of Emacs, that doesn't provide any compatibility support, but
> +only implements the public-facing API of Compat so that core packages
> +can use Compat, while also preventing the installation of Compat on
> +the most recent version of Emacs.

This NEWS entry explains the addition well, but it is a bit verbose
compared to other entries. I am not fond of the quoted word "pseudo".
Maybe say stub of Compat?

>  * Incompatible Lisp Changes in Emacs 30.1
>  
> diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
> new file mode 100644
> index 00000000000..2882974cf2d
> --- /dev/null
> +++ b/lisp/emacs-lisp/compat.el
> @@ -0,0 +1,94 @@
> +;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-

Instead of "Pseudo" maybe write "Stub of the Emacs Lisp Compatibility
Library"? We could also use "Emacs Lisp Compatibility Library" like in
ELPA Compat package, since for packages using the Compat library it
should not make a difference if the builtin compat.el stub or the ELPA
package is used.

> +;;; Commentary:

[...]

> +;; Note that Compat is NOT a core package and this file is NOT
> +;; available on GNU ELPA.

I find this sentence a bit confusing. What you want to tell here is that
the compat.el file in Emacs differs from the compat.el file in the ELPA
package. This is maybe already clear from the other comments so we can
as well remove this sentence?

[...]

> +;;;; Clever trick to avoid installing Compat if not necessary
> +
> +;; The versioning scheme of the Compat package follows that of Emacs,
> +;; to indicate what version of Emacs is being supported.  For example,
                                                 ^^^^^^^^^

Should we say "supported" here? Compat supports other version than the
one specified. Compat "provides" functionality from that Emacs version.

> +;; the Compat version number 29.2.3.9 would attempt to provide
> +;; compatibility definitions up to Emacs 29.2, while also designating
> +;; that this is the third major release and ninth minor release of
> +;; Compat, for the specific Emacs release.
> +
> +;; The package version of this file is specified programmatically,
> +;; instead of giving a fixed version in the header of this file.  This
> +;; is done to ensure that the version of compat.el provided by Emacs
> +;; is always corresponds to the current version of Emacs.  In addition
      ^^

The "is" should be removed.

> +;; to the major-minor version, a large "major release" makes sure that
> +;; the built-in version of Compat is always preferred over an external
> +;; installation.  This means that if a package specifies a dependency
> +;; on Compat which matches the current version of Emacs that is being
                              ^^^^^^^^^^^^^^^^^^^
                              
...which matches the current *or an older version* of Emacs...

> +;; used, no additional dependencies have to be downloaded.
> +;;
> +;; Further details and background on this file can be found in the
> +;; bug#66554 discussion.
> +
> +;;;###autoload (push (list 'compat
> +;;;###autoload             emacs-major-version
> +;;;###autoload             emacs-minor-version
> +;;;###autoload             1.0e+INF)
> +;;;###autoload       package--builtin-versions)

I prefer if we use 9999 here instead of 1.0e+INF. While infinity is
semantically correct, the float may lead to problems and hurt
readability in the package list.

> +(provide 'compat)
> +;;; compat.el ends here

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Daniel Mendler <mail@daniel-mendler.de>
> Cc: 66554@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>,
>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
> Date: Fri, 26 Jan 2024 11:42:30 +0100
> 
> > I have tried to update the patch to clarify some of the points in the
> > discussion, but feel free to change anything you think ought to be changed:
> 
> Thank you, Philip. I added a few comments below, mostly about some
> details of the wording. Eli, Stefans, do you think the level of
> information provided in the patch is sufficient?

It is hard to answer the question without seeing a complete patch that
incorporates your suggestions to the degree that Philip agrees with
them.

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Mendler <mail@daniel-mendler.de>
>> Cc: 66554@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>,
>>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
>> Date: Fri, 26 Jan 2024 11:42:30 +0100
>> 
>> > I have tried to update the patch to clarify some of the points in the
>> > discussion, but feel free to change anything you think ought to be changed:
>> 
>> Thank you, Philip. I added a few comments below, mostly about some
>> details of the wording. Eli, Stefans, do you think the level of
>> information provided in the patch is sufficient?
>
> It is hard to answer the question without seeing a complete patch that
> incorporates your suggestions to the degree that Philip agrees with
> them.

Okay. Philip, do you have any comments regarding my suggestions?

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 6365 bytes --]

(Sorry for the late response.)

Daniel Mendler <mail@daniel-mendler.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
> [...]
>
>>> Philip, do you plan to submit a new version of the patch or do you
>>> want
>>> me to update the patch with a more extensive explanation? We should
>>> keep
>>> in mind that the information we add to the Emacs compat.el cannot be
>>> self sufficient. Emacs developers who want to use Compat must consult
>>> the Compat manual, since that's the place where we document the
>>> available compatibility definitions. Therefore referring to the manual
>>> for further details should be okay, as long as the general mechanism
>>> (and the versioning) is explained sufficiently well in the
>>> commentary of
>>> the compat.el file in Emacs.
>>
>> I have tried to update the patch to clarify some of the points in the
>> discussion, but feel free to change anything you think ought to be
>> changed:
>
> Thank you, Philip. I added a few comments below, mostly about some
> details of the wording. Eli, Stefans, do you think the level of
> information provided in the patch is sufficient?
>
>> diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
>> index 6f52a33d194..b9239521d33 100644
>> --- a/doc/lispref/package.texi
>> +++ b/doc/lispref/package.texi
>
> [...]
>
>> +The versioning of Compat follows that of Emacs, so one can implicitly
>> +declare what range of Emacs versions a package supports like so:
>> +
>> +@example
>> +;; Package-Requires: ((emacs "27.2") (compat "29.1"))
>> +@end example
>
> The word "range" is misleading. It sounds as if the package supports
> 27.2 to 29.1, while in it actually supports 27.2 and newer and relies on
> some 29.1 APIs.

Changed this.

> [...]
>
>> diff --git a/etc/NEWS b/etc/NEWS
>> index a1874313502..46859d75aac 100644
>> --- a/etc/NEWS
>> +++ b/etc/NEWS
>> @@ -1321,6 +1321,17 @@ This minor mode generates the tags table
>> automatically based on the
>>  current project configuration, and later updates it as you edit the
>>  files and save the changes.
>>  
>> ++++
>> +** New package Compat
>> +The Compat package on GNU ELPA provides forwards-compatibility
>> +support, so that packages that still provide support for older
>> +versions of Emacs can still make use of newer definitions that can be
>> +reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
>> +part of Emacs, that doesn't provide any compatibility support, but
>> +only implements the public-facing API of Compat so that core packages
>> +can use Compat, while also preventing the installation of Compat on
>> +the most recent version of Emacs.
>
> This NEWS entry explains the addition well, but it is a bit verbose
> compared to other entries. I am not fond of the quoted word "pseudo".
> Maybe say stub of Compat?

Abbreviated it.

>>  * Incompatible Lisp Changes in Emacs 30.1
>>  
>> diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
>> new file mode 100644
>> index 00000000000..2882974cf2d
>> --- /dev/null
>> +++ b/lisp/emacs-lisp/compat.el
>> @@ -0,0 +1,94 @@
>> +;;; compat.el --- Pseudo-Compatibility for Elisp -*-
>> lexical-binding: t; -*-
>
> Instead of "Pseudo" maybe write "Stub of the Emacs Lisp Compatibility
> Library"? We could also use "Emacs Lisp Compatibility Library" like in
> ELPA Compat package, since for packages using the Compat library it
> should not make a difference if the builtin compat.el stub or the ELPA
> package is used.

I changed it to "Stub of the Compatibility Library", to avoid an
overlong line and also because Elisp should be implicit.

>> +;;; Commentary:
>
> [...]
>
>> +;; Note that Compat is NOT a core package and this file is NOT
>> +;; available on GNU ELPA.
>
> I find this sentence a bit confusing. What you want to tell here is that
> the compat.el file in Emacs differs from the compat.el file in the ELPA
> package. This is maybe already clear from the other comments so we can
> as well remove this sentence?

Done.

> [...]
>
>> +;;;; Clever trick to avoid installing Compat if not necessary
>> +
>> +;; The versioning scheme of the Compat package follows that of Emacs,
>> +;; to indicate what version of Emacs is being supported.  For example,
>                                                  ^^^^^^^^^
>
> Should we say "supported" here? Compat supports other version than the
> one specified. Compat "provides" functionality from that Emacs version.

Rephrased this, but sounds a bit clunky.

>> +;; the Compat version number 29.2.3.9 would attempt to provide
>> +;; compatibility definitions up to Emacs 29.2, while also designating
>> +;; that this is the third major release and ninth minor release of
>> +;; Compat, for the specific Emacs release.
>> +
>> +;; The package version of this file is specified programmatically,
>> +;; instead of giving a fixed version in the header of this file.  This
>> +;; is done to ensure that the version of compat.el provided by Emacs
>> +;; is always corresponds to the current version of Emacs.  In addition
>       ^^
>
> The "is" should be removed.

Right.

>> +;; to the major-minor version, a large "major release" makes sure that
>> +;; the built-in version of Compat is always preferred over an external
>> +;; installation.  This means that if a package specifies a dependency
>> +;; on Compat which matches the current version of Emacs that is being
>                               ^^^^^^^^^^^^^^^^^^^
>                               
> ...which matches the current *or an older version* of Emacs...

Ok.

>> +;; used, no additional dependencies have to be downloaded.
>> +;;
>> +;; Further details and background on this file can be found in the
>> +;; bug#66554 discussion.
>> +
>> +;;;###autoload (push (list 'compat
>> +;;;###autoload             emacs-major-version
>> +;;;###autoload             emacs-minor-version
>> +;;;###autoload             1.0e+INF)
>> +;;;###autoload       package--builtin-versions)
>
> I prefer if we use 9999 here instead of 1.0e+INF. While infinity is
> semantically correct, the float may lead to problems and hurt
> readability in the package list.

It seems the main issue is that 30.0.1.0e+INF can be displayed without
any problems, but version-to-list rejects it.  Sad, because I think this
is a neat idea but I've changed it to 9999.

>> +(provide 'compat)
>> +;;; compat.el ends here
>
> Daniel


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 8829 bytes --]

From bc1aaa2e5983d77b3ac0c3a95d73197bc9127dac Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
* doc/lispref/package.texi (Forwards-Compatibility): Mention Compat
and link to the manual.
* etc/NEWS: Document change.  (Bug#66554)
---
 doc/lispref/package.texi  | 46 ++++++++++++++++++++
 etc/NEWS                  |  7 +++
 lisp/emacs-lisp/compat.el | 92 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 4 files changed, 146 insertions(+), 1 deletion(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
index 6f52a33d194..071c87170f5 100644
--- a/doc/lispref/package.texi
+++ b/doc/lispref/package.texi
@@ -28,6 +28,7 @@ Packaging
 * Multi-file Packages::     How to package multiple files.
 * Package Archives::        Maintaining package archives.
 * Archive Web Server::      Interfacing to an archive web server.
+* Forwards-Compatibility::  Supporting older versions of Emacs.
 @end menu
 
 @node Packaging Basics
@@ -390,3 +391,48 @@ Archive Web Server
 package, or the single file for a simple package.
 
 @end table
+
+@node Forwards-Compatibility
+@section Supporting older versions of Emacs
+@cindex compatibility compat
+
+Packages that wish to support older releases of Emacs, without giving
+up on newer functionality from recent Emacs releases, one can make use
+of the Compat package on GNU ELPA.  By depending on the package, Emacs
+can provide compatibility definitions for missing functionality.
+
+The versioning of Compat follows that of Emacs, so next to the oldest
+version that a package relies on (via the @code{emacs}-package), one
+can also indicate what the newest version of Emacs is, that a package
+wishes to use definitions from:
+
+@example
+;; Package-Requires: ((emacs "27.2") (compat "29.1"))
+@end example
+
+By default, one can refer to compatibility definitions by their given
+names from future versions.  For example, one can use the function
+@code{take} on before Emacs 29 as such.  Due to changes of function
+and macro calling conventions over time, this is not always possible
+and it is occasionally necessary to explicitly refer to compatibility
+code.  To this end one can use the Compat API:
+
+@defmac compat-call fun &rest args
+This macro calls the compatibility function @var{fun} with @var{args}.
+Many functions provided by Compat can be called directly without this
+macro.  However in the case where Compat provides an alternative
+version of an existing function, the function call has to go through
+@code{compat-call}.
+@end defmac
+
+@defmac compat-function fun
+This macro returns the compatibility function symbol for @var{fun}.
+See @code{compat-call} for a more convenient macro to directly call
+compatibility functions.
+@end defmac
+
+For further details on how to make use of the package, see
+@ref{Usage,, Usage, compat, "Compat" Manual}.  In case you don't have
+the package installed, you can also read the
+@url{https://elpa.gnu.org/packages/doc/compat.html#Usage, Online
+Compat manual}.
diff --git a/etc/NEWS b/etc/NEWS
index a1874313502..7d0a202a787 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -1321,6 +1321,13 @@ This minor mode generates the tags table automatically based on the
 current project configuration, and later updates it as you edit the
 files and save the changes.
 
++++
+** New package Compat
+Emacs now comes with a stub implementation of the
+forwards-compatibility Compat package from GNU ELPA.  This allows
+built-in packages to use the library more effectively, and helps
+preventing the installation of Compat if unnecessary.
+
 \f
 * Incompatible Lisp Changes in Emacs 30.1
 
diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..f7037dc4101
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,92 @@
+;;; compat.el --- Stub of the Compatibility Library -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; A basic introduction to Compat is given in the Info node `(elisp)
+;; Forwards Compatibility'.  Further details on Compat are documented
+;; in the Info node `(compat) Top' (installed along with the Compat
+;; package) or read the same manual online:
+;; https://elpa.gnu.org/packages/doc/compat.html.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+;;;; Clever trick to avoid installing Compat if not necessary
+
+;; The versioning scheme of the Compat package follows that of Emacs,
+;; to indicate the version of Emacs, that functionality is being
+;; provided for.  For example, the Compat version number 29.2.3.9
+;; would attempt to provide compatibility definitions up to Emacs
+;; 29.2, while also designating that this is the third major release
+;; and ninth minor release of Compat, for the specific Emacs release.
+
+;; The package version of this file is specified programmatically,
+;; instead of giving a fixed version in the header of this file.  This
+;; is done to ensure that the version of compat.el provided by Emacs
+;; always corresponds to the current version of Emacs.  In addition to
+;; the major-minor version, a large "major release" makes sure that
+;; the built-in version of Compat is always preferred over an external
+;; installation.  This means that if a package specifies a dependency
+;; on Compat which matches the current or an older version of Emacs
+;; that is being used, no additional dependencies have to be
+;; downloaded.
+;;
+;; Further details and background on this file can be found in the
+;; bug#66554 discussion.
+
+;;;###autoload (push (list 'compat
+;;;###autoload             emacs-major-version
+;;;###autoload             emacs-minor-version
+;;;###autoload             9999)
+;;;###autoload       package--builtin-versions)
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..1f4a8e01294 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
-- 
2.39.2

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

[Please ignore my previous response, it was mistakenly sent prematurely.]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: 66554@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>,
>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
> Date: Fri, 02 Feb 2024 08:11:27 +0000
> 
> diff --git a/doc/lispref/package.texi b/doc/lispref/package.texi
> index 6f52a33d194..071c87170f5 100644
> --- a/doc/lispref/package.texi
> +++ b/doc/lispref/package.texi
> @@ -28,6 +28,7 @@ Packaging
>  * Multi-file Packages::     How to package multiple files.
>  * Package Archives::        Maintaining package archives.
>  * Archive Web Server::      Interfacing to an archive web server.
> +* Forwards-Compatibility::  Supporting older versions of Emacs.
     ^^^^^^^^^^^^^^^^^^^^^^
This should be "Forward Compatibility", without "s" and without the
dash.

> +@node Forwards-Compatibility

Likewise.

> +@section Supporting older versions of Emacs
> +@cindex compatibility compat

I suggest the following index entries:

  @cindex forward compatibility, in packages
  @cindex supporting older Emacs releases, in packages
  @cindex Compat package

> +Packages that wish to support older releases of Emacs, without giving
> +up on newer functionality from recent Emacs releases, one can make use
> +of the Compat package on GNU ELPA.                    ^^^

That "one" should be removed.  And the comma before it as well.

>                                      By depending on the package, Emacs
> +can provide compatibility definitions for missing functionality.

I think you mean "By depending on the Compat package, ..."

> +
> +The versioning of Compat follows that of Emacs, so next to the oldest
> +version that a package relies on (via the @code{emacs}-package), one
> +can also indicate what the newest version of Emacs is, that a package
> +wishes to use definitions from:
> +
> +@example
> +;; Package-Requires: ((emacs "27.2") (compat "29.1"))
> +@end example

That sentence is hard to grasp.  I suggest to rephrase it:

  The version of Compat included with Emacs is the same as the Emacs
  release in which it was included.  Thus, packages that want to use
  functionality from Emacs versions newer than the oldest one they
  support should @emph{also} depend on Compat from the newest Emacs
  release.  For example:

  @example
  ;; Package-Requires: ((emacs "27.2") (compat "29.1"))
  @end example

  @noindent
  The above says that the package needs at least Emacs 27.2, but uses
  functionality introduced with Emacs 29.1, which in Emacs versions
  before 29.2 will come from the Compat package.

> +By default, one can refer to compatibility definitions by their given
> +names from future versions.  For example, one can use the function
> +@code{take} on before Emacs 29 as such.

I suggest to rephrase the last sentence:

  For example, a package can call @code{take}, which was introduced in
  Emacs 29, also in older Emacs versions, and Compat will supply the
  missing function definition.

>                                            Due to changes of function
> +and macro calling conventions over time, this is not always possible
> +and it is occasionally necessary to explicitly refer to compatibility
> +code.  To this end one can use the Compat API:
> +
> +@defmac compat-call fun &rest args
> +This macro calls the compatibility function @var{fun} with @var{args}.
> +Many functions provided by Compat can be called directly without this
> +macro.  However in the case where Compat provides an alternative
> +version of an existing function, the function call has to go through
> +@code{compat-call}.
> +@end defmac

This description left me without understanding when I need to use
compat-call and when I can just call FUN.  Can you explain more?

> +;; The versioning scheme of the Compat package follows that of Emacs,
> +;; to indicate the version of Emacs, that functionality is being
                                        ^^^^
"whose", not "that"

> +;; provided for.  For example, the Compat version number 29.2.3.9
               ^^^
Please remove that "for".

Thanks.

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

(Sorry for the delay)

Eli Zaretskii <eliz@gnu.org> writes:

As usual, your way of putting it was better, so I applied the changes
and have also taken care of the typos.

>>                                            Due to changes of function
>> +and macro calling conventions over time, this is not always possible
>> +and it is occasionally necessary to explicitly refer to compatibility
>> +code.  To this end one can use the Compat API:
>> +
>> +@defmac compat-call fun &rest args
>> +This macro calls the compatibility function @var{fun} with @var{args}.
>> +Many functions provided by Compat can be called directly without this
>> +macro.  However in the case where Compat provides an alternative
>> +version of an existing function, the function call has to go through
>> +@code{compat-call}.
>> +@end defmac
>
> This description left me without understanding when I need to use
> compat-call and when I can just call FUN.  Can you explain more?

The intention was for this paragraph to catch that case,

  However in the case where Compat provides an alternative version of an
  existing function, the function call has to go through
  @code{compat-call}.

though the real information is to be found in the Compat manual, where
the functions that have to be called via compat-call are documented.

Should the above sentence be rephrased to give a general feeling for
when this is the case

  However in the case where Compat provides an alternative version of an
  existing function, the function call has to go through
  @code{compat-call}.  This is the case when, for example the signature
  changes between versions, preventing older versions of Emacs from
  using optional arguments introduced in newer releases.

or should we just refer to the external manual?

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Tue, 06 Feb 2024 19:10:32 +0000
> 
> >> +@defmac compat-call fun &rest args
> >> +This macro calls the compatibility function @var{fun} with @var{args}.
> >> +Many functions provided by Compat can be called directly without this
> >> +macro.  However in the case where Compat provides an alternative
> >> +version of an existing function, the function call has to go through
> >> +@code{compat-call}.
> >> +@end defmac
> >
> > This description left me without understanding when I need to use
> > compat-call and when I can just call FUN.  Can you explain more?
> 
> The intention was for this paragraph to catch that case,
> 
>   However in the case where Compat provides an alternative version of an
>   existing function, the function call has to go through
>   @code{compat-call}.
> 
> though the real information is to be found in the Compat manual, where
> the functions that have to be called via compat-call are documented.
> 
> Should the above sentence be rephrased to give a general feeling for
> when this is the case
> 
>   However in the case where Compat provides an alternative version of an
>   existing function, the function call has to go through
>   @code{compat-call}.  This is the case when, for example the signature
>   changes between versions, preventing older versions of Emacs from
>   using optional arguments introduced in newer releases.
> 
> or should we just refer to the external manual?

Let me try to make my point more clear: I'd prefer that the reader
emerges from reading this description with a practical way of knowing
when to call the function directly and when to call it via
'compat-call'.  If that's not easy to understand, perhaps we should
tell that 'compat-call' should always be used, to avoid some rare
corner cases where a direct call will not do, and be done?

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>   monnier@iro.umontreal.ca
>> Date: Tue, 06 Feb 2024 19:10:32 +0000
>> 
>> >> +@defmac compat-call fun &rest args
>> >> +This macro calls the compatibility function @var{fun} with @var{args}.
>> >> +Many functions provided by Compat can be called directly without this
>> >> +macro.  However in the case where Compat provides an alternative
>> >> +version of an existing function, the function call has to go through
>> >> +@code{compat-call}.
>> >> +@end defmac
>> >
>> > This description left me without understanding when I need to use
>> > compat-call and when I can just call FUN.  Can you explain more?
>> 
>> The intention was for this paragraph to catch that case,
>> 
>>   However in the case where Compat provides an alternative version of an
>>   existing function, the function call has to go through
>>   @code{compat-call}.
>> 
>> though the real information is to be found in the Compat manual, where
>> the functions that have to be called via compat-call are documented.
>> 
>> Should the above sentence be rephrased to give a general feeling for
>> when this is the case
>> 
>>   However in the case where Compat provides an alternative version of an
>>   existing function, the function call has to go through
>>   @code{compat-call}.  This is the case when, for example the signature
>>   changes between versions, preventing older versions of Emacs from
>>   using optional arguments introduced in newer releases.
>> 
>> or should we just refer to the external manual?
>
> Let me try to make my point more clear: I'd prefer that the reader
> emerges from reading this description with a practical way of knowing
> when to call the function directly and when to call it via
> 'compat-call'.  If that's not easy to understand, perhaps we should
> tell that 'compat-call' should always be used, to avoid some rare
> corner cases where a direct call will not do, and be done?

I don't think we should recommend always using `compat-call', that would
make code unreadable.  What definitions have to be called via
`compat-call' really depends on how they were defined in Compat.  I
don't know of a better rule to describe it without copying the
documentation from the Compat manual (that wouldn't be a good idea
either, because it wouldn't stay up to date).

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Philip Kaludercic <philipk@posteo.net>
>>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>>   monnier@iro.umontreal.ca
>>> Date: Tue, 06 Feb 2024 19:10:32 +0000
>>> 
>>> >> +@defmac compat-call fun &rest args
>>> >> +This macro calls the compatibility function @var{fun} with @var{args}.
>>> >> +Many functions provided by Compat can be called directly without this
>>> >> +macro.  However in the case where Compat provides an alternative
>>> >> +version of an existing function, the function call has to go through
>>> >> +@code{compat-call}.
>>> >> +@end defmac
>>> >
>>> > This description left me without understanding when I need to use
>>> > compat-call and when I can just call FUN.  Can you explain more?
>>> 
>>> The intention was for this paragraph to catch that case,
>>> 
>>>   However in the case where Compat provides an alternative version of an
>>>   existing function, the function call has to go through
>>>   @code{compat-call}.
>>> 
>>> though the real information is to be found in the Compat manual, where
>>> the functions that have to be called via compat-call are documented.
>>> 
>>> Should the above sentence be rephrased to give a general feeling for
>>> when this is the case
>>> 
>>>   However in the case where Compat provides an alternative version of an
>>>   existing function, the function call has to go through
>>>   @code{compat-call}.  This is the case when, for example the signature
>>>   changes between versions, preventing older versions of Emacs from
>>>   using optional arguments introduced in newer releases.
>>> 
>>> or should we just refer to the external manual?
>>
>> Let me try to make my point more clear: I'd prefer that the reader
>> emerges from reading this description with a practical way of knowing
>> when to call the function directly and when to call it via
>> 'compat-call'.  If that's not easy to understand, perhaps we should
>> tell that 'compat-call' should always be used, to avoid some rare
>> corner cases where a direct call will not do, and be done?
>
> I don't think we should recommend always using `compat-call', that would
> make code unreadable.  What definitions have to be called via
> `compat-call' really depends on how they were defined in Compat.  I
> don't know of a better rule to describe it without copying the
> documentation from the Compat manual (that wouldn't be a good idea
> either, because it wouldn't stay up to date).

Daniel, do have any ideas how to improve the documentation here?  It
seems to me that the authoritative source of information on what
functions to compat-call is Compat itself, right?

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Philip Kaludercic <philipk@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Philip Kaludercic <philipk@posteo.net>
>>>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>>>   monnier@iro.umontreal.ca
>>>> Date: Tue, 06 Feb 2024 19:10:32 +0000
>>>> 
>>>> >> +@defmac compat-call fun &rest args
>>>> >> +This macro calls the compatibility function @var{fun} with @var{args}.
>>>> >> +Many functions provided by Compat can be called directly without this
>>>> >> +macro.  However in the case where Compat provides an alternative
>>>> >> +version of an existing function, the function call has to go through
>>>> >> +@code{compat-call}.
>>>> >> +@end defmac
>>>> >
>>>> > This description left me without understanding when I need to use
>>>> > compat-call and when I can just call FUN.  Can you explain more?
>>>> 
>>>> The intention was for this paragraph to catch that case,
>>>> 
>>>>   However in the case where Compat provides an alternative version of an
>>>>   existing function, the function call has to go through
>>>>   @code{compat-call}.
>>>> 
>>>> though the real information is to be found in the Compat manual, where
>>>> the functions that have to be called via compat-call are documented.
>>>> 
>>>> Should the above sentence be rephrased to give a general feeling for
>>>> when this is the case
>>>> 
>>>>   However in the case where Compat provides an alternative version of an
>>>>   existing function, the function call has to go through
>>>>   @code{compat-call}.  This is the case when, for example the signature
>>>>   changes between versions, preventing older versions of Emacs from
>>>>   using optional arguments introduced in newer releases.
>>>> 
>>>> or should we just refer to the external manual?
>>>
>>> Let me try to make my point more clear: I'd prefer that the reader
>>> emerges from reading this description with a practical way of knowing
>>> when to call the function directly and when to call it via
>>> 'compat-call'.  If that's not easy to understand, perhaps we should
>>> tell that 'compat-call' should always be used, to avoid some rare
>>> corner cases where a direct call will not do, and be done?
>>
>> I don't think we should recommend always using `compat-call', that would
>> make code unreadable.  What definitions have to be called via
>> `compat-call' really depends on how they were defined in Compat.  I
>> don't know of a better rule to describe it without copying the
>> documentation from the Compat manual (that wouldn't be a good idea
>> either, because it wouldn't stay up to date).
>
> Daniel, do have any ideas how to improve the documentation here?  It
> seems to me that the authoritative source of information on what
> functions to compat-call is Compat itself, right?

The Compat manual explicitly lists how each function should be called.
This information is not static, since new functions are added with every
Compat release. Examples from the Compat manual:

   Function: compat-call plist-get plist prop &optional predicate
   Function: ntake n list

I think we should explain the general idea in the compat.el file in
Emacs itself (docstrings and commentary) and refer to the Compat manual
for the list of supported functions and their calling convention. The
explanations in the current patch look good to me.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Wed, 07 Feb 2024 17:15:45 +0000
> 
> >> Let me try to make my point more clear: I'd prefer that the reader
> >> emerges from reading this description with a practical way of knowing
> >> when to call the function directly and when to call it via
> >> 'compat-call'.  If that's not easy to understand, perhaps we should
> >> tell that 'compat-call' should always be used, to avoid some rare
> >> corner cases where a direct call will not do, and be done?
> >
> > I don't think we should recommend always using `compat-call', that would
> > make code unreadable.  What definitions have to be called via
> > `compat-call' really depends on how they were defined in Compat.  I
> > don't know of a better rule to describe it without copying the
> > documentation from the Compat manual (that wouldn't be a good idea
> > either, because it wouldn't stay up to date).
> 
> Daniel, do have any ideas how to improve the documentation here?  It
> seems to me that the authoritative source of information on what
> functions to compat-call is Compat itself, right?

What does Compat manual say about this aspect?

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>   monnier@iro.umontreal.ca
>> Date: Wed, 07 Feb 2024 17:15:45 +0000
>> 
>> >> Let me try to make my point more clear: I'd prefer that the reader
>> >> emerges from reading this description with a practical way of knowing
>> >> when to call the function directly and when to call it via
>> >> 'compat-call'.  If that's not easy to understand, perhaps we should
>> >> tell that 'compat-call' should always be used, to avoid some rare
>> >> corner cases where a direct call will not do, and be done?
>> >
>> > I don't think we should recommend always using `compat-call', that would
>> > make code unreadable.  What definitions have to be called via
>> > `compat-call' really depends on how they were defined in Compat.  I
>> > don't know of a better rule to describe it without copying the
>> > documentation from the Compat manual (that wouldn't be a good idea
>> > either, because it wouldn't stay up to date).
>> 
>> Daniel, do have any ideas how to improve the documentation here?  It
>> seems to me that the authoritative source of information on what
>> functions to compat-call is Compat itself, right?
>
> What does Compat manual say about this aspect?

--8<---------------cut here---------------start------------->8---
Note that Compat provides replacement functions with extended
functionality for functions that are already defined (@code{sort},
@code{assoc}, @dots{}).  These functions may have changed their
calling convention (additional optional arguments) or may have changed
their behavior.  These functions must be looked up explicitly with
@code{compat-function} or called explicitly with @code{compat-call}.
We call them ``Extended Definitions''.  In contrast, newly ``Added
Definitions'' can be called as usual.

@example
(compat-call assoc key alist testfn) ;; Call extended `assoc'
(mapcan fun seq)                     ;; Call newly added `mapcan'
@end example

@defmac compat-call fun &rest args
This macro calls the compatibility function @var{fun} with @var{args}.
Many functions provided by Compat can be called directly without this
macro.  However in the case where Compat provides an alternative
version of an existing function, the function call has to go through
@code{compat-call}.  This happens for example when the calling
convention of a function has changed.
@end defmac

@defmac compat-function fun
This macro returns the compatibility function symbol for @var{fun}.
See @code{compat-call} for a more convenient macro to directly call
compatibility functions.
@end defmac
--8<---------------cut here---------------end--------------->8---

and then later on, for each version there is a section like this

--8<---------------cut here---------------start------------->8---
@subsection Extended Definitions
These functions must be called explicitly via @code{compat-call},
since their calling convention or behavior was extended in Emacs 29.1:

@c copied from lispref/keymaps.texi
@defun compat-call@ set-transient-map keymap &optional keep-pred on-exit message timeout
This function adds @var{keymap} as a @dfn{transient} keymap, which
takes precedence over other keymaps for one (or more) subsequent keys.
...
--8<---------------cut here---------------end--------------->8---

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Thu, 08 Feb 2024 07:40:23 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Daniel, do have any ideas how to improve the documentation here?  It
> >> seems to me that the authoritative source of information on what
> >> functions to compat-call is Compat itself, right?
> >
> > What does Compat manual say about this aspect?
> 
> --8<---------------cut here---------------start------------->8---
> Note that Compat provides replacement functions with extended
> functionality for functions that are already defined (@code{sort},
> @code{assoc}, @dots{}).  These functions may have changed their
> calling convention (additional optional arguments) or may have changed
> their behavior.  These functions must be looked up explicitly with
> @code{compat-function} or called explicitly with @code{compat-call}.
> We call them ``Extended Definitions''.  In contrast, newly ``Added
> Definitions'' can be called as usual.

Thanks.  I suggest to copy the above paragraph to the ELisp reference
manual, and add a cross-reference to the Compat manual after it.

Btw, whenever you introduce terminology in our manuals, as in
``Extended Definitions'' above, it is best to use the @dfn markup
instead of literal quotes: the results look better in HTML and printed
output formats.

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>   monnier@iro.umontreal.ca
>> Date: Thu, 08 Feb 2024 07:40:23 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Daniel, do have any ideas how to improve the documentation here?  It
>> >> seems to me that the authoritative source of information on what
>> >> functions to compat-call is Compat itself, right?
>> >
>> > What does Compat manual say about this aspect?
>> 
>> --8<---------------cut here---------------start------------->8---
>> Note that Compat provides replacement functions with extended
>> functionality for functions that are already defined (@code{sort},
>> @code{assoc}, @dots{}).  These functions may have changed their
>> calling convention (additional optional arguments) or may have changed
>> their behavior.  These functions must be looked up explicitly with
>> @code{compat-function} or called explicitly with @code{compat-call}.
>> We call them ``Extended Definitions''.  In contrast, newly ``Added
>> Definitions'' can be called as usual.
>
> Thanks.  I suggest to copy the above paragraph to the ELisp reference
> manual, and add a cross-reference to the Compat manual after it.
>
> Btw, whenever you introduce terminology in our manuals, as in
> ``Extended Definitions'' above, it is best to use the @dfn markup
> instead of literal quotes: the results look better in HTML and printed
> output formats.

Thanks for mentioning @dfn. When we added the concept of ``Extended
Definitions'' to the Compat manual I was wondering if there is a better
way to make it stand out more. I will update the Compat manual
accordingly.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>   monnier@iro.umontreal.ca
>> Date: Thu, 08 Feb 2024 07:40:23 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Daniel, do have any ideas how to improve the documentation here?  It
>> >> seems to me that the authoritative source of information on what
>> >> functions to compat-call is Compat itself, right?
>> >
>> > What does Compat manual say about this aspect?
>> 
>> --8<---------------cut here---------------start------------->8---
>> Note that Compat provides replacement functions with extended
>> functionality for functions that are already defined (@code{sort},
>> @code{assoc}, @dots{}).  These functions may have changed their
>> calling convention (additional optional arguments) or may have changed
>> their behavior.  These functions must be looked up explicitly with
>> @code{compat-function} or called explicitly with @code{compat-call}.
>> We call them ``Extended Definitions''.  In contrast, newly ``Added
>> Definitions'' can be called as usual.
>
> Thanks.  I suggest to copy the above paragraph to the ELisp reference
> manual, 

OK, can do.

>         and add a cross-reference to the Compat manual after it.

IIUC the cross-reference will be broken, unless the user has installed
Compat manually.  Moreover, if I am not mistaken, the documentation
bundled with the Compat package might not be loaded at all, if the
package isn't activated due to the versioning trick.  I'd add a link to
the HTML manual, rendered on elpa.gnu.org, but I don't know if this is
conventional.

> Btw, whenever you introduce terminology in our manuals, as in
> ``Extended Definitions'' above, it is best to use the @dfn markup
> instead of literal quotes: the results look better in HTML and printed
> output formats.

Ok.

-- 
Philip Kaludercic

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Sat, 10 Feb 2024 16:29:00 +0000
> 
> >> Note that Compat provides replacement functions with extended
> >> functionality for functions that are already defined (@code{sort},
> >> @code{assoc}, @dots{}).  These functions may have changed their
> >> calling convention (additional optional arguments) or may have changed
> >> their behavior.  These functions must be looked up explicitly with
> >> @code{compat-function} or called explicitly with @code{compat-call}.
> >> We call them ``Extended Definitions''.  In contrast, newly ``Added
> >> Definitions'' can be called as usual.
> >
> > Thanks.  I suggest to copy the above paragraph to the ELisp reference
> > manual, 
> 
> OK, can do.
> 
> >         and add a cross-reference to the Compat manual after it.
> 
> IIUC the cross-reference will be broken, unless the user has installed
> Compat manually.

I wouldn't be worried about that: the cross-reference is readable, and
the reader will understand what it says even if it cannot be followed.
We do that for other manuals, which belong to other packages that
might not be installed.

> Moreover, if I am not mistaken, the documentation bundled with the
> Compat package might not be loaded at all, if the package isn't
> activated due to the versioning trick.

What do you mean by "not loaded"?  If a package is installed, its Info
manual is also installed, isn't it?  And if you mean the manual might
not be on the Info search path, then that again goes back to the
"package not installed" scenario, which I don't think we should worry
about.

Thanks.

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>   monnier@iro.umontreal.ca
>> Date: Sat, 10 Feb 2024 16:29:00 +0000
>> 
>> >> Note that Compat provides replacement functions with extended
>> >> functionality for functions that are already defined (@code{sort},
>> >> @code{assoc}, @dots{}).  These functions may have changed their
>> >> calling convention (additional optional arguments) or may have changed
>> >> their behavior.  These functions must be looked up explicitly with
>> >> @code{compat-function} or called explicitly with @code{compat-call}.
>> >> We call them ``Extended Definitions''.  In contrast, newly ``Added
>> >> Definitions'' can be called as usual.
>> >
>> > Thanks.  I suggest to copy the above paragraph to the ELisp reference
>> > manual, 
>> 
>> OK, can do.
>> 
>> >         and add a cross-reference to the Compat manual after it.
>> 
>> IIUC the cross-reference will be broken, unless the user has installed
>> Compat manually.
>
> I wouldn't be worried about that: the cross-reference is readable, and
> the reader will understand what it says even if it cannot be followed.
> We do that for other manuals, which belong to other packages that
> might not be installed.

OK.  I don't have access to the machine with the latest version of the
patch, but as soon as I get that going again, I'll update it and push
the change to master.

>> Moreover, if I am not mistaken, the documentation bundled with the
>> Compat package might not be loaded at all, if the package isn't
>> activated due to the versioning trick.
>
> What do you mean by "not loaded"?  If a package is installed, its Info
> manual is also installed, isn't it?  And if you mean the manual might
> not be on the Info search path, then that again goes back to the
> "package not installed" scenario, which I don't think we should worry
> about.

I might be mistaken, but from a cursory peak at package.el, specifically
`package-activate-1', the Info search path is only updated if the
package is activated.  If the core-package takes priority over an
external package, the external package shouldn't get activated.

This hasn't been a problem in the past, because core packages don't come
with their own manuals, but this is an exception since core-Compat is
not really the same code as ELPA-Compat.

One way to fix this, would be to explain how to disable a package via
`package-load-list' in the Compat manual.  I'll have to try it out, but
if it works, I think that solution should be acceptable.

> Thanks.

-- 
Philip Kaludercic

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Sat, 10 Feb 2024 16:46:30 +0000
> 
> I might be mistaken, but from a cursory peak at package.el, specifically
> `package-activate-1', the Info search path is only updated if the
> package is activated.  If the core-package takes priority over an
> external package, the external package shouldn't get activated.
> 
> This hasn't been a problem in the past, because core packages don't come
> with their own manuals, but this is an exception since core-Compat is
> not really the same code as ELPA-Compat.
> 
> One way to fix this, would be to explain how to disable a package via
> `package-load-list' in the Compat manual.  I'll have to try it out, but
> if it works, I think that solution should be acceptable.

Another way to fix this is simply to include the Compat manual in
emacs.git.

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>   monnier@iro.umontreal.ca
>> Date: Sat, 10 Feb 2024 16:46:30 +0000
>> 
>> I might be mistaken, but from a cursory peak at package.el, specifically
>> `package-activate-1', the Info search path is only updated if the
>> package is activated.  If the core-package takes priority over an
>> external package, the external package shouldn't get activated.
>> 
>> This hasn't been a problem in the past, because core packages don't come
>> with their own manuals, but this is an exception since core-Compat is
>> not really the same code as ELPA-Compat.
>> 
>> One way to fix this, would be to explain how to disable a package via
>> `package-load-list' in the Compat manual.  I'll have to try it out, but
>> if it works, I think that solution should be acceptable.
>
> Another way to fix this is simply to include the Compat manual in
> emacs.git.

No, this won't be useful. The Compat manual available at a given time is
not relevant for the Emacs version developed at that time. If we would
include the manual Compat 30 in Emacs 30, the manual will document
functionality already present and documented in the Emacs 30 manual
itself. The Compat 30 manual is only relevant for Emacs versions 29 and
older.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Philip Kaludercic <philipk@posteo.net>
>>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>>   monnier@iro.umontreal.ca
>>> Date: Sat, 10 Feb 2024 16:46:30 +0000
>>> 
>>> I might be mistaken, but from a cursory peak at package.el, specifically
>>> `package-activate-1', the Info search path is only updated if the
>>> package is activated.  If the core-package takes priority over an
>>> external package, the external package shouldn't get activated.
>>> 
>>> This hasn't been a problem in the past, because core packages don't come
>>> with their own manuals, but this is an exception since core-Compat is
>>> not really the same code as ELPA-Compat.
>>> 
>>> One way to fix this, would be to explain how to disable a package via
>>> `package-load-list' in the Compat manual.  I'll have to try it out, but
>>> if it works, I think that solution should be acceptable.
>>
>> Another way to fix this is simply to include the Compat manual in
>> emacs.git.
>
> No, this won't be useful. The Compat manual available at a given time is
> not relevant for the Emacs version developed at that time. If we would
> include the manual Compat 30 in Emacs 30, the manual will document
> functionality already present and documented in the Emacs 30 manual
> itself. The Compat 30 manual is only relevant for Emacs versions 29 and
> older.

Well, thinking more about it, for developers of :core packages (also
published on ELPA) the manual could still be helpful since the manual
will provide a list of functions with their calling conventions. However
if the Compat manual is included in Emacs itself it will get outdated
quickly, so it is better to refer to the newest version available on
ELPA.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Daniel Mendler <mail@daniel-mendler.de>
> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
> Date: Sat, 10 Feb 2024 18:47:21 +0100
> 
> Well, thinking more about it, for developers of :core packages (also
> published on ELPA) the manual could still be helpful since the manual
> will provide a list of functions with their calling conventions. However
> if the Compat manual is included in Emacs itself it will get outdated
> quickly, so it is better to refer to the newest version available on
> ELPA.

That happens automatically when Compat is installed from ELPA, doesn't
it?

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Daniel Mendler <mail@daniel-mendler.de>
> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
> Date: Sat, 10 Feb 2024 18:40:21 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Philip Kaludercic <philipk@posteo.net>
> >> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
> >>   monnier@iro.umontreal.ca
> >> Date: Sat, 10 Feb 2024 16:46:30 +0000
> >> 
> >> I might be mistaken, but from a cursory peak at package.el, specifically
> >> `package-activate-1', the Info search path is only updated if the
> >> package is activated.  If the core-package takes priority over an
> >> external package, the external package shouldn't get activated.
> >> 
> >> This hasn't been a problem in the past, because core packages don't come
> >> with their own manuals, but this is an exception since core-Compat is
> >> not really the same code as ELPA-Compat.
> >> 
> >> One way to fix this, would be to explain how to disable a package via
> >> `package-load-list' in the Compat manual.  I'll have to try it out, but
> >> if it works, I think that solution should be acceptable.
> >
> > Another way to fix this is simply to include the Compat manual in
> > emacs.git.
> 
> No, this won't be useful. The Compat manual available at a given time is
> not relevant for the Emacs version developed at that time. If we would
> include the manual Compat 30 in Emacs 30, the manual will document
> functionality already present and documented in the Emacs 30 manual
> itself. The Compat 30 manual is only relevant for Emacs versions 29 and
> older.

People who actually install and activate Compat from ELPA will have
the manual which comes with it first on the Info search path, no?  So
where's the problem, and what does "won't be useful" mean in this
context?  To recap, we are discussing ways to make the cross-reference
to the Compat manual work in as many use cases as possible, in
particular when Compat is either not installed or "not activated".

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Mendler <mail@daniel-mendler.de>
>> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
>>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
>> Date: Sat, 10 Feb 2024 18:40:21 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Philip Kaludercic <philipk@posteo.net>
>> >> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>> >>   monnier@iro.umontreal.ca
>> >> Date: Sat, 10 Feb 2024 16:46:30 +0000
>> >> 
>> >> I might be mistaken, but from a cursory peak at package.el, specifically
>> >> `package-activate-1', the Info search path is only updated if the
>> >> package is activated.  If the core-package takes priority over an
>> >> external package, the external package shouldn't get activated.
>> >> 
>> >> This hasn't been a problem in the past, because core packages don't come
>> >> with their own manuals, but this is an exception since core-Compat is
>> >> not really the same code as ELPA-Compat.
>> >> 
>> >> One way to fix this, would be to explain how to disable a package via
>> >> `package-load-list' in the Compat manual.  I'll have to try it out, but
>> >> if it works, I think that solution should be acceptable.
>> >
>> > Another way to fix this is simply to include the Compat manual in
>> > emacs.git.
>> 
>> No, this won't be useful. The Compat manual available at a given time is
>> not relevant for the Emacs version developed at that time. If we would
>> include the manual Compat 30 in Emacs 30, the manual will document
>> functionality already present and documented in the Emacs 30 manual
>> itself. The Compat 30 manual is only relevant for Emacs versions 29 and
>> older.
>
> People who actually install and activate Compat from ELPA will have
> the manual which comes with it first on the Info search path, no?  So
> where's the problem, and what does "won't be useful" mean in this
> context?  To recap, we are discussing ways to make the cross-reference
> to the Compat manual work in as many use cases as possible, in
> particular when Compat is either not installed or "not activated".

I assume it will lead to less confusion if we focus on documenting the
compat.el functionality in the Elisp manual and refer to the Compat
manual on ELPA via an http link, instead of copying the Compat manual.
There should not be a cross-reference problem then?

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Daniel Mendler <mail@daniel-mendler.de>
> Cc: philipk@posteo.net,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>   monnier@iro.umontreal.ca
> Date: Sat, 10 Feb 2024 19:14:38 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Daniel Mendler <mail@daniel-mendler.de>
> >> Cc: Philip Kaludercic <philipk@posteo.net>,  66554@debbugs.gnu.org,
> >>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
> >> Date: Sat, 10 Feb 2024 18:40:21 +0100
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> >> From: Philip Kaludercic <philipk@posteo.net>
> >> >> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
> >> >>   monnier@iro.umontreal.ca
> >> >> Date: Sat, 10 Feb 2024 16:46:30 +0000
> >> >> 
> >> >> I might be mistaken, but from a cursory peak at package.el, specifically
> >> >> `package-activate-1', the Info search path is only updated if the
> >> >> package is activated.  If the core-package takes priority over an
> >> >> external package, the external package shouldn't get activated.
> >> >> 
> >> >> This hasn't been a problem in the past, because core packages don't come
> >> >> with their own manuals, but this is an exception since core-Compat is
> >> >> not really the same code as ELPA-Compat.
> >> >> 
> >> >> One way to fix this, would be to explain how to disable a package via
> >> >> `package-load-list' in the Compat manual.  I'll have to try it out, but
> >> >> if it works, I think that solution should be acceptable.
> >> >
> >> > Another way to fix this is simply to include the Compat manual in
> >> > emacs.git.
> >> 
> >> No, this won't be useful. The Compat manual available at a given time is
> >> not relevant for the Emacs version developed at that time. If we would
> >> include the manual Compat 30 in Emacs 30, the manual will document
> >> functionality already present and documented in the Emacs 30 manual
> >> itself. The Compat 30 manual is only relevant for Emacs versions 29 and
> >> older.
> >
> > People who actually install and activate Compat from ELPA will have
> > the manual which comes with it first on the Info search path, no?  So
> > where's the problem, and what does "won't be useful" mean in this
> > context?  To recap, we are discussing ways to make the cross-reference
> > to the Compat manual work in as many use cases as possible, in
> > particular when Compat is either not installed or "not activated".
> 
> I assume it will lead to less confusion if we focus on documenting the
> compat.el functionality in the Elisp manual and refer to the Compat
> manual on ELPA via an http link, instead of copying the Compat manual.
> There should not be a cross-reference problem then?

But we will be back to the issue of having too much to say in the
ELisp manual about a relatively minor, perhaps even obscure, aspects.
Which is why I'd prefer not to have all the information in the ELisp
reference.

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Philip Kaludercic <philipk@posteo.net>
>>> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,  stefankangas@gmail.com,
>>>   monnier@iro.umontreal.ca
>>> Date: Sat, 10 Feb 2024 16:29:00 +0000
>>> 
>>> >> Note that Compat provides replacement functions with extended
>>> >> functionality for functions that are already defined (@code{sort},
>>> >> @code{assoc}, @dots{}).  These functions may have changed their
>>> >> calling convention (additional optional arguments) or may have changed
>>> >> their behavior.  These functions must be looked up explicitly with
>>> >> @code{compat-function} or called explicitly with @code{compat-call}.
>>> >> We call them ``Extended Definitions''.  In contrast, newly ``Added
>>> >> Definitions'' can be called as usual.
>>> >
>>> > Thanks.  I suggest to copy the above paragraph to the ELisp reference
>>> > manual, 
>>> 
>>> OK, can do.
>>> 
>>> >         and add a cross-reference to the Compat manual after it.
>>> 
>>> IIUC the cross-reference will be broken, unless the user has installed
>>> Compat manually.
>>
>> I wouldn't be worried about that: the cross-reference is readable, and
>> the reader will understand what it says even if it cannot be followed.
>> We do that for other manuals, which belong to other packages that
>> might not be installed.
>
> OK.  I don't have access to the machine with the latest version of the
> patch, but as soon as I get that going again, I'll update it and push
> the change to master.

I have pushed the patch to master, and will therefore close this bug
report.  Thanks to everyone involved for helping out.

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  66554@debbugs.gnu.org,
>>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
>> Date: Thu, 18 Jan 2024 19:51:27 +0000
>> 
>> Pinging this thread with an updated version of the patch:
>
> I find the documentation of this arrangement still insufficient.  The
> way this stuff works (which required Daniel to write 150 lines of
> explanation) is mostly kept out of the written docs, so we'll have to
> rely on people's memory.  Can we document this machinery better?

I can go into more detail, integrating what Daniel explained, my
question just is if this is something that really interests someone
reading the Elisp manual?  One could duplicate or adjust the
documentation for `compat-call' and `compat-function', but explaining
things like how the compat.el file prevents installing Compat
unnecessarily seems like an internal detail to me.

>> +Packages that wish to support older releases of Emacs, without giving
>> +up on newer functionality from recent Emacs releases, one can make use
>> +of the Compat package on GNU ELPA.  For details on how to make use of
>> +the package, @xref{Usage,, Usage, compat, "Compat" Manual}. In case
>                 ^^^^^                                        ^^
> This should be "see @ref" or "@pxref".  @xref is only suitable at the
> beginning of a statement.  And please leave 2 spaces between sentences
> there.

Fixed.

>> +** New package Compat
>> +The Compat package on GNU ELPA provides forwards-compatibility
>> +support, so that packages that still provide support for older
>
> I think this is known as "backward compatibility".

My understanding is that "backwards compatibility" would refer to any
measures, that would assist keeping old code working in newer versions
of Emacs.  Compat doesn't do that (let alone the compat.el from this
patch), it just allows older packages to use newer functionality that
can be replicated for older systems.  The term is also used by the
nadvice package on ELPA.

>> +versions of Emacs can still make use of newer definitions that can be
>> +reasonably re-implemented in Elisp.  Now a "pseudo" Compat package is
>> +part of Emacs, that doesn't provide any compatibility support, but
>> +only implements the public-facing API of Compat so that core packages
>> +can use Compat, while also preventing the installation of Compat on
>> +the most recent version of Emacs.
>
> Not sure this detailed description is useful.  Why not just say that
> Compat is now also available with Emacs?

I was not sure how much detail to give, but if you think it is too much,
I can of course remove it.

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> +;;;; Hack to avoid installing Compat if not necessary
>> +
>> +;; The versioning scheme of the Compat package follows that of Emacs,
>> +;; to indicate what version of Emacs is being supported.  For example,
>> +;; the Compat version number 29.2.3.9 would attempt to provide
>> +;; compatibility definitions up to Emacs 29.2, while also designating
>> +;; that this is the third major release and ninth minor release of
>> +;; Compat, for the specific Emacs release.
>> +
>> +;; To ensure that if the user is using Emacs X.Y installed, the ELPA
>> +;; package Compat X.Y.Z* (for any values of Z*) does not get
>> +;; unnecessarily installed, as there are no missing features that
>> +;; Compat could provide, we programmatically specify the version of
>> +;; the package to be that of the current Emacs version plus a high
>> +;; "major release" to exceed the major version of Compat.
>> +
>> +;;;###autoload (push (list 'compat emacs-major-version
>> emacs-minor-version most-positive-fixnum) package--builtin-versions)
>
> Hack?  Why call it a hack?
>
> By definition a `compat-NN.MM` package is attempting to provide a subset
> of the API offered by Emacs-NN.MM, so Emacs-NN.MM very much provides
> a version of `compat-NN.MM`.
> IOW
>
>     (push (list 'compat emacs-major-version emacs-minor-version ...)
>           package--builtin-versions)
>
> is not a hack at all.
>
> If you want to label the `most-positive-fixnum` as a hack, I guess
> that's OK but then the comment should clarify what it's referring to.

In this case I just meant "hack" in the sense of a "clever trick", since
it was not immediately obvious to anyone until you brought it up.

> Also, please keep the line below the 80 columns limit, e.g.:
>
>     ;;;###autoload (push (list 'compat emacs-major-version
>     ;;;###autoload              emacs-minor-version most-positive-fixnum)
>     ;;;###autoload       package--builtin-versions)

Sure, done.

>
> -- Stefan

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: mail@daniel-mendler.de,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Thu, 18 Jan 2024 20:47:38 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Philip Kaludercic <philipk@posteo.net>
> >> Cc: Eli Zaretskii <eliz@gnu.org>,  66554@debbugs.gnu.org,
> >>   monnier@iro.umontreal.ca,  stefankangas@gmail.com
> >> Date: Thu, 18 Jan 2024 19:51:27 +0000
> >> 
> >> Pinging this thread with an updated version of the patch:
> >
> > I find the documentation of this arrangement still insufficient.  The
> > way this stuff works (which required Daniel to write 150 lines of
> > explanation) is mostly kept out of the written docs, so we'll have to
> > rely on people's memory.  Can we document this machinery better?
> 
> I can go into more detail, integrating what Daniel explained, my
> question just is if this is something that really interests someone
> reading the Elisp manual?

My suggestion is to have the details in compat.el; the ELisp reference
should just mention the package as the recommended means for making
3rd-party and ELPA packages compatible with older Emacs versions.

> In this case I just meant "hack" in the sense of a "clever trick", since
> it was not immediately obvious to anyone until you brought it up.

I believe we call that a "kludge".

bug#66554: [PATCH] Add the public API of Compat to the core

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> +;;;; Hack to avoid installing Compat if not necessary
> +
> +;; The versioning scheme of the Compat package follows that of Emacs,
> +;; to indicate what version of Emacs is being supported.  For example,
> +;; the Compat version number 29.2.3.9 would attempt to provide
> +;; compatibility definitions up to Emacs 29.2, while also designating
> +;; that this is the third major release and ninth minor release of
> +;; Compat, for the specific Emacs release.
> +
> +;; To ensure that if the user is using Emacs X.Y installed, the ELPA
> +;; package Compat X.Y.Z* (for any values of Z*) does not get
> +;; unnecessarily installed, as there are no missing features that
> +;; Compat could provide, we programmatically specify the version of
> +;; the package to be that of the current Emacs version plus a high
> +;; "major release" to exceed the major version of Compat.
> +
> +;;;###autoload (push (list 'compat emacs-major-version emacs-minor-version most-positive-fixnum) package--builtin-versions)

Hack?  Why call it a hack?

By definition a `compat-NN.MM` package is attempting to provide a subset
of the API offered by Emacs-NN.MM, so Emacs-NN.MM very much provides
a version of `compat-NN.MM`.
IOW

    (push (list 'compat emacs-major-version emacs-minor-version ...)
          package--builtin-versions)

is not a hack at all.

If you want to label the `most-positive-fixnum` as a hack, I guess
that's OK but then the comment should clarify what it's referring to.

Also, please keep the line below the 80 columns limit, e.g.:

    ;;;###autoload (push (list 'compat emacs-major-version
    ;;;###autoload              emacs-minor-version most-positive-fixnum)
    ;;;###autoload       package--builtin-versions)


-- Stefan

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

>> +;;;; Hack to avoid installing Compat if not necessary
>> +
>> +;; The versioning scheme of the Compat package follows that of Emacs,
>> +;; to indicate what version of Emacs is being supported.  For example,
>> +;; the Compat version number 29.2.3.9 would attempt to provide
>> +;; compatibility definitions up to Emacs 29.2, while also designating
>> +;; that this is the third major release and ninth minor release of
>> +;; Compat, for the specific Emacs release.
>> +
>> +;; To ensure that if the user is using Emacs X.Y installed, the ELPA
>> +;; package Compat X.Y.Z* (for any values of Z*) does not get
>> +;; unnecessarily installed, as there are no missing features that
>> +;; Compat could provide, we programmatically specify the version of
>> +;; the package to be that of the current Emacs version plus a high
>> +;; "major release" to exceed the major version of Compat.
>> +
>> +;;;###autoload (push (list 'compat emacs-major-version emacs-minor-version most-positive-fixnum) package--builtin-versions)
>
> Hack?  Why call it a hack?
>
> By definition a `compat-NN.MM` package is attempting to provide a subset
> of the API offered by Emacs-NN.MM, so Emacs-NN.MM very much provides
> a version of `compat-NN.MM`.
> IOW
>
>     (push (list 'compat emacs-major-version emacs-minor-version ...)
>           package--builtin-versions)
>
> is not a hack at all.

Yes, the specified version is not a hack, since it correctly encodes the
API version provided by the compat.el in core.

> If you want to label the `most-positive-fixnum` as a hack, I guess
> that's OK but then the comment should clarify what it's referring to.

I may be wrong, but I assume that Philip considers the autoload a hack.
In this thread we discussed various alternatives to approach this
problem, e.g., the package version regexp (another hack), or my naive
idea of simply writing 29.1.9999 in the compat.el package header (yet
another hack).

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Stefan Kangas]

Daniel Mendler <mail@daniel-mendler.de> writes:

>> If you want to label the `most-positive-fixnum` as a hack, I guess
>> that's OK but then the comment should clarify what it's referring to.
>
> I may be wrong, but I assume that Philip considers the autoload a hack.
> In this thread we discussed various alternatives to approach this
> problem, e.g., the package version regexp (another hack), or my naive
> idea of simply writing 29.1.9999 in the compat.el package header (yet
> another hack).

IMHO, the "naive" (why naive?) number 9999 is more readable, and in
practice will be the same as using most-positive-fixnum.  So I'd just
use that.

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Stefan Kangas <stefankangas@gmail.com> writes:

> Daniel Mendler <mail@daniel-mendler.de> writes:
>
>>> If you want to label the `most-positive-fixnum` as a hack, I guess
>>> that's OK but then the comment should clarify what it's referring to.
>>
>> I may be wrong, but I assume that Philip considers the autoload a hack.
>> In this thread we discussed various alternatives to approach this
>> problem, e.g., the package version regexp (another hack), or my naive
>> idea of simply writing 29.1.9999 in the compat.el package header (yet
>> another hack).
>
> IMHO, the "naive" (why naive?) number 9999 is more readable, and in
> practice will be the same as using most-positive-fixnum.  So I'd just
> use that.

We can also use 1.0e+INF, unless there is a reason to avoid non-fixnums
in version strings?

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas <stefankangas@gmail.com> writes:

> Daniel Mendler <mail@daniel-mendler.de> writes:
>
>>> If you want to label the `most-positive-fixnum` as a hack, I guess
>>> that's OK but then the comment should clarify what it's referring to.
>>
>> I may be wrong, but I assume that Philip considers the autoload a hack.
>> In this thread we discussed various alternatives to approach this
>> problem, e.g., the package version regexp (another hack), or my naive
>> idea of simply writing 29.1.9999 in the compat.el package header (yet
>> another hack).
>
> IMHO, the "naive" (why naive?) number 9999 is more readable, and in
> practice will be the same as using most-positive-fixnum.  So I'd just
> use that.

The number 9999 is of course more than large enough. What I meant here -
not the choice of the number is what one may consider a hack, but the
way we register the version of the builtin compat.el package. There were
multiple ways which have been discussed in this thread:

- Write the package version of compat.el directly in the header. This
  was my first proposal. This is not ideal since it requires the
  maintainer to update the header whenever a new version of Emacs is
  released. Therefore naive.

- Stefan Monnier proposed to instead use an autoload and to dynamically
  "compute" and register the builtin compat.el version. I consider this
  a clever trick. It is maybe a bit unconventional to register builtin
  package versions like this, but it will work perfectly well and
  doesn't require any intervention. There is no risk that one can forget
  to update the package header.

If we register (emacs-major-version emacs-minor-version 9999),
(emacs-major-version emacs-minor-version most-positive-fixnum) or
(emacs-major-version emacs-minor-version 1.0e+INF) will not make any
difference in practice. As long as the number is large enough, it will
work.

We should still keep the reader of the code in mind, who may wonder
about the magic number and about the special package version
registration, so a comment is certainly justified. Philip did include
such an explanation in his patch. We should only replace the word "Hack"
with "Clever trick" in the comment.

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  66554@debbugs.gnu.org,
>   eliz@gnu.org,  Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 11 Jan 2024 20:43:11 +0000
> 
> +;; To ensure that if the user has installed Compat X.Y.Z.A on Emacs
> +;; X.Y, that the this file is preferred by package.el, we
                ^^^^^^^^
Typo.

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> --- a/lisp/subr.el
>> +++ b/lisp/subr.el
>> @@ -6873,9 +6873,9 @@ version-separator
>>  
>>  Usually the separator is \".\", but it can be any other string.")
>>  
>> -
>>  (defconst version-regexp-alist
>> -  '(("^[-._+ ]?snapshot$"                                 . -4)
>> +  `(("^[-._+ ]?hyper$"                                    . ,most-positive-fixnum)
>> +    ("^[-._+ ]?snapshot$"                                 . -4)
>>      ;; treat "1.2.3-20050920" and "1.2-3" as snapshot releases
>>      ("^[-._+]$"                                           . -4)
>>      ;; treat "1.2.3-CVS" as snapshot release
>
> Relying on a change in `version-regexp-alist` open a can of worms.

Out of curiosity, what are you thinking about?  I know this is a hack,
but the way I though about it, it should work.

> It's much simpler to make sure the GNU ELPA `compat` package uses
> versions of the form `30.0.NN` (or `30.1.NN` if it means it
> offers functionality from Emacs-30.2, tho hopefully we'll never get
> there).

This would also be an acceptable fallback strategy IMO, with the minor
annoyance that it might confuse some people into thinking that the ELPA
package is out of date.

>>>>> Is there a place where we could add a warning to make sure that
>>>>> bumping the version won't be forgotten?
>
> The addition cab done procedurally, as in
>
>     ;;;;####autoload (push (list 'compat emacs-major-version
> emacs-minor-version) package--builtin-versions)

If this work, I like this idea a lot!  I'll test it out to see if this
could simplify things.

>
> -- Stefan

bug#66554: [PATCH] Add the public API of Compat to the core

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Philip Kaludercic <philipk@posteo.net> writes:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> The addition cab done procedurally, as in
>>
>>     ;;;;####autoload (push (list 'compat emacs-major-version
>> emacs-minor-version) package--builtin-versions)
>
> If this work, I like this idea a lot!  I'll test it out to see if this
> could simplify things.

Philip, could we push (list 'compat emacs-major-version
emacs-minor-version most-positive-fixnum)? Wouldn't this solve the two
problems we have - no manual version bump and Compat could keep its
current version scheme?

Daniel

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Daniel Mendler <mail@daniel-mendler.de>,  66554@debbugs.gnu.org,
>   eliz@gnu.org,  Stefan Kangas <stefankangas@gmail.com>
> Date: Thu, 11 Jan 2024 20:24:25 +0000
> 
> > The addition cab done procedurally, as in
> >
> >     ;;;;####autoload (push (list 'compat emacs-major-version
> > emacs-minor-version) package--builtin-versions)
> 
> If this work, I like this idea a lot!  I'll test it out to see if this
> could simplify things.

I'm not sure I understand the effects of the above -- can you explain?

In general, if something needs to be done when the Emacs version is
bumped, that action should be described in make-tarball.txt.  It is a
nuisance to follow the procedure there, only to be reminded of
something else via a warning emitted by the build (and that is even
before we consider the possibility that someone launches a build and
goes for a coffee, while the warnings scroll on the screen with no one
to see them).  Likewise, if a file is modified by whatever commands we
invoke to bump an Emacs version, that file should be mentioned in
make-tarball.txt, because the person who performs that procedure will
then need to commit the changes with a suitable log message, like we
do today with configure.ac and the rest. 

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Daniel Mendler <mail@daniel-mendler.de>,  66554@debbugs.gnu.org,
>>   eliz@gnu.org,  Stefan Kangas <stefankangas@gmail.com>
>> Date: Thu, 11 Jan 2024 20:24:25 +0000
>> 
>> > The addition cab done procedurally, as in
>> >
>> >     ;;;;####autoload (push (list 'compat emacs-major-version
>> > emacs-minor-version) package--builtin-versions)
>> 
>> If this work, I like this idea a lot!  I'll test it out to see if this
>> could simplify things.
>
> I'm not sure I understand the effects of the above -- can you explain?

The idea is to manually register the file as a package while scraping
for autoloads, thereby setting the version of the built-in Compat
package to whatever the current version of Emacs is.  My patch adds a
`most-positive-fixnum' to the version number to ensure that the built-in
compat is always (or at least for a more than reasonably long while)
preferred over an external dependency.

The consequence is that we don't have to update any files, and I would
assume we don't have to document anything in make-tarball.txt, because
nothing has to be done when bumping the version of Emacs itself.

> In general, if something needs to be done when the Emacs version is
> bumped, that action should be described in make-tarball.txt.  It is a
> nuisance to follow the procedure there, only to be reminded of
> something else via a warning emitted by the build (and that is even
> before we consider the possibility that someone launches a build and
> goes for a coffee, while the warnings scroll on the screen with no one
> to see them).  Likewise, if a file is modified by whatever commands we
> invoke to bump an Emacs version, that file should be mentioned in
> make-tarball.txt, because the person who performs that procedure will
> then need to commit the changes with a suitable log message, like we
> do today with configure.ac and the rest. 

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: monnier@iro.umontreal.ca,  mail@daniel-mendler.de,
>   66554@debbugs.gnu.org,  stefankangas@gmail.com
> Date: Fri, 12 Jan 2024 07:38:20 +0000
> 
> The consequence is that we don't have to update any files, and I would
> assume we don't have to document anything in make-tarball.txt, because
> nothing has to be done when bumping the version of Emacs itself.

Thanks.

What happens when we bump Emacs version from NN.x to NN.x.90 or from
NN.x.90 to NN.x.91?  emacs-major-version and emacs-minor-version stay
the same, but the Emacs version is bumped, and there could be changes
in compat.el specific to the next release, NN.x+1 or NN+1.1, no?

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Wed, 10 Jan 2024 14:02:02 -0800
> Cc: 66554@debbugs.gnu.org, eliz@gnu.org, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>
> 
> Philip Kaludercic <philipk@posteo.net> writes:
> 
> > See https://lists.gnu.org/archive/html/emacs-devel/2023-10/msg00260.html
> > for the background behind this proposal.
> 
> I'm fine with this, and see the same benefits as described above.
> 
> I'd like to hear if Eli and Stefan M has anything to add, though, so I'm
> copying them in here.

I don't mind, but it sounds like the details of adding compat.el to
core are not yet clear to all, and are still being discussed.  Also,
it sounds like we would need to modify compat.el for every release or
something? if so, this should be in make-tarball.txt.

It might be also a good idea to mention this in the ELisp manual
somewhere.

bug#66554: [PATCH] Add the public API of Compat to the core

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't mind, but it sounds like the details of adding compat.el to
> core are not yet clear to all, and are still being discussed.

OK, thanks.  There's no rush here, I think.  I hope that people will
speak up so that we can get a handle on those details.

> Also, it sounds like we would need to modify compat.el for every
> release or something? if so, this should be in make-tarball.txt.
>
> It might be also a good idea to mention this in the ELisp manual
> somewhere.

Philip, could you take a look at this and amend the patch accordingly?
Thanks in advance.

bug#66554: [PATCH] Add the public API of Compat to the core

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 726 bytes --]

Stefan Kangas <stefankangas@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> I don't mind, but it sounds like the details of adding compat.el to
>> core are not yet clear to all, and are still being discussed.
>
> OK, thanks.  There's no rush here, I think.  I hope that people will
> speak up so that we can get a handle on those details.
>
>> Also, it sounds like we would need to modify compat.el for every
>> release or something? if so, this should be in make-tarball.txt.
>>
>> It might be also a good idea to mention this in the ELisp manual
>> somewhere.
>
> Philip, could you take a look at this and amend the patch accordingly?
> Thanks in advance.

This would integrate my previous two suggestions:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-the-public-API-of-Compat-to-the-core.patch --]
[-- Type: text/x-diff, Size: 5484 bytes --]

From 83743d8c407cf5ef8a34816b88e9b28d890bc7f6 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Wed, 13 Sep 2023 12:26:22 +0200
Subject: [PATCH] Add the public API of Compat to the core

* lisp/emacs-lisp/compat.el: Add stub file with minimal definitions,
so that core packages, that haven't been installed from ELPA, can make
use of the public API and use more recent function signatures.
* lisp/progmodes/python.el (compat): Remove 'noerror flag, because
Compat can now be required without the real package being available.
* admin/admin.el (set-version): Update the version in the package
header when setting the version.
* lisp/subr.el (version-regexp-alist): Add non-numeric version string
"compat", to ensure that the build-in compat is always
preferred. (Bug#66554)
---
 admin/admin.el            |  5 ++++
 lisp/emacs-lisp/compat.el | 61 +++++++++++++++++++++++++++++++++++++++
 lisp/progmodes/python.el  |  2 +-
 lisp/subr.el              |  4 +--
 4 files changed, 69 insertions(+), 3 deletions(-)
 create mode 100644 lisp/emacs-lisp/compat.el

diff --git a/admin/admin.el b/admin/admin.el
index 7fa2727aeb7..bcf1be6ff67 100644
--- a/admin/admin.el
+++ b/admin/admin.el
@@ -115,6 +115,11 @@ set-version
   (set-version-in-file root "nt/README.W32" version
 		       (rx (and "version" (1+ space)
 				(submatch (1+ (in "0-9."))))))
+  (set-version-in-file root "lisp/emacs-lisp/compat.el" version
+		       (rx (and bol ";; Version : " (1+ space)
+				(submatch (1+ (in "0-9.")))
+                                "compat")))
+
   ;; TODO: msdos could easily extract the version number from
   ;; configure.ac with sed, rather than duplicating the information.
   (set-version-in-file root "msdos/sed2v2.inp" version
diff --git a/lisp/emacs-lisp/compat.el b/lisp/emacs-lisp/compat.el
new file mode 100644
index 00000000000..b45b3ede633
--- /dev/null
+++ b/lisp/emacs-lisp/compat.el
@@ -0,0 +1,61 @@
+;;; compat.el --- Pseudo-Compatibility for Elisp -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2021-2024 Free Software Foundation, Inc.
+
+;; Author:								\
+;;   Philip Kaludercic <philipk@posteo.net>,				\
+;;   Daniel Mendler <mail@daniel-mendler.de>
+;; Maintainer:								\
+;;   Daniel Mendler <mail@daniel-mendler.de>,				\
+;;   Compat Development <~pkal/compat-devel@lists.sr.ht>,
+;;   emacs-devel@gnu.org
+;; URL: https://github.com/emacs-compat/compat
+;; Version: 30.1compat
+;; Keywords: lisp, maint
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;; The Compat package on ELPA provides forward-compatibility
+;; definitions for other packages.  While mostly transparent, a
+;; minimal API is necessary whenever core definitions change calling
+;; conventions (e.g. `plist-get' can be invoked with a predicate from
+;; Emacs 29.1 onward).  For core packages on ELPA to be able to take
+;; advantage of this functionality, the macros `compat-function' and
+;; `compat-call' have to be available in the core, usable even if
+;; users do not have the Compat package installed, which this file
+;; ensures.
+
+;; Note that Compat is not a core package and this file is not
+;; available on GNU ELPA.
+
+;;; Code:
+
+(defmacro compat-function (fun)
+  "Return compatibility function symbol for FUN.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  `#',fun)
+
+(defmacro compat-call (fun &rest args)
+  "Call compatibility function or macro FUN with ARGS.
+This is a pseudo-compatibility stub for core packages on ELPA,
+that depend on the Compat package, whenever the user doesn't have
+the package installed on their current system."
+  (cons fun args))
+
+(provide 'compat)
+;;; compat.el ends here
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 1148da11a06..c3f18692f61 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -267,7 +267,7 @@
 (eval-when-compile (require 'subr-x))   ;For `string-empty-p' and `string-join'.
 (require 'treesit)
 (require 'pcase)
-(require 'compat nil 'noerror)
+(require 'compat)
 (require 'project nil 'noerror)
 (require 'seq)
 
diff --git a/lisp/subr.el b/lisp/subr.el
index df28989b399..007206ebf0e 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -6873,9 +6873,9 @@ version-separator
 
 Usually the separator is \".\", but it can be any other string.")
 
-
 (defconst version-regexp-alist
-  '(("^[-._+ ]?snapshot$"                                 . -4)
+  `(("^[-._+ ]?compat$"                                   . ,most-positive-fixnum)
+    ("^[-._+ ]?snapshot$"                                 . -4)
     ;; treat "1.2.3-20050920" and "1.2-3" as snapshot releases
     ("^[-._+]$"                                           . -4)
     ;; treat "1.2.3-CVS" as snapshot release
-- 
2.39.2

bug#66554: [PATCH] Add the public API of Compat to the core

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Eli Zaretskii <eliz@gnu.org>,  66554@debbugs.gnu.org,
>   monnier@iro.umontreal.ca
> Date: Thu, 11 Jan 2024 20:07:41 +0000
> 
> > Philip, could you take a look at this and amend the patch accordingly?
> > Thanks in advance.
> 
> This would integrate my previous two suggestions:

Thanks, but please also mention this in admin/make-tarball.txt, where
it describes the invocation of set-version and the files it updates.

Should we have a NEWS entry about this?