bug#59786: Allowing arbitrary expressions in cl-labels

bug#59786: Allowing arbitrary expressions in cl-labels

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 2933 bytes --]

Tags: patch

I have found some circumstances where I'd like to write things like:

    (cl-labels ((f1 (if blabla
                        (lambda (x) (do one thing))
                      (lambda (x) (do another thing))))
                (f2 (if bleble
                        (lambda (y) (do some thing))
                      (lambda (y) (do some other thing)))))
      ...)

I.e. define two, mutually-recursive functions, but where I want to
perform some computation before "building/returning" each function.

I could rewrite the above to

    (cl-labels ((f1 (x) (if blabla
                            (do one thing)
                          (do another thing)))
                (f2 (y) (if bleble
                            (do some thing)
                          (do some other thing))))
      ...)

but then the `if` tests are repeated at each call.
I could also rewrite it to

    (letrec ((f1 (if blabla
                     (lambda (x) (do one thing))
                   (lambda (x) (do another thing))))
             (f2 (if bleble
                     (lambda (y) (do some thing))
                   (lambda (y) (do some other thing)))))
      ...)

but then I have to use (funcall f1 ..) and (funcall f2 ...) instead of
just (f1 ...) and (f2 ...).  I could add a (cl-flet ((f1 f1) (f2 f2)) ...)
but that's inconvenient, especially because I'd have to add it in
various places.

So I'd like to propose to extend `cl-labels` in the same way that
`cl-flet` was extended to allow each function to be defined by an
expression that returns a function rather than by "args + body".

One option is to use the same approach as I used in `cl-flet`,
i.e. allow each binding to be either

    (FUNC ARGS BODY...)    the normal existing syntax
or
    (FUNC EXP)             the new syntax

After I introduced this in `cl-flet` it was pointed out that it was an
incompatible change since BODY... can be the empty list.  Another option
is to use a syntax like:

    (FUNC = EXP)           the new new syntax

which should not suffer from such incompatibility since ARGS should
never be of the form `=`.

The patch below uses this "new new" syntax (and adjusts `cl-flet` to
also support this new new syntax).  It still lacks a NEWS entry (as
well as updating the CL manual), but before I do that, I'd like to hear
what other people think,


        Stefan


 In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw3d scroll bars) of 2022-11-29 built on pastel
Repository revision: 4254a4a71d5d04cfcefaedfefe5d22af55650a6a
Repository branch: work
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure -C --enable-checking --enable-check-lisp-object-type --with-modules --with-cairo --with-tiff=ifavailable
 'CFLAGS=-Wall -g3 -Og -Wno-pointer-sign'
 PKG_CONFIG_PATH=/home/monnier/lib/pkgconfig'


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: cl-labels.patch --]
[-- Type: text/patch, Size: 4600 bytes --]

diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
index 43a2ed92059..cc8c98f2264 100644
--- a/lisp/emacs-lisp/cl-macs.el
+++ b/lisp/emacs-lisp/cl-macs.el
@@ -2028,11 +2028,11 @@ cl--labels-convert
 ;;;###autoload
 (defmacro cl-flet (bindings &rest body)
   "Make local function definitions.
-Each definition can take the form (FUNC EXP) where
+Each definition can take the form (FUNC = EXP) where
 FUNC is the function name, and EXP is an expression that returns the
 function value to which it should be bound, or it can take the more common
 form (FUNC ARGLIST BODY...) which is a shorthand
-for (FUNC (lambda ARGLIST BODY)).
+for (FUNC = (lambda ARGLIST BODY)).
 
 FUNC is defined only within FORM, not BODY, so you can't write
 recursive function definitions.  Use `cl-labels' for that.  See
@@ -2055,8 +2055,12 @@ cl-flet
         (if (and (= (length args-and-body) 1) (symbolp (car args-and-body)))
             ;; Optimize (cl-flet ((fun var)) body).
             (setq var (car args-and-body))
-          (push (list var (if (= (length args-and-body) 1)
-                              (car args-and-body)
+          (push (list var (cond
+                           ((= (length args-and-body) 1) ;Obsolete syntax.
+                            (car args-and-body))
+                           ((eq '= (car args-and-body))
+                            (macroexp-progn (cdr args-and-body)))
+                           (t
                             `(cl-function (lambda . ,args-and-body))))
                 binds))
 	(push (cons (car binding)
@@ -2203,12 +2207,13 @@ cl--self-tco
 ;;;###autoload
 (defmacro cl-labels (bindings &rest body)
   "Make local (recursive) function definitions.
-+BINDINGS is a list of definitions of the form (FUNC ARGLIST BODY...) where
-FUNC is the function name, ARGLIST its arguments, and BODY the
-forms of the function body.  FUNC is defined in any BODY, as well
-as FORM, so you can write recursive and mutually recursive
-function definitions.  See info node `(cl) Function Bindings' for
-details.
+BINDINGS is a list of definitions of the form either:
+- (FUNC ARGLIST BODY...) where FUNC is the function name,
+  ARGLIST its arguments, and BODY the forms of the function body.
+- (FUNC = BODY) where BODY is an expression that evaluates to a function.
+FUNC is defined in any BODY, as well as FORM, so you can write recursive
+and mutually recursive function definitions.
+See info node `(cl) Function Bindings' for details.
 
 \(fn ((FUNC ARGLIST BODY...) ...) FORM...)"
   (declare (indent 1) (debug cl-flet))
@@ -2226,17 +2231,31 @@ cl-labels
     (unless (assq 'function newenv)
       (push (cons 'function #'cl--labels-convert) newenv))
     ;; Perform self-tail call elimination.
-    (setq binds (mapcar
-                 (lambda (bind)
-                   (pcase-let*
-                       ((`(,var ,sargs . ,sbody) bind)
-                        (`(function (lambda ,fargs . ,ebody))
-                         (macroexpand-all `(cl-function (lambda ,sargs . ,sbody))
-                                          newenv))
-                        (`(,ofargs . ,obody)
-                         (cl--self-tco var fargs ebody)))
-                     `(,var (function (lambda ,ofargs . ,obody)))))
-                 (nreverse binds)))
+    (setq binds
+          (mapcar
+           (lambda (bind)
+             (pcase-let*
+                 ((`(,var ,sargs . ,sbody) bind))
+               (list var
+                     (named-let loop
+                         ((mfunexp (macroexpand-all
+                                    (if (eq '= sargs)
+                                        (macroexp-progn sbody)
+                                      `(cl-function (lambda ,sargs . ,sbody)))
+                                    newenv)))
+                       (pcase mfunexp
+                         (`#'(lambda ,fargs . ,ebody)
+                          `#'(lambda . ,(cl--self-tco var fargs ebody)))
+                         (`(progn . ,exps)
+                          `(progn ,@(butlast exps) ,(loop (car (last exps)))))
+                         (`(let ,bindings ,exps)
+                          `(let ,bindings
+                             ,@(butlast exps) ,(loop (car (last exps)))))
+                         (`(if ,exp1 ,exp2 ,exps)
+                          `(if ,exp1 ,(loop exp2)
+                             ,@(butlast exps) ,(loop (car (last exps)))))
+                         (_ mfunexp))))))
+           (nreverse binds)))
     `(letrec ,binds
        . ,(macroexp-unprogn
            (macroexpand-all

bug#59786: Allowing arbitrary expressions in cl-labels

[Drew Adams]

> I'd like to hear what other people think,

FWIW, I think that our CL emulation library
should emulate CL.  (Shocking, I know.)  In this 
case, `cl-labels' should emulate CL's `labels'.

It's bad enough that we've added some non-CL
stuff to our CL library over the years (instead
of putting it elsewhere and not giving it prefix
`cl-').  It's worse when we co-opt a CL name to
do something other than emulate the CL thingie
that has that name.

It's fine for Emacs Lisp to add whatever we like,
including a `labels'-like function that acts
differently from CL `labels'.  What's misguided,
IMO, is for us to use the name `cl-labels' for
such non-`labels' behavior.  Add a new function
for that, without prefix `cl-'.

I'd even like to see us backtrack on the other,
divergences from CL emulation that use CL names 
(but with prefix `cl-').

Just one opinion.

bug#59786: Allowing arbitrary expressions in cl-labels

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The patch below uses this "new new" syntax (and adjusts `cl-flet` to
> also support this new new syntax).  It still lacks a NEWS entry (as
> well as updating the CL manual), but before I do that, I'd like to hear
> what other people think,

I like the idea to implement this kind of feature for `cl-labels'.  It's
a good change IMO.

I don't like the syntax I think (ugly).  The rest of this answer is
discussing this detail:

I don't recall all details about the ambiguity of the empty body case,
so forgive me if I'm missing something.

A binding like (my-fun (var1 var2)) with an empty body would give you
compiler warnings anyway.  Would this be an alternative to your "="
style syntax:

To specify a local function with an empty body one would have to use
local variable names starting with "_":

   (my-fun (_var1 _var2))

If not all variables start with an underscore or not all list members
are symbols, the binding is interpreted as specifying an expression
evaluating to the function to bind.  This assumes that "_var" never
specifies a named function.

Michael.

bug#59786: Allowing arbitrary expressions in cl-labels

[Sean Whitton]

Hello,

On Fri 02 Dec 2022 at 02:44PM -05, Stefan Monnier wrote:

> After I introduced this in `cl-flet` it was pointed out that it was an
> incompatible change since BODY... can be the empty list.  Another option
> is to use a syntax like:
>
>     (FUNC = EXP)           the new new syntax
>
> which should not suffer from such incompatibility since ARGS should
> never be of the form `=`.

I'm usually in support adding new capabilities to existing macros, but
in this case, the cost of using an equals sign is rather high.
How about just defining a new macro fletrec that works this way, like
letrec but for function cells?  That would seem like it would address
your usecase.

-- 
Sean Whitton

bug#59786: Allowing arbitrary expressions in cl-labels

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

Michael Heerdegen [2023-01-13 14:47:52] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> The patch below uses this "new new" syntax (and adjusts `cl-flet` to
>> also support this new new syntax).  It still lacks a NEWS entry (as
>> well as updating the CL manual), but before I do that, I'd like to hear
>> what other people think,
>
> I like the idea to implement this kind of feature for `cl-labels'.  It's
> a good change IMO.
>
> I don't like the syntax I think (ugly).  The rest of this answer is
> discussing this detail:
>
> I don't recall all details about the ambiguity of the empty body case,
> so forgive me if I'm missing something.
>
> A binding like (my-fun (var1 var2)) with an empty body would give you
> compiler warnings anyway.  Would this be an alternative to your "="
> style syntax:
>
> To specify a local function with an empty body one would have to use
> local variable names starting with "_":
>
>    (my-fun (_var1 _var2))
>
> If not all variables start with an underscore or not all list members
> are symbols, the binding is interpreted as specifying an expression
> evaluating to the function to bind.  This assumes that "_var" never
> specifies a named function.

I guess you're right.  So we should just use the (FUNC EXP) syntax,
exactly like we already do for `cl-flet`.
The patch below does that.


        Stefan

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: cl-labels.patch --]
[-- Type: text/x-diff, Size: 3779 bytes --]

diff --git a/etc/NEWS b/etc/NEWS
index d1c7303f976..562c9c6bdc3 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -215,6 +215,11 @@ modal editing packages.
 \f
 * Changes in Specialized Modes and Packages in Emacs 31.1
 
+** CL-Lib
++++
+*** 'cl-labels' now also accepts (FUNC EXP) bindings, like 'cl-flet'.
+Such bindings make it possible to compute which function to bind to FUNC.
+
 ** Whitespace
 
 ---
diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
index b37f744b175..388281e4b1a 100644
--- a/lisp/emacs-lisp/cl-macs.el
+++ b/lisp/emacs-lisp/cl-macs.el
@@ -2250,9 +2250,11 @@ cl--self-tco
 ;;;###autoload
 (defmacro cl-labels (bindings &rest body)
   "Make local (recursive) function definitions.
-BINDINGS is a list of definitions of the form (FUNC ARGLIST BODY...) where
+BINDINGS is a list of definitions of the form either (FUNC EXP)
+where EXP is a form that should return the function to bind to the
+function name FUNC, or (FUNC ARGLIST BODY...) where
 FUNC is the function name, ARGLIST its arguments, and BODY the
-forms of the function body.  FUNC is defined in any BODY, as well
+forms of the function body.  FUNC is defined in any BODY or EXP, as well
 as FORM, so you can write recursive and mutually recursive
 function definitions.  See info node `(cl) Function Bindings' for
 details.
@@ -2273,18 +2275,21 @@ cl-labels
     (unless (assq 'function newenv)
       (push (cons 'function #'cl--labels-convert) newenv))
     ;; Perform self-tail call elimination.
-    (setq binds (mapcar
-                 (lambda (bind)
-                   (pcase-let*
-                       ((`(,var ,sargs . ,sbody) bind)
-                        (`(function (lambda ,fargs . ,ebody))
-                         (macroexpand-all `(cl-function (lambda ,sargs . ,sbody))
-                                          newenv))
-                        (`(,ofargs . ,obody)
-                         (cl--self-tco var fargs ebody)))
-                     `(,var (function (lambda ,ofargs . ,obody)))))
-                 (nreverse binds)))
-    `(letrec ,binds
+    `(letrec ,(mapcar
+               (lambda (bind)
+                 (pcase-let* ((`(,var ,sargs . ,sbody) bind))
+                   `(,var
+                     ,(if (null sbody)
+                          ;; This is a (FUNC EXP) definition.
+                          (macroexpand-all sargs newenv)
+                        (pcase-let*
+                            ((`(function (lambda ,fargs . ,ebody))
+                              (macroexpand-all
+                               `(cl-function (lambda ,sargs . ,sbody)) newenv))
+                             (`(,ofargs . ,obody)
+                              (cl--self-tco var fargs ebody)))
+                          `(function (lambda ,ofargs . ,obody)))))))
+               (nreverse binds))
        . ,(macroexp-unprogn
            (macroexpand-all
             (macroexp-progn body)
diff --git a/test/lisp/emacs-lisp/cl-lib-tests.el b/test/lisp/emacs-lisp/cl-lib-tests.el
index 14ff8628fb8..376ccebef98 100644
--- a/test/lisp/emacs-lisp/cl-lib-tests.el
+++ b/test/lisp/emacs-lisp/cl-lib-tests.el
@@ -558,5 +558,14 @@ cl-constantly
   (should (equal (mapcar (cl-constantly 3) '(a b c d))
                  '(3 3 3 3))))
 
+(ert-deftest cl-lib-test-labels ()
+  (should (equal (cl-labels ((even (x) (if (= x 0) t (odd (1- x))))
+                             (odd (x) (if (= x 0) nil (even (1- x)))))
+                   (list (even 42) (odd 42)))
+                 '(t nil)))
+  (should (equal (cl-labels ((even (lambda (x) (if (= x 0) t (odd (1- x)))))
+                             (odd (lambda (x) (if (= x 0) nil (even (1- x))))))
+                   (list (even 42) (odd 42)))
+                 '(t nil))))
 
 ;;; cl-lib-tests.el ends here

bug#59786: Allowing arbitrary expressions in cl-labels

[Michael Heerdegen via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi Stefan,

> -forms of the function body.  FUNC is defined in any BODY, as well
> +forms of the function body.  FUNC is defined in any BODY or EXP, as well
                                                            ^^^^^^
This is unfortunately not correct, e.g.

(cl-labels ((even (if (odd 5)
                      (lambda (x) (if (= x 0) t (odd (1- x))))
                    #'ignore))
            (odd  (lambda (x) (if (= x 0) nil (even (1- x))))))
         (list (even 42) (odd 42)))

~~> funcall: Symbol's function definition is void: nil

Only preceding definitions can be used in the EXP.

Dunno how useful it is at all that the EXPs can use any of the defined
functions.


Michael.

bug#59786: Allowing arbitrary expressions in cl-labels

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> -forms of the function body.  FUNC is defined in any BODY, as well
>> +forms of the function body.  FUNC is defined in any BODY or EXP, as well
>                                                             ^^^^^^
> This is unfortunately not correct, e.g.
>
> (cl-labels ((even (if (odd 5)
>                       (lambda (x) (if (= x 0) t (odd (1- x))))
>                     #'ignore))
>             (odd  (lambda (x) (if (= x 0) nil (even (1- x))))))
>          (list (even 42) (odd 42)))
>
> ~~> funcall: Symbol's function definition is void: nil

This is normal.  Same happens with

    (letrec ((even (if (funcall odd 5)
                          (lambda (x) (if (= x 0) t (funcall odd (1- x))))
                        #'ignore))
             (odd  (lambda (x) (if (= x 0) nil (funcall even (1- x))))))
      (list (funcall even 42) (funcall odd 42)))

This construct is used for recursive functions, not recursive expressions.
Recursive expressions can't be handled without going for some kind of
lazy evaluation strategy which is quite difficult to pull off in a macro.

> Dunno how useful it is at all that the EXPs can use any of the defined
> functions.

As mentioned (admittedly, a long time ago), the purpose is to *compute*
the recursive functions, e.g.

    (cl-labels ((f1 (if (fboundp 'foo)
                        (lambda (x) ... (f1 ..) ...)
                      (lambda (x) ... (f1 ..) ...))))
      ...)

My original motivation is to define recursive oclosures (for PEG):

    (cl-labels ((f1 (oclosure-lambda ... (f1 ..) ...)))
      ...)


- Stefan

bug#59786: Allowing arbitrary expressions in cl-labels

[Michael Heerdegen via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >> -forms of the function body.  FUNC is defined in any BODY, as well
> >> +forms of the function body.  FUNC is defined in any BODY or EXP, as well

> This is normal. [...]

Ok - fine with me.

But can we avoid saying a FUNC would be defined in any EXP and try be
clearer - like "defined in any EXP _result_" or so?

EXP is the form that computes a function binding, so AFAIU it is wrong
what we are saying - at least confusing.  Do you follow what I mean?


Michael.

bug#59786: Allowing arbitrary expressions in cl-labels

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Pushed to `master`, closing,


        Stefan

bug#59786: Allowing arbitrary expressions in cl-labels

[Michael Heerdegen via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

| (cl-labels): Add support for (FUNC EXP) bindings (bug#59786)
|
| Allow `cl-labels` to use the same (FUNC EXP) bindings as were already added
| to `cl-flet` in Emacs-25.  The Info doc (mistakenly) already documented this
| new feature.
|
| * lisp/emacs-lisp/cl-macs.el (cl--self-tco-on-form): New function.
| (cl-labels): Use it to add support for (FUNC EXP) bindings.
|
| * test/lisp/emacs-lisp/cl-macs-tests.el (cl-macs--labels): Add test for
| tail-recursive (FUNC EXP) bindings.

Thanks... but what happened to your `cl-lib-test-labels' ert test for
"cl-lib-tests.el"?


Michael.

bug#59786: Allowing arbitrary expressions in cl-labels

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> | (cl-labels): Add support for (FUNC EXP) bindings (bug#59786)
> |
> | Allow `cl-labels` to use the same (FUNC EXP) bindings as were already added
> | to `cl-flet` in Emacs-25.  The Info doc (mistakenly) already documented this
> | new feature.
> |
> | * lisp/emacs-lisp/cl-macs.el (cl--self-tco-on-form): New function.
> | (cl-labels): Use it to add support for (FUNC EXP) bindings.
> |
> | * test/lisp/emacs-lisp/cl-macs-tests.el (cl-macs--labels): Add test for
> | tail-recursive (FUNC EXP) bindings.
>
> Thanks... but what happened to your `cl-lib-test-labels' ert test for
> "cl-lib-tests.el"?

I moved it to `cl-macs--labels` in `cl-macs-tests.el` after noticing
that this is where the other `cl-labels` tests were.


        Stefan

bug#59786: Allowing arbitrary expressions in cl-labels

[Michael Heerdegen via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I moved it to `cl-macs--labels` in `cl-macs-tests.el` after noticing
> that this is where the other `cl-labels` tests were.

Ah ok, I had missed it.

Thx,

Michael.