Why is FUNC in cl-callf not allowed to be an expression?

Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Hello,

I wonder why the FUNC argument in cl-callf is restricted to symbols (and
lambda expressions, though this is not documented).  I don't see any
reason to disallow arbitrary expressions, besides, maybe, that symbols
would become ambiguous.

I really would like to have an gv-callf that would simply interpret the
function argument as an expression.  cl-callf just saves the programmer
from typing the #' but forbids many use cases.

I think I would even like it most like this:

(defmacro gv-callf (call &optional n)
  (gv-letplace (_getter setter) (nth (or n 1) call)
    (funcall setter call)))

Example:

(let ((l (list 1 2)))
  (gv-callf (append l (list 3 4)))
  l)

=> (1 2 3 4)

(let ((l (list (list 'c 'd))))
  (gv-callf (append (list 'a 'b) (car l) (list 'e 'f)) 2)
  l)

=> ((a b c d e f))

That's simpler than current cl-callf, unites cl-callf and cl-callf2 in
something more general, and even eldoc still works in the CALL.


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> I wonder why the FUNC argument in cl-callf is restricted to symbols (and
> lambda expressions, though this is not documented).  I don't see any
> reason to disallow arbitrary expressions, besides, maybe, that symbols
> would become ambiguous.

`callf` has been in cl.el "for ever" but it's not in Common-Lisp, so
I don't really know where it comes from.  I agree that when I moved it
to cl-lib, I missed an opportunity to fix it and make it take a normal
expression as first argument.

> I really would like to have an gv-callf that would simply interpret the
> function argument as an expression.  cl-callf just saves the programmer
> from typing the #' but forbids many use cases.

FWIW, you can circumvent this with

    (cl-flet ((f <EXPR>))
      (cl-callf f ...))

> I think I would even like it most like this:
>
> (defmacro gv-callf (call &optional n)
>   (gv-letplace (_getter setter) (nth (or n 1) call)
>     (funcall setter call)))

It's cute, tho I'm not too fond of specifying the place via a number,
personally,


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > (defmacro gv-callf (call &optional n)
> >   (gv-letplace (_getter setter) (nth (or n 1) call)
> >     (funcall setter call)))
>
> It's cute, tho I'm not too fond of specifying the place via a number,
> personally,

[I should use GETTER btw for more efficient code:

(defmacro gv-callf (call &optional n)
  (unless n (setq n 1))
  (gv-letplace (getter setter) (nth n call)
    (setf (nth n call) getter)
    (funcall setter call)))
]

Yes, but still better than having it as part of the macro name as in
cl-callf2... I don't have any better idea currently.  We could still
extend cl-callf however.


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

>> > (defmacro gv-callf (call &optional n)
>> >   (gv-letplace (_getter setter) (nth (or n 1) call)
>> >     (funcall setter call)))
>>
>> It's cute, tho I'm not too fond of specifying the place via a number,
>> personally,
>
> [I should use GETTER btw for more efficient code:
>
> (defmacro gv-callf (call &optional n)
>   (unless n (setq n 1))
>   (gv-letplace (getter setter) (nth n call)
>     (setf (nth n call) getter)
>     (funcall setter call)))
> ]
>
> Yes, but still better than having it as part of the macro name as in
> cl-callf2... I don't have any better idea currently.  We could still
> extend cl-callf however.

While it doesn't solve the problem, while playing with it I noticed that
I'm less annoyed by the use of a number if I can put it first, as in:

    (gv-callf 2
      (append head (car x) (cdr x)))
      
I'm not sure I can say why, tho.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> While it doesn't solve the problem, while playing with it I noticed that
> I'm less annoyed by the use of a number if I can put it first, as in:
>
>     (gv-callf 2
>       (append head (car x) (cdr x)))
>       
> I'm not sure I can say why, tho.

Now that I look at it again, I guess an "anamorphic" version would be
probably more obvious:

    (gv-modify (nth i l)
       (append head it (cdr x)))

Implementation left as an exercise for the reader,


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> Now that I look at it again, I guess an "anamorphic" version would be
> probably more obvious:
>
>     (gv-modify (nth i l)
>        (append head it (cdr x)))

And I see it solves another downside of (gv-callf N EXP), which is that in

    (gv-callf 2 (append HEAD (nth INDEX LIST) TAIL))
    
the evaluation order will actually end up being

    INDEX; LIST; HEAD; TAIL
    
which is counter-intuitive.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > Now that I look at it again, I guess an "anamorphic" version would be
> > probably more obvious:
> >
> >     (gv-modify (nth i l)
> >        (append head it (cdr x)))

I guess I would rather like it with explicit naming:

(defmacro gv-modify (binding call)
  (declare (indent 1))
  (pcase-let ((`(,var ,place) binding))
    (gv-letplace (getter setter) place
      (funcall setter `(let ((,var ,getter)) ,call)))))

Example:

(let ((l '(x ((2 3)))))
  (gv-modify (plc (car (cadr l)))
    (append '(0 1) plc '(4 5)))
  l)

=> (x ((0 1 2 3 4 5)))

> And I see it solves another downside of (gv-callf N EXP), which is that in
>
>     (gv-callf 2 (append HEAD (nth INDEX LIST) TAIL))
>
> the evaluation order will actually end up being
>
>     INDEX; LIST; HEAD; TAIL
>
> which is counter-intuitive.

The disadvantage is that the `append' call looks a bit pulled apart, but
I think it would be ok for me.


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> (defmacro gv-modify (binding call)

Why not (gv-modify VAR PLACE EXP) ?
I mean, is there any hope/possibility we'll ever want to extend
`binding` to anything else than "a single VAR and a single PLACE"?

Or do you actually like having the extra set of parens around the
VAR-PLACE pair?


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > (defmacro gv-modify (binding call)
>
> Why not (gv-modify VAR PLACE EXP) ?
> I mean, is there any hope/possibility we'll ever want to extend
> `binding` to anything else than "a single VAR and a single PLACE"?

Yeah - I was on a trip, and the first thing that occurred to my when I
was traveling was that these extra parens are nonsense.

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > (defmacro gv-modify (binding call)

Here is another idea:

(defmacro gv-with-place (place f)
  (declare (indent 1))
  (gv-letplace (getter setter) place
    (let ((v (make-symbol "v")))
      `(funcall ,f ,getter (lambda (,v) ,(funcall setter v))))))

Example:

(let ((x '((a))))
  (gv-with-place (car x)
    (lambda (v set)
      (funcall set (append '(1 2) v '(3 4)))))
  x)

=> ((1 2 a 3 4))

The usage looks a bit longer than in all other proposals, but I find it
cleaner.  It's also more general, you can e.g. make setting conditional
as in your (commented) `gv-pushnew!'.  The second arg of `gv-with-place'
being a function also gives a chance for factoring.  More function calls
at run time involved, though.

I see that the suggestion comes near to just use `gv-letplace' in the
first place, but I think it's easier to understand for the end
programmer.


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> (defmacro gv-with-place (place f)
>   (declare (indent 1))
>   (gv-letplace (getter setter) place
>     (let ((v (make-symbol "v")))
>       `(funcall ,f ,getter (lambda (,v) ,(funcall setter v))))))

We could at least avoid the overall `funcall`:

    (defmacro gv-with-place (val setfun place &rest body)
      (declare (indent 3))
      (gv-letplace (getter setter) place
        `(let ((,val ,getter)
               (,setfun (lambda (v) ,(funcall setter 'v))))
           ,@body)))

Along the same lines maybe we could provide a (gv-ref VAL SETFUN) 
pcase-macro, so you could do

    (pcase-let (((gv-ref val setter) (gv-ref PLACE)))
      (unless (member X val) (funcall setter (cons X val))))

Even better would be if we could do

    (pcase-let (((gv-ref p) (gv-ref PLACE)))
      (unless (member X p) (setf p (cons X p))))

but it seems this would require non-trivial changes to pcase (since
basically `p` should not be bound there with a `let` but with
a `cl-symbol-macrolet`).

Still, I think the `gv-modify` discussed earlier hits a sweeter spot.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> > (defmacro gv-with-place (place f)
> >   (declare (indent 1))
> >   (gv-letplace (getter setter) place
> >     (let ((v (make-symbol "v")))
> >       `(funcall ,f ,getter (lambda (,v) ,(funcall setter v))))))
>
> We could at least avoid the overall `funcall`:
>
>     (defmacro gv-with-place (val setfun place &rest body)
>       (declare (indent 3))
>       (gv-letplace (getter setter) place
>         `(let ((,val ,getter)
>                (,setfun (lambda (v) ,(funcall setter 'v))))
>            ,@body)))

Yes, or maybe even

(defmacro gv-with-place (val-sym setfun-sym place &rest body)
  (declare (indent 3))
  (gv-letplace (getter setter) place
    (let ((v (make-symbol "v")))
      `(cl-flet ((,setfun-sym (lambda (,v) ,(funcall setter v))))
         (let ((,val-sym ,getter))
           ,@body)))))

Example:

  (let ((l '(2 3 4)))
    (gv-with-place p set (cdr l)
      (unless (memq 1 p)
        (set (cons 1 p))))
    l)

  ==> (2 1 3 4)

> Along the same lines maybe we could provide a (gv-ref VAL SETFUN)
> pcase-macro, so you could do
>
>     (pcase-let (((gv-ref val setter) (gv-ref PLACE)))
>       (unless (member X val) (funcall setter (cons X val))))
>
> Even better would be if we could do
>
>     (pcase-let (((gv-ref p) (gv-ref PLACE)))
>       (unless (member X p) (setf p (cons X p))))
>
> but it seems this would require non-trivial changes to pcase (since
> basically `p` should not be bound there with a `let` but with
> a `cl-symbol-macrolet`).

I tried to implement more or less that as a normal macro:

(defun gv-ad-hoc-place (val _setter)
  (declare (compiler-macro (lambda (_) val))
           (gv-expander funcall))
  val)

(defmacro gv-do-place (place-sym place &rest body)
  (declare (indent 2))
  (gv-letplace (getter setter) place
    `(let ((val ,getter))
       (cl-symbol-macrolet ((,place-sym (gv-ad-hoc-place val ,setter)))
         ,@body))))

Example:
  (let ((l '(2 3 4)))
    (gv-do-place p (cdr l)
      (unless (memq 1 p)
        (setf p (cons 1 p))))
    l)

  ==> (2 1 3 4)

Note that `gv-synthetic-place' can't be used here because in setf it
would just expand to (setq val ...) with val being the let-bound symbol
in the implementation, instead of something setf'ing the PLACE.

> Still, I think the `gv-modify` discussed earlier hits a sweeter spot.

Is it still your favorite?


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> (defmacro gv-do-place (place-sym place &rest body)
>   (declare (indent 2))
>   (gv-letplace (getter setter) place
>     `(let ((val ,getter))
>        (cl-symbol-macrolet ((,place-sym (gv-ad-hoc-place val ,setter)))
>          ,@body))))

A better name for this might be "gv-place-let", though that could be
confused with "gv-letplace".

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> A better name for this might be "gv-place-let", though that could be
> confused with "gv-letplace".

I played with it a bit.  It may make sense to make this "gv-place-let"
accept a binding list and implement a "gv-place-let*".  Here are two
examples where this would be useful:

(let ((x '(2 . 1)))
  (gv-place-let a (car x)
    (gv-place-let b (cdr x)
      (when (< b a) (cl-rotatef a b))))
  x)

==> (1 . 2)

(let ((l '(1 (2 3) 4)))
  (gv-place-let p1 (nth 1 l)
    (gv-place-let p2 (car p1)
      (setq p2 (list p2 (- p2)))))
  l)
==> (1 ((2 -2) 3) 4)


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> (defun gv-ad-hoc-place (val _setter)
>   (declare (compiler-macro (lambda (_) val))
>            (gv-expander funcall))
>   val)
[...]
> Note that `gv-synthetic-place' can't be used here because in setf it
> would just expand to (setq val ...) with val being the let-bound symbol
> in the implementation, instead of something setf'ing the PLACE.

Hmm... that's indeed what I see, but I haven't yet understood why that
is (or rather, why that doesn't happen to your gv-ad-hoc-place).
I guess that qualifies as a bug in gv-synthetic-place (not that it
matters too much: I can't find a single use of it).


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Hmm... that's indeed what I see, but I haven't yet understood why that
> is (or rather, why that doesn't happen to your gv-ad-hoc-place).  I
> guess that qualifies as a bug in gv-synthetic-place (not that it
> matters too much: I can't find a single use of it).

Well, let's see:

;; works:
(let ((l '(0)))
  (cl-symbol-macrolet ((p (gv-synthetic-place (car l) (lambda (v) `(setcar l ,v)))))
    (setf p 1))
  l) ;; ==> (1)

;; dosn't work:
(let ((l '((0))))
  (let ((cl (car l)))
    (cl-symbol-macrolet ((p (gv-synthetic-place cl (lambda (v) `(setcar cl ,v)))))
      (setf p 1)))
  l) ;; ==> ((0))

;; but this works
(let ((l '((0))))
  (let ((cl (car l)))
    (setf (gv-synthetic-place cl (lambda (v) `(setcar cl ,v))) 1))
  l) ;; ((1))

The problem in the second case is that `cl-symbol-macrolet' is too
eager: it also macroexpands the symbol expansion `p' inside the `setf',
i.e. it expands the `gv-synthetic-place' macro call, to just `cl', so
you get (setf cl 1).

That just doesn't happen to my version since it's a function instead of
a macro (and the compiler macro seems to be applied later).

Any suggestions?


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> The problem in the second case is that `cl-symbol-macrolet' is too
> eager: it also macroexpands the symbol expansion `p' inside the `setf',
> i.e. it expands the `gv-synthetic-place' macro call, to just `cl', so
> you get (setf cl 1).

Something like this would help (i.e. gv-synthetic-place would
work as expected):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-WIP-Small-fix-in-cl-sm-macroexpand.patch --]
[-- Type: text/x-diff, Size: 963 bytes --]

From e6c6e2a360e7970c8c687af580b0fcabc11aaae7 Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <michael_heerdegen@web.de>
Date: Thu, 16 May 2019 17:05:07 +0200
Subject: [PATCH] WIP: Small fix in cl--sm-macroexpand

---
 lisp/emacs-lisp/cl-macs.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
index 8af8fccde7..ce4be75d2b 100644
--- a/lisp/emacs-lisp/cl-macs.el
+++ b/lisp/emacs-lisp/cl-macs.el
@@ -2152,7 +2152,8 @@ cl--sm-macroexpand
              ;; Perform symbol-macro expansion.
              (let ((symval (assq exp venv)))
                (when symval
-                 (setq exp (cadr symval)))))
+                 (setq exp (cadr symval))
+                 nil)))
             (`(setq . ,_)
              ;; Convert setq to setf if required by symbol-macro expansion.
              (let* ((args (mapcar (lambda (f) (macroexpand f env))
--
2.20.1


[-- Attachment #3: Type: text/plain, Size: 131 bytes --]


but I have no clue if it's correct or a good idea.  Symbol macro
bindings to macro forms are rare in the Emacs sources.

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> Something like this would help (i.e. gv-synthetic-place would
> work as expected):
[...]
> but I have no clue if it's correct or a good idea.

It's not correct in general, no.  But it pointed me to the origin of the
problem.  I installed a fix.

This said, having `gv-expander` on a macro is always risky business
because it makes the result of macroexpansion sensitive to the order in
which things are expanded.  Of course, having both compiler-macro and
gv-expander suffers fundamentally from the same problem, tho to
a lesser extent.

Related to your patch, we should change `macroexpand` to call
`macroexpand-1` and then make `cl-symbol-macrolet` advise
`macroexpand-1` so it doesn't need the `while` loop.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > Something like this would help (i.e. gv-synthetic-place would
> > work as expected):
> [...]
> > but I have no clue if it's correct or a good idea.
>
> It's not correct in general, no.  But it pointed me to the origin of the
> problem.  I installed a fix.

I had a hard time to understand why that works until I found that
(setf p ...) with symbol p symbol-macro bound is first expanded into
setq (treating p as a simple symbol) and that it treated by your fix
(and gets transformed into a setf again).  Ok, why not.

> Related to your patch, we should change `macroexpand` to call
> `macroexpand-1` and then make `cl-symbol-macrolet` advise
> `macroexpand-1` so it doesn't need the `while` loop.

Sounds reasonable, but I don't volunteer to do it.

What do think now about `gv-place-let'?  Does that look acceptable?

Thanks,

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> What do think now about `gv-place-let'?  Does that look acceptable?

The existence of both `gv-letplace` and `gv-place-let` is kind of
problematic for the occasional user who'll struggle to remember which
is which.

So I think it needs a name which clarifies the difference.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> So I think it needs a name which clarifies the difference.

Hmm, ok.  Here is a draft supporting multiple (parallel) bindings.  The
macro builds code by recursively calling `gv-letplace', the innermost
expression is one big symbol-macrolet that makes use of the bound
getters and setters.

#+begin_src emacs-lisp
(defmacro gv-place-bind (bindings &rest body)
  "Make place expression bindings.
BINDINGS is a list of elements of the form (VAR PLACE).

Eval BODY with...

The effect is very similar to `cl-symbol-macrolet'
but preferred for place expressions since it produces more
efficient code.

\(fn ((VAR PLACE) ...) FORM...)"
  (declare (indent 1))
  (letrec ((helper
            (lambda (bindings symbols+getters+setters body)
              (if bindings
                  (let ((binding (car bindings)))
                    (gv-letplace (getter setter) (cadr binding)
                      (funcall helper
                               (cdr bindings)
                               (cons (list (car binding) getter setter)
                                     symbols+getters+setters)
                               body)))
                `(cl-symbol-macrolet
                     ,(mapcar
                       (lambda (entry)
                         `(,(car entry)
                           (gv-synthetic-place ,(cadr entry) ,(caddr entry))))
                       (nreverse symbols+getters+setters))
                   ,@body)))))
    (funcall helper bindings '() body)))
#+end_src

;; Example:
(let ((l '(2 3 4)))
    (gv-place-bind ((p (cdr l)))
      (unless (memq 1 p)
        (setf p (cons 1 p))))
    l)
;; ==> (2 1 3 4)

Writing the docstring made me thoughtful though - how is this different
from symbol-macrolet?  An advantage is that it generates a bit more
efficient code for "complicated" (nested) place expressions.

OTOH, the purpose of symbol-macrolet is, at the end, more or less
defining abbreviations of place expressions.  So I wonder now if the
right thing to do is rather to improve symbol-macrolet instead to make
it generate better code by consulting getters and setters itself,
instead of blindly substituting.

My second point in this message: thinking once more about callf, we
could also support a syntax like (callf (with EXPR) PLACE ARG) or
something like that to support expressions as first arg.

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> (defmacro gv-place-bind (bindings &rest body)

Hmm... "bind", eh?  I don't think that's different enough.
How 'bout something more like a "gv-alias"?

> Writing the docstring made me thoughtful though - how is this different
> from symbol-macrolet?

Indeed, they're closely related but symbol-macrolet does not "evaluates"
anything at the "binding" site.  Also it can also be used for purposes
different from "places", e.g. to keep a log of all the accessed to
a "variable" (tho, we could just as well create a (journaled PLACE)
pseudo place which would record accesses in the very same way).

Yeah, maybe a "place alias" is a generalization of symbol-macro.

Actually, no: symbol-macro can also expand to other macro-calls which
are supposed to be re-expanded at each use-site, AFAIK, so it can
macro-expand to something different every time the "variable" is
referenced, which is not the case of places, I think.

Maybe people over on comp.lang.lisp (or other Lisp discussion area)
would have some insight about this.

> My second point in this message: thinking once more about callf, we
> could also support a syntax like (callf (with EXPR) PLACE ARG) or
> something like that to support expressions as first arg.

You can already do that, except that `with` is spelled `lambda (it)`:

    (cl-callf (lambda (it) EXPR) PLACE)
  
Notice that the gv-callf we discussed are equivalent to the above rather
than to something where the EXPR is evaluated to decide which function
to call.  To me, the benefit of

    (gv-modify it PLACE EXPR)
    
[with or without the `it`] is that expressions appear in the order in
which they're evaluated, and also that it looks more like `setf`, which
is right since this is about doing a `setf` just with the added twist
that it can refer to the old value of the variable to compute the
new one.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > My second point in this message: thinking once more about callf, we
> > could also support a syntax like (callf (with EXPR) PLACE ARG) or
> > something like that to support expressions as first arg.
>
> You can already do that, except that `with` is spelled `lambda (it)`:
>
>     (cl-callf (lambda (it) EXPR) PLACE)

Sorry that I haven't been clear: I want EXPR to eval to the function to
be applied, not to the expression to set to.  Like in

  (cl-callf (with (if flag #'1+ #'1-)) my-number)

That's something I occasionally wanted.  Yes, I know I can paraphrase
this with cl-flet.  It's still disappointing that cl-callf can't do it
out of the box.


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> Sorry that I haven't been clear: I want EXPR to eval to the function to
> be applied, not to the expression to set to.  Like in
>
>   (cl-callf (with (if flag #'1+ #'1-)) my-number)

Note that the gv-modify macro we discussed does not allow that either:
it doesn't funcall the result of an EXPR.

So, what you want can be written

    (cl-callf (lambda (x) (if flag (1+ x) (1- x))) my-number)
or
    (cl-callf (lambda (x) (funcall (if flag #'1+ #'1-) x)) my-number)

Just like you'd have to write

    (gv-modify my-number (if flag (1+ it) (1- it)))
or
    (gv-modify my-number (funcall (if flag #'1+ #'1-) it))


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >   (cl-callf (with (if flag #'1+ #'1-)) my-number)
>
> Note that the gv-modify macro we discussed does not allow that either:
> it doesn't funcall the result of an EXPR.

Yeah, I know.  But cl-callf already exists, and the first argument is
interpreted as a function, so would it hurt to remove the restriction to
fbound symbols?

OTOH, it probably wouldn't be good idea to complicate cl-callf if we
also want to add other stuff.

Would you want to see both gv-modify and gv-place-bind (or however we
call it) added?  I tend to be more a fan of gv-place-bind.

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Noam Postavsky]

On Mon, 20 May 2019 at 22:47, Michael Heerdegen
<michael_heerdegen@web.de> wrote:

> > >   (cl-callf (with (if flag #'1+ #'1-)) my-number)

> OTOH, it probably wouldn't be good idea to complicate cl-callf if we
> also want to add other stuff.

Yeah, why not use something like

(funcallf (if flag #'1+ #'1-) my-number)

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Noam Postavsky <npostavs@gmail.com> writes:

> > > >   (cl-callf (with (if flag #'1+ #'1-)) my-number)
>
> > OTOH, it probably wouldn't be good idea to complicate cl-callf if we
> > also want to add other stuff.
>
> Yeah, why not use something like
>
> (funcallf (if flag #'1+ #'1-) my-number)

FWIW, at least you can already write this as

  (cl-callf2 funcall (if flag #'1+ #'1-) my-number)

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> Yeah, I know.  But cl-callf already exists, and the first argument is
> interpreted as a function, so would it hurt to remove the restriction to
> fbound symbols?

FWIW, I don't care what happens to cl-callf ;-)
I just assume that

    (cl-callf (lambda (x) (funcall (if flag #'1+ #'1-) x)) my-number)
    
should compile to the same bytecode as

    (cl-callf (with (if flag #'1+ #'1-)) my-number)

I think the discussion whether the first arg can be an expression or
not is just an indication of a design problem in `cl-callf`.

IOW, I'm more interested in providing a better replacement than in
trying to extend it.

> Would you want to see both gv-modify and gv-place-bind (or however we
> call it) added?  I tend to be more a fan of gv-place-bind.

I don't really have a clear opinion formed on this.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Andy Moreton]

On Mon 20 May 2019, Stefan Monnier wrote:

>> Sorry that I haven't been clear: I want EXPR to eval to the function to
>> be applied, not to the expression to set to.  Like in
>>
>>   (cl-callf (with (if flag #'1+ #'1-)) my-number)
>
> Note that the gv-modify macro we discussed does not allow that either:
> it doesn't funcall the result of an EXPR.
>
> So, what you want can be written
>
>     (cl-callf (lambda (x) (if flag (1+ x) (1- x))) my-number)
> or
>     (cl-callf (lambda (x) (funcall (if flag #'1+ #'1-) x)) my-number)

The documentation for `cl-callf' says:

    Set PLACE to (FUNC PLACE ARGS...).
    FUNC should be an unquoted function name.  PLACE may be a symbol,
    or any generalized variable allowed by ‘setf’.

Please update this (and `cl-callf2') to show that FUNC can be a lambda.

Thanks,

    AndyM

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> The documentation for `cl-callf' says:
>
>     Set PLACE to (FUNC PLACE ARGS...).
>     FUNC should be an unquoted function name.  PLACE may be a symbol,
>     or any generalized variable allowed by ‘setf’.
>
> Please update this (and `cl-callf2') to show that FUNC can be a lambda.

Done in a564d6e8bb (master).  I didn't touch cl-callf2 because its
docstring only relegates to that of cl-callf.

The manual already lists all cases.  Actually it also explicitly
mentions macro names.  I guess I should add that to the doc string as
well?

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Yeah, maybe a "place alias" is a generalization of symbol-macro.
>
> Actually, no: [...]

I see Graham using symbol macros to abbreviate normal code in "On Lisp".
That's enough evidence for me not to touch symbol-macrolet.

> [...] symbol-macro can also expand to other macro-calls which are
> supposed to be re-expanded at each use-site, AFAIK, so it can
> macro-expand to something different every time the "variable" is
> referenced, which is not the case of places, I think.

With our current implementation of local macros, only the expansion
(compile) time matters, so this not so simple to do:

(let ((l '(0 1 2 3))
      (i '(0 1 2 3))
      (flag nil))
  (cl-macrolet ((yyy () (if flag '(nth 2 l) '(nth 3 i))))
    (cl-symbol-macrolet ((xxx (yyy)))
      (setf xxx 0)
      (list l i))))
|- void-variable: flag


(defvar flag t)

(defmacro zzz () (if flag '(nth 2 l) '(nth 3 i)))

(let ((l '(0 1 2 3))
      (i '(0 1 2 3))
      (flag nil))
  (cl-symbol-macrolet ((xxx (zzz)))
    (setf xxx 0)
    (list l i)))

==> ((0 1 2 3)
     (0 1 2 0)) ;; FLAG has been checked at expansion time

I could move the check of FLAG into the expansion, but that's not what
you meant, and since you made `if` forms place expressions, it would not
even be a counterexample any more.

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 711 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> With our current implementation of local macros, only the expansion
> (compile) time matters, so this not so simple to do:
>
> (let ((l '(0 1 2 3))
>       (i '(0 1 2 3))
>       (flag nil))
>   (cl-macrolet ((yyy () (if flag '(nth 2 l) '(nth 3 i))))
>     (cl-symbol-macrolet ((xxx (yyy)))
>       (setf xxx 0)
>       (list l i))))
> |- void-variable: flag

Most Common Lisps seem to barf in this case (macro definition
referencing local variables).  I only found it explicitly described
here:

  http://www.lispworks.com/documentation/HyperSpec/Body/s_flet_.htm#macrolet

Anyway, I would like to speak out that one should not do that, how about
this?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-doc-misc-cl.texi-Macro-Bindings-State-expansion-time.patch --]
[-- Type: text/x-diff, Size: 768 bytes --]

From ae8e42cf23e4883aed0177aad30d56f0e3e5086b Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <michael_heerdegen@web.de>
Date: Thu, 23 May 2019 03:33:04 +0200
Subject: [PATCH] * doc/misc/cl.texi (Macro Bindings): State expansion time

---
 doc/misc/cl.texi | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/misc/cl.texi b/doc/misc/cl.texi
index eb06791ba9..c96b9b6c3e 100644
--- a/doc/misc/cl.texi
+++ b/doc/misc/cl.texi
@@ -1360,6 +1360,8 @@ Macro Bindings
 affect only calls that appear physically within the body
 @var{forms}, possibly after expansion of other macros in the
 body.
+
+Like global macros local macros are expanded at compile-time.
 @end defmac

 @defmac cl-symbol-macrolet (bindings@dots{}) forms@dots{}
--
2.20.1


[-- Attachment #3: Type: text/plain, Size: 11 bytes --]



Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> @@ -1360,6 +1360,8 @@ Macro Bindings
>  affect only calls that appear physically within the body
>  @var{forms}, possibly after expansion of other macros in the
>  body.
> +
> +Like global macros local macros are expanded at compile-time.
>  @end defmac

Of course, if you don't compile the code, there's no "compile-time", so
this description becomes a bit weird.

Macros can be expanded at any time before executing the code, so indeed
they can't reliably make use of variables bound in the code surrounding
the macro call.  Whether they're local or global macros makes no
difference in this respect.

In practice, you basically can't compile code without first expanding
the macros it invokes, so when the code is compiled, the macros are
usually expanded at that time.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Macros can be expanded at any time before executing the code, so indeed
> they can't reliably make use of variables bound in the code surrounding
> the macro call.  Whether they're local or global macros makes no
> difference in this respect.

I would like to speak that out somewhere because it's not obvious, and
also explicitly for local macros (because the current docstring saying
"This is like `cl-flet'[...]" somehow made me think that they might be
different from global macros in this regard).

Can we say that macros are "expanded in the global environment", or
should we just say that the expander functions are not allowed to refer
to local variables (though, that would still leave some questions open,
e.g. what about local (flet-bound) local functions or other local
macros?).

My patch spoke about when these macros are expanded because of such
subtleties, I thought it could be better to imagine with that wording.


Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

> I would like to speak that out somewhere because it's not obvious, and

Agreed.

> also explicitly for local macros (because the current docstring saying

I don't see them as any different, but clearly some users don't see it
this way, so we can put a reminder there, indeed.

> Can we say that macros are "expanded in the global environment", or

Sounds good.

> should we just say that the expander functions are not allowed to refer
> to local variables (though, that would still leave some questions open,
> e.g. what about local (flet-bound) local functions or other local
> macros?).

Something like it, yes (using "refer" can be tricky because they can
emit code which then refers to it, and this can be seen as "the macro
refers to" even though technically it's different).

> My patch spoke about when these macros are expanded because of such
> subtleties, I thought it could be better to imagine with that wording.

In macros.texi (@node Compiling Macros) as well as in loading.texi
(@cindex eager macro expansion) we allude to when macros are expanded,
but I think we should be more upfront about it somewhere.

A separate @node about when macro expansion can take place and what
environment it can rely on would be welcome.


        Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 164 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > Can we say that macros are "expanded in the global environment", or
>
> Sounds good.

Ok, I installed this:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-doc-misc-cl.texi-Macro-Bindings-Add-a-detail.patch --]
[-- Type: text/x-diff, Size: 935 bytes --]

From 997ac9f829059bf37c81fd586910c834394951e6 Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <michael_heerdegen@web.de>
Date: Thu, 23 May 2019 03:33:04 +0200
Subject: [PATCH] * doc/misc/cl.texi (Macro Bindings): Add a detail

Say that calls of 'cl-macrolet' bound macros are expanded in the
global environment.
---
 doc/misc/cl.texi | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/doc/misc/cl.texi b/doc/misc/cl.texi
index eb06791ba9..ee73c65b78 100644
--- a/doc/misc/cl.texi
+++ b/doc/misc/cl.texi
@@ -1359,7 +1359,8 @@ Macro Bindings
 scoped.  The @code{cl-macrolet} binding will
 affect only calls that appear physically within the body
 @var{forms}, possibly after expansion of other macros in the
-body.
+body.  Calls of @code{cl-macrolet} bound macros are expanded in the
+global environment.
 @end defmac

 @defmac cl-symbol-macrolet (bindings@dots{}) forms@dots{}
--
2.20.1


[-- Attachment #3: Type: text/plain, Size: 215 bytes --]


> A separate @node about when macro expansion can take place and what
> environment it can rely on would be welcome.

That would be good, yes, though, I will probably not be the one writing
it.


Thanks,

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Indeed, they're closely related but symbol-macrolet does not "evaluates"
> anything at the "binding" site.  Also it can also be used for purposes
> different from "places", e.g. to keep a log of all the accessed to
> a "variable" (tho, we could just as well create a (journaled PLACE)
> pseudo place which would record accesses in the very same way).

I also come to the conclusion that symbol-macrolet and place binding are
two different things.

E.g. in

(macroexpand-all
 '(gv-place-bind ((p (car (cdr x))))
    (progn (cl-incf p) p)))

==>
 (let*
    ((#1=#:v
      (cdr x)))
  (progn
    (setcar #1#
            (1+ #2=(car #1#)))
    #2#))

we want that the (cdr x) "calculation" is factored out because it is
always the same (an "inner place" reference so to say).

OTOH, `symbol-macrolet' might be used for arbitrary calculations.  It
would be surprising (wrong) if in

(symbol-macrolet ((r (list (random))))
  (list r r))

the `random' function would be called only once.

One can't reliably guess from the bound expression which case the coder
wants, so I'll leave s-macrolet as is.

Michael.

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Stefan Monnier]

>> Indeed, they're closely related but symbol-macrolet does not "evaluates"
>> anything at the "binding" site.  Also it can also be used for purposes
>> different from "places", e.g. to keep a log of all the accessed to
>> a "variable" (tho, we could just as well create a (journaled PLACE)
>> pseudo place which would record accesses in the very same way).
>
> I also come to the conclusion that symbol-macrolet and place binding are
> two different things.

Right.  I think you can always use one to implement the functionality of
the other, but they're different.

> OTOH, `symbol-macrolet' might be used for arbitrary calculations.  It
> would be surprising (wrong) if in
>
>     (symbol-macrolet ((r (list (random))))
>       (list r r))
>
> the `random' function would be called only once.

Indeed, with gv-place-bind you'd need to use something like

    (gv-place-bind ((r (gv-cbn (list (random)))))
      (list r r))

for some definition of `gv-cbn` such as

    (defun gv-cbn (x)
      (declare (gv-expander (lambda (do)
                 (funcall do x (lambda (v) `(setf ,x ,v))))))
      x)


-- Stefan

Re: Why is FUNC in cl-callf not allowed to be an expression?

[Michael Heerdegen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> Even better would be if we could do
>
>     (pcase-let (((gv-ref p) (gv-ref PLACE)))
>       (unless (member X p) (setf p (cons X p))))
>
> but it seems this would require non-trivial changes to pcase (since
> basically `p` should not be bound there with a `let` but with
> a `cl-symbol-macrolet`).

FWIW I'm all for enhancing pcase so that it can establish other types of
bindings in general.

Michael.