lexical-let detail semantics

lexical-let detail semantics

[Daniel Kraft]

Hi,

I'm working on an implementation of elisp for GNU Guile, and want to 
include the lexical-let construct from the Common Lisp Extensions of 
elisp.  There are some details of its semantics I'm not sure about and 
that are not clarified in the documentation of lexical-let, so I had to 
do some experiments.  Here are two questions that came up during these 
where I'd love to hear comments from regular elisp users:


1) let within lexical-let:

(setq a 1)
(defun dyna () a)
(lexical-let ((a 2))
   (let ((a 3))
     (print (dyna))))
   => 1

My first thought was that a let within the lexical scope of another 
lexical-let would revert the symbols to dynamic scoping again, but it 
seems that let behaves just as if it was lexical-let for symbols already 
lexically bound.

Is this 'expected behaviour' or something 'by chance'?  Do you think it 
is necessary for compatibility with (most) existing code to mimic this 
behaviour or would it be ok for the code above to print 3?

In contrast, the code:

(setq a 1)
(defun dyna () a)
(lexical-let ((a 2))
   ((lambda (a)
      (print (dyna))) 3))
   => 3

does indeed revert a to dynamic binding...  This seems somewhat 
inconsistent to me (although of course argument-lists and let's are not 
really the same thing).


2) Closures:

I'm happy that lexical-let works well to build closures (and in fact it 
seems that this is the main intention for lexical-let at all); however 
this code does not work as expected:

(setq a 1)
(lexical-let ((a 2))
   ((lambda () (print a))))
   => 1

I don't know why, but it seems that calling a closure directly fails, 
while storing it and calling it later succeeds (as in the examples at 
http://www.delorie.com/gnu/docs/emacs/cl_21.html for instance).  Is this 
a bug or again something expected?  If the latter, what's the exact 
rationale and semantics then?

Thank you very much for your help!
Daniel

Re: lexical-let detail semantics

[A.Politz]

On Jul 27, 11:50 am, Daniel Kraft <d...@domob.eu> wrote:
[...]
> I'm happy that lexical-let works well to build closures (and in fact it
> seems that this is the main intention for lexical-let at all); however
> this code does not work as expected:
>
> (setq a 1)
> (lexical-let ((a 2))
>    ((lambda () (print a))))
>    => 1
>

I don't know, but this prints `2' in my emacs (GNU Emacs 22.3.1).

-ap
> I don't know why, but it seems that calling a closure directly fails,
> while storing it and calling it later succeeds (as in the examples athttp://www.delorie.com/gnu/docs/emacs/cl_21.htmlfor instance).  Is this
> a bug or again something expected?  If the latter, what's the exact
> rationale and semantics then?
>
> Thank you very much for your help!
> Daniel

Re: lexical-let detail semantics

[Daniel Kraft]

A.Politz wrote:
> On Jul 27, 11:50 am, Daniel Kraft <d...@domob.eu> wrote:
> [...]
>> I'm happy that lexical-let works well to build closures (and in fact it
>> seems that this is the main intention for lexical-let at all); however
>> this code does not work as expected:
>>
>> (setq a 1)
>> (lexical-let ((a 2))
>>    ((lambda () (print a))))
>>    => 1
>>
> 
> I don't know, but this prints `2' in my emacs (GNU Emacs 22.3.1).

I'm using 21.3.1, and there it is 1; but I checked with a development 
build, and yes, then I get 2.  So it seems that was really a bug that 
got fixed, glad to know this!

Thanks for pointing this out!

Daniel

Re: lexical-let detail semantics

[Pascal J. Bourguignon]

Daniel Kraft <d@domob.eu> writes:

> Hi,
>
> I'm working on an implementation of elisp for GNU Guile, and want to
> include the lexical-let construct from the Common Lisp Extensions of
> elisp.  There are some details of its semantics I'm not sure about and
> that are not clarified in the documentation of lexical-let, so I had
> to do some experiments.  Here are two questions that came up during
> these where I'd love to hear comments from regular elisp users:
>
>
> 1) let within lexical-let:
>
> (setq a 1)
> (defun dyna () a)
> (lexical-let ((a 2))
>   (let ((a 3))
>     (print (dyna))))
>   => 1

If you have a look at the macro expansion of lexical-let, it looks
like it is the intended behavior.

(macroexpand '(lexical-let ((a 2))
               (let ((a 3))
                 (print (dyna)))))
--> (let ((--cl-a-- 2)) (letf (((symbol-value (quote --cl-a--)) 3)) (print (dyna))))



> My first thought was that a let within the lexical scope of another

> lexical-let would revert the symbols to dynamic scoping again, but it
> seems that let behaves just as if it was lexical-let for symbols
> already lexically bound.
>
> Is this 'expected behaviour' or something 'by chance'?  Do you think
> it is necessary for compatibility with (most) existing code to mimic
> this behaviour or would it be ok for the code above to print 3?

I agree, it's not what I'd expect either.

Have a look at the Common Lisp reference:
http://www.lispworks.com/documentation/HyperSpec/Body/s_let_l.htm
http://www.lispworks.com/documentation/HyperSpec/Body/d_specia.htm


I would translate the above forms in Common Lisp as:

(SETF (SYMBOL-VALUE 'A) 1)

(DEFUN DYNA () 
  (DECLARE (SPECIAL A))
  A)

(LET ((A 2))
  (LET ((A 3))
    (DECLARE (SPECIAL A))
    (PRINT (DYNA))))
prints: 3
--> 3



You can avoid the problem by putting the dynaminc bindings outside of
reach of lexical-let:

(defun call-dyna ()
  (let ((a 3)) (print (dyna))))

(lexical-let ((a 2))
        (call-dyna))
prints: 3
--> 3


Notice also that in languages that have both special variables and
lexical variables, it is found worthwhile to keep them in separate
name spaces.  In ISO-LISP, this is done with the (dynamic var) form
for special variables.  In Common Lisp it's done with the *earmuff*
convention.


(defvar *a* 1)
(defun dyna ()
    (print *a*))
(lexical-let ((a 2))
   (let ((*a* 3))
     (dyna)))
prints: 3
--> 3

which is what we expect:

(macroexpand '(lexical-let ((a 2))
                 (let ((*a* 3))
                   (dyna))))
--> (let ((--cl-a-- 2)) (let ((*a* 3)) (dyna)))





You may report the bug to the maintainers, but I'm not sure it's
worthwhile.  If you want a real language, perhaps you could use
emacs-cl?  http://www.lisp.se/emacs-cl/


> In contrast, the code:
>
> (setq a 1)
> (defun dyna () a)
> (lexical-let ((a 2))
>   ((lambda (a)
>      (print (dyna))) 3))
>   => 3
>
> does indeed revert a to dynamic binding...  This seems somewhat
> inconsistent to me (although of course argument-lists and let's are
> not really the same thing).

Yes, there's a (theorical) equivalence between lambda and let.

In this case again the macroexpansion explains why it works:

(macroexpand '(lexical-let ((a 2))
               ((lambda (a)
                  (print (dyna))) 3)))
--> (let ((--cl-a-- 2)) (funcall (function (lambda (a) (print (dyna)))) 3))



> 2) Closures:
>
> I'm happy that lexical-let works well to build closures (and in fact
> it seems that this is the main intention for lexical-let at all);
> however this code does not work as expected:
>
> (setq a 1)
> (lexical-let ((a 2))
>   ((lambda () (print a))))
>   => 1
>
> I don't know why, but it seems that calling a closure directly fails,
> while storing it and calling it later succeeds (as in the examples at
> http://www.delorie.com/gnu/docs/emacs/cl_21.html for instance).  Is
> this a bug or again something expected?  If the latter, what's the
> exact rationale and semantics then?

I guess you have a bug in your version.  Mine works ok.
Again, the macroexpansion explains what lexical-let does in this case:

(macroexpand '(lexical-let ((a 2))
               ((lambda () (print a)))))
--> (let ((--cl-a-- (make-symbol "--a--")))
      (setf (symbol-value --cl-a--) 2)
      (funcall (list (quote lambda) (quote (&rest --cl-rest--))
                     (list (quote apply)
                           (function (lambda (G93796) (print (symbol-value G93796))))
                           (list (quote quote) --cl-a--)
                           (quote --cl-rest--)))))

In emacs-version "22.2.1", I get the right result:

(lexical-let ((a 2))
    ((lambda () (print a))))
prints: 2
--> 2


-- 
__Pascal Bourguignon__

Re: lexical-let detail semantics

[Daniel Kraft]

Hi Pascal,

thanks for your nice answers!

Pascal J. Bourguignon wrote:
> You can avoid the problem by putting the dynaminc bindings outside of
> reach of lexical-let:
> 
> (defun call-dyna ()
>   (let ((a 3)) (print (dyna))))
> 
> (lexical-let ((a 2))
>         (call-dyna))
> prints: 3
> --> 3
> 
> 
> Notice also that in languages that have both special variables and
> lexical variables, it is found worthwhile to keep them in separate
> name spaces.  In ISO-LISP, this is done with the (dynamic var) form
> for special variables.  In Common Lisp it's done with the *earmuff*
> convention.

That's true of course (and good advice), but unfortunatly I'm working on 
another implementation of elisp which should include lexical-let 
natively (instead of being a user) and thus I'm forced to care about 
such special cases, too.

> You may report the bug to the maintainers, but I'm not sure it's
> worthwhile.  If you want a real language, perhaps you could use
> emacs-cl?  http://www.lisp.se/emacs-cl/

Hm...  My main point was whether this is expected behaviour I should 
mimic in my implementation, but I take your response that you would 
prefer to have my implementation behave differently (like what I 
described as my expectation)?

Do you think this would lead to compatibility problems with existing code?

>> 2) Closures:
>>
>> I'm happy that lexical-let works well to build closures (and in fact
>> it seems that this is the main intention for lexical-let at all);
>> however this code does not work as expected:
>>
>> (setq a 1)
>> (lexical-let ((a 2))
>>   ((lambda () (print a))))
>>   => 1
>>
>> I don't know why, but it seems that calling a closure directly fails,
>> while storing it and calling it later succeeds (as in the examples at
>> http://www.delorie.com/gnu/docs/emacs/cl_21.html for instance).  Is
>> this a bug or again something expected?  If the latter, what's the
>> exact rationale and semantics then?
> 
> I guess you have a bug in your version.  Mine works ok.
> Again, the macroexpansion explains what lexical-let does in this case:

Yes, that's right; I checked with a current development version of emacs 
and there it works as expected.

Yours,
Daniel

Re: lexical-let detail semantics

[Pascal J. Bourguignon]

Daniel Kraft <d@domob.eu> writes:
> Hm...  My main point was whether this is expected behaviour I should
> mimic in my implementation, but I take your response that you would
> prefer to have my implementation behave differently (like what I
> described as my expectation)?

Yes.  


   Personnaly, the closer it's to Common Lisp, the better.  Instead of
   providing a lexical-let, I would provide a (with-cl-semantics ...)
   macro where let, let*, lambda, etc, would behave like in CL, and
   where (declare (special var)) would be needed to get dynamic
   binding like in emacs lisp.  defun* would include an implicit
   with-cl-semantics...


> Do you think this would lead to compatibility problems with existing code?

Be sure to document clearly how it works.  


You may try to "grep" all the .el you can find to see how lexical-let
is used.

# locate -r '\.el*$'> /tmp/els
# wc -l /tmp/els
58179 /tmp/els
# xargs cat < /tmp/els | grep lexical-let | wc -l
3485

That's more uses of lexical-let that I would have expected.  You could
use a function such as my map-sexps (in
http://darcs.informatimago.com/public/emacs/pjb-sources.el
) to find all the occurences of lexical-let and analyse what's in
their bodies.




As you can see from the various comments, it seems the behavior of
lexical-let is changing depending on the version, so it's most
probable that no code depend on this yet.


-- 
__Pascal Bourguignon__