Are there any problems with lexical-let or other cl-macros???

Are there any problems with lexical-let or other cl-macros???

[LanX]

Hi

http://steve-yegge.blogspot.com/2008/11/ejacs-javascript-interpreter-for-emacs.html
steve yegge complains about missing lexical vars roughly giving this
example

(require 'cl)
(defun foo ()
  (setq x 7))

(defun bar ()
  (let ((x 6))
    (foo)
    x))  ; you would expect x to still be 6 here

(message (number-to-string (bar)))

What I don't understand is that simply replacing let with lexical-let
will solve the problem.

IMHO "let" corresponds to "local" in perl while "lexical-let" is  "my"
and everything seems to work as expected...

What am I missing?

Whats the problem with using cl macros, do they introduce any new and
ugly side effects better avoided?

Cheers
  Rolf

Re: Are there any problems with lexical-let or other cl-macros???

[Pascal J. Bourguignon]

LanX <lanx.perl@googlemail.com> writes:

> Hi
>
> http://steve-yegge.blogspot.com/2008/11/ejacs-javascript-interpreter-for-emacs.html
> steve yegge complains about missing lexical vars roughly giving this
> example
>
> (require 'cl)
> (defun foo ()
>   (setq x 7))
>
> (defun bar ()
>   (let ((x 6))
>     (foo)
>     x))  ; you would expect x to still be 6 here
>
> (message (number-to-string (bar)))
>
> What I don't understand is that simply replacing let with lexical-let
> will solve the problem.

Esthetically, but the implementation is not efficient, and cannot be
since the emacs VM (which is a very high level lisp VM) doesn't
provide a way to implement lexical binding at all AFAIK (well perhaps
using a lisp vector and mapping variables to offsets, but this
probably would not be more efficient either).

Let's compare the emacs VM (I introduce a (setf x (+ x x)) form to
have both compiler generate more comparable code, clisp compilers
optimizes out the original let):

    (disassemble (byte-compile (defun* bar ()
                                 (lexical-let ((x 3))
                                   (setf x (+ x x))
                                   (foo)
                                   x))))
    byte code:
      args: nil
    0       constant  3
    1       varbind   --cl-x--
    2       constant  --cl-x--
    3       symbol-value 
    4       constant  --cl-x--
    5       symbol-value 
    6       plus      
    7       varset    --cl-x--
    8       constant  foo
    9       call      0
    10      discard   
    11      constant  --cl-x--
    12      symbol-value 
    13      unbind    1
    14      return    

    (Note that these occurences of --cl-x-- above are all the same
    symbol, but it's not interned, so in a different (let ((x ...))
    ...) it would be a different --cl-x-- symbol; we could have used a
    symbol named x too).


With the clisp VM:

    (disassemble (compile (defun bar ()
                            (let ((x 3))
                              (setf x (+ x x))
                              (foo)
                              x))))
    WARNING in BAR :
    Function FOO is not defined

    Disassembly of function BAR
    (CONST 0) = 3
    (CONST 1) = FOO
    0 required arguments
    0 optional arguments
    No rest parameter
    No keyword parameters
    7 byte-code instructions:
    0     (CONST&PUSH 0)                      ; 3
    1     (LOAD&PUSH 0)
    2     (LOAD&PUSH 1)
    3     (CALLSR&STORE 2 55 0)               ; +
    7     (CALL0 1)                           ; FOO
    9     (POP)
    10    (SKIP&RET 1)
    NIL


In the case of the emacs VM, it has to establish a dynamic binding
with varbind, which is rather costly.  References to variables must go
thru the symbol: you need to keep the symbols around (space), and it
goes thru the symbol (time, one indirection).

In the case of the clisp VM,  for lexical references  the compiler can
drop the variable name (spare the symbol), and the references to the
lexical variable is done by direct addressing (relative to the frame,
or in the case of clisp, the stack).  

clisp byte code is of lower level than emacs byte code, but it is
bound to be more efficient at run-time.  On the other hand, the clisp
compiler is more complex than the emacs byte-compiler.


-- 
__Pascal Bourguignon__
http://www.informatimago.com

Re: Are there any problems with lexical-let or other cl-macros???

[LanX]

> Esthetically, but the implementation is not efficient, and cannot be
> since the emacs VM (which is a very high level lisp VM) doesn't
> provide a way to implement lexical binding at all AFAIK (well perhaps
> using a lisp vector and mapping variables to offsets, but this
> probably would not be more efficient either).
>

So it's only a performance issue? Isn't Elisp supposed to be slow
anyway?

he also writes

"(elisp...) at the very least, it should support reader macros."

could you please explain whats missing in defmacro?

I always thought they are expanded when the parser reads the code to
be compiled... or what is the definition of "reader macro" (couldn't
google good explanations and the elisp info has only one mention at
"16.5 Evaluation During Compilation")

Thanks
  rolf

Re: Are there any problems with lexical-let or other cl-macros???

[Helmut Eller]

* LanX [2010-06-01 16:20+0200] writes:

>> Esthetically, but the implementation is not efficient, and cannot be
>> since the emacs VM (which is a very high level lisp VM) doesn't
>> provide a way to implement lexical binding at all AFAIK (well perhaps
>> using a lisp vector and mapping variables to offsets, but this
>> probably would not be more efficient either).
>>
>
> So it's only a performance issue? Isn't Elisp supposed to be slow
> anyway?
>
> he also writes
>
> "(elisp...) at the very least, it should support reader macros."
>
> could you please explain whats missing in defmacro?
>
> I always thought they are expanded when the parser reads the code to
> be compiled... or what is the definition of "reader macro" (couldn't
> google good explanations and the elisp info has only one mention at
> "16.5 Evaluation During Compilation")

"Reader macros" are not the same as macros defined with defmacro.  In
Common Lisp the reader ie. the function that turns text into
s-expressions can be customized.  A typical reader macro is #' which is
responsible for converting #'foo to (function foo).  Emacs' reader is
not customizable.  Even ' is which turns 'foo into (quote foo) is a
reader macro.

Traditionally reader macros start with # and some are standard, e.g. #(
is for reading vectors and #s for reading structures.  But in principle
every character can have a reader macro e.g. { could be used to parse
hash tables or #/ to parse regexps etc.

A reader macro has access to the text input stream and can construct
arbitrary objects at read time.  The compiler takes the objects returned
by the reader and the compiler expands the macros defined with defmacro.

Helmut

Re: Are there any problems with lexical-let or other cl-macros???

[LanX]

> "Reader macros" are not the same as macros defined with defmacro.  In
> Common Lisp the reader ie. the function that turns text into
> s-expressions can be customized.  A typical reader macro is #' which is
> responsible for converting #'foo to (function foo).  Emacs' reader is
> not customizable.  Even ' is which turns 'foo into (quote foo) is a
> reader macro.

(please correct me if I don't get it right in my words)

So defmacro is restricted to defining macros with "function syntax" -
ie "(macro ...)" -  while reader macros could be triggered by any
character, opening the possibility to even extend the syntax to have
special markup for different data structures?

Interesting... :)

Re: Are there any problems with lexical-let or other cl-macros???

[Helmut Eller]

* LanX [2010-06-01 17:58+0200] writes:

> (please correct me if I don't get it right in my words)
>
> So defmacro is restricted to defining macros with "function syntax" -
> ie "(macro ...)" -  while reader macros could be triggered by any
> character, opening the possibility to even extend the syntax to have
> special markup for different data structures?

Yes right.  defmacro defines a transformation from s-exps to s-exps.  A
reader macro reads text from a stream and returns an s-exp.

> Interesting... :)

Reader macros have been used to write inline XML, SQL, and of course
Python or C like syntaxes, but the problem is that the editor (usually
Emacs) doesn't know how to indent reader macros.  Also the read table
(the data structure which controls the reader) must be set up properly
before compiling; that complicates the build process and interaction
with other tools like the debugger.  It's sometimes useful to parse data
files, but I never use reader macros in source files.

Helmut

Re: Are there any problems with lexical-let or other cl-macros???

[Pascal J. Bourguignon]

Helmut Eller <eller.helmut@gmail.com> writes:

> * LanX [2010-06-01 17:58+0200] writes:
>
>> (please correct me if I don't get it right in my words)
>>
>> So defmacro is restricted to defining macros with "function syntax" -
>> ie "(macro ...)" -  while reader macros could be triggered by any
>> character, opening the possibility to even extend the syntax to have
>> special markup for different data structures?
>
> Yes right.  defmacro defines a transformation from s-exps to s-exps.  A
> reader macro reads text from a stream and returns an s-exp.
>
>> Interesting... :)
>
> Reader macros have been used to write inline XML, SQL, and of course
> Python or C like syntaxes, but the problem is that the editor (usually
> Emacs) doesn't know how to indent reader macros.  Also the read table
> (the data structure which controls the reader) must be set up properly
> before compiling; that complicates the build process and interaction
> with other tools like the debugger.  It's sometimes useful to parse data
> files, but I never use reader macros in source files.

That's just because your source programs have always dealt only with
standard lisp objects.

If you started to  write a lisp program that would have  to deal a lot
with higher-level  data types (ie.  problem oriented data  types), and
you'd  need  to write  often  literals of  that  type,  you would  use
profitably reader macros then.

-- 
__Pascal Bourguignon__
http://www.informatimago.com

Re: Are there any problems with lexical-let or other cl-macros???

[Elena]

On 1 Giu, 15:01, LanX <lanx.p...@googlemail.com> wrote:
> What I don't understand is that simply replacing let with lexical-let
> will solve the problem.

To be more precise, it would exchange such a problem with a memory
leak problem (`lexical-let' leaks memory).

Re: Are there any problems with lexical-let or other cl-macros???

[Stefan Monnier]

>> What I don't understand is that simply replacing let with lexical-let
>> will solve the problem.
> To be more precise, it would exchange such a problem with a memory
> leak problem (`lexical-let' leaks memory).

In which sense does lexical-let leak memory?


        Stefan

Re: Are there any problems with lexical-let or other cl-macros???

[Elena]

On Jul 19, 10:44 pm, Stefan Monnier <monn...@iro.umontreal.ca> wrote:
> In which sense does lexical-let leak memory?

From http://c2.com/cgi/wiki?EmacsLisp :

"Note that variables bound with lexical-let are never released, even
if they are never used. Try

 (loop for i from 1 to 100000 collect (lexical-let ((x i)) '()))

and watch it eat memory. So making infinity (ZeroOneInfinity) lexical
variables is out of the question except for very small values of
infinity."