Surrounding Lexical Variable Reference in the Body of defun

Surrounding Lexical Variable Reference in the Body of defun

[Deokhwan Kim]

Hi there,

I'm having trouble understanding lexical binding in Emacs 24. I came across the following sentence in the Emacs Lisp manual <http://www.gnu.org/software/emacs/manual/html_node/elisp/Lexical-Binding.html>:

> the code in the body of a defun or defmacro cannot refer to surrounding lexical variables.

It was a great shock to me because it sounded quite awkward and Common Lisp does not have such restriction AFAIK. Rather, I suspected that I might misunderstand what the sentence really meant. So I decided to make some experiments with the following code stored in foo.el:

  ;;; -*- lexical-binding: t -*-
  (let ((x 0))
    (defun counter ()
      (setq x (1+ x))))

  (message "%d" (counter))
  (message "%d" (counter))

Surprisingly, when I ran it in the form of source code, it worked:

  $ emacs -Q -batch -l foo.el
  1
  2

On the other hand, when I tried to byte-compile it, I got the following warning messages:

  $ emacs -Q -batch -f batch-byte-compile foo.el
  In toplevel form:
  foo.el:2:1:Warning: Function counter will ignore its context (x)
  foo.el:2:1:Warning: Unused lexical variable `x'
  foo.el:4:11:Warning: reference to free variable `x'
  foo.el:4:17:Warning: assignment to free variable `x'

  In end of data:
  foo.el:8:1:Warning: the function `counter' is not known to be defined.
  Wrote foo.elc

When I ran the resulting byte-compiled code, I got an error as the manual claims:

  $ emacs -Q -batch -l foo.elc
  Symbol's value as variable is void: x

Now I'm so confused. Here are my two questions:

  1. Why does this restriction exists? Is it inevitable because of some design decision of Emacs? Or is it temporary and removed in a (near) future release?
  2. Why does the original source code behave differently from its compiled code?

Best regards,
Deokhwan Kim

Re: Surrounding Lexical Variable Reference in the Body of defun

[Andreas Röhler]

Am 21.08.2012 08:07, schrieb Deokhwan Kim:
> Hi there,
>
> I'm having trouble understanding lexical binding in Emacs 24. I came across the following sentence in the Emacs Lisp manual <http://www.gnu.org/software/emacs/manual/html_node/elisp/Lexical-Binding.html>:
>
>> the code in the body of a defun or defmacro cannot refer to surrounding lexical variables.
>
> It was a great shock to me because it sounded quite awkward and Common Lisp does not have such restriction AFAIK. Rather, I suspected that I might misunderstand what the sentence really meant. So I decided to make some experiments with the following code stored in foo.el:
>
>    ;;; -*- lexical-binding: t -*-
>    (let ((x 0))
>      (defun counter ()
>        (setq x (1+ x))))
>
>    (message "%d" (counter))
>    (message "%d" (counter))
>
> Surprisingly, when I ran it in the form of source code, it worked:
>
>    $ emacs -Q -batch -l foo.el
>    1
>    2
>
> On the other hand, when I tried to byte-compile it, I got the following warning messages:
>
>    $ emacs -Q -batch -f batch-byte-compile foo.el
>    In toplevel form:
>    foo.el:2:1:Warning: Function counter will ignore its context (x)
>    foo.el:2:1:Warning: Unused lexical variable `x'
>    foo.el:4:11:Warning: reference to free variable `x'
>    foo.el:4:17:Warning: assignment to free variable `x'
>
>    In end of data:
>    foo.el:8:1:Warning: the function `counter' is not known to be defined.
>    Wrote foo.elc
>
> When I ran the resulting byte-compiled code, I got an error as the manual claims:
>
>    $ emacs -Q -batch -l foo.elc
>    Symbol's value as variable is void: x
>
> Now I'm so confused. Here are my two questions:
>
>    1. Why does this restriction exists? Is it inevitable because of some design decision of Emacs? Or is it temporary and removed in a (near) future release?
>    2. Why does the original source code behave differently from its compiled code?
>
> Best regards,
> Deokhwan Kim
>

as this question isn't raised first time, maybe Emacs developers consider to revert that change?

Re: Surrounding Lexical Variable Reference in the Body of defun

[Stefan Monnier]

>   1. Why does this restriction exists? Is it inevitable because of
>      some design decision of Emacs? Or is it temporary and removed in
>      a (near) future release?

It is because of a technical limitation in the byte-compiler's handling
of dynamically loaded docstrings.

You can work around this by replacing your defun
with (defalias counter (lambda () (setq x (1+ x)))).

I believe this is fixed in the Emacs trunk (where defun is just a macro
that expands to a defalias+lambda and where the docstrings handling was
improved accordingly).


        Stefan

Re: Surrounding Lexical Variable Reference in the Body of defun

[justinhj]

Does this mean that if you build from emacs trunk then the code above should compile and run just fine? In which case is the manual wrong (or at least out of date) about warning you not to refer to the surrounding lexical environment from inside a defun?

Justin

> I believe this is fixed in the Emacs trunk (where defun is just a macro
> 
> that expands to a defalias+lambda and where the docstrings handling was
> 
> improved accordingly).
> 
> 
> 
> 
> 
>         Stefan

Re: Surrounding Lexical Variable Reference in the Body of defun

[Deokhwan Kim]

For reference, Revision 108427 (especially, lisp/emacs-lisp/byte-run.el) looks like bringing that change:

  http://bzr.savannah.gnu.org/lh/emacs/trunk/revision/108427

- Deokhwan Kim


On Wednesday, August 22, 2012 11:11:06 AM UTC-4, Stefan Monnier wrote:
> I believe this is fixed in the Emacs trunk (where defun is just a macro
> 
> that expands to a defalias+lambda and where the docstrings handling was
> 
> improved accordingly).