`pp-eval-expression' behaves strangely under lexical binding

`pp-eval-expression' behaves strangely under lexical binding

[Marcin Borkowski]

Hi all,

put this in an empty buffer, call `normal-mode' (so that
`lexical-binding' is set to t) and evaluate:

--8<---------------cut here---------------start------------->8---
;;; -*- lexical-binding: t; -*-
(setq spam 'eggs)
(let ((spam 'bacon))
  (pp-eval-expression 'spam)
  (message "%s" spam))
--8<---------------cut here---------------end--------------->8---

Then look at *Messages*: "eggs" is printed, then "bacon".

I wanted to use `pp-eval-expression' when debugging (basically as
a pretty-printing variant of `message'), and this behavior hit me.  Why
does this behave in this way, and is it a feature or a bug?

Best

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: `pp-eval-expression' behaves strangely under lexical binding

[Marcin Borkowski]

On 2016-04-17, at 13:43, Marcin Borkowski <mbork@mbork.pl> wrote:

> Hi all,
>
> put this in an empty buffer, call `normal-mode' (so that
> `lexical-binding' is set to t) and evaluate:
>
> --8<---------------cut here---------------start------------->8---
> ;;; -*- lexical-binding: t; -*-
> (setq spam 'eggs)
> (let ((spam 'bacon))
>   (pp-eval-expression 'spam)
>   (message "%s" spam))
> --8<---------------cut here---------------end--------------->8---
>
> Then look at *Messages*: "eggs" is printed, then "bacon".
>
> I wanted to use `pp-eval-expression' when debugging (basically as
> a pretty-printing variant of `message'), and this behavior hit me.  Why
> does this behave in this way, and is it a feature or a bug?

OK, never mind: `pp-eval-expression' is defined in Lisp, so it cannot
work any other way.

But isn't my question still valid for `symbol-value'?

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: `pp-eval-expression' behaves strangely under lexical binding

[Stefan Monnier]

> But isn't my question still valid for `symbol-value'?

No, whether it's implemented in C or in Lisp is irrelevant.  As soon as
you put a symbol into a `quote`, it can't be a reference to a lexical
variable any more because the compiler will treat it as data and data
cannot refer to lexical variables, only to other data.

Dynamically scoped variables are data, on the other hand (they're just
a particular field of a symbol object, which is a kind of "struct").


        Stefan

Re: `pp-eval-expression' behaves strangely under lexical binding

[Marcin Borkowski]

On 2016-04-17, at 16:15, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> But isn't my question still valid for `symbol-value'?
>
> No, whether it's implemented in C or in Lisp is irrelevant.  As soon as
> you put a symbol into a `quote`, it can't be a reference to a lexical
> variable any more because the compiler will treat it as data and data
> cannot refer to lexical variables, only to other data.
>
> Dynamically scoped variables are data, on the other hand (they're just
> a particular field of a symbol object, which is a kind of "struct").

Stefan, thanks for your answer.  I'm not entirely sure I understand it,
but I'm going to give it some thinking, and if it is still unclear, I'll
get back here.

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Faculty of Mathematics and Computer Science
Adam Mickiewicz University

Re: `pp-eval-expression' behaves strangely under lexical binding

[Andy Moreton]

On Sun 17 Apr 2016, Stefan Monnier wrote:

>> But isn't my question still valid for `symbol-value'?
>
> No, whether it's implemented in C or in Lisp is irrelevant.  As soon as
> you put a symbol into a `quote`, it can't be a reference to a lexical
> variable any more because the compiler will treat it as data and data
> cannot refer to lexical variables, only to other data.
>
> Dynamically scoped variables are data, on the other hand (they're just
> a particular field of a symbol object, which is a kind of "struct").

Hi Stefan,

This short description is very helpful. Could you add some similar wording
to the elisp manual to help clarify usage of lexical binding ?

Thanks,

    AndyM

Re: `pp-eval-expression' behaves strangely under lexical binding

[Stefan Monnier]

>>> But isn't my question still valid for `symbol-value'?
>> No, whether it's implemented in C or in Lisp is irrelevant.  As soon as
>> you put a symbol into a `quote`, it can't be a reference to a lexical
>> variable any more because the compiler will treat it as data and data
>> cannot refer to lexical variables, only to other data.
>> Dynamically scoped variables are data, on the other hand (they're just
>> a particular field of a symbol object, which is a kind of "struct").
> This short description is very helpful. Could you add some similar wording
> to the elisp manual to help clarify usage of lexical binding ?

I don't really see how/where to put it in the manual, sadly.
If you give me a sample patch, I promise I'll try to improve it ;-)


        Stefan


PS: Another way to look at those issues is from the "α-renaming" point
of view.  A lexical variable's name should have no impact on the code,
in the sense that the compiler should be able to rename it or get rid of
the name (and indeed Emacs's Elisp compiler gets rid of the lexical
variables's names).  So the compiler can take

    (let ((a 1))
      (foo 'a (+ a 1)))

and treat it as if you had written

    (let ((b 1))
      (foo 'a (+ b 1)))

or even

    (let ((<noname> 1))
      (foo 'a (+ <the-last-var-defined> 1)))

But clearly, it can treat it as if you had written

    (let ((b 1))
      (foo 'b (+ b 1)))

[ Unless maybe the compiler is being told that `foo` takes
  a variable-identifier as first argument.  ]

and even less

    (let ((<noname> 1))
      (foo <reference-to-the-last-var-defined> (+ <the-last-var-defined> 1)))

since `foo` now wouldn't even receive a *symbol* any more but some other
kind of object (lexical variables, at run-time, aren't referenced via
symbols at all).


        Stefan